mypy and nested generics with type variables

[SimpleArt]

I was trying to create subclasses of collections.abc.ItemsView, implementing my own __and__ with my own Set, and I was running into an error with mypy:

Return type "EmptySet[Tuple[KT, VT]]" of "__and__" incompatible with return type "Set[Tuple[KT, VT]]" in supertype "ItemsView"

MCVE:

from __future__ import annotations
from abc import ABC
from collections.abc import ItemsView, Set
from typing import Any, Generic, Iterable, Iterator, Tuple, TypeVar

T = TypeVar("T")
KT = TypeVar("KT")
VT = TypeVar("VT")


class EmptySet(Set[T], Generic[T]):

    def __and__(self: EmptySet[T], other: Iterable[Any], /) -> EmptySet[T]:
        return self

    def __contains__(self, value: Any) -> bool:
        return False

    def __iter__(self) -> Iterator[T]:
        x: list[T] = []
        return iter(x)

    def __len__(self) -> int:
        return 0


x: Set[Tuple[int, int]] = EmptySet[Tuple[int, int]]()


class EmptyItemsView(ItemsView[KT, VT], EmptySet[Tuple[KT, VT]], ABC, Generic[KT, VT]):

    def __and__(self: EmptyItemsView[KT, VT], other: Iterable[Any], /) -> EmptySet[Tuple[KT, VT]]:
        ...

I assume this is just some strange behavior with nested type variables. How would I go about fixing this?

[erictraut]

The problem is that EmptySet derives from collections.abc.Set[T] rather than typing.Set[T] or set[T]. The typeshed stubs define collections.abc.Set differently from the builtins set. This is probably a bug in the stubs. I think these should be the same.

I'll also note that the __and__ method in the base class ItemsView has an other parameter that can be specified as a keyword, so pyright flags your use of a position-only parameter separator (/) as invalid when overriding this method. I don't think mypy performs this check.

[AlexWaygood]

The typeshed stubs define collections.abc.Set differently from the builtins set. This is probably a bug in the stubs.

collections.abc.Set is the same as typing.AbstractSet in terms of the interface it represents, not the same as typing.Set or builtins.set.

It's unfortunate that the writers of the collections.abc module (which is much older than Python's static-typing framework) chose to give collections.abc.Set such a confusing name. AbstractSet is a much better name than Set for this interface. It's for this very reason that typeshed has a policy (enforced by our linter, the flake8-pyi plugin) to never have from collections.abc import Set at the top of a stub file, and to always do from collections.abc import Set as AbstractSet instead.

[JelleZijlstra]

We should probably make the argument to __and__ positional-only in the stubs, though.

[SimpleArt]

Ah, so ItemsView must return a typing.Set, not a collections.abc.Set. It's impossible to see the difference from the error message alone. Does that mean I should use typing.Set over collections.abc.Set or perhaps just return a typing.Set instead of EmptySet?