Generic types and isinstance results in Any type parameter

[stephenfin]

I have two classes - a Loader class and a Plugin class - that I want to indicate a relationship for. For example:

import abc
from typing import Any, Generic, TypeVar


class BasePlugin:

    def get_token(self) -> str:
        return 'foo'


class BaseLoader(metaclass=abc.ABCMeta):

    @property
    def plugin_class(self) -> type[BasePlugin]:
        raise NotImplementedError()

    @abc.abstractmethod
    def load(self, **kwargs: Any) -> BasePlugin:
        return self.plugin_class(**kwargs)

There are multiple Plugin classes and there is a distinct Loader class for each Plugin class. To indicate that relationship, I have made the base loader a generic class:

class BasePlugin:

    def get_token(self) -> str:
        return 'foo'


T = TypeVar('T', bound='BasePlugin')


class BaseLoader(Generic[T], metaclass=abc.ABCMeta):

    @property
    def plugin_class(self) -> type[T]:
        raise NotImplementedError()

    @abc.abstractmethod
    def load(self, **kwargs: Any) -> T:
        return self.plugin_class(**kwargs)


class PasswordPlugin(BasePlugin):

    def __init__(self, auth_url: str, username: str, password: str):
        super().__init__()
        self.auth_url = auth_url
        self.username = username
        self.password = password


class PasswordLoader(BaseLoader[PasswordPlugin]):

    @property
    def plugin_class(self) -> type[PasswordPlugin]:
        return PasswordPlugin

    def load(self, **kwargs: Any) -> PasswordPlugin:
        if 'username' not in kwargs:
            raise Exception('need username')

        if 'password' not in kwargs:
            raise Exception('need password')

        return super().load(**kwargs)

And this is mostly happy. However, the loaders are configured and sourced from opaque objects (in this case, an argparse.Namespace object returned after parsing CLI opts, but that is an implementation detail).

import argparse

def load_from_parser(
    namespace: argparse.Namespace, **kwargs: Any
) -> BasePlugin | None:
    if not hasattr(namespace, 'os_auth_type'):
        return None

    # narrow type
    if isinstance(namespace.os_auth_type, BaseAuthLoader):
        plugin = namespace.os_auth_type
    else:
        return None

    return plugin.load(**kwargs)

I check this with --strict mode, mypy complains:

Returning Any from function declared to return "BasePlugin | None"  [no-any-return]

Likewise, pyright with strict mode enabled complains:

/tmp/foo.py
  /tmp/foo.py:75:9 - error: Type of "plugin" is partially unknown
    Type of "plugin" is "BaseAuthLoader[Unknown]" (reportUnknownVariableType)
  /tmp/foo.py:75:18 - error: Type of "os_auth_type" is partially unknown
    Type of "os_auth_type" is "BaseAuthLoader[Unknown]" (reportUnknownMemberType)
  /tmp/foo.py:79:12 - error: Return type is unknown (reportUnknownVariableType)
3 errors, 0 warnings, 0 informations

However, BaseLoader.plugin returns T which is bound to BasePlugin, so I would expect to get an instance of BasePlugin or a subclass, not Any. Clearly not though. I also can't narrow the type further using isinstance since that doesn't allow the use of subscripted generics.

I'm guessing my mental model isn't quite right here, so could someone advise me as to what I'm missing and how I can achieve the coupling between two different classes?

[Daverball]

If you don't specify the type of a type parameter it's always considered to be Any, that's part of the gradual guarantee of Python's type system, setting an upper bound does not change that. If you want the default to be something else you need to use PEP-696 and set a default type for the TypeVar.

The default parameter is new however and only available starting with Python 3.13, in earlier versions you will need to use typing_extensions.TypeVar.

The other way to do it would be to add an annotation to plugin i.e.

import argparse

def load_from_parser(
    namespace: argparse.Namespace, **kwargs: Any
) -> BasePlugin | None:
    if not hasattr(namespace, 'os_auth_type'):
        return None

    # narrow type
    if isinstance(namespace.os_auth_type, BaseAuthLoader):
        plugin: BaseLoader[BasePlugin] = namespace.os_auth_type
    else:
        return None

    return plugin.load(**kwargs)

It's also worth noting that, at least based on your example, that you might want to change T to be covariant, otherwise you may run into surprising issues with assignability. E.g. you would not be able to assign PasswordLoader to BaseLoader[BasePlugin], because BaseLoader[PasswordPlugin] is not assignable to BaseLoader[BasePlugin] unless T is covariant.

You may also want to change the name to T_co at that point, to follow naming conventions. Or better yet, switch to a more descriptive name like PluginT, which is generally preferred for type vars that have upper bounds and/or defaults.

[stephenfin]

Thanks for the comprehensive reply. I'll try this out. I have one follow-up question for now, if you'd be so kind. You said:

If you don't specify the type of a type parameter it's always considered to be Any

Is there currently a way to specify a type parameter in this case? ininstance does not accept parameterized generics, so it appears I am stuck with cast or type: ignore comments?

[Daverball]

You could write your own TypeIs/TypeGuard function:

def is_baseloader(x: object) -> TypeIs[BaseLoader[BasePlugin]]:
    return isinstance(x, BaseLoader) and issubclass(x.plugin_class(), BasePlugin)

Although setting a default for the TypeVar seems more sensible here as long as you can change it to be covariant. Otherwise I would stick to just annotating plugin to force Any to be specialized to BasePlugin in the assignment.

You could also just add an additional runtime check to your function to get rid of the error:

def load_from_parser(
    namespace: argparse.Namespace, **kwargs: Any
) -> BasePlugin | None:
    if not hasattr(namespace, 'os_auth_type'):
        return None

    # narrow type
    if isinstance(namespace.os_auth_type, BaseAuthLoader):
        plugin = namespace.os_auth_type
    else:
        return None

    instance = plugin.load(**kwargs)
    assert isinstance(instance, BasePlugin)  # narrows from Any to BasePlugin
    return instance