Getting signature in runtime

[yaitskov]

Hi,

I am experimenting with calling Python function from Haskell via C-API.
I need types to generate correspondent function stubs with Template Haskell.
I launch an embedded Python at compilation stage via Py_Initialize, then PyImport_ImportModule etc.

I found inspect.signature function returns type for function arguments and its result if they are specified via type annotations in Python source. Also default values are returned for optional arguments, which is very helpful for inferring argument types even without annotations. Though Python library - pickle, I am interested in, does not have such annotations at all.

I found typeshed package contains type information for pickle.dump.
Is there an example explaining how to get this type representation after Python interpreter is started i.e. after Py_Initialize?

[Daverball]

Generally this isn't going to be trivial since type annotations often contain forward-references to types that haven't been declared yet. So you will run into issues with runtime introspection even in modules that do have inline type annotations. The reference may never be resolved at runtime, because e.g. the symbol is imported in a if TYPE_CHECKING: block, which only gets evaluated inside type checkers and never at runtime, so you will have a heck of a time resolving that symbol at runtime (you would probably need to parse the module again and analyze the AST to retrieve all the imports and declarations inside if TYPE_CHECKING: blocks).

This is currently almost unavoidable because CPython will not be able to resolve cyclic dependencies. (There is from __future__ import annotations which will turn all annotations implicitly into forward declarations, which helps, but won't solve all cyclic dependency woes). Then on top of that you will have more complex typing constructs like Protocol, Generic and TypeVar, which can be inspected at runtime, but it's difficult to cover all the possible use-cases at runtime.

I don't think there's an easy way to import pyi stubs and merge them with a module at runtime. You will either need to write some static analysis code yourself that makes use of typeshed or try to integrate with something like mypy, to do it for you.

It's also worth noting that the type annotations aren't a hard requirement and quite likely contain some inconsistencies with the runtime code, especially with external stubs. There are some tools like stubtest to help ensure that doesn't happen, but even they can't catch all the errors that may happen and sometimes inconsistencies may be intentional to make the public API less cumbersome to use (but it would break the internal API if it were actually enforced).