Add baseline stubs for Python Tensor types

[klecki]

Category: New feature

Description:

Base for the Python Tensor and TensorList types interface files,
generated using stubgen.

C-based types (or types not guaranteed to be present, like np.array),
are removed from signatures and left as a comment or replaced with Any.

This doesn't necessary add new information, just enables static tools
to access the types.

    imgs, labels = pipe.run()
    reveal_type(imgs)

Before:

type_annotations/test_typing_pipelines.py:63: note: Revealed type is "Union[Any, Any]"

After:

type_annotations/test_typing_pipelines.py:63: note: Revealed type is "Union[nvidia.dali.backend_impl.TensorListCPU, nvidia.dali.backend_impl.TensorListGPU]"

Note that the backend contains many more types, like OpSchema, OpSpec, Pipeline, etc.
They are internal, so I didn't expose them, but we should consider adding them to aid our development.

This file is generated and than manually adjusted, I don't see a clear automated workflow for this.
It appears harder to maintain than a list of supported enum values.

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

• Existing tests apply
• New tests added
  • Python tests
  • GTests
  • Benchmark
  • Other
• N/A

Checklist

Documentation

• Existing documentation applies
• Documentation updated
  • Docstring
  • Doxygen
  • RST
  • Jupyter
  • Other
• N/A

DALI team only

Requirements

• Implements new requirements
• Affects existing requirements
• N/A

REQ IDs: N/A

JIRA TASK: N/A

[klecki]

!build

[klecki]

After the last changes, mypy declares the return types of Pipeline.run() as

Revealed type is "builtins.tuple[Union[nvidia.dali.tensors.TensorListCPU, nvidia.dali.tensors.TensorListGPU], ...]"

I am intentionally not adjusting any other imports to see if this change doesn't break anything in the existing code.
It would be good to check the generated docs as well.

[dali-automaton]

CI MESSAGE: [10788983]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [10788983]: BUILD PASSED

[klecki]

It would be good to check the generated docs as well.

the docs set the module explicitly, but still, even though we adjusted the module path, the signatures utilize the backend_impl variant. This is generated by the pybind and inserted into the docstring, I don't know if we can easily override it in the backend or maybe we should replace it when it is loaded into python. Both approaches are kinda weird.

[klecki]

!build

[klecki]

It works

[dali-automaton]

CI MESSAGE: [10841126]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [10841126]: BUILD FAILED

[klecki]

!build

[dali-automaton]

CI MESSAGE: [10842296]: BUILD STARTED

[klecki]

!build

[dali-automaton]

CI MESSAGE: [10842605]: BUILD STARTED

[stiepan]

FYI: I did not look into the documentation as it's not built yet.

[klecki]

!build

[dali-automaton]

CI MESSAGE: [10842605]: BUILD PASSED

[dali-automaton]

CI MESSAGE: [10845923]: BUILD STARTED

[dali-automaton]

CI MESSAGE: [10845923]: BUILD PASSED