-
-
Notifications
You must be signed in to change notification settings - Fork 250
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PEP 621 support for dynamic metadata #1537
Comments
|
Implementing I guess if it's a must have feature, we have to require a working Python interpreter for simplicity. |
See also scikit-build/scikit-build-core#230 |
Thanks @messense. I am a lot better at python than I am at Rust currently. I am learning Rust to speedup our bottlenecks as described above. So I am not (yet) very proficient in it. That said, it is a beautiful language and my first impressions are very good. It will take me a little bit of time to read through the codebase and to understand what parts I would need to change. Thanks a lot for linking the relevant sections. Is it worth waiting a little while before we implement if the PEP is still being finalized? Or are you happy with a BETA implementation like setuptools has? I do like the suggested approach in scikit-build/scikit-build-core#230 - that is much simpler than the backend specific # After
[project.dynamic]
version = {attr="mymod.__version__"}
dependencies = {file="requeriments.in"}
optional-dependencies.dev = {file="dev-requeriments.in"}
optional-dependencies.test = {file="test-requeriments.in"} I would also personally be happy to omit |
I'm not sure if that can cover your use case, but two notes from my side:
For the requirements, pip-tools supports PEP 621. Would keeping all dependencies in pyproject.toml and exporting them with pip-compile (or any other tool of your choice) work for you? I know it's suboptimal to have this duplication, but i expect that eventually all relevant tools will be able to install from pyproject.toml so it's only a temporary solution. In general the big disadvantages of the PEP proposal are that we need to install the requirements beforehand (while currently we can just build an abi3 rust crate and put in an archive, no python interpreter involved), it's performance overhead with extra error surface (launching the provider in a new python interpreter) and static analysis (including someone looking at pyproject.toml) can't see the metadata anymore. That's why i try hard to find solutions that work without the dynamic provider syntax and only use it if there's no other way. |
One use-case not covered with statically putting it in pyproject.toml is automatically deriving version from the output of __version__ = subprocess.check_output(['git', 'describe']).decode('utf8').lstrip('v').strip() (I've found this ticket while trying to simplify version field in my project). |
I would like to see this as well, it matches my current workflow for writing python packages and is particularly nice when configuring optional dependencies. A maturin-specific feature that might be nice is to pull the package version from |
Maybe I'm misreading what you're trying to do, but I would recommend to maybe overthink your deployment strategy a little bit, and possibly change it to something that cargo and poetry are doing. In a gist: Your library should have flexible dependencies. That means in your
These flexibly declare what version you need to have for this library to run. Your deployments on the other side should have locked dependencies, to ensure reproducible builds and rollbacks. That means your
That way you ensure that your library remains more flexible and open for reuse in other projects with other deployments, and your deployments are reproducible using lockfiles. As soon as you start putting the content of Now, of course your library still can contain a lockfile or |
Hello, We also require setting dynamic version for CI to be able to set the version components. Do you know any other method of passing CI settings into the build process without overwriting git managed files? Regards, |
Letsql uses maturin as the build-backend. To handle our dependencies we use poetry. As of now, and in the future, maturin does not recognize the dependencies specified by poetry see this issue: PyO3/maturin#632 It also does not provide an alternative way to support dynamic dependencies The following issue is still open PyO3/maturin#1537 On the other side poetry will support PEP-621 project style dependencies in the version 2.0 python-poetry/poetry#3332 Therefore one simple solution is to duplicate the dependencies section, as in the package: https://github.com/tmtenbrink/rustfrc/blob/main/pyproject.toml To do so, a semi-automated approach is to generate the dependencies using poetry export poetry export -f requirements.txt --without="test,dev,docs" / --all-extras --without-hashes --output requirements.txt And then update the dependencies section in the pyproject.toml file. Additionally this commit solves a few inconsistencies regarding packages listed as optional (duckdb, ibis), but when running the code it gives errors.
Letsql uses maturin as the build-backend. To handle our dependencies we use poetry. As of now, and in the future, maturin does not recognize the dependencies specified by poetry see this issue: PyO3/maturin#632 It also does not provide an alternative way to support dynamic dependencies The following issue is still open PyO3/maturin#1537 On the other side poetry will support PEP-621 project style dependencies in the version 2.0 python-poetry/poetry#3332 Therefore one simple solution is to duplicate the dependencies section, as in the package: https://github.com/tmtenbrink/rustfrc/blob/main/pyproject.toml To do so, a semi-automated approach is to generate the dependencies using poetry export poetry export -f requirements.txt --without="test,dev,docs" / --all-extras --without-hashes --output requirements.txt And then update the dependencies section in the pyproject.toml file. For more details on how to express poetry optional dependencies as PEP-621 optional dependencies, see the following resources: https://astarvienna.github.io/howtotoml.html#extras https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements https://python-poetry.org/docs/pyproject/#extras Additionally this commit solves a few inconsistencies regarding packages listed as optional (duckdb, ibis), but when running the code it raises ImportError. See the penguins_example.py for code that was raising ImportError
* chore: prepare for release 0.1.4 Letsql uses maturin as the build-backend. To handle our dependencies we use poetry. As of now, and in the future, maturin does not recognize the dependencies specified by poetry see this issue: PyO3/maturin#632 It also does not provide an alternative way to support dynamic dependencies The following issue is still open PyO3/maturin#1537 On the other side poetry will support PEP-621 project style dependencies in the version 2.0 python-poetry/poetry#3332 Therefore one simple solution is to duplicate the dependencies section, as in the package: https://github.com/tmtenbrink/rustfrc/blob/main/pyproject.toml To do so, a semi-automated approach is to generate the dependencies using poetry export poetry export -f requirements.txt --without="test,dev,docs" / --all-extras --without-hashes --output requirements.txt And then update the dependencies section in the pyproject.toml file. For more details on how to express poetry optional dependencies as PEP-621 optional dependencies, see the following resources: https://astarvienna.github.io/howtotoml.html#extras https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements https://python-poetry.org/docs/pyproject/#extras Additionally this commit solves a few inconsistencies regarding packages listed as optional (duckdb, ibis), but when running the code it raises ImportError. See the penguins_example.py for code that was raising ImportError * fix: failed logging without git
In PEP 621 is the dynamic stuff, is there any plan to support it for dependencies and optional-dependencies?
git pull
andconda env update --file environment.dev.yml --prune
andpip install -e .[dev,test]
to ensure we are all developing on the latest code in the latest environment with the latest dependencies. We pin all versions of our primary packages in 3 requirements filesrequirements.txt
(prod),requirements.dev.txt
(dev), andrequirements.test.txt
(test) so that we can separate out the requirements. Right now setuptools handles this well for a pure python program, such thatpip install -e .
(prod/base requirements) orpip install -e[dev,test]
(prod + dev + test requirements) both work really well.some examples/links:
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata
https://peps.python.org/pep-0621/#dynamic
pyproject.toml:
There is something very similar that setuptools supports in setup.cfg files too (https://setuptools.pypa.io/en/latest/userguide/declarative_config.html#options):
the analogous idea in the older setup.py was (and the code to implement this in maturin may not need to be much more complicated than this I guess - probably some filtering of comment/pip flag lines etc.):
The text was updated successfully, but these errors were encountered: