-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use the [project] section in pyproject.toml according to PEP-621 #3332
Comments
The status of PEP 621 is still "Draft". But yes, I guess it's a valid request (assuming it actually gets accepted and depending under which form). It might take a while though... the way I see it, there are still too many uncertainties |
Uh, interesting. My bad. However, PEP-631 is accepted. Do you see uncertainties there as well? |
At some point, I thought there was an intention to merge the text of 631 into 621, to avoid having to refer to two documents, since 631 is rather short and unsurprising. Don't know if it will happen, or if it stays as 2 separate documents. I lost track of that conversation. And there are some more points of 621 that are still being discussed. Now, if I put myself in the shoes of poetry:
Personally I am curious to see how poetry will handle this. There is always the dynamic escape hatch for dependencies. If I were poetry, I would probably try to offer both dependency notations (the existing poetry notation as dynamic and the new/old PEP 631 one). I had suggested a compromise hybrid notation here but it didn't gain any traction at all (probably because too complex to parse, I don't know). |
It's mentioned in the abstract of PEP-0631. However, I don't know when this will happen and where the discussion around that is.
I'm surprised by that. Why do you think so? To me, it seems like a step towards more standardization and thus positive.
Yes, I think that would be desirable just to not break a working setup for anybody. Is the poetry dependency notation more powerful or even different at all? |
It's all in the discussions, particularly this one, I don't want to repeat things and take them out of context: |
The discussion about how the dependencies should be defined within PEP-621 was quite said and frustrating for us as poetry maintainers. But I will not go into details now... As @sdispater has said somewhere in the discussion, poetry will probably not be one of the earlier adopters of PEP-621 now PEP-631 is accepted. We have a problem here now: On the one hand, we want to support as many standards as possible and on the other, user experience is an important topic for us. And we believe that PEP-631 will be a step backwards compared to what we have now. A dilemma we have to discuss. :( |
PEP 621's status is now provisionally accepted as of 8 days ago. From the point of view of a package dev., having a build tool-independent metadata specification in pyproject.toml is useful to be able to parse metadata from the file without adding build tool-specific references elsewhere, and without duplicating information. Use case examples:
I have been using/parsing all of the above based on the PEP 621/631 drafts, and duplicating this info in pyproject.toml for now from the Fundamentally, project metadata is not only used by build tools, so having Poetry support (provisionally) accepted standards for these allows the developer to use the metadata for other cases without duplication, and having this support from Poetry would be fantastic :) |
Additionally, PEP 621 is now referenced at https://packaging.python.org/specifications/declaring-project-metadata/. |
As part of this, |
Status of PEP 621 is now: Final.
|
The way I see it, poetry can start by supporting PEP 621. PEP 631 (actually PEP 508) requires some changes in the way versions are dealt with in Poetry (e.g |
PEP 621 was updated to incorporate the decision on PEP 631 vs. PEP 633; it requires PEP 508 version strings. |
Hi poetry maintainers, do you think is there a way to start supporting everything else in pep 621 while the dependency list discussion happens? I don't want to misrepresent anyone so correct me if I'm wrong, as I understood the discussion it looks like pretty much everything else (name, version, python_requires, etc) was generally accepted and shouldn't impact poetry usability. PS. Thank you for this awesome piece of software. |
If even permitting users to specify PEP compliant dependency versions is a dealbreaker (remember, people wanting poetry style versions can just specify that dependencies is a "Error: poetry requires specifying dynamic dependencies in order to manage it differently". Then people could use the PEP, have a PEP compliant pyproject.toml, and get some benefits of the PEP (and test out using it in poetry), even if poetry doesn't accept every possible way of following the PEP. |
Last time I checked (which was a long time ago already, I must admit), I also came to a similar conclusion, poetry could relatively easily move to PEP621 and use the To the people in this thread asking for support of PEP621, if I may, honest question: What is the motivation for you to see PEP621 adopted by poetry? Is there any advantage already today to have PEP621? What would be the benefits for your project today? (Medium- long-term, I see the advantages, obviously) |
The solution seems pretty straightforward to me in principle - fully/properly support PEP621 (and that includes dependency specification per PEP631), and if there is some form of dependency specification that can't be expressed in PEP631 format, then either that can be expressed in a PEP621 doesn't have to support everything Poetry's config. supports for it to be supported by Poetry, it doesn't make sense to look at this issue with that assumption. It just needs to be able to sufficiently define project metadata in at least some cases for Poetry to be able to use sinoroc, see #3332 (comment) in answer to your last question. |
Honestly, this seems like the worst of all worlds. |
I'm not irrationally suggesting that poetry becomes worse as a result of the PEP because of the disagreement over version descriptions. Using Given some people do seem to think that merely offering the option to use PEP 631 is "bad", I figured a partial implementation is better than no implementation, because a partial implementation is partially useful whereas no implementation has no usefulness. I don't actually think this is a remotely ideal situation... |
@sinoroc I'm personally more interested in having the non-dependencies metadata in a single place so that several tools/scripts can reuse it without the hassle of keep it in sync. For example for a couple of projects I use poetry and towncrier: [tool.poetry]
name = "mypackage"
version = "0.1.0"
description = "My awesome package."
[tool.towncrier]
package = "mypackage"
package_dir = "src"
filename = "NEWS.rst"
version = "0.1.0" Like this example I have more related to documentation and publishing tools, albeit mostly for custom scripts atm. |
One of the big advantages of using PEP621 would also be that 3rd party tools would understand the dependencies. I for example have a project that is doing static analysis of a code and also dependencies (flagging outdated ones, potentially malicious, vulnerable packages, etc). If every tool has its own format of how dependencies are declared then the static analyzer must have a separate support for each of these tools which are quite hard to maintain over time. If all the tools start migrating to use the PEP621 then that would mean also a more secure python ecosystem as the static analyzers can all use the same metadata information regardless if the project is using poetry, setuptools, flit, or anything else. There are several other metadata info in the toml file that is very helpful for static analyzers and auditing tools which is very painful to support and maintain given all the different formats that each packaging tool has. |
PEP621 in general is good idea and can even make poetry's live easier. @sdispater is one of the authors of this PEP. The hard question is how the dependency specification in poetry project should look like. poetry projects doesn't have just run time dependencies, they also have dev-dependencies and more dependency groups are planned. How do we want to be able to use poetry specific flags like |
@finswimmer I answered that exact question in #3332 (comment). |
For sure |
On this one specific thing, I wish packaging tools would agree on some conventions here. I for example like and use the convention of having a |
Is there any summary what we will lose when moving project dependencies into the I know optional dependencies/extras will be defined quite different and wouldn't allow to automatically re-use the same packet version in multiple extras groups in the same way, but there is another option for that and should already work in other build backends. Basically, you can define your dependencies like:
And the Another thing I know of is defining the source of specific packages - I understand why |
An example of a project migrating from Poetry to PDM in part of the missing PEP 621 support |
I think Poetry's syntax is better and more powerful, but it's important to support PEP621. I propose that:
The upshot of my suggestion is to not try to support some complicated blend. Let people pick one or the other, and raise an error if it's not just one or the other. Should be much easier to develop. |
I think I can summarize your fairly lengthy description.
This has already been suggested but I believe poetry didn't want to do that. |
I don't know if I understood this proposal correctly, but if the idea is allow It should:
|
Have we already discussed the following option: Whenever Poetry performs dependency resolution, i.e., when it makes changes to If all other Metadata is equally well specifiable in This proposal becomes difficult if there are Poetry dependency expressions which are inexpressible in PEP 621. I do not really know a solution to this if these cases have actual usefulness and are not just weird corner cases. |
pyproject is not meant to be a lock file – dependencies in pyproject are not a full list of pinned dependencies. |
That’s subjective, and false, respectively. Regarding “better”, I think it’s worse since Regarding “more powerful”, see above the distinction between abstract and concrete dependencies.
|
You may want to watch this proposal (or its descendants when it grows them) so that you can participate in it. |
Thanks. I personally don’t need this that much. I think people who do should contribute there, so they can get all the Poetry resolution niceness in a standardized way, everywhere. |
Huh, and I thought you were going to say something about how poetry's dependency syntax cannot be more powerful than the PEP 621 syntax since the poetry syntax is, quite literally, compiled by poetry into the PEP 621 syntax (PEP 508 strings) in the process of being written into the wheel metadata. This was, after all, a major factor in poetry's PEP 633 being rejected. |
You’re right. The metadata part (abstract dependencies) is just as powerful. Poetry mixes that with package sources and path dependencies, which are resolution features (concrete dependencies), and therefore shouldn’t be part of the dependency array. Of course, package sources and path dependencies are super useful, but extricating them from the |
#9135 should be in a testable state now. (Consider it an alpha version.) Please try it out and report issues you encounter in the PR. |
Letsql uses maturin as the build-backend. To handle our dependencies we use poetry. As of now, and in the future, maturin does not recognize the dependencies specified by poetry see this issue: PyO3/maturin#632 It also does not provide an alternative way to support dynamic dependencies The following issue is still open PyO3/maturin#1537 On the other side poetry will support PEP-621 project style dependencies in the version 2.0 python-poetry/poetry#3332 Therefore one simple solution is to duplicate the dependencies section, as in the package: https://github.com/tmtenbrink/rustfrc/blob/main/pyproject.toml To do so, a semi-automated approach is to generate the dependencies using poetry export poetry export -f requirements.txt --without="test,dev,docs" / --all-extras --without-hashes --output requirements.txt And then update the dependencies section in the pyproject.toml file. Additionally this commit solves a few inconsistencies regarding packages listed as optional (duckdb, ibis), but when running the code it gives errors.
Letsql uses maturin as the build-backend. To handle our dependencies we use poetry. As of now, and in the future, maturin does not recognize the dependencies specified by poetry see this issue: PyO3/maturin#632 It also does not provide an alternative way to support dynamic dependencies The following issue is still open PyO3/maturin#1537 On the other side poetry will support PEP-621 project style dependencies in the version 2.0 python-poetry/poetry#3332 Therefore one simple solution is to duplicate the dependencies section, as in the package: https://github.com/tmtenbrink/rustfrc/blob/main/pyproject.toml To do so, a semi-automated approach is to generate the dependencies using poetry export poetry export -f requirements.txt --without="test,dev,docs" / --all-extras --without-hashes --output requirements.txt And then update the dependencies section in the pyproject.toml file. For more details on how to express poetry optional dependencies as PEP-621 optional dependencies, see the following resources: https://astarvienna.github.io/howtotoml.html#extras https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements https://python-poetry.org/docs/pyproject/#extras Additionally this commit solves a few inconsistencies regarding packages listed as optional (duckdb, ibis), but when running the code it raises ImportError. See the penguins_example.py for code that was raising ImportError
* chore: prepare for release 0.1.4 Letsql uses maturin as the build-backend. To handle our dependencies we use poetry. As of now, and in the future, maturin does not recognize the dependencies specified by poetry see this issue: PyO3/maturin#632 It also does not provide an alternative way to support dynamic dependencies The following issue is still open PyO3/maturin#1537 On the other side poetry will support PEP-621 project style dependencies in the version 2.0 python-poetry/poetry#3332 Therefore one simple solution is to duplicate the dependencies section, as in the package: https://github.com/tmtenbrink/rustfrc/blob/main/pyproject.toml To do so, a semi-automated approach is to generate the dependencies using poetry export poetry export -f requirements.txt --without="test,dev,docs" / --all-extras --without-hashes --output requirements.txt And then update the dependencies section in the pyproject.toml file. For more details on how to express poetry optional dependencies as PEP-621 optional dependencies, see the following resources: https://astarvienna.github.io/howtotoml.html#extras https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#dependencies-and-requirements https://python-poetry.org/docs/pyproject/#extras Additionally this commit solves a few inconsistencies regarding packages listed as optional (duckdb, ibis), but when running the code it raises ImportError. See the penguins_example.py for code that was raising ImportError * fix: failed logging without git
Feature Request
PEP621 (Storing project metadata in pyproject.toml) and PEP631 (Dependency specification in pyproject.toml based on PEP 508) define a clear way how metadata and dependencies should be stored. Both of them have the status "accepted". It would be good if poetry would use that.
For example, I've noticed that
poetry init
generates a pretty similarpyproject.toml
, but puts the metadata in a poetry-specific section.The text was updated successfully, but these errors were encountered: