-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Poetry #796
Comments
Our team is contemplating switching to Poetry (from stock pip), however are blocked on this too. |
Perhaps a |
Hey there! It's on the product roadmap to add experimental Poetry support - there's a user submitted PR #801 that I might be able to iterate on. I'll circle back in a few weeks with a progress update, but keep an eye on that pr for more details for now 👍 |
I think one wouldn't need explicit poetry support but simply the latest version of pip (>19.). See python-poetry/poetry#321. Would be nice if one could easily configure which pip version is used from a buildpack runtime.txt or similar. |
It looks like #801 was closed by the user. Any update on the status of Poetry support? |
Not sure if helpful but another PR (by me) also adds Poetry Support |
Looks like progress about feature is frozen :( |
Here is a series of PRs which implement support for Poetry and other PEP-517 compliant build backends: |
Curious as to the status of this. There seems to be numerous community PR to do this, but none of them are merged. |
Thanks for checking in on the status y'all - the first of @cjolowicz 's prs (pip update to 20) has been merged and released, and fully plan on working through the last two of that trio to provide poetry support. I'll leave this issue open until resolved, but will close other Poetry feature requests so we can track progress and questions in one place. |
Good news everyone! Prepare yourselves for a rambly update... So, while I've been pushing out smaller improvements, this issue has been hanging around in the back of my mind. I think one of the lessons from the way pipenv support was added is that a tight coupling between pre-python install (path updates, necessary system packages, flags, more sysadmin-type setup) and post-python install (pip installs, package installs, any sort of venv setups) leads to a lot of user and maintenance friction. Especially as the package changes, grows and adapts. We do some magic to reach into the pipenv config files and do the right thing with them in the sysadmin step, because it as a tool doesn't cause deterministic builds at the python install level, it just documents what the determined python version for the build should be. The proposed solution in #835 is good work, but follows the same convention - and, I'm concerned the convention sets us up for that same friction. From Poetry's documentation, the solution to that is to rely on pyenv for pre-python setup, and then poetry for post-python setup, with a clearer divide between the two. This is the path forward here. Poetry has a lot of promise and I'm very excited about it, but I don't believe Heroku should be restrictive - it should be possible to use the tooling of your choice, and it should be plug and play. So I want to be able to easily plug and play new systems as they gain ground and maturity (and may no one wait this long again 😂). That's the benefit of a strict pre/post python divide using pyenv (or at first, just a .python-version file) -- it's a simple and straightforward base to build on, whether it's for Poetry, standard pip workflow flow or other tools. The downside is that it's yet another file to be specified on Heroku, albeit a community-standard one. So while I hate to ask for duplicate specifiers or files, I think we need both. With this decision formed, though, expect to see progress towards this speed up a bit in the next several weeks. Especially after I'm able to figure out how to record my PyCon talks from my home 😬 |
Hi @CaseyFaist !!, has there been any update regarding this?. Not trying to be pushy, just curious :-) |
@alfonsoperez Hi! I've just taken over from Casey as Python owner. Exploring support for alternative package managers is definitely on my list of things to do, though the overall list is pretty long at the moment, so not sure as to timeframe :-) |
I just spotted that I advocated for poetry support back when I was a customer (#796 (comment)) - I guess I definitely have to add support now 😆 |
Hi @edmorley, great to see all the things you are cleaning up! Just wanted to check back and see if you have any sense of a likely timeline to support or if this is still a ways off? |
@nextmat I don't have a timeline at the moment, but this is definitely bubbling up closer to the top of the list, now some of the more urgent issues have been addressed :-) |
It'd be great if the buidpack could leverage |
@apollo13 it should do the same thing other tools do if a range is given they try to use the latest in the range but if I'm deploying my app to heroku I'm going to specify exactly the python version I want in that field, and if I don't it's on me if I get a surprise |
@anentropic Fair enough, but not every app is ment to be deployed to just heroku, in that sense I think |
If the existing info is too vague, a |
Hey @edmorley, any update on an estimated timeline for this? No worries if not, I'm mainly curious because I've been putting off switching from Pipenv to Poetry for when Heroku has official Poetry support. But if that's still likely to be several months off, then I'll just go ahead and use the community Poetry buildpack. |
@michaelhays Hi! Non-Python projects consumed most of my time the last few months (eg Heroku-20 beta/GA/default stack, Cedar-14 EOL + sunset, Heroku-16 deprecation, amongst others), but I'm now able to return to the Python buildpack for the next few months. Top of my list is fixing the somewhat dismal test coverage (and non-production-representative tests), after which it will be much safer to make further changes. I would use the community buildpack for now and I'll update this issue when I start working on native support :-) |
Hi @edmorley , Author of the Poetry buildpack here :) I hope it won't come across as an unwelcome intrusion - my intention is not to advertise our stuff or interfere with your plans... I understand that you're up to investing quite some work into the official Python buildpack, so maybe it would be a good thing to have a discussion as to which direction should we take? Back when we migrated to In as far as our buildpack is concerned, I think it's pretty feature-complete by now (there is a number of environment variables you can set, and it can optionally take care of generating a Back when we were using Then we switched to On the other hand, there are obvious advantages to the "official" support - (1) one only needs one buildpack and (2) one doesn't need to rely on third-parties that are not affiliated with Heroku and can in theory stop taking their pills anytime without warning ;-) One thing that I think fits nicely into the official buildpack is the support for pinning Python versions or picking the correct runtime - this would be rather a Anyways, I would be quite interested in hearing your thoughts. Do you think build tool support belongs to the buildpack at all, or maybe there should be a Python buildpack that does Cheers! |
https://github.com/moneymeets/python-poetry-buildpack is the buildpack @zyv means. I had to go find it, since i wasn't aware it existed. |
@sidmitra Thanks! I think the poetry buildpack should be promoted more. |
Different tools store dependencies in different sections of pyproject.toml. poetry is not PEP 631 compliant (python-poetry/poetry#3332): https://github.com/python-poetry/poetry/blob/4ec09d4e6b8ca007e67bb873c96277b54986fbdb/pyproject.toml#L31-L52 |
I will note that PEP 621 was accepted less late last year, and PDM is younger than Covid-19. |
@zyv Hi! Sorry for not replying sooner :-) Glad to hear that the current approach with the supplementary Poetry buildpack is working well for now. I do believe there is still value in first-party support for Poetry for what it's worth. However we also want to make it easier to extend buildpacks too, which is one of the benefits we'll get once we migrate to Cloud Native Buildpacks (https://buildpacks.io) due to its support of buildpack composition. For example having a "python-engine" buildpack that's consumed by a pip buildpack, but could equally be consumed by a poetry/pipenv/... buildpack. We're in the process of writing new CNBs for each language (initially for the Salesforce Functions product, but eventually they'll replace the existing Heroku buildpacks) - Node.js and Java have a native CNB, Python is next. The Python repository is currently empty, but development will be occurring here: |
@edmorley Thanks for the answer, glad to hear that Heroku is still working on innovative things (including in Python land). |
No problem at all, and thanks for the update! So, if I understood it correctly, the plan is to leave the classic buildpack as is for now, and provide official Poetry support for the CNB buildpack later, because it would compose well? This certainly feels like a sound idea to me. We'll continue to maintain our buildpack as long as we are still using Heroku, but quite honestly with all the outages that happened in the last year and a half, and annoying changes like 2FA + reduced login session lifetime, Salesforce banner in the dashboard, difficult GDPR story and stuff like this are pushing us to move directly to AWS, even if it's quite a lot of work :-/ So at some point we might have to put it up for adoption... |
Hi :-)
Yeah that's correct. Since the CNBs will eventually replace the classic buildpacks, we don't want to invest too much time in the latter - though once the CNBs exist there may be some amount of backporting to the classic buildpacks, to make the transition easier. |
I'm curious if there's any ETA on the Python native CNB. Sounds pretty cool BTW! |
Hi! We've been working on libcnb.rs, which will power our CNBs moving forwards. That library is now in a good shape (as of the last few weeks), so work on the Python CNB will be starting imminently. |
@edmorley Thanks for the update! Is there any resource-channnel we can track the progress of Python CNB implementation? |
I would repo watch:
|
Also, if you have opinions on ways to specify Python version, feel free to comment on: |
Poetry is a python package manager, similar to Pipenv.
At the moment, heroku-buildpack-python supports both vanilla pip (via requirements.txt) and Pipenv (via Pipfile, Pipfile.lock). Unfortunately, there's been a lot of backlash with regards to Pipenv in the Python community, and on the back of that, Poetry was born to serve as a better Pipenv.
That being said, it would be good if this buildpack can detect and install dependencies using Poetry. The package manager uses two files:
pyproject.toml
(which has been provisionally accepted in PEP 518) andpoetry.lock
.The text was updated successfully, but these errors were encountered: