You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Python community seems to be adopting PEP621 for storing project metadata in pyproject.toml. We have been using Poetry from the start of FawltyDeps, which uses a different Poetry-specific format. Although it's not vitally important for us to follow PEP621, it would be nice to follow best practices in the Python community here.
There is some movement in Poetry to allow support for PEP621 metadata, but we don't know if or when this would be merged:
Our shell.nix perform similar commands to create/enter the Poetry dev shell.
Setting up CI action environments (most files under .github/workflows/ follow the same pattern):
Use Gr1N/setup-poetry@v8
Cache the Poetry virtualenv at ~/.cache/pypoetry/virtualenvs, the cache key includes a hash of the poetry.lock file.
Use poetry install --no-interaction --sync --with=$DEPENDENCY_GROUPS or similar to setup environment (this is instead of Nox' setup).
Use poetry run nox --no-venv --no-install --non-interactive -s ... to run the Nox action.
Accessing Poetry dependency groups when running Nox sessions:
We need to convert the Poetry dependency group that corresponds to a Nox session into a requirements.txt file that can be consumed by Nox's session.install():
poetry export --format=requirements.txt --output=.../reqs_from_poetry.txt --with=... or similar
The resulting reqs_from_poetry.txt file is passed to Nox' session.install("-r", ...)
In CI we are setting up the test venv with Poetry instead of Nox. Is this only to avoid duplicate/nested venvs, or is it easier to cache Poetry venvs than Nox venvs?
Creating/locking the poetry.lock file is sometimes really slow. Have to run with --no-update to prevent unrelated updates to dependencies.
How to support test scenarios where we want to automatically test FawltyDeps against the oldest/newest set of acceptable dependencies on some platform?
uv can be used to create/install venvs very quickly.
uv only produces resolutions for a single platform and Python version at a time. Unlike Poetry (and PDM), uv does not yet produce a machine-agnostic lockfile. What changes does this mean for our workflow?
Flit is another PEP621-compatible solution for building and publishing Python packages.
Remaining Questions?
Does PEP621 support the kind of dependency groups we are currently using in Poetry, and how we are using these from Nox? If not, what is the most appropriate way to express our various Nox session dependencies in PEP621?
Tangential: What is the optimal cache behavior on GitHub CI?
Cache full virtualenvs and re-install as little as possible between runs?
Only cache (uninstalled) Python packages and rely on uv to recreate virtualenvs really quickly?
Nix??? Currently we're not using Nix in CI at all.
Run bechmarks between various cache scenarios.
Always try to stay as close to local development as possible. It should be easy to reproduce CI failures locally with Nix and Nox.
The text was updated successfully, but these errors were encountered:
The Python community seems to be adopting PEP621 for storing project metadata in
pyproject.toml
. We have been using Poetry from the start of FawltyDeps, which uses a different Poetry-specific format. Although it's not vitally important for us to follow PEP621, it would be nice to follow best practices in the Python community here.There is some movement in Poetry to allow support for PEP621 metadata, but we don't know if or when this would be merged:
project
section (PEP 621) python-poetry/poetry#9135What are we using Poetry for?
CONTRIBUTING.md
:shell.nix
perform similar commands to create/enter the Poetry dev shell..github/workflows/
follow the same pattern):Gr1N/setup-poetry@v8
~/.cache/pypoetry/virtualenvs
, the cache key includes a hash of thepoetry.lock
file.poetry install --no-interaction --sync --with=$DEPENDENCY_GROUPS
or similar to setup environment (this is instead of Nox' setup).poetry run nox --no-venv --no-install --non-interactive -s ...
to run the Nox action.requirements.txt
file that can be consumed by Nox'ssession.install()
:poetry export --format=requirements.txt --output=.../reqs_from_poetry.txt --with=...
or similarreqs_from_poetry.txt
file is passed to Nox'session.install("-r", ...)
poetry config pypi-token.pypi <token>
[tool.poetry]
section inpyproject.toml
.poetry build
andpoetry publish
accomplishes the actual release.publish
CI action performs similar steps to publish a test release to https://test.pypi.org/project/fawltydeps/Problems with Poetry?
poetry.lock
file is sometimes really slow. Have to run with--no-update
to prevent unrelated updates to dependencies.uv
: https://github.com/astral-sh/uv?tab=readme-ov-file#resolution-strategyAlternatives?
uv
can be used to create/install venvs very quickly.uv
only produces resolutions for a single platform and Python version at a time. Unlike Poetry (and PDM),uv
does not yet produce a machine-agnostic lockfile. What changes does this mean for our workflow?Remaining Questions?
uv
to recreate virtualenvs really quickly?The text was updated successfully, but these errors were encountered: