-E ignored when using poetry.group

[Conchylicultor]

Reposting #4891 with more info, as I don't think the issue has been solved.

• OS version and name: Debian
• Poetry version: 1.2.0a2

[tool.poetry]
name = "tmp-poetry"
version = "0.1.0"
description = ""
authors = ["xyz <xyz@gmail.com>"]

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

# Bellow is the bug

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.group.group0]
optional = true

[tool.poetry.group.group0.dependencies]
numpy = "*"
matplotlib = "*"

[tool.poetry.extras]
extra_deps = ["numpy", "matplotlib"]

Issue

When [tool.poetry.extras] is used to refer to deps defined inside [tool.poetry.group.group0.dependencies], those deps are ignored.

Reproduction instrutions:

poetry install -E extra_deps

I would expect the command to install tmp-poetry[extra_deps] which should install numpy. However, this silently fail to install the package.
The extra deps is ignored and not installed, poetry show will display nothing.
poetry run python -c "import numpy" will fail with ModuleNotFoundError

Note: removing the groups and using instead numpy={version="*", optional=true} will work.

Why using dependency group with extra ?

The reason I'm trying to use dependency group is that it greatly reduce boilerplate of defining deps

Without dependency group:

[tool.poetry.dependencies]
absl-py = {version="*", optional=true}
importlib_resources = {version="*", optional=true}
jax = {version="*", optional=true}
jupyter = {version="*", optional=true}
matplotlib = {version="*", optional=true}
numpy = {version=">=1.17", optional=true}
tf-nightly = {version="*", optional=true}
tqdm = {version="*", optional=true}
tree = {version="*", optional=true}
zipp = {version="*", optional=true}

With dependency group:

[tool.poetry.group.group0]
optional = true

[tool.poetry.group.group0.dependencies]
absl-py = "*"
importlib_resources = "*"
jax = "*"
jupyter = "*"
matplotlib = "*"
numpy = "*"
tf-nightly = "*"
tqdm = "*"
tree = "*"
zipp ="*"

Resolving this issue

There are 2 ways to resolve this issue:

1. Either extras and group are incompatible features, in which case poetry should raise an explicit error message (rather than silently skipping deps installation)
2. Either extras can use deps defined in group (best), in which case the deps should be correctly installed.

Of course 2) should be greatly preferred.

[Conchylicultor]

To answer the previous issue objection:

At no point in the doc is explained that extras and group are mutually exclusive: https://python-poetry.org/docs/master/managing-dependencies/#dependency-groups

Poetry provides a way to organize your dependencies by groups. For instance, you might have dependencies that are only needed to test your project or to build the documentation.
Think of dependency groups as labels associated with your dependencies: they don’t have any bearings on whether their dependencies will be resolved and installed by default, they are simply a way to organize the dependencies logically.

The doc say those deps can be used for tests but not that they have to.

Additionally, even global dependencies have a "default" group, which indicates group are not limited to developpement. From the doc:

The dependencies declared in tool.poetry.dependencies are part of an implicit default group.

Additionally, other users seems to understand like me that group and extras features are complementary features which can be used jointly:
#4842

[tasansal]

I am not sure if this is the right issue to ask this, but here it goes.

I am a little confused about this. I am on the 1.2.0b2 and using dependency groups.

My use case is the following:
1- have minimum dependencies in the default section.
2- have .group.dev.dependencies for dev.
3- have group.my_group.dependencies as optional for installing packages using the --with argument

The groups are not seen as extras if I want to pip install my_package[my_group] from PyPI.

Extras were installed as extras when groups didn't exist using pip. I migrated all my extras into groups, but they are not pip installable.

What is the correct procedure here? Thanks!

[finswimmer]

Hello,

extras are a python concept to bundle multiple dependencies into one group that can be installed to extend the functionality of the library/app.

groups are a Poetry-only things to organize your non-runtime dependencies. This is why you cannot reference a dependencies listed in an explicite group for an extra.

A PR to improve this in the documentation is always welcome 😃

fin swimmer

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.