Adding connectedk8s extension failing with Pip failed with status code 1 on 2.55.0

[christianjedroCDT]

Describe the bug

We´re using mcr.microsoft.com/azure-cli and trying to install connectedk8s. This gives the following error:

docker run --rm -it mcr.microsoft.com/azure-cli:2.55.0 az extension add --name connectedk8s
❌ An error occurred. Pip failed with status code 1. Use --debug for more information.

Related command

docker run --rm -it mcr.microsoft.com/azure-cli:2.55.0 az extension add --name connectedk8s

Errors

An error occurred. Pip failed with status code 1

Issue script & Debug output

docker run --rm -it mcr.microsoft.com/azure-cli:2.55.0 az extension add --name connectedk8s --debug

(full ouput too big for github)

Successfully built pyyaml
Failed to build pycryptodome
ERROR: Could not build wheels for pycryptodome, which is required to install pyproject.toml-based projects
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 429, in run
    raise InstallationError(
pip._internal.exceptions.InstallationError: Could not build wheels for pycryptodome, which is required to install pyproject.toml-based projects
Removed build tracker: '/tmp/pip-build-tracker-d23qzhxs'

cli.azure.cli.core.extension.operations: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--target', '/root/.azure/cliextensions/connectedk8s', '/tmp/tmp56q4bjnr/connectedk8s-1.5.6-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /root/.azure/cliextensions/connectedk8s
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
    return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 344, in add_extension
    extension_name = _add_whl_ext(cli_ctx=cmd_cli_ctx, source=source, ext_sha256=ext_sha256,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 173, in _add_whl_ext
    raise CLIError('An error occurred. Pip failed with status code {}. '
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.

Expected behavior

Succeeding to install extension connectedk8s

Environment Summary

azure-cli 2.55.0

Additional context

We´ve tested it with 2.53.1 and there it´s working. 2.54.0 is also not working.

Tested on Mac M1 and Linux/AMD

[yonzhan]

Thank you for opening this issue, we will look into it.

[jkroepke]

It seems like an issue with pycryptodome, found other issues which has issues with pycryptodome and revert to 2.53 is the solution.

Ref: #27866 (comment)

[bebound]

This issue is caused by this change #27567.
You need to manually add these packages to build pycryptodome.

apk add musl-dev gcc

[jkroepke]

Is this a temporary or permanent solution?

@bebound @jiasli

The thing is that I'm running mcr.microsoft.com/azure-cli inside Kubernetes which denies runAsRoot and I'm unable to use apk as non root.

% docker run -u 1000 mcr.microsoft.com/azure-cli:2.55.0 apk add musl-dev gcc
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied

Can you publish a full flavor which includes the devel packages? Or can you modify the filesystem permissions that I'm able to use apk as non root out of the box?

[bebound]

The easiest solution I come up with is to build a custom image based on our image.

[jkroepke]

The easiest solution I come up with is to build a custom image based on our image.

I guess the easiest solution is revert #27567 which avoids that a lot of user has to build a custom image.

[jkroepke]

@bebound @jiasli Is it possible to include the pycryptodome dependency into the standard image? On build time, all build tools are installed.

[jkroepke]

Hi, the error still exists with 2.56