cffi relies on deprecated (now removed) distutils.msvc9compiler module

[alex]

In pypa/distutils#287 this module was removed and in pypa/setuptools#4606 that was incorporated into setuptools 74.

As a result, building cffi now fails with:

    Traceback (most recent call last):
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\_shimmed_dist_utils.py", line 33, in <module>
        from distutils.msvc9compiler import MSVCCompiler
    ModuleNotFoundError: No module named 'distutils.msvc9compiler'

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "D:\a\cryptography\cryptography\src\_cffi_src\build_openssl.py", line 60, in <module>
        ffi.emit_c_code(c_file)
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\api.py", line 695, in emit_c_code
        recompile(self, module_name, source,
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\recompiler.py", line 1546, in recompile
        ext = ffiplatform.get_extension(ext_c_file, module_name, **kwds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\ffiplatform.py", line 9, in get_extension
        from cffi._shimmed_dist_utils import Extension
      File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-bywibcpm\overlay\Lib\site-packages\cffi\_shimmed_dist_utils.py", line 36, in <module>
        raise Exception("This CFFI feature requires setuptools on Python >= 3.12. Please install the setuptools package.") from ex
    Exception: This CFFI feature requires setuptools on Python >= 3.12. Please install the setuptools package.

[MarkCallow]

Given this issue is breaking my CI builds and the delays getting one or other of the PRs to fix it landed and released, I am looking to workaround the issue temporarily for my project. I did not write our Python interface and I lack Python expertise so I am seeking help here to find a workaround.

My project's requirements.txt has

cffi==1.15.1
setuptools==70.0.0

I would have thought the latter line would get me an older version of setuptools that still has distutils.mscv9compiler but I am getting the ModuleNotFound error.

The setup function in my project's setup.py which does from setuptools import setup has

    cffi_modules=["buildscript.py:ffibuilder"],
    ...
    install_requires=["cffi>=1.15.1"],
    ...
    setup_requires=["cffi>=1.15.1"],

It seems that cffi is unaware of my project's setuptools choice and is using the latest one.

How can I make it use my project's selected setuptools or prevent it using the latest one?

To the maintainers of cffi, please release a fix a.s.a.p.

[Flow86]

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

[arigo]

@nitzmahone see comments above.

[BLACKNOTES]

Pyclarity package depends on cffi and 2024 challenges of clarity is ongoing right now.
Only solution I find was installing pyclarity on Linux and running codes there. As cffi has no problem on Linux but this problem persists on windows.

[MarkCallow]

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

Thank you @Flow86. This worked for me too. I recommend anyone who is running their build from 'CMake' in GitHub Actions to use a full path for constraint.txt. I used $env:RUNNER_TEMP/constraint.txt.

[LeftMonster]

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

Same worked for me, Thanks. I'm curious why?

[Flow86]

Hi, I temporarily fixed my issues by using an environmentvariable PIP_CONSTRAINT=constraint.txt and adding setuptools<74 into the file constraint.txt. that way the constraints also apply to any install/setup requirements

Same worked for me, Thanks. I'm curious why?

you can specify constraint files for the "simple" install (with --constraint file.txt) on the pip install command, but: and thats a big but: its not applied by pip to any build requirements - e.g if the package needs to be build locally since no binary is available, a "venv" is created for the build, and there the constraints do not apply at the moment and would use the broken combination of setuptools and cffi. if you use the environment variable, the venv also picks up the constraints and therefor it works.