Python 3.13 free-threading compatibility

[lysnikolaou]

Hey all! 👋

This aims to be a tracking issue for all work necessary to support the free-threaded build of Python 3.13. A very high-level list of steps is (more details to follow as I investigate more issues):

• Set up CI for free-threading
  • Test 3.13 free-threaded build on CI #8200
• Move away from APIs that use borrowed references under the free-threaded build
  • Move away from APIs that use borrowed references under the free-threaded build #8216
• Audit C extension module for thread-safety issues
  • Lock around usages of imaging memory arenas #8238
• Mark C extension modules as thread-safe with the Py_mod_gil slot (or PyUnstable_Module_SetGIL)
  • Move away from APIs that use borrowed references under the free-threaded build #8216
• Upload wheels for the free-threaded build. Other projects have started uploading nightlies for the free-threaded on the scientific-python-nightly-wheels index so that we can enable faster iteration, we can discuss whether we wanna do this for Pillow as well.
  • Build free-threaded wheels #8219
  • Request access for Pillow scientific-python/upload-nightly-action#84
  • Upload wheels to scientific-python-nightly-wheels index #8236
  • Skip QEMU-emulated wheels on workflow dispatch event #8243
  • Move uploading nightly wheels to different job #8244

[hugovk]

Thanks for opening this issue!

As some background, the acceptance of PEP 703 lays out the three phases for disabling the GIL. Python 3.13 is the first in the experimental phase I, where a free-threaded build is available as an option in the Windows and macOS installers, and can be built for Linux (and the other platforms) using an extra flag.

So now is a good time to start testing on free-threaded builds, and getting the code ready. It will be good to produce wheels at some point, to help others test and integrate. These will be experimental and not for production.

I've made a start on the first two steps, I'll open a PR for the CI first.

Some handy references:

• https://dev.to/hugovk/help-us-test-free-threaded-python-without-the-gil-1hgf
• https://docs.python.org/3.13/howto/free-threading-extensions.html
• https://github.com/Quansight-Labs/free-threaded-compatibility

[hugovk]

Please see PR #8200 to set up CI for free-threading.

[hugovk]

I'm working on this today with @lysnikolaou at the EuroPython sprints.

It would help the ecosystem if we also upload nightly wheels so others can test the current code without needing to wait for quarterly releases.

@lysnikolaou has requested access at scientific-python/upload-nightly-action#84 for us to upload to the Scientific Python Nightly Wheels index, where other projects like SciPy are uploading.

See also https://labs.quansight.org/blog/free-threaded-python-rollout and https://py-free-threading.github.io/.

[radarhere]

If anyone would like wheels generated after #8216, here you go - Linux.zip, macOS.zip, Windows.zip

[lysnikolaou]

Nightly wheels for Linux, macOS and Windows are now available on https://anaconda.org/scientific-python-nightly-wheels/pillow/.

[rgommers]

https://anaconda.org/scientific-python-nightly-wheels/pillow rather than pandas. And, yay thank you both! 🎉

[radarhere]

Every task in this issue has been completed - is this just open now as a tracking issue until Python 3.13 is finalized?

[lysnikolaou]

is this just open now as a tracking issue until Python 3.13 is finalized?

Yes, I'd leave it open until at least the final release of 3.13 and see how it goes from there.