Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.11.8 update #6040

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Python 3.11.8 update #6040

wants to merge 15 commits into from

Conversation

th0ma7
Copy link
Contributor

@th0ma7 th0ma7 commented Mar 18, 2024

Description

Python 3.11.8 update

Fixes #

Checklist

  • Build rule all-supported completed successfully
  • New installation of package completed successfully
  • Package upgrade completed successfully (Manually install the package again)
  • Package functionality was tested
  • Any needed documentation is updated/created

Type of change

  • Bug fix
  • New Package
  • Package update
  • Includes small framework changes
  • This change requires a documentation update (e.g. Wiki)

@hgy59
Copy link
Contributor

hgy59 commented Mar 20, 2024

@th0ma7 in my local homeassistant work, I have some pending issues.


  1. Additional wheels for crossenv

    Some wheels require expandvars and one requires pybind11 in the crossenv
    I temporary added the following lines in cross/python311/Makefile to solve this.

    # stdc++ bindings for webrtc_noise_gain
    CROSSENV_WHEELS += pybind11==2.11.1
    # for building frozenlist, yarl, ...
    CROSSENV_WHEELS += expandvars==0.12.0
    

    wouldn't it be possible to define additional dependencies to install into crossenv

    for example:
    define the following in spk/homeassistant/Makefile

    ADD_CROSSENV_WHEELS = pybind11==2.11.1 expandvars==0.12.0
    

    Than before cross-compiling wheels for a package, all wheels in ADD_CROSSENV_WHEELS are added to crossenv.


  1. cross compile pandas wheel
    pandas==2.1.4 cannot be cross compiled, because it is lacking numpy
    Is it possible, to install cross built numpy into crossenv to solve this?

@th0ma7
Copy link
Contributor Author

th0ma7 commented Mar 21, 2024

  1. Additional wheels for crossenv
    Some wheels require expandvars and one requires pybind11 in the crossenv
    I temporary added the following lines in cross/python311/Makefile to solve this.
    # stdc++ bindings for webrtc_noise_gain
    CROSSENV_WHEELS += pybind11==2.11.1
    # for building frozenlist, yarl, ...
    CROSSENV_WHEELS += expandvars==0.12.0
    

@hgy59 Feature now added. Quite of a nice addition I must say 🚀

  1. cross compile pandas wheel
    pandas==2.1.4 cannot be cross compiled, because it is lacking numpy
    Is is possible, to install cross built numpy into crossenv to solve this?

Wow I tried quite a bit and no luck. The crossenv does have all the minimal requirements whereas numpy was already included by default from cross/python311 and remaining using the new functionality:

# for pandas
ADDITIONAL_CROSSENV_WHEELS += python-dateutil==2.9.0.post0
ADDITIONAL_CROSSENV_WHEELS += pytz==2024.1
ADDITIONAL_CROSSENV_WHEELS += tzdata==2024.1

No luck. I ended-up creating a bug entry on pandas-dev/pandas#57945

Wait and see what comes next.

@th0ma7 th0ma7 mentioned this pull request Mar 21, 2024
10 tasks
@th0ma7 th0ma7 force-pushed the py311-update branch 2 times, most recently from 20a491e to b33b529 Compare March 25, 2024 10:56
@th0ma7
Copy link
Contributor Author

th0ma7 commented Mar 25, 2024

@hgy59 I do have an almost working solution for pandas in my local branch. Issue becomes that updating Cython to >= 3.0 pandas-dev/pandas#57945 (comment) cause Pyyaml to fail to build, something I had already documented in the code but that still exist to this day yaml/pyyaml#601.

With the new functionality to allow adding wheels to the crossenv, my thinking is now the following:

  1. Move the crossenv creation to a call into spksrc.wheel.mk
  2. When ADDITIONAL_CROSSENV_WHEELS is captured for a specific wheel, generate a "new" crossenv specific for that wheel in particular

This bescomes way more intrusive but it would allow potentially all worlds to work out properly, what ever is needed (besides gcc version limitations).

That i don't have code for yet and I'd need to get #6002 merged first.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Apr 1, 2024

Now rebased against master, time to confirm status hasn't changed between the two (i.e. still fails at the same places).

@th0ma7 th0ma7 mentioned this pull request Apr 1, 2024
10 tasks
th0ma7 added a commit that referenced this pull request Apr 2, 2024
* c-ares: Update from version 1.19.1 to 1.27.0

* libsodium: Update from version 1.0.18 to 1.0.19

* libxslt: Update from version 1.1.38 to 1.1.39

* sqlite: Update from version 3420000 to 3450200

* xz: Update to version 5.6.1+really5.4.5 (debian security fixed)

* gdbm: Update from version 1.22 to 1.23

* libexpat: Update from version 2.5.0 to 2.6.2

* libffi: Update from version 3.4.2 to 3.4.6
@hgy59
Copy link
Contributor

hgy59 commented Apr 10, 2024

@th0ma7 it is not worth any effort for the home assistant package.

Homeassistant dropped Python 3.11 support with v2024.4 and now requires Python 3.12.

If version 2023.12 or 2024.1 works with Python 3.11.8 out of the box, I'll probably take the time to build one of these. Otherwise I'd rather wait for a Python 3.12 package.

@th0ma7
Copy link
Contributor Author

th0ma7 commented Apr 10, 2024

Been lacking cycles lately but i intend to include py312 as well. But most importantly is the ability to generate as needed a crossenv for wheels that requires specific dependencies that may be an issue to be added in the default build (like panda). I got prototype code working locally, I'll let you know once i have something to play with.

@hgy59
Copy link
Contributor

hgy59 commented Apr 24, 2024

@th0ma7 BTW they released Python 3.11.9 on April 2, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants