Numpy 2.0 Wheels #15

mikedh · 2024-05-11T00:04:29Z

Thanks for maintaining these bindings!! I'm working on supporting the Numpy 2.0 release and I noticed that the current wheels + Numpy 2.0 do some funky stuff. I was able to fix it with a simple re-build, although on further investigation I'm not sure why that worked (maybe it used a new pybind11 from my environment?) haha.

It appears that the minimal thing we need to do is pin pybind11>=2.12 which supports both Numpy 1.x and 2.x and re-build. However I was poking and it looks like tests may not have been running on the wheels so I ended up also changing:

I moved the cibuildwheel logic and project information into pyproject.toml and only left the extension stuff in setup.py.
I don't think cibuildwheel was running tests previously? I added a test-command that runs pytest with a default install (Numpy 1.x), and then also tests with a Numpy 2.0 prerelease if one is available for that platform (this did previously fail).
I changed the build version pin to pybind11==2.12.0
Wheels are currently building.

If the infrastructure changes are too much I'd be also happy to re-open this as the minimal PR which would (I think) be:

pin pybind11==2.12.0
bump version

ChrisBarker-NOAA · 2024-07-18T19:24:27Z

NOTE: we have it building on conda-forge now. Not the same, but it's something, and shows it can work.

By the way, it would be good to put the tests in the sdist -- or better yet, in the package (they're small). That way, the buitl wheels/conda packages can be easily tested.

ChrisBarker-NOAA

thanks for getting this started.

@skogler: any change you're still active and can merge some of this?

ChrisBarker-NOAA · 2024-07-22T17:45:45Z

pyproject.toml

@@ -2,6 +2,49 @@
 requires = [
    "setuptools>=42",
    "wheel",
-    "pybind11~=2.6",
+    "pybind11==2.12.0",


maybe put a >= in here?

ChrisBarker-NOAA · 2024-07-22T17:46:45Z

pyproject.toml

+[project]
+name = "mapbox_earcut"
+version = "1.0.2"
+requires-python = ">=3.7"


I think this should be >=3.9:

3.7 is getting pretty old

numpy 2.0 isn't available for < 3.9 anyway.

ChrisBarker-NOAA · 2024-07-22T17:47:37Z

pyproject.toml

+authors = [{name = "Samuel Kogler", email = "samuel.kogler@gmail.com"}]
+license =  {file = "LICENSE.md"}
+description = "Python bindings for the mapbox earcut C++ polygon triangulation library."
+dependencies = ["numpy>=1.19.0"]


maybe > 1.24? -- not entirely sure how far back the 2.0 compatibility goes -- but no on e needs an ancient numpy :-)

ChrisBarker-NOAA · 2024-07-22T17:48:32Z

pyproject.toml

+	        "pip install --force-reinstall --upgrade --pre numpy",
+		"pytest {package}/tests"]
+
+# don't test on PyPy as it will re-build numpy


does this workin PyPy at all? I give up on that on the conda-forge build.

ChrisBarker-NOAA · 2024-07-22T17:53:00Z

pyproject.toml

+# Run the package tests using `pytest`
+# also test against pre-release Numpy
+# TODO : when numpy 2.0 releases this can be reduced to just one pytest
+test-command = ["pytest {package}/tests",


does this work? the tests aren't in the sdist -- I had to put acopy of the test fijles in the conda recipe to get the tests to run.

really, they should be installed with the package -- they are small, and it's realy good to be able to directly test the installed distro with a compiled package like this.

ChrisBarker-NOAA · 2024-07-22T17:55:01Z

setup.py

@@ -3,40 +3,44 @@
 from setuptools import setup


we really should move to a "modern" build -- e.g. not putting code in setup.py.

maybe even scikit build?

ChrisBarker-NOAA · 2024-07-29T23:28:37Z

How important is it to change the module name?

That's going to break downstream packages -- how many? I have no idea, but mine, anyway :-)

If you do decide to change it, please put in an alias to the old name that raises a deprecation warning ...

mikedh · 2024-07-29T23:52:24Z

Hey, yeah I needed wheels on PyPi for something today so I forked it into pip install earcutx, the name change is just because in the past I've not done that and ended up extremely confused as to which fork I was installing 😆. I think the minimal things the project here needs to change are just pybind11>=2.12.0 and possibly un-pin the OS versions for the CI runners.

FWIW earcut.hpp wins the triangulator wars pretty thoroughly for most casual users when I did some quick benchmarks.

ChrisBarker-NOAA · 2024-07-30T00:51:09Z

Ahh -- so no word from @skogler yet? darn.

OK -- for now, I'm using my conda-forge packages, so no need for any changes.

Thanks for the benchmarks -- interesting, and good to know this is worth using.

I wonder what's up with shapely on your "quality" check -- what the heck?

skogler · 2024-07-30T21:03:16Z

Hey there, sorry for being absent, I do not currently use Python anymore, so I don't check this often!

Thanks for the PR, I will take a look this week!

ChrisBarker-NOAA · 2024-07-30T21:12:22Z

great, thanks! Perhaps you want to give one of us permissions on the rep for the future?

…

-CHB

On Tue, Jul 30, 2024 at 2:03 PM Samuel Kogler ***@***.***> wrote: Hey there, sorry for being absent, I do not currently use Python anymore, so I don't check this often! Thanks for the PR, I will take a look this week! — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG7YYETAIG5TY6J5UGX3CDZO75SVAVCNFSM6AAAAABHRM7Y4CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJZGIYDKMZVGY> . You are receiving this because you commented.Message ID: ***@***.***>

-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception ***@***.***

skogler · 2024-07-30T22:12:12Z

@mikedh Are you okay with me just integrating some of your changes manually? This PR includes a lot, and I would rather go step-by-step and include only what is necessary.

mikedh · 2024-07-30T23:32:56Z

That would be great! I think you just need to bump the pybind11 version and re-build the wheels.

ChrisBarker-NOAA · 2024-07-30T23:35:20Z

Can I put a plug in for adding the tests to the sdist, too?

That way, I can run them in a conda-build script -- which is a really good way to make sure it's actually built correctly.

skogler · 2024-07-31T05:58:22Z

@ChrisBarker-NOAA Tests are now included in the sdist

ChrisBarker-NOAA · 2024-07-31T15:49:35Z

Great, thanks! When you get a new release out, I'll update the conda-forge feedstock.

https://github.com/conda-forge/mapbox_earcut-feedstock
Let me now if either of you want to be involved with that.

skogler · 2024-08-03T15:10:03Z

Release with new builds is out! Could you check if it fits your needs?
I had to remove some variants from the wheels, but I tried to match whatever numpy publishes as close as possible.

skogler · 2024-08-04T08:32:53Z

Closing this for now, feel free to open issues/PRs if you have problems!

mikedh · 2024-08-13T19:59:10Z

Awesome thank you!! Tested in docker and locally and looks like it's working great!

mikedh added 10 commits May 9, 2024 14:29

move cibuildwheel to pyproject and test on numpy 2.0

6fefcb2

trigger action

921f369

try oldest numpy on python 3.6

93e4ad3

move project info to pyproject

3bb9b85

relax versions

d322389

set minimum python to 3.7

2d7a36a

skip tests where numpy isn't available as wheels

7865c68

skip pypy

9abaf37

try pinning pybind

db28557

use tomli for old python

2b8fb94

mikedh mentioned this pull request May 17, 2024

Release: Numpy 2.0 Compatibility mikedh/trimesh#2220

Merged

This was referenced Jun 9, 2024

Feature: Numpy 2 Support thomwolf/pyVHACD#4

Closed

Feature: Numpy 2 Support. trimesh/vhacdx#5

Merged

zh-pan mentioned this pull request Jul 6, 2024

Erro result when I test readme example! #16

Closed

mikedh mentioned this pull request Jul 7, 2024

Release: Numpy 2 Followup + Drop Python 3.7 mikedh/trimesh#2238

Merged

This was referenced Jul 12, 2024

Numpy 2.0 support Deltares/xugrid#259

Merged

mapbox_earcut is broken for numpy 2.0 conda-forge/mapbox_earcut-feedstock#23

Open

ChrisBarker-NOAA reviewed Jul 22, 2024

View reviewed changes

mikedh added 6 commits July 29, 2024 13:01

address suggestions and release as earcutx

145dd08

skip pypy

784c5b8

try building on ubuntu latest

6fdbf6d

update macos runner

bb1281b

release on main

1d379cd

update test module ref

85c0fd6

mikedh closed this Jul 29, 2024

skogler reopened this Jul 30, 2024

skogler closed this Aug 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Numpy 2.0 Wheels #15

Numpy 2.0 Wheels #15

mikedh commented May 11, 2024

ChrisBarker-NOAA commented Jul 18, 2024

ChrisBarker-NOAA left a comment

ChrisBarker-NOAA Jul 22, 2024

ChrisBarker-NOAA Jul 22, 2024

ChrisBarker-NOAA Jul 22, 2024

ChrisBarker-NOAA Jul 22, 2024

ChrisBarker-NOAA Jul 22, 2024

ChrisBarker-NOAA Jul 22, 2024

ChrisBarker-NOAA commented Jul 29, 2024

mikedh commented Jul 29, 2024

ChrisBarker-NOAA commented Jul 30, 2024

skogler commented Jul 30, 2024

ChrisBarker-NOAA commented Jul 30, 2024 via email

skogler commented Jul 30, 2024

mikedh commented Jul 30, 2024

ChrisBarker-NOAA commented Jul 30, 2024

skogler commented Jul 31, 2024

ChrisBarker-NOAA commented Jul 31, 2024

skogler commented Aug 3, 2024

skogler commented Aug 4, 2024

mikedh commented Aug 13, 2024

Numpy 2.0 Wheels #15

Numpy 2.0 Wheels #15

Conversation

mikedh commented May 11, 2024

ChrisBarker-NOAA commented Jul 18, 2024

ChrisBarker-NOAA left a comment

Choose a reason for hiding this comment

ChrisBarker-NOAA Jul 22, 2024

Choose a reason for hiding this comment

ChrisBarker-NOAA Jul 22, 2024

Choose a reason for hiding this comment

ChrisBarker-NOAA Jul 22, 2024

Choose a reason for hiding this comment

ChrisBarker-NOAA Jul 22, 2024

Choose a reason for hiding this comment

ChrisBarker-NOAA Jul 22, 2024

Choose a reason for hiding this comment

ChrisBarker-NOAA Jul 22, 2024

Choose a reason for hiding this comment

ChrisBarker-NOAA commented Jul 29, 2024

mikedh commented Jul 29, 2024

ChrisBarker-NOAA commented Jul 30, 2024

skogler commented Jul 30, 2024

ChrisBarker-NOAA commented Jul 30, 2024 via email

skogler commented Jul 30, 2024

mikedh commented Jul 30, 2024

ChrisBarker-NOAA commented Jul 30, 2024

skogler commented Jul 31, 2024

ChrisBarker-NOAA commented Jul 31, 2024

skogler commented Aug 3, 2024

skogler commented Aug 4, 2024

mikedh commented Aug 13, 2024