Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use PyType_GetDict to safely access tp_dict [3.12] #408

Closed
wants to merge 3 commits into from
Closed

Use PyType_GetDict to safely access tp_dict [3.12] #408

wants to merge 3 commits into from

Conversation

cdce8p
Copy link
Contributor

@cdce8p cdce8p commented Jul 23, 2023

This resolve the Segmenation fault issues with Python 3.12.0b1 - 3.12.0b4. My particular test case, although others in the existing test suite failed as well (when tested with Python 3.12.0b4).

import orjson

def default(obj):
    if isinstance(obj, set):
        return list(obj)
    raise TypeError

print(orjson.dumps({1, 2}, default=default))

For 3.12 it's no longer safe to assume tp_dict is always a PyObject as it can be NULL too for static builtin types now. Instead use PyType_GetDict (new in 3.12) to access it safely.

Requires: PyO3/pyo3#3339

Refs:
https://docs.python.org/3.12/whatsnew/changelog.html?highlight=pytype_getdict#c-api
https://docs.python.org/3.12/c-api/type.html#c.PyType_GetDict
https://docs.python.org/3.12/c-api/typeobj.html#c.PyTypeObject.tp_dict

python/cpython#103912

@ijl
Copy link
Owner

ijl commented Jul 26, 2023

Thanks. This will wait on the first release candidate and PyO3.

@cdce8p
Copy link
Contributor Author

cdce8p commented Aug 1, 2023

PyO3 was released today: https://github.com/PyO3/pyo3/releases/tag/v0.19.2

I've updated the PR to use v0.19.2 now. Besides the change with PyType_GetDict the ref counting in pyo3 for Python 3.12 was changed a bit. See PyO3/pyo3#3335. Pushed another commit to address this.

@ijl
Copy link
Owner

ijl commented Aug 3, 2023

Thanks, this is on master cherry-picked as 5c8f93d, fa2df4d.

@ijl ijl closed this Aug 3, 2023
@cdce8p cdce8p deleted the fix-tp_dict branch August 3, 2023 15:52
@cdce8p cdce8p restored the fix-tp_dict branch August 3, 2023 15:52
@cdce8p cdce8p deleted the fix-tp_dict branch August 6, 2023 10:25
@ijl
Copy link
Owner

ijl commented Aug 6, 2023

@cdce8p
Copy link
Contributor Author

cdce8p commented Aug 6, 2023

Awesome! Thanks 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants