"Collecting" hangs forever if mark.parametrize is used with scope session or module #12355
Closed
4 tasks done
Labels
topic: fixtures
anything involving fixtures directly or indirectly
topic: parametrize
related to @pytest.mark.parametrize
type: performance
performance or memory problem/improvement
Comments
|
Thanks for the report. By gradually increasing the number of parameters, this looks like an "accidentally quadratic" (or worse...) situation. A (not so...) quick profile run puts the blame on Profile output
I don't have time to look into it right now and will look at it later, but this looks like a fun issue if someone else would like to try to figure it out. There are probably some python-level optimizations which could bring down the processing time, and algorithmic-level optimizations which could eliminate it entirely. Both are welcome. |
bluetech
added
topic: parametrize
bluetech
added a commit
to bluetech/pytest
that referenced
this issue
Jun 2, 2024
Fix pytest-dev#12355. In the issue, it was reported that the `reorder_items` has quadratic (or worse...) behavior with certain simple parametrizations. After some debugging I found that the problem happens because the "Fix items_by_argkey order" loop keeps adding the same item to the deque, and it reaches epic sizes which causes the slowdown. I don't claim to understand how the `reorder_items` algorithm works, but if as far as I understand, if an item already exists in the deque, the correct thing to do is to move it to the front. Since a deque doesn't have such an (efficient) operation, this switches to `OrderedDict` which can efficiently append from both sides, deduplicate and move to front.
bluetech
added a commit
to bluetech/pytest
that referenced
this issue
Jun 4, 2024
Fix pytest-dev#12355. In the issue, it was reported that the `reorder_items` has quadratic (or worse...) behavior with certain simple parametrizations. After some debugging I found that the problem happens because the "Fix items_by_argkey order" loop keeps adding the same item to the deque, and it reaches epic sizes which causes the slowdown. I don't claim to understand how the `reorder_items` algorithm works, but if as far as I understand, if an item already exists in the deque, the correct thing to do is to move it to the front. Since a deque doesn't have such an (efficient) operation, this switches to `OrderedDict` which can efficiently append from both sides, deduplicate and move to front.
bluetech
added a commit
to bluetech/pytest
that referenced
this issue
Jun 4, 2024
…ems` Manual minimal backport from commit e89d23b. Fix pytest-dev#12355. In the issue, it was reported that the `reorder_items` has quadratic (or worse...) behavior with certain simple parametrizations. After some debugging I found that the problem happens because the "Fix items_by_argkey order" loop keeps adding the same item to the deque, and it reaches epic sizes which causes the slowdown. I don't claim to understand how the `reorder_items` algorithm works, but if as far as I understand, if an item already exists in the deque, the correct thing to do is to move it to the front. Since a deque doesn't have such an (efficient) operation, this switches to `OrderedDict` which can efficiently append from both sides, deduplicate and move to front.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If you parametrize a test function with several parameters with
@mark.parametrizeand set scope to "session" or "module" the collection step will take forever*. Debugging shows that_pytest/fixtures.py:fix_cache_order()is called repeatedly:*forever scales with the number of parameters.
pip listfrom the virtual environment you are usingpytest and operating system versions
minimal example if possible
The issue is reproducible with pytest 7.4.0 and 8.2.1 with the following minimal example.
The text was updated successfully, but these errors were encountered: