diff --git a/changelog/11706.bugfix.rst b/changelog/11706.bugfix.rst deleted file mode 100644 index a86db5ef66a..00000000000 --- a/changelog/11706.bugfix.rst +++ /dev/null @@ -1,4 +0,0 @@ -Fix reporting of teardown errors in higher-scoped fixtures when using `--maxfail` or `--stepwise`. - -Originally added in pytest 8.0.0, but reverted in 8.0.2 due to a regression in pytest-xdist. -This regression was fixed in pytest-xdist 3.6.1. diff --git a/changelog/11771.contrib.rst b/changelog/11771.contrib.rst deleted file mode 100644 index a3c1ed1099e..00000000000 --- a/changelog/11771.contrib.rst +++ /dev/null @@ -1,5 +0,0 @@ -The PyPy runtime version has been updated to 3.9 from 3.8 that introduced -a flaky bug at the garbage collector which was not expected to fix there -as the V3.8 is EoL. - --- by :user:`x612skm` diff --git a/changelog/11797.bugfix.rst b/changelog/11797.bugfix.rst deleted file mode 100644 index 94b72da00fd..00000000000 --- a/changelog/11797.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -:func:`pytest.approx` now correctly handles :class:`Sequence `-like objects. diff --git a/changelog/12153.doc.rst b/changelog/12153.doc.rst deleted file mode 100644 index ac36becf9a7..00000000000 --- a/changelog/12153.doc.rst +++ /dev/null @@ -1 +0,0 @@ -Documented using :envvar:`PYTEST_VERSION` to detect if code is running from within a pytest run. diff --git a/changelog/12204.bugfix.rst b/changelog/12204.bugfix.rst deleted file mode 100644 index 099ad70610a..00000000000 --- a/changelog/12204.bugfix.rst +++ /dev/null @@ -1,11 +0,0 @@ -Fixed a regression in pytest 8.0 where tracebacks get longer and longer when multiple -tests fail due to a shared higher-scope fixture which raised -- by :user:`bluetech`. - -Also fixed a similar regression in pytest 5.4 for collectors which raise during setup. - -The fix necessitated internal changes which may affect some plugins: - -* ``FixtureDef.cached_result[2]`` is now a tuple ``(exc, tb)`` - instead of ``exc``. -* ``SetupState.stack`` failures are now a tuple ``(exc, tb)`` - instead of ``exc``. diff --git a/changelog/12231.feature.rst b/changelog/12231.feature.rst deleted file mode 100644 index dad04bc20c1..00000000000 --- a/changelog/12231.feature.rst +++ /dev/null @@ -1,11 +0,0 @@ -Added `--xfail-tb` flag, which turns on traceback output for XFAIL results. - -* If the `--xfail-tb` flag is not sent, tracebacks for XFAIL results are NOT shown. -* The style of traceback for XFAIL is set with `--tb`, and can be `auto|long|short|line|native|no`. -* Note: Even if you have `--xfail-tb` set, you won't see them if `--tb=no`. - -Some history: - -With pytest 8.0, `-rx` or `-ra` would not only turn on summary reports for xfail, but also report the tracebacks for xfail results. This caused issues with some projects that utilize xfail, but don't want to see all of the xfail tracebacks. - -This change detaches xfail tracebacks from `-rx`, and now we turn on xfail tracebacks with `--xfail-tb`. With this, the default `-rx`/ `-ra` behavior is identical to pre-8.0 with respect to xfail tracebacks. While this is a behavior change, it brings default behavior back to pre-8.0.0 behavior, which ultimately was considered the better course of action. diff --git a/changelog/12264.bugfix.rst b/changelog/12264.bugfix.rst deleted file mode 120000 index e5704e6e819..00000000000 --- a/changelog/12264.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -12204.bugfix.rst \ No newline at end of file diff --git a/changelog/12275.bugfix.rst b/changelog/12275.bugfix.rst deleted file mode 100644 index 2d040a3a063..00000000000 --- a/changelog/12275.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix collection error upon encountering an :mod:`abstract ` class, including abstract `unittest.TestCase` subclasses. diff --git a/changelog/12281.feature.rst b/changelog/12281.feature.rst deleted file mode 100644 index c6e8e3b3098..00000000000 --- a/changelog/12281.feature.rst +++ /dev/null @@ -1,8 +0,0 @@ -Added support for keyword matching in marker expressions. - -Now tests can be selected by marker keyword arguments. -Supported values are :class:`int`, (unescaped) :class:`str`, :class:`bool` & :data:`None`. - -See :ref:`marker examples ` for more information. - --- by :user:`lovetheguitar` diff --git a/changelog/12328.bugfix.rst b/changelog/12328.bugfix.rst deleted file mode 100644 index f334425850b..00000000000 --- a/changelog/12328.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix a regression in pytest 8.0.0 where package-scoped parameterized items were not correctly reordered to minimize setups/teardowns in some cases. diff --git a/changelog/12424.bugfix.rst b/changelog/12424.bugfix.rst deleted file mode 100644 index 7ad1126858b..00000000000 --- a/changelog/12424.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fix crash with `assert testcase is not None` assertion failure when re-running unittest tests using plugins like pytest-rerunfailures. Regressed in 8.2.2. diff --git a/changelog/12467.improvement.rst b/changelog/12467.improvement.rst deleted file mode 100644 index b1e0581ed16..00000000000 --- a/changelog/12467.improvement.rst +++ /dev/null @@ -1,3 +0,0 @@ -Migrated all internal type-annotations to the python3.10+ style by using the `annotations` future import. - --- by :user:`RonnyPfannschmidt` diff --git a/changelog/12469.doc.rst b/changelog/12469.doc.rst deleted file mode 100644 index 2340315353c..00000000000 --- a/changelog/12469.doc.rst +++ /dev/null @@ -1,6 +0,0 @@ -The external plugin mentions in the documentation now avoid mentioning -:std:doc:`setuptools entry-points ` as the concept is -much more generic nowadays. Instead, the terminology of "external", -"installed", or "third-party" plugins (or packages) replaces that. - --- by :user:`webknjaz` diff --git a/changelog/12469.improvement.rst b/changelog/12469.improvement.rst deleted file mode 100644 index a90fb1e6610..00000000000 --- a/changelog/12469.improvement.rst +++ /dev/null @@ -1,4 +0,0 @@ -The console output now uses the "third-party plugins" terminology, -replacing the previously established but confusing and outdated -reference to :std:doc:`setuptools ` --- by :user:`webknjaz`. diff --git a/changelog/12472.bugfix.rst b/changelog/12472.bugfix.rst deleted file mode 100644 index f08e9d1f90b..00000000000 --- a/changelog/12472.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed a crash when returning category ``"error"`` or ``"failed"`` with a custom test status from :hook:`pytest_report_teststatus` hook -- :user:`pbrezina`. diff --git a/changelog/12493.contrib.rst b/changelog/12493.contrib.rst deleted file mode 100644 index db3d045697e..00000000000 --- a/changelog/12493.contrib.rst +++ /dev/null @@ -1,13 +0,0 @@ -The change log draft preview integration has been refactored to use a -third party extension ``sphinxcontib-towncrier``. The previous in-repo -script was putting the change log preview file at -:file:`doc/en/_changelog_towncrier_draft.rst`. Said file is no longer -ignored in Git and might show up among untracked files in the -development environments of the contributors. To address that, the -contributors can run the following command that will clean it up: - -.. code-block:: console - - $ git clean -x -i -- doc/en/_changelog_towncrier_draft.rst - --- by :user:`webknjaz` diff --git a/changelog/12498.contrib.rst b/changelog/12498.contrib.rst deleted file mode 100644 index 436c6f0e9ed..00000000000 --- a/changelog/12498.contrib.rst +++ /dev/null @@ -1,5 +0,0 @@ -All the undocumented ``tox`` environments now have descriptions. -They can be listed in one's development environment by invoking -``tox -av`` in a terminal. - --- by :user:`webknjaz` diff --git a/changelog/12501.contrib.rst b/changelog/12501.contrib.rst deleted file mode 100644 index 6f434c287b3..00000000000 --- a/changelog/12501.contrib.rst +++ /dev/null @@ -1,11 +0,0 @@ -The changelog configuration has been updated to introduce more accurate -audience-tailored categories. Previously, there was a ``trivial`` -change log fragment type with an unclear and broad meaning. It was -removed and we now have ``contrib``, ``misc`` and ``packaging`` in -place of it. - -The new change note types target the readers who are downstream -packagers and project contributors. Additionally, the miscellaneous -section is kept for unspecified updates that do not fit anywhere else. - --- by :user:`webknjaz` diff --git a/changelog/12502.contrib.rst b/changelog/12502.contrib.rst deleted file mode 100644 index 940a2d7a120..00000000000 --- a/changelog/12502.contrib.rst +++ /dev/null @@ -1,7 +0,0 @@ -The UX of the GitHub automation making pull requests to update the -plugin list has been updated. Previously, the maintainers had to close -the automatically created pull requests and re-open them to trigger the -CI runs. From now on, they only need to click the `Ready for review` -button instead. - --- by :user:`webknjaz` diff --git a/changelog/12505.bugfix.rst b/changelog/12505.bugfix.rst deleted file mode 100644 index f55a8a17e4b..00000000000 --- a/changelog/12505.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Improve handling of invalid regex patterns in :func:`pytest.raises(match=r'...') ` by providing a clear error message. diff --git a/changelog/12522.contrib.rst b/changelog/12522.contrib.rst deleted file mode 100644 index dd994317165..00000000000 --- a/changelog/12522.contrib.rst +++ /dev/null @@ -1,4 +0,0 @@ -The ``:pull:`` RST role has been replaced with a shorter -``:pr:`` due to starting to use the implementation from -the third-party :pypi:`sphinx-issues` Sphinx extension --- by :user:`webknjaz`. diff --git a/changelog/12531.contrib.rst b/changelog/12531.contrib.rst deleted file mode 100644 index 12083fc320e..00000000000 --- a/changelog/12531.contrib.rst +++ /dev/null @@ -1,6 +0,0 @@ -The coverage reporting configuration has been updated to exclude -pytest's own tests marked as expected to fail from the coverage -report. This has an effect of reducing the influence of flaky -tests on the resulting number. - --- by :user:`webknjaz` diff --git a/changelog/12533.contrib.rst b/changelog/12533.contrib.rst deleted file mode 100644 index 3da7007a0fd..00000000000 --- a/changelog/12533.contrib.rst +++ /dev/null @@ -1,7 +0,0 @@ -The ``extlinks`` Sphinx extension is no longer enabled. The ``:bpo:`` -role it used to declare has been removed with that. BPO itself has -migrated to GitHub some years ago and it is possible to link the -respective issues by using their GitHub issue numbers and the -``:issue:`` role that the ``sphinx-issues`` extension implements. - --- by :user:`webknjaz` diff --git a/changelog/12544.improvement.rst b/changelog/12544.improvement.rst deleted file mode 100644 index 41125f5d939..00000000000 --- a/changelog/12544.improvement.rst +++ /dev/null @@ -1,3 +0,0 @@ -The ``_in_venv()`` function now detects Python virtual environments by -checking for a :file:`pyvenv.cfg` file, ensuring reliable detection on -various platforms -- by :user:`zachsnickers`. diff --git a/changelog/12545.improvement.rst b/changelog/12545.improvement.rst deleted file mode 120000 index 41a1e6bfa49..00000000000 --- a/changelog/12545.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -12544.improvement.rst \ No newline at end of file diff --git a/changelog/12557.contrib.rst b/changelog/12557.contrib.rst deleted file mode 120000 index c036c519093..00000000000 --- a/changelog/12557.contrib.rst +++ /dev/null @@ -1 +0,0 @@ -11771.contrib.rst \ No newline at end of file diff --git a/changelog/12562.contrib.rst b/changelog/12562.contrib.rst deleted file mode 100644 index 0d30495983a..00000000000 --- a/changelog/12562.contrib.rst +++ /dev/null @@ -1,2 +0,0 @@ -Possible typos in using the ``:user:`` RST role is now being linted -through the pre-commit tool integration -- by :user:`webknjaz`. diff --git a/changelog/12567.feature.rst b/changelog/12567.feature.rst deleted file mode 100644 index 3690d7aff68..00000000000 --- a/changelog/12567.feature.rst +++ /dev/null @@ -1,7 +0,0 @@ -Added ``--no-fold-skipped`` command line option - -If this option is set, then skipped tests in short summary are no longer grouped -by reason but all tests are printed individually with correct nodeid in the same -way as other statuses. - --- by :user:`pbrezina` diff --git a/changelog/12577.doc.rst b/changelog/12577.doc.rst deleted file mode 100644 index 0bd427e177d..00000000000 --- a/changelog/12577.doc.rst +++ /dev/null @@ -1,3 +0,0 @@ -`CI` and `BUILD_NUMBER` environment variables role is discribed in -the reference doc. They now also appears when doing `pytest -h` --- by :user:`MarcBresson`. diff --git a/changelog/12580.bugfix.rst b/changelog/12580.bugfix.rst deleted file mode 100644 index 9186ef1a4c9..00000000000 --- a/changelog/12580.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed a crash when using the cache class on Windows and the cache directory was created concurrently. diff --git a/changelog/2871.improvement.rst b/changelog/2871.improvement.rst deleted file mode 100644 index 1ba399550c7..00000000000 --- a/changelog/2871.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -Do not truncate arguments to functions in output when running with `-vvv`. diff --git a/changelog/389.improvement.rst b/changelog/389.improvement.rst deleted file mode 100644 index f8e2c19fde0..00000000000 --- a/changelog/389.improvement.rst +++ /dev/null @@ -1,38 +0,0 @@ -The readability of assertion introspection of bound methods has been enhanced --- by :user:`farbodahm`, :user:`webknjaz`, :user:`obestwalter`, :user:`flub` -and :user:`glyphack`. - -Earlier, it was like: - -.. code-block:: console - - =================================== FAILURES =================================== - _____________________________________ test _____________________________________ - - def test(): - > assert Help().fun() == 2 - E assert 1 == 2 - E + where 1 = >() - E + where > = .fun - E + where = Help() - - example.py:7: AssertionError - =========================== 1 failed in 0.03 seconds =========================== - - -And now it's like: - -.. code-block:: console - - =================================== FAILURES =================================== - _____________________________________ test _____________________________________ - - def test(): - > assert Help().fun() == 2 - E assert 1 == 2 - E + where 1 = fun() - E + where fun = .fun - E + where = Help() - - test_local.py:13: AssertionError - =========================== 1 failed in 0.03 seconds =========================== diff --git a/changelog/6962.bugfix.rst b/changelog/6962.bugfix.rst deleted file mode 100644 index 030b6e06392..00000000000 --- a/changelog/6962.bugfix.rst +++ /dev/null @@ -1,2 +0,0 @@ -Parametrization parameters are now compared using `==` instead of `is` (`is` is still used as a fallback if the parameter does not support `==`). -This fixes use of parameters such as lists, which have a different `id` but compare equal, causing fixtures to be re-computed instead of being cached. diff --git a/changelog/7166.bugfix.rst b/changelog/7166.bugfix.rst deleted file mode 100644 index 98e6821f2ff..00000000000 --- a/changelog/7166.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed progress percentages (the ``[ 87%]`` at the edge of the screen) sometimes not aligning correctly when running with pytest-xdist ``-n``. diff --git a/changelog/7662.improvement.rst b/changelog/7662.improvement.rst deleted file mode 100644 index b6ae1ba7e4c..00000000000 --- a/changelog/7662.improvement.rst +++ /dev/null @@ -1 +0,0 @@ -Added timezone information to the testsuite timestamp in the JUnit XML report. diff --git a/doc/en/announce/index.rst b/doc/en/announce/index.rst index c65eb5f3613..aa5dd8c87b4 100644 --- a/doc/en/announce/index.rst +++ b/doc/en/announce/index.rst @@ -6,6 +6,7 @@ Release announcements :maxdepth: 2 + release-8.3.0 release-8.2.2 release-8.2.1 release-8.2.0 diff --git a/doc/en/announce/release-8.3.0.rst b/doc/en/announce/release-8.3.0.rst new file mode 100644 index 00000000000..ec5cd3d0db9 --- /dev/null +++ b/doc/en/announce/release-8.3.0.rst @@ -0,0 +1,60 @@ +pytest-8.3.0 +======================================= + +The pytest team is proud to announce the 8.3.0 release! + +This release contains new features, improvements, and bug fixes, +the full list of changes is available in the changelog: + + https://docs.pytest.org/en/stable/changelog.html + +For complete documentation, please visit: + + https://docs.pytest.org/en/stable/ + +As usual, you can upgrade from PyPI via: + + pip install -U pytest + +Thanks to all of the contributors to this release: + +* Anita Hammer +* Ben Brown +* Brian Okken +* Bruno Oliveira +* Cornelius Riemenschneider +* Farbod Ahmadian +* Florian Bruhin +* Hynek Schlawack +* James Frost +* Jason R. Coombs +* Jelle Zijlstra +* Josh Soref +* Marc Bresson +* Michael Vogt +* Nathan Goldbaum +* Nicolas Simonds +* Oliver Bestwalter +* Pavel Březina +* Pierre Sassoulas +* Pradyun Gedam +* Ran Benita +* Ronny Pfannschmidt +* SOUBHIK KUMAR MITRA +* Sam Jirovec +* Stavros Ntentos +* Sviatoslav Sydorenko +* Sviatoslav Sydorenko (Святослав Сидоренко) +* Tomasz Kłoczko +* Virendra Patil +* Yutian Li +* Zach Snicker +* dj +* holger krekel +* joseph-sentry +* lovetheguitar +* neutraljump + + +Happy testing, +The pytest Development Team diff --git a/doc/en/builtin.rst b/doc/en/builtin.rst index 8dfffb0828a..b4c0f91f8e5 100644 --- a/doc/en/builtin.rst +++ b/doc/en/builtin.rst @@ -22,7 +22,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a cachedir: .pytest_cache rootdir: /home/sweet/project collected 0 items - cache -- .../_pytest/cacheprovider.py:560 + cache -- .../_pytest/cacheprovider.py:558 Return a cache object that can persist state between testing sessions. cache.get(key, default) @@ -33,7 +33,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a Values can be any object handled by the json stdlib module. - capsysbinary -- .../_pytest/capture.py:1003 + capsysbinary -- .../_pytest/capture.py:1005 Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``. The captured output is made available via ``capsysbinary.readouterr()`` @@ -43,6 +43,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a Returns an instance of :class:`CaptureFixture[bytes] `. Example: + .. code-block:: python def test_output(capsysbinary): @@ -50,7 +51,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a captured = capsysbinary.readouterr() assert captured.out == b"hello\n" - capfd -- .../_pytest/capture.py:1030 + capfd -- .../_pytest/capture.py:1033 Enable text capturing of writes to file descriptors ``1`` and ``2``. The captured output is made available via ``capfd.readouterr()`` method @@ -60,6 +61,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a Returns an instance of :class:`CaptureFixture[str] `. Example: + .. code-block:: python def test_system_echo(capfd): @@ -67,7 +69,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a captured = capfd.readouterr() assert captured.out == "hello\n" - capfdbinary -- .../_pytest/capture.py:1057 + capfdbinary -- .../_pytest/capture.py:1061 Enable bytes capturing of writes to file descriptors ``1`` and ``2``. The captured output is made available via ``capfd.readouterr()`` method @@ -77,6 +79,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a Returns an instance of :class:`CaptureFixture[bytes] `. Example: + .. code-block:: python def test_system_echo(capfdbinary): @@ -84,7 +87,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a captured = capfdbinary.readouterr() assert captured.out == b"hello\n" - capsys -- .../_pytest/capture.py:976 + capsys -- .../_pytest/capture.py:977 Enable text capturing of writes to ``sys.stdout`` and ``sys.stderr``. The captured output is made available via ``capsys.readouterr()`` method @@ -94,6 +97,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a Returns an instance of :class:`CaptureFixture[str] `. Example: + .. code-block:: python def test_output(capsys): @@ -101,7 +105,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a captured = capsys.readouterr() assert captured.out == "hello\n" - doctest_namespace [session scope] -- .../_pytest/doctest.py:738 + doctest_namespace [session scope] -- .../_pytest/doctest.py:740 Fixture that returns a :py:class:`dict` that will be injected into the namespace of doctests. @@ -115,7 +119,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a For more details: :ref:`doctest_namespace`. - pytestconfig [session scope] -- .../_pytest/fixtures.py:1338 + pytestconfig [session scope] -- .../_pytest/fixtures.py:1344 Session-scoped fixture that returns the session's :class:`pytest.Config` object. @@ -125,7 +129,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a if pytestconfig.getoption("verbose") > 0: ... - record_property -- .../_pytest/junitxml.py:284 + record_property -- .../_pytest/junitxml.py:280 Add extra properties to the calling test. User properties become part of the test report and are available to the @@ -139,13 +143,13 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a def test_function(record_property): record_property("example_key", 1) - record_xml_attribute -- .../_pytest/junitxml.py:307 + record_xml_attribute -- .../_pytest/junitxml.py:303 Add extra xml attributes to the tag for the calling test. The fixture is callable with ``name, value``. The value is automatically XML-encoded. - record_testsuite_property [session scope] -- .../_pytest/junitxml.py:345 + record_testsuite_property [session scope] -- .../_pytest/junitxml.py:341 Record a new ```` tag as child of the root ````. This is suitable to writing global information regarding the entire test @@ -170,10 +174,10 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a `pytest-xdist `__ plugin. See :issue:`7767` for details. - tmpdir_factory [session scope] -- .../_pytest/legacypath.py:303 + tmpdir_factory [session scope] -- .../_pytest/legacypath.py:298 Return a :class:`pytest.TempdirFactory` instance for the test session. - tmpdir -- .../_pytest/legacypath.py:310 + tmpdir -- .../_pytest/legacypath.py:305 Return a temporary directory path object which is unique to each test function invocation, created as a sub directory of the base temporary directory. @@ -192,7 +196,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a .. _legacy_path: https://py.readthedocs.io/en/latest/path.html - caplog -- .../_pytest/logging.py:602 + caplog -- .../_pytest/logging.py:600 Access and control log capturing. Captured logs are available through the following properties/methods:: @@ -203,7 +207,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a * caplog.record_tuples -> list of (logger_name, level, message) tuples * caplog.clear() -> clear captured records and formatted log output string - monkeypatch -- .../_pytest/monkeypatch.py:33 + monkeypatch -- .../_pytest/monkeypatch.py:31 A convenient fixture for monkey-patching. The fixture provides these methods to modify objects, dictionaries, or @@ -227,7 +231,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a To undo modifications done by the fixture in a contained scope, use :meth:`context() `. - recwarn -- .../_pytest/recwarn.py:32 + recwarn -- .../_pytest/recwarn.py:35 Return a :class:`WarningsRecorder` instance that records all warnings emitted by test functions. See https://docs.pytest.org/en/latest/how-to/capture-warnings.html for information diff --git a/doc/en/changelog.rst b/doc/en/changelog.rst index 8e3efd0479b..12302ffe0e4 100644 --- a/doc/en/changelog.rst +++ b/doc/en/changelog.rst @@ -31,6 +31,263 @@ with advance notice in the **Deprecations** section of releases. .. towncrier release notes start +pytest 8.3.0 (2024-07-20) +========================= + +New features +------------ + +- `#12231 `_: Added `--xfail-tb` flag, which turns on traceback output for XFAIL results. + + * If the `--xfail-tb` flag is not sent, tracebacks for XFAIL results are NOT shown. + * The style of traceback for XFAIL is set with `--tb`, and can be `auto|long|short|line|native|no`. + * Note: Even if you have `--xfail-tb` set, you won't see them if `--tb=no`. + + Some history: + + With pytest 8.0, `-rx` or `-ra` would not only turn on summary reports for xfail, but also report the tracebacks for xfail results. This caused issues with some projects that utilize xfail, but don't want to see all of the xfail tracebacks. + + This change detaches xfail tracebacks from `-rx`, and now we turn on xfail tracebacks with `--xfail-tb`. With this, the default `-rx`/ `-ra` behavior is identical to pre-8.0 with respect to xfail tracebacks. While this is a behavior change, it brings default behavior back to pre-8.0.0 behavior, which ultimately was considered the better course of action. + + +- `#12281 `_: Added support for keyword matching in marker expressions. + + Now tests can be selected by marker keyword arguments. + Supported values are :class:`int`, (unescaped) :class:`str`, :class:`bool` & :data:`None`. + + See :ref:`marker examples ` for more information. + + -- by :user:`lovetheguitar` + + +- `#12567 `_: Added ``--no-fold-skipped`` command line option + + If this option is set, then skipped tests in short summary are no longer grouped + by reason but all tests are printed individually with correct nodeid in the same + way as other statuses. + + -- by :user:`pbrezina` + + + +Improvements in existing functionality +-------------------------------------- + +- `#12467 `_: Migrated all internal type-annotations to the python3.10+ style by using the `annotations` future import. + + -- by :user:`RonnyPfannschmidt` + + +- `#12469 `_: The console output now uses the "third-party plugins" terminology, + replacing the previously established but confusing and outdated + reference to :std:doc:`setuptools ` + -- by :user:`webknjaz`. + + +- `#12544 `_, `#12545 `_: The ``_in_venv()`` function now detects Python virtual environments by + checking for a :file:`pyvenv.cfg` file, ensuring reliable detection on + various platforms -- by :user:`zachsnickers`. + + +- `#2871 `_: Do not truncate arguments to functions in output when running with `-vvv`. + + +- `#389 `_: The readability of assertion introspection of bound methods has been enhanced + -- by :user:`farbodahm`, :user:`webknjaz`, :user:`obestwalter`, :user:`flub` + and :user:`glyphack`. + + Earlier, it was like: + + .. code-block:: console + + =================================== FAILURES =================================== + _____________________________________ test _____________________________________ + + def test(): + > assert Help().fun() == 2 + E assert 1 == 2 + E + where 1 = >() + E + where > = .fun + E + where = Help() + + example.py:7: AssertionError + =========================== 1 failed in 0.03 seconds =========================== + + + And now it's like: + + .. code-block:: console + + =================================== FAILURES =================================== + _____________________________________ test _____________________________________ + + def test(): + > assert Help().fun() == 2 + E assert 1 == 2 + E + where 1 = fun() + E + where fun = .fun + E + where = Help() + + test_local.py:13: AssertionError + =========================== 1 failed in 0.03 seconds =========================== + + +- `#7662 `_: Added timezone information to the testsuite timestamp in the JUnit XML report. + + + +Bug fixes +--------- + +- `#11706 `_: Fix reporting of teardown errors in higher-scoped fixtures when using `--maxfail` or `--stepwise`. + + Originally added in pytest 8.0.0, but reverted in 8.0.2 due to a regression in pytest-xdist. + This regression was fixed in pytest-xdist 3.6.1. + + +- `#11797 `_: :func:`pytest.approx` now correctly handles :class:`Sequence `-like objects. + + +- `#12204 `_, `#12264 `_: Fixed a regression in pytest 8.0 where tracebacks get longer and longer when multiple + tests fail due to a shared higher-scope fixture which raised -- by :user:`bluetech`. + + Also fixed a similar regression in pytest 5.4 for collectors which raise during setup. + + The fix necessitated internal changes which may affect some plugins: + + * ``FixtureDef.cached_result[2]`` is now a tuple ``(exc, tb)`` + instead of ``exc``. + * ``SetupState.stack`` failures are now a tuple ``(exc, tb)`` + instead of ``exc``. + + +- `#12275 `_: Fix collection error upon encountering an :mod:`abstract ` class, including abstract `unittest.TestCase` subclasses. + + +- `#12328 `_: Fix a regression in pytest 8.0.0 where package-scoped parameterized items were not correctly reordered to minimize setups/teardowns in some cases. + + +- `#12424 `_: Fix crash with `assert testcase is not None` assertion failure when re-running unittest tests using plugins like pytest-rerunfailures. Regressed in 8.2.2. + + +- `#12472 `_: Fixed a crash when returning category ``"error"`` or ``"failed"`` with a custom test status from :hook:`pytest_report_teststatus` hook -- :user:`pbrezina`. + + +- `#12505 `_: Improve handling of invalid regex patterns in :func:`pytest.raises(match=r'...') ` by providing a clear error message. + + +- `#12580 `_: Fixed a crash when using the cache class on Windows and the cache directory was created concurrently. + + +- `#6962 `_: Parametrization parameters are now compared using `==` instead of `is` (`is` is still used as a fallback if the parameter does not support `==`). + This fixes use of parameters such as lists, which have a different `id` but compare equal, causing fixtures to be re-computed instead of being cached. + + +- `#7166 `_: Fixed progress percentages (the ``[ 87%]`` at the edge of the screen) sometimes not aligning correctly when running with pytest-xdist ``-n``. + + + +Improved documentation +---------------------- + +- `#12153 `_: Documented using :envvar:`PYTEST_VERSION` to detect if code is running from within a pytest run. + + +- `#12469 `_: The external plugin mentions in the documentation now avoid mentioning + :std:doc:`setuptools entry-points ` as the concept is + much more generic nowadays. Instead, the terminology of "external", + "installed", or "third-party" plugins (or packages) replaces that. + + -- by :user:`webknjaz` + + +- `#12577 `_: `CI` and `BUILD_NUMBER` environment variables role is discribed in + the reference doc. They now also appears when doing `pytest -h` + -- by :user:`MarcBresson`. + + + +Contributor-facing changes +-------------------------- + +- `#11771 `_, `#12557 `_: The PyPy runtime version has been updated to 3.9 from 3.8 that introduced + a flaky bug at the garbage collector which was not expected to fix there + as the V3.8 is EoL. + + -- by :user:`x612skm` + + +- `#12493 `_: The change log draft preview integration has been refactored to use a + third party extension ``sphinxcontib-towncrier``. The previous in-repo + script was putting the change log preview file at + :file:`doc/en/_changelog_towncrier_draft.rst`. Said file is no longer + ignored in Git and might show up among untracked files in the + development environments of the contributors. To address that, the + contributors can run the following command that will clean it up: + + .. code-block:: console + + $ git clean -x -i -- doc/en/_changelog_towncrier_draft.rst + + -- by :user:`webknjaz` + + +- `#12498 `_: All the undocumented ``tox`` environments now have descriptions. + They can be listed in one's development environment by invoking + ``tox -av`` in a terminal. + + -- by :user:`webknjaz` + + +- `#12501 `_: The changelog configuration has been updated to introduce more accurate + audience-tailored categories. Previously, there was a ``trivial`` + change log fragment type with an unclear and broad meaning. It was + removed and we now have ``contrib``, ``misc`` and ``packaging`` in + place of it. + + The new change note types target the readers who are downstream + packagers and project contributors. Additionally, the miscellaneous + section is kept for unspecified updates that do not fit anywhere else. + + -- by :user:`webknjaz` + + +- `#12502 `_: The UX of the GitHub automation making pull requests to update the + plugin list has been updated. Previously, the maintainers had to close + the automatically created pull requests and re-open them to trigger the + CI runs. From now on, they only need to click the `Ready for review` + button instead. + + -- by :user:`webknjaz` + + +- `#12522 `_: The ``:pull:`` RST role has been replaced with a shorter + ``:pr:`` due to starting to use the implementation from + the third-party :pypi:`sphinx-issues` Sphinx extension + -- by :user:`webknjaz`. + + +- `#12531 `_: The coverage reporting configuration has been updated to exclude + pytest's own tests marked as expected to fail from the coverage + report. This has an effect of reducing the influence of flaky + tests on the resulting number. + + -- by :user:`webknjaz` + + +- `#12533 `_: The ``extlinks`` Sphinx extension is no longer enabled. The ``:bpo:`` + role it used to declare has been removed with that. BPO itself has + migrated to GitHub some years ago and it is possible to link the + respective issues by using their GitHub issue numbers and the + ``:issue:`` role that the ``sphinx-issues`` extension implements. + + -- by :user:`webknjaz` + + +- `#12562 `_: Possible typos in using the ``:user:`` RST role is now being linted + through the pre-commit tool integration -- by :user:`webknjaz`. + + pytest 8.2.2 (2024-06-04) ========================= diff --git a/doc/en/example/parametrize.rst b/doc/en/example/parametrize.rst index d540bf08337..42dd1e2e416 100644 --- a/doc/en/example/parametrize.rst +++ b/doc/en/example/parametrize.rst @@ -162,7 +162,7 @@ objects, they are still using the default pytest representation: rootdir: /home/sweet/project collected 8 items - + @@ -239,7 +239,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia rootdir: /home/sweet/project collected 4 items - + @@ -318,7 +318,7 @@ Let's first see how it looks like at collection time: rootdir: /home/sweet/project collected 2 items - + @@ -505,8 +505,8 @@ Running it results in some skips if we don't have all the python interpreters in . $ pytest -rs -q multipython.py ssssssssssss...ssssssssssss [100%] ========================= short test summary info ========================== - SKIPPED [12] multipython.py:65: 'python3.9' not found - SKIPPED [12] multipython.py:65: 'python3.11' not found + SKIPPED [12] multipython.py:67: 'python3.9' not found + SKIPPED [12] multipython.py:67: 'python3.11' not found 3 passed, 24 skipped in 0.12s Parametrization of optional implementations/imports diff --git a/doc/en/example/pythoncollection.rst b/doc/en/example/pythoncollection.rst index 39b799ed934..2c8dbbb10d2 100644 --- a/doc/en/example/pythoncollection.rst +++ b/doc/en/example/pythoncollection.rst @@ -152,7 +152,7 @@ The test collection would look like this: configfile: pytest.ini collected 2 items - + @@ -215,7 +215,7 @@ You can always peek at the collection tree without running tests like this: configfile: pytest.ini collected 3 items - + diff --git a/doc/en/example/reportingdemo.rst b/doc/en/example/reportingdemo.rst index 2c34cc2b00d..0da58d0490e 100644 --- a/doc/en/example/reportingdemo.rst +++ b/doc/en/example/reportingdemo.rst @@ -25,7 +25,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > assert param1 * 2 < param2 E assert (3 * 2) < 6 - failure_demo.py:19: AssertionError + failure_demo.py:21: AssertionError _________________________ TestFailing.test_simple __________________________ self = @@ -42,7 +42,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + where 42 = .f at 0xdeadbeef0002>() E + and 43 = .g at 0xdeadbeef0003>() - failure_demo.py:30: AssertionError + failure_demo.py:32: AssertionError ____________________ TestFailing.test_simple_multiline _____________________ self = @@ -50,7 +50,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: def test_simple_multiline(self): > otherfunc_multi(42, 6 * 9) - failure_demo.py:33: + failure_demo.py:35: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ a = 42, b = 54 @@ -59,7 +59,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > assert a == b E assert 42 == 54 - failure_demo.py:14: AssertionError + failure_demo.py:16: AssertionError ___________________________ TestFailing.test_not ___________________________ self = @@ -72,7 +72,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert not 42 E + where 42 = .f at 0xdeadbeef0006>() - failure_demo.py:39: AssertionError + failure_demo.py:41: AssertionError _________________ TestSpecialisedExplanations.test_eq_text _________________ self = @@ -84,7 +84,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E - eggs E + spam - failure_demo.py:44: AssertionError + failure_demo.py:46: AssertionError _____________ TestSpecialisedExplanations.test_eq_similar_text _____________ self = @@ -98,7 +98,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + foo 1 bar E ? ^ - failure_demo.py:47: AssertionError + failure_demo.py:49: AssertionError ____________ TestSpecialisedExplanations.test_eq_multiline_text ____________ self = @@ -112,7 +112,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + spam E bar - failure_demo.py:50: AssertionError + failure_demo.py:52: AssertionError ______________ TestSpecialisedExplanations.test_eq_long_text _______________ self = @@ -130,7 +130,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + 1111111111a222222222 E ? ^ - failure_demo.py:55: AssertionError + failure_demo.py:57: AssertionError _________ TestSpecialisedExplanations.test_eq_long_text_multiline __________ self = @@ -150,7 +150,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E E ...Full output truncated (7 lines hidden), use '-vv' to show - failure_demo.py:60: AssertionError + failure_demo.py:62: AssertionError _________________ TestSpecialisedExplanations.test_eq_list _________________ self = @@ -162,7 +162,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E At index 2 diff: 2 != 3 E Use -v to get more diff - failure_demo.py:63: AssertionError + failure_demo.py:65: AssertionError ______________ TestSpecialisedExplanations.test_eq_list_long _______________ self = @@ -176,7 +176,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E At index 100 diff: 1 != 2 E Use -v to get more diff - failure_demo.py:68: AssertionError + failure_demo.py:70: AssertionError _________________ TestSpecialisedExplanations.test_eq_dict _________________ self = @@ -194,7 +194,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E {'d': 0} E Use -v to get more diff - failure_demo.py:71: AssertionError + failure_demo.py:73: AssertionError _________________ TestSpecialisedExplanations.test_eq_set __________________ self = @@ -212,7 +212,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E 21 E Use -v to get more diff - failure_demo.py:74: AssertionError + failure_demo.py:76: AssertionError _____________ TestSpecialisedExplanations.test_eq_longer_list ______________ self = @@ -224,7 +224,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E Right contains one more item: 3 E Use -v to get more diff - failure_demo.py:77: AssertionError + failure_demo.py:79: AssertionError _________________ TestSpecialisedExplanations.test_in_list _________________ self = @@ -233,7 +233,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > assert 1 in [0, 2, 3, 4, 5] E assert 1 in [0, 2, 3, 4, 5] - failure_demo.py:80: AssertionError + failure_demo.py:82: AssertionError __________ TestSpecialisedExplanations.test_not_in_text_multiline __________ self = @@ -252,7 +252,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E and a E tail - failure_demo.py:84: AssertionError + failure_demo.py:86: AssertionError ___________ TestSpecialisedExplanations.test_not_in_text_single ____________ self = @@ -266,7 +266,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E single foo line E ? +++ - failure_demo.py:88: AssertionError + failure_demo.py:90: AssertionError _________ TestSpecialisedExplanations.test_not_in_text_single_long _________ self = @@ -280,7 +280,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E head head foo tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail E ? +++ - failure_demo.py:92: AssertionError + failure_demo.py:94: AssertionError ______ TestSpecialisedExplanations.test_not_in_text_single_long_term _______ self = @@ -294,7 +294,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E head head fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffftail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail E ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - failure_demo.py:96: AssertionError + failure_demo.py:98: AssertionError ______________ TestSpecialisedExplanations.test_eq_dataclass _______________ self = @@ -321,7 +321,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E - c E + b - failure_demo.py:108: AssertionError + failure_demo.py:110: AssertionError ________________ TestSpecialisedExplanations.test_eq_attrs _________________ self = @@ -348,7 +348,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E - c E + b - failure_demo.py:120: AssertionError + failure_demo.py:122: AssertionError ______________________________ test_attribute ______________________________ def test_attribute(): @@ -360,7 +360,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert 1 == 2 E + where 1 = .Foo object at 0xdeadbeef0018>.b - failure_demo.py:128: AssertionError + failure_demo.py:130: AssertionError _________________________ test_attribute_instance __________________________ def test_attribute_instance(): @@ -372,7 +372,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + where 1 = .Foo object at 0xdeadbeef0019>.b E + where .Foo object at 0xdeadbeef0019> = .Foo'>() - failure_demo.py:135: AssertionError + failure_demo.py:137: AssertionError __________________________ test_attribute_failure __________________________ def test_attribute_failure(): @@ -385,7 +385,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: i = Foo() > assert i.b == 2 - failure_demo.py:146: + failure_demo.py:148: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = .Foo object at 0xdeadbeef001a> @@ -394,7 +394,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > raise Exception("Failed to get attrib") E Exception: Failed to get attrib - failure_demo.py:141: Exception + failure_demo.py:143: Exception _________________________ test_attribute_multiple __________________________ def test_attribute_multiple(): @@ -411,7 +411,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + and 2 = .Bar object at 0xdeadbeef001c>.b E + where .Bar object at 0xdeadbeef001c> = .Bar'>() - failure_demo.py:156: AssertionError + failure_demo.py:158: AssertionError __________________________ TestRaises.test_raises __________________________ self = @@ -421,7 +421,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > raises(TypeError, int, s) E ValueError: invalid literal for int() with base 10: 'qwe' - failure_demo.py:166: ValueError + failure_demo.py:168: ValueError ______________________ TestRaises.test_raises_doesnt _______________________ self = @@ -430,7 +430,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > raises(OSError, int, "3") E Failed: DID NOT RAISE - failure_demo.py:169: Failed + failure_demo.py:171: Failed __________________________ TestRaises.test_raise ___________________________ self = @@ -439,7 +439,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > raise ValueError("demo error") E ValueError: demo error - failure_demo.py:172: ValueError + failure_demo.py:174: ValueError ________________________ TestRaises.test_tupleerror ________________________ self = @@ -448,7 +448,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > a, b = [1] # noqa: F841 E ValueError: not enough values to unpack (expected 2, got 1) - failure_demo.py:175: ValueError + failure_demo.py:177: ValueError ______ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ______ self = @@ -459,7 +459,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > a, b = items.pop() E TypeError: cannot unpack non-iterable int object - failure_demo.py:180: TypeError + failure_demo.py:182: TypeError --------------------------- Captured stdout call --------------------------- items is [1, 2, 3] ________________________ TestRaises.test_some_error ________________________ @@ -470,7 +470,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > if namenotexi: # noqa: F821 E NameError: name 'namenotexi' is not defined - failure_demo.py:183: NameError + failure_demo.py:185: NameError ____________________ test_dynamic_compile_shows_nicely _____________________ def test_dynamic_compile_shows_nicely(): @@ -486,7 +486,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: sys.modules[name] = module > module.foo() - failure_demo.py:202: + failure_demo.py:204: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > ??? @@ -506,9 +506,9 @@ Here is a nice run of several failures and how ``pytest`` presents things: > somefunc(f(), g()) - failure_demo.py:213: + failure_demo.py:215: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - failure_demo.py:10: in somefunc + failure_demo.py:12: in somefunc otherfunc(x, y) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ @@ -518,7 +518,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > assert a == b E assert 44 == 43 - failure_demo.py:6: AssertionError + failure_demo.py:8: AssertionError ___________________ TestMoreErrors.test_z1_unpack_error ____________________ self = @@ -528,7 +528,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > a, b = items E ValueError: not enough values to unpack (expected 2, got 0) - failure_demo.py:217: ValueError + failure_demo.py:219: ValueError ____________________ TestMoreErrors.test_z2_type_error _____________________ self = @@ -538,7 +538,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > a, b = items E TypeError: cannot unpack non-iterable int object - failure_demo.py:221: TypeError + failure_demo.py:223: TypeError ______________________ TestMoreErrors.test_startswith ______________________ self = @@ -551,7 +551,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + where False = ('456') E + where = '123'.startswith - failure_demo.py:226: AssertionError + failure_demo.py:228: AssertionError __________________ TestMoreErrors.test_startswith_nested ___________________ self = @@ -570,7 +570,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + where '123' = .f at 0xdeadbeef0029>() E + and '456' = .g at 0xdeadbeef002a>() - failure_demo.py:235: AssertionError + failure_demo.py:237: AssertionError _____________________ TestMoreErrors.test_global_func ______________________ self = @@ -581,7 +581,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E + where False = isinstance(43, float) E + where 43 = globf(42) - failure_demo.py:238: AssertionError + failure_demo.py:240: AssertionError _______________________ TestMoreErrors.test_instance _______________________ self = @@ -592,7 +592,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert 42 != 42 E + where 42 = .x - failure_demo.py:242: AssertionError + failure_demo.py:244: AssertionError _______________________ TestMoreErrors.test_compare ________________________ self = @@ -602,7 +602,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert 11 < 5 E + where 11 = globf(10) - failure_demo.py:245: AssertionError + failure_demo.py:247: AssertionError _____________________ TestMoreErrors.test_try_finally ______________________ self = @@ -613,7 +613,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: > assert x == 0 E assert 1 == 0 - failure_demo.py:250: AssertionError + failure_demo.py:252: AssertionError ___________________ TestCustomAssertMsg.test_single_line ___________________ self = @@ -628,7 +628,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert 1 == 2 E + where 1 = .A'>.a - failure_demo.py:261: AssertionError + failure_demo.py:263: AssertionError ____________________ TestCustomAssertMsg.test_multiline ____________________ self = @@ -647,7 +647,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert 1 == 2 E + where 1 = .A'>.a - failure_demo.py:268: AssertionError + failure_demo.py:270: AssertionError ___________________ TestCustomAssertMsg.test_custom_repr ___________________ self = @@ -669,7 +669,7 @@ Here is a nice run of several failures and how ``pytest`` presents things: E assert 1 == 2 E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a - failure_demo.py:281: AssertionError + failure_demo.py:283: AssertionError ========================= short test summary info ========================== FAILED failure_demo.py::test_generative[3-6] - assert (3 * 2) < 6 FAILED failure_demo.py::TestFailing::test_simple - assert 42 == 43 diff --git a/doc/en/example/simple.rst b/doc/en/example/simple.rst index d4ace3f0413..a5e2e78c397 100644 --- a/doc/en/example/simple.rst +++ b/doc/en/example/simple.rst @@ -645,31 +645,6 @@ If we run this: E assert 0 test_step.py:11: AssertionError - ================================ XFAILURES ================================= - ______________________ TestUserHandling.test_deletion ______________________ - - item = - - def pytest_runtest_setup(item): - if "incremental" in item.keywords: - # retrieve the class name of the test - cls_name = str(item.cls) - # check if a previous test has failed for this class - if cls_name in _test_failed_incremental: - # retrieve the index of the test (if parametrize is used in combination with incremental) - parametrize_index = ( - tuple(item.callspec.indices.values()) - if hasattr(item, "callspec") - else () - ) - # retrieve the name of the first test function to fail for this class name and index - test_name = _test_failed_incremental[cls_name].get(parametrize_index, None) - # if name found, test has failed for the combination of class name & test name - if test_name is not None: - > pytest.xfail(f"previous test failed ({test_name})") - E _pytest.outcomes.XFailed: previous test failed (test_modification) - - conftest.py:47: XFailed ========================= short test summary info ========================== XFAIL test_step.py::TestUserHandling::test_deletion - reason: previous test failed (test_modification) ================== 1 failed, 2 passed, 1 xfailed in 0.12s ================== diff --git a/doc/en/getting-started.rst b/doc/en/getting-started.rst index 85bee729ba1..c8efdeaf745 100644 --- a/doc/en/getting-started.rst +++ b/doc/en/getting-started.rst @@ -22,7 +22,7 @@ Install ``pytest`` .. code-block:: bash $ pytest --version - pytest 8.2.2 + pytest 8.3.0 .. _`simpletest`: diff --git a/doc/en/how-to/fixtures.rst b/doc/en/how-to/fixtures.rst index ecd297867c5..2dc37f7c061 100644 --- a/doc/en/how-to/fixtures.rst +++ b/doc/en/how-to/fixtures.rst @@ -1418,7 +1418,7 @@ Running the above tests results in the following test IDs being used: rootdir: /home/sweet/project collected 12 items - + diff --git a/doc/en/how-to/output.rst b/doc/en/how-to/output.rst index 4994ad1af69..8b15f95f0fd 100644 --- a/doc/en/how-to/output.rst +++ b/doc/en/how-to/output.rst @@ -444,14 +444,6 @@ Example: E assert 0 test_example.py:14: AssertionError - ================================ XFAILURES ================================= - ________________________________ test_xfail ________________________________ - - def test_xfail(): - > pytest.xfail("xfailing this test") - E _pytest.outcomes.XFailed: xfailing this test - - test_example.py:26: XFailed ================================= XPASSES ================================== ========================= short test summary info ========================== SKIPPED [1] test_example.py:22: skipping this test diff --git a/doc/en/reference/reference.rst b/doc/en/reference/reference.rst index d1222728e13..099c8a00260 100644 --- a/doc/en/reference/reference.rst +++ b/doc/en/reference/reference.rst @@ -1939,18 +1939,19 @@ All the command-line flags can be obtained by running ``pytest --help``:: general: -k EXPRESSION Only run tests which match the given substring expression. An expression is a Python evaluable - expression where all names are substring-matched against - test names and their parent classes. Example: -k - 'test_method or test_other' matches all test functions - and classes whose name contains 'test_method' or - 'test_other', while -k 'not test_method' matches those - that don't contain 'test_method' in their names. -k 'not - test_method and not test_other' will eliminate the - matches. Additionally keywords are matched to classes - and functions containing extra names in their - 'extra_keyword_matches' set, as well as functions which - have names assigned directly to them. The matching is - case-insensitive. + expression where all names are substring-matched + against test names and their parent classes. + Example: -k 'test_method or test_other' matches all + test functions and classes whose name contains + 'test_method' or 'test_other', while -k 'not + test_method' matches those that don't contain + 'test_method' in their names. -k 'not test_method + and not test_other' will eliminate the matches. + Additionally keywords are matched to classes and + functions containing extra names in their + 'extra_keyword_matches' set, as well as functions + which have names assigned directly to them. The + matching is case-insensitive. -m MARKEXPR Only run tests matching given mark expression. For example: -m 'mark1 and not mark2'. --markers show markers (builtin, plugin and per-project ones). @@ -1968,28 +1969,28 @@ All the command-line flags can be obtained by running ``pytest --help``:: --trace Immediately break when running each test --capture=method Per-test capturing method: one of fd|sys|no|tee-sys -s Shortcut for --capture=no - --runxfail Report the results of xfail tests as if they were not - marked - --lf, --last-failed Rerun only the tests that failed at the last run (or all - if none failed) - --ff, --failed-first Run all tests, but run the last failures first. This may - re-order tests and thus lead to repeated fixture + --runxfail Report the results of xfail tests as if they were + not marked + --lf, --last-failed Rerun only the tests that failed at the last run (or + all if none failed) + --ff, --failed-first Run all tests, but run the last failures first. This + may re-order tests and thus lead to repeated fixture setup/teardown. --nf, --new-first Run tests from new files first, then the rest of the tests sorted by file mtime --cache-show=[CACHESHOW] - Show cache contents, don't perform collection or tests. - Optional argument: glob (default: '*'). + Show cache contents, don't perform collection or + tests. Optional argument: glob (default: '*'). --cache-clear Remove all cache contents at start of test run --lfnf={all,none}, --last-failed-no-failures={all,none} - With ``--lf``, determines whether to execute tests when - there are no previously (known) failures or when no - cached ``lastfailed`` data was found. ``all`` (the - default) runs the full test suite again. ``none`` just - emits a message about no known failures and exits - successfully. - --sw, --stepwise Exit on test failure and continue from last failing test - next time + With ``--lf``, determines whether to execute tests + when there are no previously (known) failures or + when no cached ``lastfailed`` data was found. + ``all`` (the default) runs the full test suite + again. ``none`` just emits a message about no known + failures and exits successfully. + --sw, --stepwise Exit on test failure and continue from last failing + test next time --sw-skip, --stepwise-skip Ignore the first failing test but stop on the next failing test. Implicitly enables --stepwise. @@ -2001,20 +2002,22 @@ All the command-line flags can be obtained by running ``pytest --help``:: -v, --verbose Increase verbosity --no-header Disable header --no-summary Disable summary + --no-fold-skipped Do not fold skipped tests in short summary. -q, --quiet Decrease verbosity --verbosity=VERBOSE Set verbosity. Default: 0. -r chars Show extra test summary info as specified by chars: (f)ailed, (E)rror, (s)kipped, (x)failed, (X)passed, (p)assed, (P)assed with output, (a)ll except passed - (p/P), or (A)ll. (w)arnings are enabled by default (see - --disable-warnings), 'N' can be used to reset the list. - (default: 'fE'). + (p/P), or (A)ll. (w)arnings are enabled by default + (see --disable-warnings), 'N' can be used to reset + the list. (default: 'fE'). --disable-warnings, --disable-pytest-warnings Disable warnings summary -l, --showlocals Show locals in tracebacks (disabled by default) - --no-showlocals Hide locals in tracebacks (negate --showlocals passed - through addopts) - --tb=style Traceback print mode (auto/long/short/line/native/no) + --no-showlocals Hide locals in tracebacks (negate --showlocals + passed through addopts) + --tb=style Traceback print mode + (auto/long/short/line/native/no) --xfail-tb Show tracebacks for xfail (as long as --tb != no) --show-capture={no,stdout,stderr,log,all} Controls how captured stdout/stderr/log is shown on @@ -2022,37 +2025,39 @@ All the command-line flags can be obtained by running ``pytest --help``:: --full-trace Don't cut any tracebacks (default is to cut) --color=color Color terminal output (yes/no/auto) --code-highlight={yes,no} - Whether code should be highlighted (only if --color is - also enabled). Default: yes. + Whether code should be highlighted (only if --color + is also enabled). Default: yes. --pastebin=mode Send failed|all info to bpaste.net pastebin service --junit-xml=path Create junit-xml style report file at given path --junit-prefix=str Prepend prefix to classnames in junit-xml output pytest-warnings: -W PYTHONWARNINGS, --pythonwarnings=PYTHONWARNINGS - Set which warnings to report, see -W option of Python - itself + Set which warnings to report, see -W option of + Python itself --maxfail=num Exit after first num failures or errors --strict-config Any warnings encountered while parsing the `pytest` section of the configuration file raise errors - --strict-markers Markers not registered in the `markers` section of the - configuration file raise errors + --strict-markers Markers not registered in the `markers` section of + the configuration file raise errors --strict (Deprecated) alias to --strict-markers -c FILE, --config-file=FILE Load configuration from `FILE` instead of trying to locate one of the implicit configuration files. --continue-on-collection-errors Force test execution even if collection errors occur - --rootdir=ROOTDIR Define root directory for tests. Can be relative path: - 'root_dir', './root_dir', 'root_dir/another_dir/'; - absolute path: '/home/user/root_dir'; path with - variables: '$HOME/root_dir'. + --rootdir=ROOTDIR Define root directory for tests. Can be relative + path: 'root_dir', './root_dir', + 'root_dir/another_dir/'; absolute path: + '/home/user/root_dir'; path with variables: + '$HOME/root_dir'. collection: --collect-only, --co Only collect tests, don't execute them --pyargs Try to interpret all arguments as Python packages --ignore=path Ignore path during collection (multi-allowed) - --ignore-glob=path Ignore path pattern during collection (multi-allowed) + --ignore-glob=path Ignore path pattern during collection (multi- + allowed) --deselect=nodeid_prefix Deselect item (via node id prefix) during collection (multi-allowed) @@ -2062,8 +2067,8 @@ All the command-line flags can be obtained by running ``pytest --help``:: --collect-in-virtualenv Don't ignore tests in a local virtualenv directory --import-mode={prepend,append,importlib} - Prepend/append to sys.path when importing test modules - and conftest files. Default: prepend. + Prepend/append to sys.path when importing test + modules and conftest files. Default: prepend. --doctest-modules Run doctests in all .py modules --doctest-report={none,cdiff,ndiff,udiff,only_first_failure} Choose another output format for diffs on doctest @@ -2076,37 +2081,38 @@ All the command-line flags can be obtained by running ``pytest --help``:: failure test session debugging and configuration: - --basetemp=dir Base temporary directory for this test run. (Warning: - this directory is removed if it exists.) - -V, --version Display pytest version and information about plugins. - When given twice, also display information about - plugins. + --basetemp=dir Base temporary directory for this test run. + (Warning: this directory is removed if it exists.) + -V, --version Display pytest version and information about + plugins. When given twice, also display information + about plugins. -h, --help Show help message and configuration info -p name Early-load given plugin module name or entry point - (multi-allowed). To avoid loading of plugins, use the - `no:` prefix, e.g. `no:doctest`. + (multi-allowed). To avoid loading of plugins, use + the `no:` prefix, e.g. `no:doctest`. --trace-config Trace considerations of conftest.py files --debug=[DEBUG_FILE_NAME] Store internal tracing debug information in this log - file. This file is opened with 'w' and truncated as a - result, care advised. Default: pytestdebug.log. + file. This file is opened with 'w' and truncated as + a result, care advised. Default: pytestdebug.log. -o OVERRIDE_INI, --override-ini=OVERRIDE_INI - Override ini option with "option=value" style, e.g. `-o - xfail_strict=True -o cache_dir=cache`. + Override ini option with "option=value" style, e.g. + `-o xfail_strict=True -o cache_dir=cache`. --assert=MODE Control assertion debugging tools. 'plain' performs no assertion debugging. - 'rewrite' (the default) rewrites assert statements in - test modules on import to provide assert expression - information. + 'rewrite' (the default) rewrites assert statements + in test modules on import to provide assert + expression information. --setup-only Only setup fixtures, do not execute tests --setup-show Show setup of fixtures while executing tests - --setup-plan Show what fixtures and tests would be executed but don't - execute anything + --setup-plan Show what fixtures and tests would be executed but + don't execute anything logging: - --log-level=LEVEL Level of messages to catch/display. Not set by default, - so it depends on the root/parent log handler's effective - level, where it is "WARNING" by default. + --log-level=LEVEL Level of messages to catch/display. Not set by + default, so it depends on the root/parent log + handler's effective level, where it is "WARNING" by + default. --log-format=LOG_FORMAT Log format used by the logging module --log-date-format=LOG_DATE_FORMAT @@ -2130,13 +2136,8 @@ All the command-line flags can be obtained by running ``pytest --help``:: Auto-indent multiline messages passed to the logging module. Accepts true|on, false|off or an integer. --log-disable=LOGGER_DISABLE - Disable a logger by name. Can be passed multiple times. - - Custom options: - --lsof Run FD checks if lsof is available - --runpytest={inprocess,subprocess} - Run pytest sub runs in tests using an 'inprocess' or - 'subprocess' (python -m main) method + Disable a logger by name. Can be passed multiple + times. [pytest] ini-options in the first pytest.ini|tox.ini|setup.cfg|pyproject.toml file found: @@ -2151,33 +2152,37 @@ All the command-line flags can be obtained by running ``pytest --help``:: warnings.filterwarnings. Processed after -W/--pythonwarnings. consider_namespace_packages (bool): - Consider namespace packages when resolving module names - during import - usefixtures (args): List of default fixtures to be used with this project + Consider namespace packages when resolving module + names during import + usefixtures (args): List of default fixtures to be used with this + project python_files (args): Glob-style file patterns for Python test module discovery python_classes (args): - Prefixes or glob names for Python test class discovery + Prefixes or glob names for Python test class + discovery python_functions (args): Prefixes or glob names for Python test function and method discovery disable_test_id_escaping_and_forfeit_all_rights_to_community_support (bool): - Disable string escape non-ASCII characters, might cause - unwanted side effects(use at your own risk) + Disable string escape non-ASCII characters, might + cause unwanted side effects(use at your own risk) console_output_style (string): - Console output: "classic", or with additional progress - information ("progress" (percentage) | "count" | - "progress-even-when-capture-no" (forces progress even - when capture=no) + Console output: "classic", or with additional + progress information ("progress" (percentage) | + "count" | "progress-even-when-capture-no" (forces + progress even when capture=no) verbosity_test_cases (string): Specify a verbosity level for test case execution, - overriding the main level. Higher levels will provide - more detailed information about each test case executed. - xfail_strict (bool): Default for the strict parameter of xfail markers when - not given explicitly (default: False) + overriding the main level. Higher levels will + provide more detailed information about each test + case executed. + xfail_strict (bool): Default for the strict parameter of xfail markers + when not given explicitly (default: False) tmp_path_retention_count (string): How many sessions should we keep the `tmp_path` - directories, according to `tmp_path_retention_policy`. + directories, according to + `tmp_path_retention_policy`. tmp_path_retention_policy (string): Controls which directories created by the `tmp_path` fixture are kept around, based on test outcome. @@ -2186,9 +2191,9 @@ All the command-line flags can be obtained by running ``pytest --help``:: Enables the pytest_assertion_pass hook. Make sure to delete any previously generated pyc cache files. verbosity_assertions (string): - Specify a verbosity level for assertions, overriding the - main level. Higher levels will provide more detailed - explanation when an assertion fails. + Specify a verbosity level for assertions, overriding + the main level. Higher levels will provide more + detailed explanation when an assertion fails. junit_suite_name (string): Test suite name for JUnit report junit_logging (string): @@ -2210,8 +2215,8 @@ All the command-line flags can be obtained by running ``pytest --help``:: log_format (string): Default value for --log-format log_date_format (string): Default value for --log-date-format - log_cli (bool): Enable log display during test run (also known as "live - logging") + log_cli (bool): Enable log display during test run (also known as + "live logging") log_cli_level (string): Default value for --log-cli-level log_cli_format (string): @@ -2231,18 +2236,16 @@ All the command-line flags can be obtained by running ``pytest --help``:: Default value for --log-auto-indent pythonpath (paths): Add paths to sys.path faulthandler_timeout (string): - Dump the traceback of all threads if a test takes more - than TIMEOUT seconds to finish + Dump the traceback of all threads if a test takes + more than TIMEOUT seconds to finish addopts (args): Extra command line options minversion (string): Minimally required pytest version required_plugins (args): Plugins that must be present for pytest to run - pytester_example_dir (string): - Directory to take the pytester example files from Environment variables: CI When set (regardless of value), pytest knows it is running in a CI process and does not truncate summary info - BUILD_NUMBER equivalent to CI + BUILD_NUMBER Equivalent to CI PYTEST_ADDOPTS Extra command line options PYTEST_PLUGINS Comma-separated plugins to load during startup PYTEST_DISABLE_PLUGIN_AUTOLOAD Set to disable plugin auto-loading