Nowadays, the prepend and append import modes use importlib.import_module() instead of __import__().
There was a phrase “which avoids having to use `__import__`”, in which I couldn’t just replace `__import__` by `importlib.import_module` because the latter is used (in insert_missing_modules()) also when using importlib mode. Therefore I removed the part from the sentence.
This makes it possible to correlate pytest stages with external events, and also makes it readable when TestReports are exported externall (for example with pytest-reportlog).
Closes#10710
Instead of `SysCapture`/`FDCapture` inheriting from
`SysCaptureBinary`/`FDCaptureBinary`, have both inherit from a common
`SysCaptureBase`/`FDCaptureBase`. This fixes a Liskov substitution
violation.
Since pytest now requires Python>=3.7, we can use the stdlib attrs
clone, dataclasses, instead of the OG package.
attrs is still somewhat nicer than dataclasses and has some extra
functionality, but for pytest usage there's not really a justification
IMO to impose the extra dependency on users when a standard alternative
exists.
Makes `AbstractContextManager` the shared base class between "raises" and other context managers.
The motivation is for type checkers to narrow `pytest.raises(...) if x else nullcontext()` to a `ContextManager` rather than `object`.
Fixes https://github.com/pytest-dev/pytest/issues/10604 which could intermittently display unexpected behavior between checking if the path exists and requesting creation. This was fairly prevalent when pytest was being invoked in parallel by another test runner (CTest) and trying to create the same parent-folder for multiple XMLs. A modest amount of testing did not reproduce other filesystem race conditions.
This notably does not work around an edge case where the parent path of the XML could be created as a file instead of a folder or link. That vanishingly rare case should cause file creation to fail on the next line, with a fairly obvious exception message.
The latest 'packaging' release has dropped support for `LegacyVersion`, meaning
that version strings which do not conform to the standard now raise an error.
Packaging >=22 no longer supports 'latest' versions, which is the version that is assigned when building the docs.
https://github.com/pytest-dev/pytest/pull/10578#issuecomment-1348249045
Also first install pytest, then `docs/en/requirements.txt`:
Installing the requirements last will honor pins of libraries
shared between pytest and the docs build.
Closes#10473
Python <3.11 versions depend on `exceptiongroup>=1.0.0rc8`, and they released version `1.0.1`
6 days ago (2022/11/03) that as a side-effect changed the output of exceptions.
Otherwise we see this warning:
Warning: You are using "pypa/gh-action-pypi-publish@master". The "master" branch of this project has been sunset and will not receive any updates, not even security bug fixes. Please, make sure to use a supported version. If you want to pin to v1 major version, use "pypa/gh-action-pypi-publish@release/v1". If you feel adventurous, you may opt to use use "pypa/gh-action-pypi-publish@unstable/v1" instead. A more general recommendation is to pin to exact tags or commit shas.
Tags from the main repository are required for testing to work properly, but the current documentation does not reflect this, and at least one new contributor has been tripped up by this. Update the documentation with a short explanation and directions.
mypy stubs recently changed warnings.catch_warnings to a Generic, in
order to have proper overloads depending on the parameters passed to it, whihc
triggers this mypy error now when we subclass it:
src/_pytest/recwarn.py:170: error: Missing type parameters for generic type "catch_warnings" [type-arg]
For our porpuses the parameter is not relevant (we always use record=True), so
decided to just ignore the type error.
The builtin actions token is not allowed to trigger further workflows.
So when we create a PR using that token, the PR checks don't run, and a
maintainer must close & open the PR to make them run. This is quite
annoying, so switch back to the "machine user" approach.
Fixes#9554.
* Recommend importlib import mode for new projects
* Recommend src layout more strongly
* Switch to hatchling as the packaging tool in the example (following PyPA)
* Add explanation about the different import modes
It is unnecessary and has been deleted from the setuptools' docs too.
The setuptools' PEP 517 build backend implementation has been
auto-adding the `wheel` dependency since it's first been implemented.
* Squashed commit of the following:
commit 41d339c46763bbe26123e1e6504b6e32290e33e1
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 17:01:04 2022 +0800
test in all py versions
commit b3572a5a12672228c3276fc8c8e05980dfb7888a
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 16:41:06 2022 +0800
add test
commit 7166a2a51e4f99046b028b663c193d8b558c7fd4
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 16:00:07 2022 +0800
update changelog
commit b958c73d489157f0c0d4e46425083a5e2e2bc851
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date: Thu Jun 23 07:50:52 2022 +0000
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
commit ea7f376c6ca37c40c83df0e4a1cfaaedb34bae91
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 15:48:21 2022 +0800
Fix MyPy
commit 97469beb1da40257e9a061a5e19548546c9312c4
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 15:03:48 2022 +0800
fix if ExceptionGroup not exist
commit 84e553642cd69b4d499231d733df91ebfa84c7ad
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date: Thu Jun 23 03:43:27 2022 +0000
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
commit 76bbef449b88bbd74fb5cca3b5293337a624ef03
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 11:40:41 2022 +0800
adding changelog
commit db82bebc5a4969e2083adcd97bdfd2a63bb17d98
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 11:33:10 2022 +0800
fall back to native when handeling to exception groups
* Typed ExceptionGroupTypes and changed to BaseExceptionGroup, fixed exceptionchain (excinfo->excinfo_, set reprcrash. Extended tests, though they're wip.
* added exceptiongroup to pre-commit-config, moved away from tuple to directly defining BaseExceptionGroup, added block comment, added match line for inner exception, changked mark.skipif to importorskip to not need top-level import, changed tox.ini a bit - only uncovered should now be py37 without exceptiongroup, due to hypothesis
* added py311-exceptiongroup to github CI, exceptiongroup is now a hard dependency on py<3.11, renamed bad variable names
* added use_coverage to ubuntu-py311
* import BaseExceptionGroup with explicit version check instead of try/catch
* removed from CI, added comments to tox and pre-commit
While taking a look at the docs I noticed they were not properly sorted.
This just moves the sections around so they are sorted alphabetically, without any other changes.
This should help users with the common issue of patching the wrong place.
Also took the opportunity of using proper links in the monkeypatch introduction.
Related to #10216
Co-authored-by: Ran Benita <ran@unusedvar.com>
The main motivation for this change is to simplify the type shown in
code editors -- `Sequence[str]` is easier to follow than
`Union[list[str], tuple[str, ...]]`.
It also permits using other types if desired. It might lead to problems
if someone uses some oddball sequence type, but hopefully they won't do
that.
New Sphinx added support for overloads and always displays them all with
full type annotations etc. This regresses the API reference for
overloaded functions like `fixture()`, `warns()`, `raises()` and friends
to become impossible to read.
I tried various workarounds but none worked except this one.
New versions of sphinx starting showing `__init__` parameters even when
we don't want them to show because they are private (have `_ispytest`
argument).
The only working solution I found was to switch to
`autodoc_typehints_description_target = "documented"` and explicitly
document parameters for which we want to show the types. It's a little
tedious and repetitive in some simple cases, but overall it results in
nicer API docs.
This started to fail recently with:
```
mypy.....................................................................Failed
- hook id: mypy
- exit code: 1
src/_pytest/assertion/rewrite.py:284: error: Returning Any from function declared to return "TraversableResources" [no-any-return]
Found 1 error in 1 file (checked 219 source files)
```
Not sure why that started failing, but seems like ignoring that error specifically is OK.
Given we are already creating a `PurePath`, just pass the parts directly to it.
This avoids using `os.path.sep`, that although is an official API, seems not to be available in all systems.
Fix#9791
`usage.rst` has been moved from `doc/en` to `doc/en/how-to`, so the `man_pages` configuration
value needs to be updated to the new location, so that we dont get this warning:
writing... WARNING: "man_pages" config value references unknown document usage
`usage.rst` has been moved from `doc/en` to `doc/en/how-to`, so the `man_pages` configuration
value needs to be updated to the new location, so that we dont get this warning:
writing... WARNING: "man_pages" config value references unknown document usage
- Remove the index page https://docs.pytest.org/en/7.1.x/genindex.html
Such an index is reasonable in books but not so much in HTML pages.
IMHO the integrated search is much more user-friendly and sufficient.
- Remove the PDF link from the entry page. The PDF is already linked
in the sidebar. I anticipate that getting the PDF documentation is
not a major concern for users.
- Remove the sections "Changelog" and "License" from the entry page.
These section only contain links to the respective pages.
These topics are purely technical. They should be easily accessible
(which they still are through sidebar section "About the project"),
but they don't need additional advertisement through a dedicated
section on the entry page.
This is especially useful for large repositories (e.g. monorepos) that
use a hierarchical file system organization for nested test paths.
src/*/tests
The implementation uses the standard `glob` module to perform wildcard
expansion in Config.parse().
The related logic that determines whether or not to include 'testpaths'
in the terminal header was previously relying on a weak heuristic: if
Config.args matched 'testpaths', then its value was printed. That
generally worked, but it could also print when the user explicitly used
the same arguments on the command-line as listed in 'testpaths'. Not a
big deal, but it shows that the check was logically incorrect.
Now that 'testpaths' can contain wildcards, it's no longer possible to
perform this simple comparison, so this change also introduces a public
Config.ArgSource enum and Config.args_source attribute that explicitly
names the "source" of the arguments: the command line, the invocation
directory, or the 'testdata' configuration value.
This changes the order to:
- API Reference
- Fixtures reference
- Configuration
- Exit codes
- Plugin List
which is approximately sorted from general to specific and often used to less used.
Plugin List ist at the end because it points to further external resources.
This extensive explanation is not need anymore because they are now
unsupported for a long time.
Instead add as short section on Python version compatibility in the
backward compatibility docs.
It's better to have the documentation in one place, instead
of having some in the docstring and some additional
information added to the reference documentation in
`reference.rst`.
Up to now, some had the prefix, some didn't. I think it's good to have
this prefix to give more context, otherwise the links sometimes blend
into the description.
In one case, the link goes to an example, so I used `**Example**:` there.
Up to now, some had the prefix, some didn't. I think it's good to have
this prefix to give more context, otherwise the links sometimes blend
into the description.
In one case, the link goes to an example, so I used `**Example**:` there.
These pytester utility methods were annotated to only receive `str`
names, but they naturally support os.PathLike values, as well.
This makes writing some pytester calls a little nicer, such as when
creating a directory based on a `.joinpath()` call. We previously needed
to cast that intermediate value to a `str`.
As discussed in https://github.com/pytest-dev/pytest/pull/9800, this uses
monkeypatch to change directories for two tests in `test_collection.py`, to preserve
their original purpose (even if just removing it doesn't seem to affect the outcome).
Getting the current working directory has no side-effects, so these context managers were no-ops.
Discovered because Path.__enter__ is deprecated in Python 3.11, for removal in 3.13.
This will trigger a deploy directly when a tag is pushed, avoiding
having to wait for another lenghty build.
This is safe since we only push release tags after the release PR has passed all its tests.
See comments in #9793.
The left/right operands produced when `verbose > 1` should not contain newlines, because they are used to
build the `summary` string. The `assertrepr_compare` function returns a list of lines, and the summary is one of those lines and should not contain newlines itself.
Fix#9742
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
The change from `path not in confuctdir.parents` to the `relative_to`
check in 0c98f19231 broke picking up
conftest files when running against an installed package/site-packages.
See the issue for more details.
Fix#9767.
Including the file name is enough to let the user know what the problem is.
The same is not needed for `.ini` files because the error message includes the path to the file by default.
Fix#9730
It is tempting to use `monkeypatch` to replace the other mechanisms in pytester which change global
state: `CwdSnapshot`, `SysModulesSnapshot`, `SysPathsSnapshot`, however those are more delicate
than they look at first glance so leaving those alone for now.
Close#9708
The dummy modules we introduce in `insert_missing_modules` (due to #7856 and #7859)
would cause problems if the dummy modules actually end up replacing modules
which could be imported normally because they are available in `PYTHONPATH`.
Now we attempt to first import the module via normal mechanisms, and only
introduce the dummy modules if the intermediary modules don't actually exist.
Close#9645
In the following
@pytest.mark.parametrize(..., ids=[val])
the ID values are only allowed to be `str`, `float`, `int` or `bool`.
In the following
@pytest.mark.parametrize(..., [val])
@pytest.mark.parametrize(..., [pytest.param(..., id=val])
a different code path is used, which also allows `bytes`, `complex`,
`re.Pattern`, `Enum` and anything with a `__name__`.
In the interest of consistency, use the latter code path for all cases.
originally reproduced with this pollution set:
```
testing/test_assertrewrite.py::TestEarlyRewriteBailout::test_pattern_contains_subdirectories
testing/test_assertrewrite.py::TestRewriteOnImport::test_remember_rewritten_modules
```
This fixes#9610.
pytest 7.0.0 (unintentionally) changed `UnitTestFunction.obj`'s' behavior
to match `Function.obj`. That is probably a good thing to have, however
it evidently causes some regressions as described in the issue, so
restore the previous behavior for now. In the future we might want to
make this change again, but with proper consideration.
re: review from @asottile that this should only get imported in the function
modify the else/if logic since inside the function we already know the python version is >= 3.10, and just have to know if it is 3.11 or greater
I still think it makes sense to always serve docs from the release branch (rather than the latest tagged release), but that means we need to update it on new releases.
The docs already assume an 'upstream' remote, so we can only fetch from there instead of fetching all remotes. We also don't need to hardcode the remote URL.
alinsa_vix in Discord noticed that we are not testing Python 3.9 and 3.10 on macOS, which seems strange. Maybe this is due how to macOS CI resources were quite scarce for a while, but I believe this has improved since.
* releasing: Add template for major releases
With pytest 6.0.0, we still used a manual releasing workflow (at least if I
remember correctly), and apparently we never wrote a release announcement
template for major releases. Instead, the minor release template claimed that
the release would contain "breaking changes", which doesn't seem reasonable.
Thus, this adds a new major template based on the former minor template, and
adjusts the latter to only mention fixes and new features instead.
* Update scripts/release.minor.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* Add additional docs for uncooperative ctor deprecation
Fixes#9488
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Break up long line
* Recommend kwonly args
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit only refactors, it does not change or add functionality yet. Public
API is retained. Reason or refactoring:
User provided parameter IDs (e.g. Metafunc.parametrize(ids=...)) had so far
only been used to calculate a unique test ID for each test invocation. That
test ID was a joined string where each parameter contributed some partial ID.
We're soon going to reuse functionality to generate parameter keys for
reorder_items and FixtureDef cache. We will be interested in the partial
IDs, and only if they originate from explicit user information. Refactoring
makes logic and data accessible for reuse, and increases cohesion in general.
The docstring (and function name itself) described things as if IDs are
being assigned to the argnames, but actually they're assigned to the
parameter sets.
If we do the `update`s in the right order, we can avoid the `mark.name
not in self.keywords` check, since `self.keywords` starts out clean and
`update` will override previously set keywords.
By my analysis, this deleted code block has no effect:
1. `self.keywords` is `update`d with `callspec.marks`.
2. `self.own_markers` is `update`d with `callspec.marks`.
3. `self.keywords` is `update`d with `self.own_markers`.
So together steps 2+3 completely undo step 1.
This does have a slight semantic change: in a node hierarchy parent ->
child, if parent has a marker applied, then child is constructed, then
`parent.themarker = "overridden"`, previously
`child.keywords['themarker']` would return `True`, now it returns
`"overridden"`. But that's actually what I would have expected so I see
it as more of a bugfix.
(except `Instance`)
Currently, `Function` does this manually, but other node types don't get
their markers added to their `keywords`, but they should, if only for
consistency.
Marks are added to keywords in three places:
- `Node.add_marker`: name -> `Mark`
- `Function.__init__(callspec)`: name -> `Mark`
- `Function.__init__ iter_markers`: name -> True
I think it should be consistent, which will also help with some upcoming
code cleaning. The `Mark` seems more useful than just a `True`, so
switch to that.
* Add docs on pytest.warns(None) deprecation
* Add new section for common warnings use cases
* Fix references for warnings use cases
* Fix reference link
I think this named function makes the code a bit easier to understand.
Also change the check to explicitly check for "is a sub-path of" instead
of the previous check which only worked assuming that path is within
confcutdir or a direct parent of it.
This made the cache not work as intended, causing a major slowdown.
See #9478 for discussion and context.
Authored-by: Anthony Sottile <asottile@umich.edu>
Before 7.0.0rc1, the function accepted `Union[str, py.path.local]`, and
`py.path.local` compares equal to the string path, so a user was able to
pass the path as a string and it would work. In 7.0.0rc1 we changed the
`py.path.local` to `Path` which doesn't compare equal to the string
path, which breaks compatibility (e.g. the `sybil` package).
This restores compatibility for this function by accepting any
`os.PathLike[str]` and only comparing the string representations.
Since commit 89f0b5b5a2 cases as in the
added test started to fail, like they do for the standard pytest names
(`setup_module` etc). But the name `setup` in particular is way too
common for us to start taking it over more aggressively, so restore the
previous behavior which required the object to be callable.
Fix#9391.
To backport a PR, e.g. 1000, to another branch, e.g. `7.0.x`, add a
label `backport 7.0.x` to the PR. This will trigger a workflow which
will create a branch `backport-1000-to-7.0.x` based on the `7.0.x`
branch with a cherry-pick of the PR's merge commit, and create a new PR
for it against the `7.0.x` branch.
It is very simplistic, for instance it doesn't handle cherry-pick
failure gracefully, doesn't validate the state of the PR, doesn't check
if the branch already exists, etc. But we can improve on it later as
needed.
Finally, PRs created by github actions do not themselves trigger further
actions, i.e. the PR isn't checked. You need to close & reopen the PR
for the checks to trigger. There are workarounds for this but they are
either less secure or require more setup.
The `pytest_pycollector_makeitem` argument `collector` is currently
annotated with type `PyCollector`. As part of #7469, that would have
required us to expose it in the public API. But really it's an
implementation detail, not something we want to expose. So replace the
annotation with the concrete python collector types that are passed.
Strictly speaking, `pytest_pycollector_makeitem` is called from
`PyCollector.collect()`, so the new type annotation is incorrect if
another type subclasses `PyCollector`. But the set of python collectors
is closed (mapping to language constructs), and the type is private, so
there shouldn't be any other deriving classes, and we can consider it
effectively sealed (unfortunately Python does not provide a way to
express this - yet?).
* scripts: Use release branch for changelog URL
With a prerelease, /stable won't show the correct changelog.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Look through the `GitHub issues for bugs <https://github.com/pytest-dev/pytest/labels/type:%20bug>`_.
See also the `"status: easy" issues <https://github.com/pytest-dev/pytest/labels/status%3A%20easy>`_
that are friendly to new contributors.
:ref:`Talk <contact>` to developers to find out how you can fix specific bugs. To indicate that you are going
to work on a particular issue, add a comment to that effect on the specific issue.
@@ -221,7 +223,7 @@ changes you want to review and merge. Pull requests are stored on
Once you send a pull request, we can discuss its potential modifications and
even add more commits to it later on. There's an excellent tutorial on how Pull
Requests work in the
`GitHub Help Center <https://help.github.com/articles/using-pull-requests/>`_.
`GitHub Help Center <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>`_.
Here is a simple overview, with pytest-specific bits:
@@ -242,6 +244,11 @@ Here is a simple overview, with pytest-specific bits:
be released in micro releases whereas features will be released in
minor releases and incompatible changes in major releases.
You will need the tags to test locally, so be sure you have the tags from the main repository. If you suspect you don't, set the main repository as upstream and fetch the tags::
#. For major and minor releases, change the default version in the `Read the Docs Settings <https://readthedocs.org/dashboard/pytest/advanced/>`_ to the new branch.
#. Send an email announcement with the contents from::
Test methods decorated with ``@classmethod`` can now be discovered as tests, following the same rules as normal methods. This fills the gap that static methods were discoverable as tests but not class methods.
Added :confval:`tmp_path_retention_count` and :confval:`tmp_path_retention_policy` configuration options to control how directories created by the :fixture:`tmp_path` fixture are kept.
Features currently deprecated and removed in previous releases can be found in :ref:`deprecations`.
We track future deprecation and removal of features using milestones and the `deprecation <https://github.com/pytest-dev/pytest/issues?q=label%3A%22type%3A+deprecation%22>`_ and `removal <https://github.com/pytest-dev/pytest/labels/type%3A%20removal>`_ labels on GitHub.
Python version support
======================
Released pytest versions support all Python versions that are actively maintained at the time of the release:
@@ -28,6 +28,401 @@ with advance notice in the **Deprecations** section of releases.
.. towncrier release notes start
pytest 7.2.1 (2023-01-13)
=========================
Bug Fixes
---------
-`#10452 <https://github.com/pytest-dev/pytest/issues/10452>`_: Fix 'importlib.abc.TraversableResources' deprecation warning in Python 3.12.
-`#10457 <https://github.com/pytest-dev/pytest/issues/10457>`_: If a test is skipped from inside a fixture, the test summary now shows the test location instead of the fixture location.
-`#10506 <https://github.com/pytest-dev/pytest/issues/10506>`_: Fix bug where sometimes pytest would use the file system root directory as :ref:`rootdir <rootdir>` on Windows.
-`#10607 <https://github.com/pytest-dev/pytest/issues/10607>`_: Fix a race condition when creating junitxml reports, which could occur when multiple instances of pytest execute in parallel.
-`#10641 <https://github.com/pytest-dev/pytest/issues/10641>`_: Fix a race condition when creating or updating the stepwise plugin's cache, which could occur when multiple xdist worker nodes try to simultaneously update the stepwise plugin's cache.
pytest 7.2.0 (2022-10-23)
=========================
Deprecations
------------
-`#10012 <https://github.com/pytest-dev/pytest/issues/10012>`_: Update :class:`pytest.PytestUnhandledCoroutineWarning` to a deprecation; it will raise an error in pytest 8.
-`#10396 <https://github.com/pytest-dev/pytest/issues/10396>`_: pytest no longer depends on the ``py`` library. ``pytest`` provides a vendored copy of ``py.error`` and ``py.path`` modules but will use the ``py`` library if it is installed. If you need other ``py.*`` modules, continue to install the deprecated ``py`` library separately, otherwise it can usually be removed as a dependency.
-`#4562 <https://github.com/pytest-dev/pytest/issues/4562>`_: Deprecate configuring hook specs/impls using attributes/marks.
Instead use :py:func:`pytest.hookimpl` and :py:func:`pytest.hookspec`.
For more details, see the :ref:`docs <legacy-path-hooks-deprecated>`.
-`#9886 <https://github.com/pytest-dev/pytest/issues/9886>`_: The functionality for running tests written for ``nose`` has been officially deprecated.
This includes:
* Plain ``setup`` and ``teardown`` functions and methods: this might catch users by surprise, as ``setup()`` and ``teardown()`` are not pytest idioms, but part of the ``nose`` support.
* Setup/teardown using the `@with_setup <with-setup-nose>`_ decorator.
For more details, consult the :ref:`deprecation docs <nose-deprecation>`.
-`#7337 <https://github.com/pytest-dev/pytest/issues/7337>`_: A deprecation warning is now emitted if a test function returns something other than `None`. This prevents a common mistake among beginners that expect that returning a `bool` (for example `return foo(a, b) == result`) would cause a test to pass or fail, instead of using `assert`. The plan is to make returning non-`None` from tests an error in the future.
Features
--------
-`#9897 <https://github.com/pytest-dev/pytest/issues/9897>`_: Added shell-style wildcard support to ``testpaths``.
Improvements
------------
-`#10218 <https://github.com/pytest-dev/pytest/issues/10218>`_: ``@pytest.mark.parametrize()`` (and similar functions) now accepts any ``Sequence[str]`` for the argument names,
instead of just ``list[str]`` and ``tuple[str, ...]``.
(Note that ``str``, which is itself a ``Sequence[str]``, is still treated as a
comma-delimited name list, as before).
-`#10381 <https://github.com/pytest-dev/pytest/issues/10381>`_: The ``--no-showlocals`` flag has been added. This can be passed directly to tests to override ``--showlocals`` declared through ``addopts``.
-`#3426 <https://github.com/pytest-dev/pytest/issues/3426>`_: Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expressed instead.
-`#8508 <https://github.com/pytest-dev/pytest/issues/8508>`_: Introduce multiline display for warning matching via :py:func:`pytest.warns` and
enhance match comparison for :py:func:`_pytest._code.ExceptionInfo.match` as returned by :py:func:`pytest.raises`.
-`#8646 <https://github.com/pytest-dev/pytest/issues/8646>`_: Improve :py:func:`pytest.raises`. Previously passing an empty tuple would give a confusing
error. We now raise immediately with a more helpful message.
-`#9741 <https://github.com/pytest-dev/pytest/issues/9741>`_: On Python 3.11, use the standard library's :mod:`tomllib` to parse TOML.
:mod:`tomli` is no longer a dependency on Python 3.11.
-`#9742 <https://github.com/pytest-dev/pytest/issues/9742>`_: Display assertion message without escaped newline characters with ``-vv``.
-`#9823 <https://github.com/pytest-dev/pytest/issues/9823>`_: Improved error message that is shown when no collector is found for a given file.
-`#9873 <https://github.com/pytest-dev/pytest/issues/9873>`_: Some coloring has been added to the short test summary.
-`#9883 <https://github.com/pytest-dev/pytest/issues/9883>`_: Normalize the help description of all command-line options.
-`#9920 <https://github.com/pytest-dev/pytest/issues/9920>`_: Display full crash messages in ``short test summary info``, when running in a CI environment.
-`#9987 <https://github.com/pytest-dev/pytest/issues/9987>`_: Added support for hidden configuration file by allowing ``.pytest.ini`` as an alternative to ``pytest.ini``.
Bug Fixes
---------
-`#10150 <https://github.com/pytest-dev/pytest/issues/10150>`_: :data:`sys.stdin` now contains all expected methods of a file-like object when capture is enabled.
-`#10382 <https://github.com/pytest-dev/pytest/issues/10382>`_: Do not break into pdb when ``raise unittest.SkipTest()`` appears top-level in a file.
-`#7792 <https://github.com/pytest-dev/pytest/issues/7792>`_: Marks are now inherited according to the full MRO in test classes. Previously, if a test class inherited from two or more classes, only marks from the first super-class would apply.
When inheriting marks from super-classes, marks from the sub-classes are now ordered before marks from the super-classes, in MRO order. Previously it was the reverse.
When inheriting marks from super-classes, the `pytestmark` attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; use :func:`pytest.Node.iter_markers` instead.
-`#9159 <https://github.com/pytest-dev/pytest/issues/9159>`_: Showing inner exceptions by forcing native display in ``ExceptionGroups`` even when using display options other than ``--tb=native``. A temporary step before full implementation of pytest-native display for inner exceptions in ``ExceptionGroups``.
-`#9877 <https://github.com/pytest-dev/pytest/issues/9877>`_: Ensure ``caplog.get_records(when)`` returns current/correct data after invoking ``caplog.clear()``.
Improved Documentation
----------------------
-`#10344 <https://github.com/pytest-dev/pytest/issues/10344>`_: Update information on writing plugins to use ``pyproject.toml`` instead of ``setup.py``.
-`#9248 <https://github.com/pytest-dev/pytest/issues/9248>`_: The documentation is now built using Sphinx 5.x (up from 3.x previously).
-`#9291 <https://github.com/pytest-dev/pytest/issues/9291>`_: Update documentation on how :func:`pytest.warns` affects :class:`DeprecationWarning`.
Trivial/Internal Changes
------------------------
-`#10313 <https://github.com/pytest-dev/pytest/issues/10313>`_: Made ``_pytest.doctest.DoctestItem`` export ``pytest.DoctestItem`` for
type check and runtime purposes. Made `_pytest.doctest` use internal APIs
to avoid circular imports.
-`#9906 <https://github.com/pytest-dev/pytest/issues/9906>`_: Made ``_pytest.compat`` re-export ``importlib_metadata`` in the eyes of type checkers.
-`#9910 <https://github.com/pytest-dev/pytest/issues/9910>`_: Fix default encoding warning (``EncodingWarning``) in ``cacheprovider``
-`#9984 <https://github.com/pytest-dev/pytest/issues/9984>`_: Improve the error message when we attempt to access a fixture that has been
torn down.
Add an additional sentence to the docstring explaining when it's not a good
idea to call ``getfixturevalue``.
pytest 7.1.3 (2022-08-31)
=========================
Bug Fixes
---------
-`#10060 <https://github.com/pytest-dev/pytest/issues/10060>`_: When running with ``--pdb``, ``TestCase.tearDown`` is no longer called for tests when the *class* has been skipped via ``unittest.skip`` or ``pytest.mark.skip``.
-`#10190 <https://github.com/pytest-dev/pytest/issues/10190>`_: Invalid XML characters in setup or teardown error messages are now properly escaped for JUnit XML reports.
-`#10230 <https://github.com/pytest-dev/pytest/issues/10230>`_: Ignore ``.py`` files created by ``pyproject.toml``-based editable builds introduced in `pip 21.3 <https://pip.pypa.io/en/stable/news/#v21-3>`__.
-`#3396 <https://github.com/pytest-dev/pytest/issues/3396>`_: Doctests now respect the ``--import-mode`` flag.
-`#9514 <https://github.com/pytest-dev/pytest/issues/9514>`_: Type-annotate ``FixtureRequest.param`` as ``Any`` as a stop gap measure until :issue:`8073` is fixed.
-`#9791 <https://github.com/pytest-dev/pytest/issues/9791>`_: Fixed a path handling code in ``rewrite.py`` that seems to work fine, but was incorrect and fails in some systems.
-`#9917 <https://github.com/pytest-dev/pytest/issues/9917>`_: Fixed string representation for :func:`pytest.approx` when used to compare tuples.
Improved Documentation
----------------------
-`#9937 <https://github.com/pytest-dev/pytest/issues/9937>`_: Explicit note that :fixture:`tmpdir` fixture is discouraged in favour of :fixture:`tmp_path`.
Trivial/Internal Changes
------------------------
-`#10114 <https://github.com/pytest-dev/pytest/issues/10114>`_: Replace `atomicwrites <https://github.com/untitaker/python-atomicwrites>`__ dependency on windows with `os.replace`.
pytest 7.1.2 (2022-04-23)
=========================
Bug Fixes
---------
-`#9726 <https://github.com/pytest-dev/pytest/issues/9726>`_: An unnecessary ``numpy`` import inside :func:`pytest.approx` was removed.
-`#9820 <https://github.com/pytest-dev/pytest/issues/9820>`_: Fix comparison of ``dataclasses`` with ``InitVar``.
-`#9869 <https://github.com/pytest-dev/pytest/issues/9869>`_: Increase ``stacklevel`` for the ``NODE_CTOR_FSPATH_ARG`` deprecation to point to the
user's code, not pytest.
-`#9871 <https://github.com/pytest-dev/pytest/issues/9871>`_: Fix a bizarre (and fortunately rare) bug where the `temp_path` fixture could raise
an internal error while attempting to get the current user's username.
pytest 7.1.1 (2022-03-17)
=========================
Bug Fixes
---------
-`#9767 <https://github.com/pytest-dev/pytest/issues/9767>`_: Fixed a regression in pytest 7.1.0 where some conftest.py files outside of the source tree (e.g. in the `site-packages` directory) were not picked up.
pytest 7.1.0 (2022-03-13)
=========================
Breaking Changes
----------------
-`#8838 <https://github.com/pytest-dev/pytest/issues/8838>`_: As per our policy, the following features have been deprecated in the 6.X series and are now
removed:
*``pytest._fillfuncargs`` function.
*``pytest_warning_captured`` hook - use ``pytest_warning_recorded`` instead.
*``-k -foobar`` syntax - use ``-k 'not foobar'`` instead.
*``-k foobar:`` syntax.
*``pytest.collect`` module - import from ``pytest`` directly.
For more information consult
`Deprecations and Removals <https://docs.pytest.org/en/latest/deprecations.html>`__ in the docs.
-`#9437 <https://github.com/pytest-dev/pytest/issues/9437>`_: Dropped support for Python 3.6, which reached `end-of-life <https://devguide.python.org/#status-of-python-branches>`__ at 2021-12-23.
Improvements
------------
-`#5192 <https://github.com/pytest-dev/pytest/issues/5192>`_: Fixed test output for some data types where ``-v`` would show less information.
Also, when showing diffs for sequences, ``-q`` would produce full diffs instead of the expected diff.
-`#9362 <https://github.com/pytest-dev/pytest/issues/9362>`_: pytest now avoids specialized assert formatting when it is detected that the default ``__eq__`` is overridden in ``attrs`` or ``dataclasses``.
-`#9536 <https://github.com/pytest-dev/pytest/issues/9536>`_: When ``-vv`` is given on command line, show skipping and xfail reasons in full instead of truncating them to fit the terminal width.
-`#9644 <https://github.com/pytest-dev/pytest/issues/9644>`_: More information about the location of resources that led Python to raise :class:`ResourceWarning` can now
be obtained by enabling :mod:`tracemalloc`.
See :ref:`resource-warnings` for more information.
-`#9678 <https://github.com/pytest-dev/pytest/issues/9678>`_: More types are now accepted in the ``ids`` argument to ``@pytest.mark.parametrize``.
Previously only `str`, `float`, `int` and `bool` were accepted;
now `bytes`, `complex`, `re.Pattern`, `Enum` and anything with a `__name__` are also accepted.
-`#9692 <https://github.com/pytest-dev/pytest/issues/9692>`_: :func:`pytest.approx` now raises a :class:`TypeError` when given an unordered sequence (such as :class:`set`).
Note that this implies that custom classes which only implement ``__iter__`` and ``__len__`` are no longer supported as they don't guarantee order.
Bug Fixes
---------
-`#8242 <https://github.com/pytest-dev/pytest/issues/8242>`_: The deprecation of raising :class:`unittest.SkipTest` to skip collection of
tests during the pytest collection phase is reverted - this is now a supported
feature again.
-`#9493 <https://github.com/pytest-dev/pytest/issues/9493>`_: Symbolic link components are no longer resolved in conftest paths.
This means that if a conftest appears twice in collection tree, using symlinks, it will be executed twice.
For example, given
tests/real/conftest.py
tests/real/test_it.py
tests/link -> tests/real
running ``pytest tests`` now imports the conftest twice, once as ``tests/real/conftest.py`` and once as ``tests/link/conftest.py``.
This is a fix to match a similar change made to test collection itself in pytest 6.0 (see :pull:`6523` for details).
-`#9626 <https://github.com/pytest-dev/pytest/issues/9626>`_: Fixed count of selected tests on terminal collection summary when there were errors or skipped modules.
If there were errors or skipped modules on collection, pytest would mistakenly subtract those from the selected count.
-`#9645 <https://github.com/pytest-dev/pytest/issues/9645>`_: Fixed regression where ``--import-mode=importlib`` used together with :envvar:`PYTHONPATH` or :confval:`pythonpath` would cause import errors in test suites.
-`#9708 <https://github.com/pytest-dev/pytest/issues/9708>`_: :fixture:`pytester` now requests a :fixture:`monkeypatch` fixture instead of creating one internally. This solves some issues with tests that involve pytest environment variables.
-`#9730 <https://github.com/pytest-dev/pytest/issues/9730>`_: Malformed ``pyproject.toml`` files now produce a clearer error message.
pytest 7.0.1 (2022-02-11)
=========================
Bug Fixes
---------
-`#9608 <https://github.com/pytest-dev/pytest/issues/9608>`_: Fix invalid importing of ``importlib.readers`` in Python 3.9.
-`#9610 <https://github.com/pytest-dev/pytest/issues/9610>`_: Restore `UnitTestFunction.obj` to return unbound rather than bound method.
Fixes a crash during a failed teardown in unittest TestCases with non-default `__init__`.
Regressed in pytest 7.0.0.
-`#9636 <https://github.com/pytest-dev/pytest/issues/9636>`_: The ``pythonpath`` plugin was renamed to ``python_path``. This avoids a conflict with the ``pytest-pythonpath`` plugin.
-`#9642 <https://github.com/pytest-dev/pytest/issues/9642>`_: Fix running tests by id with ``::`` in the parametrize portion.
-`#9643 <https://github.com/pytest-dev/pytest/issues/9643>`_: Delay issuing a :class:`~pytest.PytestWarning` about diamond inheritance involving :class:`~pytest.Item` and
:class:`~pytest.Collector` so it can be filtered using :ref:`standard warning filters <warnings>`.
pytest 7.0.0 (2022-02-03)
=========================
(**Please see the full set of changes for this release also in the 7.0.0rc1 notes below**)
Deprecations
------------
-`#9488 <https://github.com/pytest-dev/pytest/issues/9488>`_: If custom subclasses of nodes like :class:`pytest.Item` override the
``__init__`` method, they should take ``**kwargs``. See
:ref:`uncooperative-constructors-deprecated` for details.
Note that a deprection warning is only emitted when there is a conflict in the
arguments pytest expected to pass. This deprecation was already part of pytest
7.0.0rc1 but wasn't documented.
Bug Fixes
---------
-`#9355 <https://github.com/pytest-dev/pytest/issues/9355>`_: Fixed error message prints function decorators when using assert in Python 3.8 and above.
-`#9396 <https://github.com/pytest-dev/pytest/issues/9396>`_: Ensure :attr:`pytest.Config.inifile` is available during the :func:`pytest_cmdline_main <_pytest.hookspec.pytest_cmdline_main>` hook (regression during ``7.0.0rc1``).
Improved Documentation
----------------------
-`#9404 <https://github.com/pytest-dev/pytest/issues/9404>`_: Added extra documentation on alternatives to common misuses of `pytest.warns(None)` ahead of its deprecation.
-`#9505 <https://github.com/pytest-dev/pytest/issues/9505>`_: Clarify where the configuration files are located. To avoid confusions documentation mentions
that configuration file is located in the root of the repository.
Trivial/Internal Changes
------------------------
-`#9521 <https://github.com/pytest-dev/pytest/issues/9521>`_: Add test coverage to assertion rewrite path.
:func:`unittest.skip` in unittest test cases is fully supported.
..note:: This deprecation has been reverted in pytest 7.1.0.
-`#8315 <https://github.com/pytest-dev/pytest/issues/8315>`_: Several behaviors of :meth:`Parser.addoption <pytest.Parser.addoption>` are now
scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):
@@ -156,7 +553,7 @@ Deprecations
See :ref:`the deprecation note <diamond-inheritance-deprecated>` for full details.
-`#8592 <https://github.com/pytest-dev/pytest/issues/8592>`_: :func:`pytest_cmdline_preparse <_pytest.hookspec.pytest_cmdline_preparse>` has been officially deprecated. It will be removed in a future release. Use :func:`pytest_load_initial_conftests <_pytest.hookspec.pytest_load_initial_conftests>` instead.
-`#8592 <https://github.com/pytest-dev/pytest/issues/8592>`_: :hook:`pytest_cmdline_preparse` has been officially deprecated. It will be removed in a future release. Use :hook:`pytest_load_initial_conftests` instead.
See :ref:`the deprecation note <cmdline-preparse-deprecated>` for full details.
@@ -173,6 +570,12 @@ Deprecations
This was changed for consistency with :func:`pytest.mark.skip <pytest.mark.skip>` and :func:`pytest.mark.xfail <pytest.mark.xfail>` which both accept
``reason`` as an argument.
-`#8174 <https://github.com/pytest-dev/pytest/issues/8174>`_: The following changes have been made to types reachable through :attr:`pytest.ExceptionInfo.traceback`:
- The ``path`` property of ``_pytest.code.Code`` returns ``Path`` instead of ``py.path.local``.
- The ``path`` property of ``_pytest.code.TracebackEntry`` returns ``Path`` instead of ``py.path.local``.
There was no deprecation period for this change (sorry!).
Features
@@ -203,11 +606,11 @@ Features
-``pytest.Mark`` for :class:`marks <pytest.Mark>`.
-``pytest.MarkDecorator`` for :class:`mark decorators <pytest.MarkDecorator>`.
-``pytest.MarkGenerator`` for the :class:`pytest.mark <pytest.MarkGenerator>` singleton.
-``pytest.Metafunc`` for the :class:`metafunc <pytest.MarkGenerator>` argument to the :func:`pytest_generate_tests <pytest.hookspec.pytest_generate_tests>` hook.
-``pytest.Metafunc`` for the :class:`metafunc <pytest.MarkGenerator>` argument to the :hook:`pytest_generate_tests` hook.
-``pytest.CallInfo`` for the :class:`CallInfo <pytest.CallInfo>` type passed to various hooks.
-``pytest.PytestPluginManager`` for :class:`PytestPluginManager <pytest.PytestPluginManager>`.
-``pytest.ExceptionInfo`` for the :class:`ExceptionInfo <pytest.ExceptionInfo>` type returned from :func:`pytest.raises` and passed to various hooks.
-``pytest.Parser`` for the :class:`Parser <pytest.Parser>` type passed to the :func:`pytest_addoption <pytest.hookspec.pytest_addoption>` hook.
-``pytest.Parser`` for the :class:`Parser <pytest.Parser>` type passed to the :hook:`pytest_addoption` hook.
-``pytest.OptionGroup`` for the :class:`OptionGroup <pytest.OptionGroup>` type returned from the :func:`parser.addgroup <pytest.Parser.getgroup>` method.
-``pytest.HookRecorder`` for the :class:`HookRecorder <pytest.HookRecorder>` type returned from :class:`~pytest.Pytester`.
-``pytest.RecordedHookCall`` for the :class:`RecordedHookCall <pytest.HookRecorder>` type returned from :class:`~pytest.HookRecorder`.
@@ -228,11 +631,11 @@ Features
-`#8144 <https://github.com/pytest-dev/pytest/issues/8144>`_: The following hooks now receive an additional ``pathlib.Path`` argument, equivalent to an existing ``py.path.local`` argument:
-:func:`pytest_ignore_collect <_pytest.hookspec.pytest_ignore_collect>` - The ``collection_path`` parameter (equivalent to existing ``path`` parameter).
-:func:`pytest_collect_file <_pytest.hookspec.pytest_collect_file>` - The ``file_path`` parameter (equivalent to existing ``path`` parameter).
-:func:`pytest_pycollect_makemodule <_pytest.hookspec.pytest_pycollect_makemodule>` - The ``module_path`` parameter (equivalent to existing ``path`` parameter).
-:func:`pytest_report_header <_pytest.hookspec.pytest_report_header>` - The ``start_path`` parameter (equivalent to existing ``startdir`` parameter).
-:func:`pytest_report_collectionfinish <_pytest.hookspec.pytest_report_collectionfinish>` - The ``start_path`` parameter (equivalent to existing ``startdir`` parameter).
-:hook:`pytest_ignore_collect` - The ``collection_path`` parameter (equivalent to existing ``path`` parameter).
-:hook:`pytest_collect_file` - The ``file_path`` parameter (equivalent to existing ``path`` parameter).
-:hook:`pytest_pycollect_makemodule` - The ``module_path`` parameter (equivalent to existing ``path`` parameter).
-:hook:`pytest_report_header` - The ``start_path`` parameter (equivalent to existing ``startdir`` parameter).
-:hook:`pytest_report_collectionfinish` - The ``start_path`` parameter (equivalent to existing ``startdir`` parameter).
..note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes (the
@@ -473,10 +876,9 @@ Trivial/Internal Changes
-`#8174 <https://github.com/pytest-dev/pytest/issues/8174>`_: The following changes have been made to internal pytest types/functions:
- The ``path`` property of ``_pytest.code.Code`` returns ``Path`` instead of ``py.path.local``.
- The ``path`` property of ``_pytest.code.TracebackEntry`` returns ``Path`` instead of ``py.path.local``.
- The ``_pytest.code.getfslineno()`` function returns ``Path`` instead of ``py.path.local``.
- The ``_pytest.python.path_matches_patterns()`` function takes ``Path`` instead of ``py.path.local``.
- The ``_pytest._code.Traceback.cut()`` function accepts any ``os.PathLike[str]``, not just ``py.path.local``.
-`#8248 <https://github.com/pytest-dev/pytest/issues/8248>`_: Internal Restructure: let ``python.PyObjMixin`` inherit from ``nodes.Node`` to carry over typing information.
@@ -491,7 +893,7 @@ Trivial/Internal Changes
-`#8913 <https://github.com/pytest-dev/pytest/issues/8913>`_: The private ``CallSpec2._arg2scopenum`` attribute has been removed after an internal refactoring.
-`#8967 <https://github.com/pytest-dev/pytest/issues/8967>`_: :func:`pytest_assertion_pass <_pytest.hookspec.pytest_assertion_pass>` is no longer considered experimental and
-`#8967 <https://github.com/pytest-dev/pytest/issues/8967>`_: :hook:`pytest_assertion_pass` is no longer considered experimental and
future changes to it will be considered more carefully.
@@ -538,7 +940,7 @@ Bug Fixes
the ``tmp_path``/``tmpdir`` fixture). Now the directories are created with
private permissions.
pytest used to silenty use a pre-existing ``/tmp/pytest-of-<username>`` directory,
pytest used to silently use a pre-existing ``/tmp/pytest-of-<username>`` directory,
even if owned by another user. This means another user could pre-create such a
directory and gain control of another user's temporary directory. Now such a
condition results in an error.
@@ -853,8 +1255,8 @@ Deprecations
if you use this and want a replacement.
-:issue:`7255`: The :func:`pytest_warning_captured <_pytest.hookspec.pytest_warning_captured>` hook is deprecated in favor
of :func:`pytest_warning_recorded <_pytest.hookspec.pytest_warning_recorded>`, and will be removed in a future version.
-:issue:`7255`: The :hook:`pytest_warning_captured` hook is deprecated in favor
of :hook:`pytest_warning_recorded`, and will be removed in a future version.
-:issue:`7648`: The ``gethookproxy()`` and ``isinitpath()`` methods of ``FSCollector`` and ``Package`` are deprecated;
@@ -962,7 +1364,7 @@ Trivial/Internal Changes
process, pytest now ignores builtin attributes (like ``__class__``,
``__delattr__`` and ``__new__``) without consulting the :confval:`python_classes` and
:confval:`python_functions` configuration options and without passing them to plugins
using the :func:`pytest_pycollect_makeitem <_pytest.hookspec.pytest_pycollect_makeitem>` hook.
using the :hook:`pytest_pycollect_makeitem` hook.
pytest 6.0.2 (2020-09-04)
@@ -1304,7 +1706,7 @@ Improvements
is not displayed by default for passing tests. This change makes the mistake
visible during testing.
You may supress this behavior temporarily or permanently by setting
You may suppress this behavior temporarily or permanently by setting
``logging.raiseExceptions = False``.
@@ -1751,7 +2153,7 @@ Bug Fixes
-:issue:`6646`: Assertion rewriting hooks are (re)stored for the current item, which fixes them being still used after e.g. pytester's :func:`testdir.runpytest <_pytest.pytester.Testdir.runpytest>` etc.
-:issue:`6660`: :py:func:`pytest.exit` is handled when emitted from the :func:`pytest_sessionfinish <_pytest.hookspec.pytest_sessionfinish>` hook. This includes quitting from a debugger.
-:issue:`6660`: :py:func:`pytest.exit` is handled when emitted from the :hook:`pytest_sessionfinish` hook. This includes quitting from a debugger.
-:issue:`6752`: When :py:func:`pytest.raises` is used as a function (as opposed to a context manager),
@@ -1759,7 +2161,7 @@ Bug Fixes
it was swallowed and ignored (regression in pytest 5.1.0).
-:issue:`6801`: Do not display empty lines inbetween traceback for unexpected exceptions with doctests.
-:issue:`6801`: Do not display empty lines inbetween traceback for unexpected exceptions with doctests.
-:issue:`6802`: The :fixture:`testdir fixture <testdir>` works within doctests now.
@@ -1863,7 +2265,7 @@ Improvements
-:issue:`6231`: Improve check for misspelling of :ref:`pytest.mark.parametrize ref`.
-:issue:`6257`: Handle :py:func:`pytest.exit` being used via :py:func:`~_pytest.hookspec.pytest_internalerror`, e.g. when quitting pdb from post mortem.
-:issue:`6257`: Handle :func:`pytest.exit` being used via :hook:`pytest_internalerror`, e.g. when quitting pdb from post mortem.
@@ -2421,7 +2823,8 @@ Important
This release is a Python3.5+ only release.
For more details, see our :std:doc:`Python 2.7 and 3.4 support plan <py27-py34-deprecation>`.
For more details, see our `Python 2.7 and 3.4 support plan
-:issue:`3457`: New :func:`~_pytest.hookspec.pytest_assertion_pass`
-:issue:`3457`: New :hook:`pytest_assertion_pass`
hook, called with context information when an assertion *passes*.
This hook is still **experimental** so use it with caution.
@@ -2645,7 +3048,11 @@ Features
-:issue:`6870`: New ``Config.invocation_args`` attribute containing the unchanged arguments passed to ``pytest.main()``.
Remark: while this is technically a new feature and according to our:ref:`policy <what goes into 4.6.x releases>` it should not have been backported, we have opened an exception in this particular case because it fixes a serious interaction with ``pytest-xdist``, so it can also be considered a bugfix.
Remark: while this is technically a new feature and according to our
@@ -81,11 +236,11 @@ no matter what argument was used in the constructor. We expect to deprecate the
In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
*:func:`pytest_ignore_collect(collection_path: pathlib.Path) <_pytest.hookspec.pytest_ignore_collect>` as equivalent to ``path``
*:func:`pytest_collect_file(file_path: pathlib.Path) <_pytest.hookspec.pytest_collect_file>` as equivalent to ``path``
*:func:`pytest_pycollect_makemodule(module_path: pathlib.Path) <_pytest.hookspec.pytest_pycollect_makemodule>` as equivalent to ``path``
*:func:`pytest_report_header(start_path: pathlib.Path) <_pytest.hookspec.pytest_report_header>` as equivalent to ``startdir``
*:func:`pytest_report_collectionfinish(start_path: pathlib.Path) <_pytest.hookspec.pytest_report_collectionfinish>` as equivalent to ``startdir``
*:hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
*:hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
*:hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
*:hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
*:hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
@@ -127,7 +282,7 @@ Passing ``msg=`` to ``pytest.skip``, ``pytest.fail`` or ``pytest.exit``
Passing the keyword argument ``msg`` to :func:`pytest.skip`, :func:`pytest.fail` or :func:`pytest.exit`
is now deprecated and ``reason`` should be used instead. This change is to bring consistency between these
functions and the``@pytest.mark.skip`` and ``@pytest.mark.xfail`` markers which already accept a ``reason`` argument.
functions and the``@pytest.mark.skip`` and ``@pytest.mark.xfail`` markers which already accept a ``reason`` argument.
..code-block::python
@@ -157,8 +312,8 @@ Implementing the ``pytest_cmdline_preparse`` hook
..deprecated:: 7.0
Implementing the :func:`pytest_cmdline_preparse <_pytest.hookspec.pytest_cmdline_preparse>` hook has been officially deprecated.
Implement the :func:`pytest_load_initial_conftests <_pytest.hookspec.pytest_load_initial_conftests>` hook instead.
Implementing the :hook:`pytest_cmdline_preparse` hook has been officially deprecated.
Implement the :hook:`pytest_load_initial_conftests` hook instead.
..code-block::python
@@ -191,6 +346,40 @@ Instead, a separate collector node should be used, which collects the item. See
:func:`unittest.skip` in unittest test cases is fully supported.
Using ``pytest.warns(None)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.0
:func:`pytest.warns(None) <pytest.warns>` is now deprecated because many people used
it to mean "this code does not emit warnings", but it actually had the effect of
checking that the code emits at least one warning of any type - like ``pytest.warns()``
:func:`pytest.warns(None) <pytest.warns>` is now deprecated because it was frequently misused.
Its correct usage was checking that the code emits at least one warning of any type - like ``pytest.warns()``
or ``pytest.warns(Warning)``.
See :ref:`warns use cases` for examples.
Returning non-None value in test functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.2
A :class:`pytest.PytestReturnNotNoneWarning` is now emitted if a test function returns something other than `None`.
This prevents a common mistake among beginners that expect that returning a `bool` would cause a test to pass or fail, for example:
..code-block::python
@pytest.mark.parametrize(
["a","b","result"],
[
[1,2,5],
[2,3,8],
[5,3,18],
],
)
deftest_foo(a,b,result):
returnfoo(a,b)==result
Given that pytest ignores the return value, this might be surprising that it will never fail.
The proper fix is to change the `return` to an `assert`:
..code-block::python
@pytest.mark.parametrize(
["a","b","result"],
[
[1,2,5],
[2,3,8],
[5,3,18],
],
)
deftest_foo(a,b,result):
assertfoo(a,b)==result
The ``--strict`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -250,29 +467,42 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Removed Features
----------------
..deprecated:: 6.0
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recored`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest.collect`` module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
The ``pytest.collect`` module is no longer part of the public API, all its names
should now be imported from ``pytest`` directly instead.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recorded`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest._fillfuncargs`` function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
This function was kept for backward compatibility with an older plugin.
@@ -281,12 +511,6 @@ it, use `function._request._fillfixtures()` instead, though note this is not
a public API and may break in the future.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
``--no-print-logs`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -324,8 +548,8 @@ at some point, depending on the plans for the plugins and number of users using
..versionremoved:: 6.0
The ``pytest_collect_directory`` has not worked properly for years (it was called
but the results were ignored). Users may consider using :func:`pytest_collection_modifyitems <_pytest.hookspec.pytest_collection_modifyitems>` instead.
The ``pytest_collect_directory`` hook has not worked properly for years (it was called
but the results were ignored). Users may consider using :hook:`pytest_collection_modifyitems` instead.
TerminalReporter.writer
~~~~~~~~~~~~~~~~~~~~~~~
@@ -828,7 +1052,7 @@ that are then turned into proper test methods. Example:
..code-block::python
defcheck(x,y):
assertx**x==y
assertx**x==y
deftest_squared():
@@ -843,7 +1067,7 @@ This form of test function doesn't support fixtures properly, and users should s
@pytest.mark.parametrize("x, y",[(2,4),(3,9)])
deftest_squared(x,y):
assertx**x==y
assertx**x==y
.._internal classes accessed through node deprecated:
@@ -246,9 +246,9 @@ You can ask which markers exist for your test suite - the list includes our just
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
For an example on how to add and work with markers from a plugin, see
@@ -346,7 +346,7 @@ Custom marker and command line option to control test runs
Plugins can provide custom markers and implement specific behaviour
based on it. This is a self-contained example which adds a command
line option and a parametrized test function marker to run tests
specifies via named environments:
specified via named environments:
..code-block::python
@@ -375,7 +375,7 @@ specifies via named environments:
pytest.skip("test requires env in {!r}".format(envnames))
pytest.skip(f"test requires env in {envnames!r}")
A test file using this local plugin:
@@ -438,9 +438,9 @@ The ``--markers`` option always gives you a list of available markers:
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
.._`passing callables to custom markers`:
@@ -528,7 +528,7 @@ test function. From a conftest file we can read it like this:
Here is an example ``conftest.py`` (extracted from Ali Afshar's special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yaml`` files and will execute the yaml-formatted content as custom tests:
@@ -94,7 +94,7 @@ Mark Lapierre discusses the `Pros and Cons of Quarantined Tests <https://dev.to/
CI tools that rerun on failure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/azure/devops/release-notes/2017/dec-11-vsts#identify-flaky-tests>`_ and rerun failed tests.
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/previous-versions/azure/devops/2017/dec-11-vsts?view=tfs-2017#identify-flaky-tests>`_ and rerun failed tests.
@@ -12,41 +12,26 @@ For development, we recommend you use :mod:`venv` for virtual environments and
as well as the ``pytest`` package itself.
This ensures your code and dependencies are isolated from your system Python installation.
Next, place a ``pyproject.toml`` file in the root of your package:
Create a ``pyproject.toml`` file in the root of your repository as described in
:doc:`packaging:tutorials/packaging-projects`.
The first few lines should look like this:
..code-block::toml
[build-system]
requires=["setuptools>=42","wheel"]
build-backend="setuptools.build_meta"
requires=["hatchling"]
build-backend="hatchling.build"
and a ``setup.cfg`` file containing your package's metadata with the following minimum content:
..code-block::ini
[metadata]
name=PACKAGENAME
[options]
packages=find:
[project]
name="PACKAGENAME"
where ``PACKAGENAME`` is the name of your package.
..note::
If your pip version is older than ``21.3``, you'll also need a ``setup.py`` file:
..code-block::python
fromsetuptoolsimportsetup
setup()
You can then install your package in "editable" mode by running from the same directory:
..code-block::bash
pip install -e .
pip install -e .
which lets you change your source code (both tests and application) and rerun tests at will.
@@ -65,8 +50,8 @@ Conventions for Python test discovery
* In those directories, search for ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_.
* From those files, collect test items:
*``test`` prefixed test functions or methods outside of class
*``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method)
*``test`` prefixed test functions or methods outside of class.
*``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method). Methods decorated with ``@staticmethod`` and ``@classmethods`` are also considered.
For examples of how to customize your test discovery :doc:`/example/pythoncollection`.
@@ -89,11 +74,11 @@ to keep tests separate from actual application code (often a good idea):
..code-block::text
pyproject.toml
setup.cfg
mypkg/
__init__.py
app.py
view.py
src/
mypkg/
__init__.py
app.py
view.py
tests/
test_app.py
test_view.py
@@ -103,84 +88,57 @@ This has the following benefits:
* Your tests can run against an installed version after executing ``pip install .``.
* Your tests can run against the local copy with an editable install after executing ``pip install --editable .``.
* If you don't use an editable install and are relying on the fact that Python by default puts the current
directory in ``sys.path`` to import your package, you can execute ``python -m pytest`` to execute the tests against the
local copy directly, without using ``pip``.
For new projects, we recommend to use ``importlib``:ref:`import mode <import-modes>`
(see which-import-mode_ for a detailed explanation).
To this end, add the following to your ``pyproject.toml``:
..code-block::toml
[tool.pytest.ini_options]
addopts=[
"--import-mode=importlib",
]
.._src-layout:
Generally, but especially if you use the default import mode ``prepend``,
it is **strongly** suggested to use a ``src`` layout.
Here, your application root package resides in a sub-directory of your root,
i.e. ``src/mypkg/`` instead of ``mypkg``.
This layout prevents a lot of common pitfalls and has many benefits,
which are better explained in this excellent `blog post`_ by Ionel Cristian Mărieș.
@@ -16,7 +16,7 @@ import process can be controlled through the ``--import-mode`` command-line flag
these values:
*``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of :py:data:`sys.path` if not already there, and then imported with the :func:`__import__ <__import__>` builtin.
of :py:data:`sys.path` if not already there, and then imported with the :func:`importlib.import_module <importlib.import_module>` function.
This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in :py:data:`sys.modules` after importing.
@@ -24,7 +24,7 @@ these values:
This is the classic mechanism, dating back from the time Python 2 was still supported.
*``append``: the directory containing each module is appended to the end of :py:data:`sys.path` if not already
there, and imported with ``__import__``.
there, and imported with :func:`importlib.import_module <importlib.import_module>`.
This better allows to run test modules against installed versions of a package even if the
package under test has the same import root. For example:
@@ -43,12 +43,21 @@ these values:
Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in :py:data:`sys.modules` after importing.
*``importlib``: new in pytest-6.0, this mode uses :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
*``importlib``: new in pytest-6.0, this mode uses more fine control mechanisms provided by :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
For this reason this doesn't require test module names to be unique, but also makes test
modules non-importable by each other.
For this reason this doesn't require test module names to be unique.
One drawback however is that test modules are non-importable by each other. Also, utility
modules in the tests directories are not automatically importable because the tests directory is no longer
added to :py:data:`sys.path`.
Initially we intended to make ``importlib`` the default in future releases, however it is clear now that
it has its own set of drawbacks so the default will remain ``prepend`` for the foreseeable future.
..seealso::
The :confval:`pythonpath` configuration variable.
We intend to make ``importlib`` the default in future releases, depending on feedback.
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_finalizers.py test_bar
.finalizer_1
finalizer_2
============================ 1 passed in 0.12s =============================
This is so because yield fixtures use `addfinalizer` behind the scenes: when the fixture executes, `addfinalizer` registers a function that resumes the generator, which in turn calls the teardown code.
.._`safe teardowns`:
Safe teardowns
@@ -752,10 +835,10 @@ above):
..code-block::python
# content of test_emaillib.py
importpytest
fromemaillibimportEmail,MailAdminClient
importpytest
@pytest.fixture
defsetup():
@@ -1030,16 +1113,17 @@ read an optional server URL from the test module which uses our fixture:
``pytest`` has basic support for running tests written for nose_.
..warning::
This functionality has been deprecated and is likely to be removed in ``pytest 8.x``.
.._nosestyle:
Usage
@@ -23,8 +26,8 @@ make use of pytest's capabilities.
Supported nose Idioms
----------------------
*setup and teardown at module/class/method level
* SkipTest exceptions and markers
*``setup()`` and ``teardown()`` at module/class/method level: any function or method called ``setup`` will be called during the setup phase for each test, same for ``teardown``.
*``SkipTest`` exceptions and markers
* setup/teardown decorators
*``__test__`` attribute on modules/classes/functions
-`Professional Testing with Python <https://www.python-academy.com/courses/specialtopics/python_course_testing.html>`_, via `Python Academy <https://www.python-academy.com/>`_, February 1st to 3rd, 2022, Leipzig (Germany) and remote.
-`Professional Testing with Python <https://python-academy.com/courses/python_course_testing.html>`_, via `Python Academy <https://www.python-academy.com/>`_, March 7th to 9th 2023 (3 day in-depth training), Remote
Also see `previous talks and blogposts <talks.html>`_.
Also see :doc:`previous talks and blogposts <talks>`.
.._features:
@@ -17,12 +17,10 @@ The ``pytest`` framework makes it easy to write small, readable tests, and can
scale to support complex functional testing for applications and libraries.
**Pythons**: ``pytest`` requires: Python 3.6, 3.7, 3.8, 3.9, or PyPy3.
``pytest`` requires: Python 3.7+ or PyPy3.
**PyPI package name**: :pypi:`pytest`
**Documentation as PDF**: `download latest <https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf>`_
A quick example
---------------
@@ -78,7 +76,7 @@ Features
- Can run :ref:`unittest <unittest>` (including trial) and :ref:`nose <noseintegration>` test suites out of the box
- Python 3.6+ and PyPy 3
- Python 3.7+ or PyPy 3
- Rich plugin architecture, with over 800+ :ref:`external plugins <plugin-list>` and thriving community
@@ -98,11 +96,6 @@ Bugs/Requests
Please use the `GitHub issue tracker <https://github.com/pytest-dev/pytest/issues>`_ to submit bugs or request features.
Changelog
---------
Consult the :ref:`Changelog <changelog>` page for fixes and enhancements of each version.
Support pytest
--------------
@@ -135,13 +128,3 @@ Security
pytest has never been associated with a security vulnerability, but in any case, to report a
security vulnerability please use the `Tidelift security contact <https://tidelift.com/security>`_.
Tidelift will coordinate the fix and disclosure.
License
-------
Copyright Holger Krekel and others, 2004.
Distributed under the terms of the `MIT`_ license, pytest is free and open source software.
Many :ref:`pytest settings <ini options ref>` can be set in a *configuration file*, which
by convention resides on the root of your repository or in your
tests folder.
by convention resides in the root directory of your repository.
A quick example of the configuration files supported by pytest:
@@ -30,9 +29,11 @@ pytest.ini
``pytest.ini`` files take precedence over other files, even when empty.
Alternatively, the hidden version ``.pytest.ini`` can be used.
..code-block::ini
# pytest.ini
# pytest.ini or .pytest.ini
[pytest]
minversion=6.0
addopts = -ra -q
@@ -89,7 +90,7 @@ and can also be used to hold pytest configuration if they have a ``[pytest]`` se
setup.cfg
~~~~~~~~~
``setup.cfg`` files are general purpose configuration files, used originally by :doc:`distutils <distutils/configfile>`, and can also be used to hold pytest configuration
``setup.cfg`` files are general purpose configuration files, used originally by :doc:`distutils <python:distutils/configfile>`, and can also be used to hold pytest configuration
- Training: `pytest - simple, rapid and fun testing with Python <https://www.youtube.com/watch?v=ofPHJrAOaTE>`_, Florian Bruhin, PyConDE 2022
-`pytest: Simple, rapid and fun testing with Python, <https://youtu.be/cSJ-X3TbQ1c?t=15752>`_ (@ 4:22:32), Florian Bruhin, WeAreDevelopers World Congress 2021
- Webinar: `pytest: Test Driven Development für Python (German) <https://bruhin.software/ins-pytest/>`_, Florian Bruhin, via mylearning.ch, 2020
- Webinar: `Simplify Your Tests with Fixtures <https://blog.jetbrains.com/pycharm/2020/08/webinar-recording-simplify-your-tests-with-fixtures-with-oliver-bestwalter/>`_, Oliver Bestwalter, via JetBrains, 2020
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
