* Fix handling empty values of NO_COLOR and FORCE_COLOR
Fix handling NO_COLOR and FORCE_COLOR environment variables to correctly
be ignored when they are set to an empty value, as defined
in the specification:
> Command-line software which adds ANSI color to its output by default
> should check for a NO_COLOR environment variable that, when present
> *and not an empty string* (regardless of its value), prevents
> the addition of ANSI color.
(emphasis mine, https://no-color.org/)
The same is true of FORCE_COLOR, https://force-color.org/.
* Streamline testing for FORCE_COLOR and NO_COLOR
Streamline the tests for FORCE_COLOR and NO_COLOR variables, and cover
all possible cases (unset, set to empty, set to "1"). Combine the two
assert functions into one taking boolean parameters. Mock file.isatty
in all circumstances to ensure that the environment variables take
precedence over the fallback value resulting from isatty check (or that
the fallback is actually used, in the case of both FORCE_COLOR
and NO_COLOR being unset).
* Put a 'reset' color in front of the highlighting
When doing the highlighting, some lexers will not set the initial color
explicitly, which may lead to the red from the errors being propagated
to the start of the expression
* Add syntactic highlighting to the error explanations
This updates the various error reporting to highlight python code when
displayed, to increase readability and make it easier to understand
--- Current main
In current main (before pervious commit), calls to gethookproxy/ihook
are the trigger for loading non-initial conftests. This basically means
that conftest loading is done almost as a random side-effect,
uncontrolled and very non-obvious. And it also dashes any hope of making
gethookproxy faster (gethookproxy shows up prominently in pytest
profiles).
I've wanted to improve this for a while, #11268 was the latest step
towards that.
--- PR before change
In this PR, I ran into a problem.
Previously, Session and Package did all of the directory traversals
inside of their collect, which loaded the conftests as a side effect. If
the conftest loading failed, it will occur inside of the collect() and
cause it to be reported as a failure.
Now I've changed things such that Session.collect and Package.collect no
longer recurse, but just collect their immediate descendants, and
genitems does the recursive expansion work.
The problem though is that genitems() doesn't run inside of specific
collector's collect context. So when it loads a conftest, and the
conftest loading fails, the exception isn't handled by any CollectReport
and causes an internal error instead.
The way I've fixed this problem is by loading the conftests eagerly in a
pytest_collect_directory post-wrapper, but only during genitems to make
sure the directory is actually selected.
This solution in turn caused the conftests to be collected too early;
specifically, the plugins are loaded during the parent's collect(), one
after the other as the directory entries are collected. So when the
ihook is hoisted out of the loop, new plugins are loaded inside the
loop, and due to the way the hook proxy works, they are added to the
ihook even though they're supposed to be scoped to the child collectors.
So no hoisting.
--- PR after change
Now I've come up with a better solution: since now the collection tree
actually reflects the filesystem tree, what we really want is to load
the conftest of a directory right before we run its collect(). A
conftest can affect a directory's collect() (e.g. with a
pytest_ignore_collect hookimpl), but it cannot affect how the directory
node itself is collected. So I just moved the conftest loading to be
done right before calling collect, but still inside the CollectReport
context.
This allows the hoisting, and also removes conftest loading from
gethookproxy since it's no longer necessary. And it will probably enable
further cleanups. So I'm happy with it.
Previously the error report would have all sections glued together:
- The assertion representation
- The error explanation
- The full diff
This makes it hard to see at a glance where which one starts and ends.
One of the representation (dataclasses, tuples, attrs) does display a
newlines at the start already.
Let's add a newlines before the error explanation and before the full
diff, so we get an easier to read report.
This has one disadvantage: we get one line less in the least verbose
mode, where the output gets truncated.
The default for `_prepareconfig` is to use `sys.argv`, which in this
case are the flags passed to (top-level) `pytest`. This is not the
intention, the tests themselves should not be affected by it.
When running `tox -e py-lsof` I get a deluge of this warning:
```
src/pytest/.tox/py-lsof-numpy-pexpect/lib/python3.11/site-packages/_pytest/pytester.py:130: EncodingWarning: UTF-8 Mode affects locale.getpreferredencoding(). Consider locale.getencoding() instead.
```
Use `locale.getencoding` instead.
The normal default pretty printer is not great when objects are nested
and it can get hard to read the diff.
Instead, provide a pretty printer that behaves more like when json get
indented, which allows for smaller, more meaningful differences, at
the expense of a slightly longer diff.
This does not touch the other places where the pretty printer is used,
and only updated the full diff one.
Reset color-related environment variables in a fixture to prevent them
from affecting test results. Otherwise, some of the tests fail
e.g. if NO_COLOR is set in the calling environment.
We already have the AlwaysDispatchingPrettyPrinter override of the
default pretty printer. In order to make more in depth changes, we
need to copy the upstream version in, as it doesn't lend itself well to
being extended.
This does a verbatime copy, adding provenance information at the top.
if we were ever to change the `name:` field in `test.yml` this would suddenly break -- this references the workflow file directly (and is what github generates now for badges)
The tested py.path.local.make_numbered_dir function is *not*
multiprocess safe, because is uses os.listdir which itself is not.
The os.listdir documentation explicitly states that:
> If a file is removed from or added to the directory during the call
> of this function, whether a name for that file be included is unspecified.
This can lead to a race when:
1. process A attempts to create directory N
2. the creation fails, as another process already created it in the meantime
3. process A calls listdir to determine a more recent maxnum
4. processes B+ repeatedly create newer directories and they delete directory N
5. process A doesn't have directory N or any newer directory in listdir result
6. process A attempts to create directory N again and raises
For details, see https://github.com/pytest-dev/pytest/issues/11603#issuecomment-1805708144
and bellow.
Additionally, the test itself has a race in batch_make_numbered_dirs.
When this functions attempts to write to repro-N/foo,
repro-N may have already been removed by another process.
For details, see https://github.com/pytest-dev/pytest/issues/11603#issuecomment-1804714313
and bellow.
---
The tested py.path.local.make_numbered_dir function is not used in pytest.
There is a different implementation in _pytest.pathlib.
Closes#11603
Previously, the sidebar TOC had unlimited depth, making it useless and
interfering with the content. This seems to have regressed in pytest
7.2.x going by the RTD version selector.
Traditionally pytest publishes tags in the form `X.Y.Z`, however the deploy workflow (copied from somewhere else) published tags in the form `vX.Y.Z`.
This is the root cause of #11548, because it tried to publish the release notes for tag `X.Y.Z` (which did not exist).
Fix#11548
Logging has many global states, and we did foresee this by creating a ``cleanup_disabled_logging`` fixture,
however one might still forget to use it and failures leak later -- sometimes not even in the same PR, because the order
of the tests might change in the future, specially when running under xdist.
This problem surfaced during pytest-dev/pytest#11530, where tests unrelated to the change started to fail.
The current method as the following problem, described by Sadra
Barikbin:
The tests that request both `pytester` and `monkeypatch` and use
`monkeypatch.chdir` without context, relying on `monkeypatch`'s teardown
to restore cwd. This doesn't work because the following sequence of
actions take place:
- `monkeypatch` is set up.
- `pytester` is set up. It saves the original cwd and changes it to a
new one dedicated to the test function.
- Test function calls `monkeypatch.chdir()` without context.
`monkeypatch` saves cwd, which is not the original one, before
changing it.
- `pytester` is torn down. It restores the cwd to the original one.
- `monkeypatch` is torn down. It restores cwd to what it has saved.
The solution here is to have pytester use `monkeypatch.chdir()` itself,
then everything is handled correctly.
Tests if a captured exception group contains an expected exception.
Will raise `AssertionError` if the wrapped exception is not an exception group.
Supports recursive search into nested exception groups.
Starting with `resolve_package_path` and its associated tests,
this refactoring seeks to make path concatenation more
readable and consistent within tests/functions.
As discussed in #11413:
- code is free to use either `/` and `joinpath`
- consistency within a function is more important than consistency across the codebase
- it is nice to use `/` when it is more readable
- it is nice to use `joinpath` when there is little context
- be mindful that `joinpath` may be clearer when joining multiple segments
There used to be two callers to `_setup_fixtures()`, now there's only
one, so inline it and make `DoctestItem` more similar to `Function`.
(Eventually we may want to generalize `TopRequest` from taking
`Function` directly to some "fixture-supporting item", removing the
remaining `type: ignore` here and allowing plugins to do it in a stable
manner).
As discussed in #11408:
* Improve documentation for the release process.
* Fix the description for the PRs created by the `prepare release pr` workflow.
* Fix pushing tag in the `deploy` workflow.
For packages, `import_path` receives the path to the package's `__init__.py` file, however module names (as they live in `sys.modules`) should not include the `__init__` part.
For example, `app/core/__init__.py` should be imported as `app.core`, not as `app.core.__init__`.
Fix#11306
I could find 2 plugins that would be broken by this (pytest-play and
pytest-wdl), but they will be better served by just copying
`_fillfixtures` instead of use the private function.
`_pyfuncitem.fixturenames` is just an alias for
`_pyfuncitem._fixtureinfo.names_closure` (at least in core pytest), so
let's do the less abstraction-breaking thing.
Move handling of user_properties to `finalize()`.
Previously if a fixture failed during teardown, `pytest_runtest_logreport` would not be called with "teardown", resulting in the user properties not being saved on the JUnit XML file.
Fixes: #11367
* Build the package only once, and test on all platforms.
* Deploy is now triggered manually via an Action, which is then responsible for tagging the repository after the package has been uploaded successfully.
* Drop 'docs': we nowadays rely on readthedocs preview PR builds.
This reverts commit e938580257.
Revert "improve error msg and test"
This reverts commit c0cf822ca1.
Revert "error msg"
This reverts commit ec1053cc16.
Revert "changelog"
This reverts commit d2dc8a70b5.
Revert "simplify code / take out user-gen typeerror case"
This reverts commit b9cb87d862.
Setting `norecursedirs` overrides the default, so we end up scanning
dot-directories and such which slows down collection unnecessarily
(150ms on my working directory).
To remove fixtures.py::add_funcargs_pseudo_fixture_def and add its logic
i.e. registering funcargs as params and making corresponding fixturedefs,
right to Metafunc.parametrize in which parametrization takes place.
To remove funcargs from metafunc attributes as we populate metafunc
params and make pseudo fixturedefs simultaneously and there's no need to
keep funcargs separately.
Previously, when assigning a scope for a fully-indirect parameter set,
when there are multiple fixturedefs for a param (i.e. same-name fixture
chain), the highest scope was used, but it should be the lowest scope,
since that's the effective scope of the fixture.
What one typically actually wants in such a case is both, checking for some
resulting values *and* checking for some expected exception.
Since this is easily possible with the `nullcontext` context manager, adapt the
example accordingly (needlessly using a different name rather just confuses people).
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Previously, the `_getconftestmodules` function was used both to load
conftest modules for a path (during `pytest_load_initial_conftests`),
and to retrieve conftest modules for a path (during hook dispatch and
for fetching `collect_ignore`). This made things muddy - it is usually
nicer to have clear separation between "command" and "query" functions,
when they occur in separate phases.
So split into "load" and "get".
Currently, `gethookproxy` still loads conftest itself. I hope to change
this in the future.
this change
* switches from html index usage to json index usage
* adds using requests-cache as http cache to trim down runtime
* adds gh action caches for pip and the new http cache
This test makes clear the need for the `_check_scope()` call in the
`pytest_setup_fixture` impl in fixtures.py, which otherwise seems
redundant with the one in `_compute_fixture_value`.
FixtureDef is used in the `pytest_fixture_setup` hook so needs to be
public. However since its current internals are quite dubious (and not
all marked with `_` prefix) I've added an explicit note that only
documented fields/methods are considered public.
Refs #7469.
From understanding the code better I see this is the correct fix.
The fixturedefs can be None if `request.getfixturevalue("doesnotexist")`
is used.
In practice there is no change in behavior because this mapping is used
as `self._arg2fixturedefs.get(argname, None)` which ends up the same.
It makes for a more debuggable repr. Before:
<function FixtureRequest._schedule_finalizers.<locals>.<lambda> at 0x7fe4ae32d440>
After:
functools.partial(<bound method FixtureDef.finish of <FixtureDef argname='hello_package' scope='package' baseid=''>>, request=<SubRequest 'hello_package' for <Function test_hello>>)
Add explicit mentions of the default behavior to both API reference and
how-to section about ways to invoke pytest.
Co-authored-by: Ran Benita <ran@unusedvar.com>
Dict comparsion in the ApproxMapping class did not check if values were None before attempting to subtract for max_abs_diff stat, which was throwing an TypeError instead of being handled by pytest error assertion. Check for None has been added before these calculations, so that None will properly show as Obtained/Expected in pytest assert message
The initial implementation (in #7246) introduced the `importlib` mode, which
never added the imported module to `sys.modules`, so it included a test
to ensure calling `import_path` twice would yield different modules.
Not adding modules to `sys.modules` proved problematic, so we began to add the imported module to `sys.modules`
in #7870, but failed to realize that given we are now changing `sys.modules`, we might
as well avoid importing it more than once.
Then #10088 came along, passing `importlib` also when importing application modules
(as opposed to only test modules before), which caused problems due to imports
having side-effects and the expectation being that they are imported only once.
With this PR, `import_path` returns the module immediately if already in
`sys.modules`.
Fix#10811Fix#10341
Fixes#11104.
See the issue for a description of the problem.
Now, we use the same logic for initial conftest paths as we do for
deciding the initial args, which was the idea behind checking
`namespace.file_or_dir` and `testpaths` previously.
This fixes the issue of `testpaths` being considered for initial
conftests even when it's not used for the args.
(Another issue in faeb16146b was that the
`testpaths` were not glob-expanded, this is also fixed.)
I think it's helpful to separate the node classes from the other
objects, as they have their own unique usage.
I've chosen not to alphabetize the order, but to use a logical order
instead.
Also slightly improve the docstrings.
Currently, if `--confcutdir` is not set, `inipath.parent` is used, and
if `initpath` is not set, then `confcutdir` is None, which means there
is no cutoff.
Having no cutoff is not great, it means we potentially start probing
stuff all the way up to the filesystem root directory. So let's add
another fallback, to `rootpath`, which is always something reasonable.
In #10758 we introduced the support for the use of the walrus operator in the test cases. There was a case which was not handled that caused a bug report #11028. This PR aims to fix the issue and also to improve how the walrus operator is handled in the AssertionRewriter class.
Closes#11028
Added handling of %f directive to print microseconds in log format options, such as log-date-format. It is impossible to do with a standard logging.Formatter because it uses time.strftime which doesn't know about milliseconds and %f. In this PR I added a custom Formatter which converts LogRecord to a datetime.datetime object and formats it with %f flag. This behaviour is enabled only if a microsecond flag is specified in a format string.
Also added a few tests to check the standard and changed behavior.
Closes#10991
`--lf` has a feature where if a certain `Module` (python file) does not
contain any failed tests, it is skipped entirely at the collector level
instead of being collected and each item skipped individually. When this
happens the collection summary looks like this:
run-last-failure: rerun previous 1 failure (skipped 1 file)
However, this feature didn't work for `Module`s inside of `Package`s,
only for those directly beneath the `Session`.
Fix#11054.
The definition of `repr_failure` on the `YamlItem` subclass only handled the custom `YamlException` class, which hides all other errors from the user. By adding in the `super` call we ensure all other exception types also appropriately handled by `repr_failure`.
Forces requested `caplog` logging levels to be enabled if they were disabled via `logging.disable()`
`[attr-defined]` mypy error ignored in `logging.py` because there were existing errors with the imports
and `loggin.Logger.manager` is an attr set at runtime. Since it's in the standard lib I can't really fix that.
Ignored an attr-defined error in `src/_pytest/config/__init__.py` because the re-export is necessary.
Fixes#8711
`_set_initial_conftests` could break on some systems if a very long
option was passed, because the `Path.exists()` call raises an
`OSError` instead of returning `False`.
Fix#10169
This introduces a workflow to automatically close issues with the label
`status: needs information` after a number of days of inactivity.
This work has been done manually for a number of years, but I think it is safe
to close issues with this label automatically.
Not tested yet, but it is in `debug-only` mode so we can watch what it does before
deciding to turn it on (however it needs to be in `main` for it to run).
Fixes a pytest-xdist regression after
762bb61562 (not yet released).
pytest-xdist patches sys.stderr with an object which doesn't have
`encoding`. Strictly speaking, this should be fixed there (or more
precisely, in execnet), but it will drop support for older versions
which don't want.
But in any case, the fix turns out to simplify the code, using FD
support added in Python 3.5, so it's good anyway!
Refs: https://github.com/pytest-dev/pytest-xdist/pull/900
* faulthandler: fix an EncodingWarning
* _py/path: tiny change to `ensure` to silence EncodingWarning
We're not supposed to diverge here, but make this change to fix an
unavoidable EncodingWarning that is otherwise raised in pytest's test
suite. The behavior should be exactly the same besides the warning,
hopefully that won't cause confusion.
Since `Traceback.getcrashentry` takes the `ExceptionInfo`, it is not
really independent of it and is in the wrong layer. Prevent nonsensical
mistakes by inlining it.
TracebackEntry needs the excinfo for the `__tracebackhide__ = callback`
functionality, where `callback` accepts the excinfo.
Currently it achieves this by storing a weakref to the excinfo which
created it. I think this is not great, mixing layers and bloating the
objects.
Instead, have `ishidden` (and transitively, `Traceback.filter()`) take
the excinfo as a parameter.
Closes#10831.
This fixes a small bug where running tests that contained
`pytest.fail(pytrace=False)` with the `--tb=line` flag set results in
an output of "None" in the Failures section of the output, and adds
a test to ensure the behavior is correct.
The problem is that we would loop over all directories of the basetemp directory searching for dead symlinks, for each test, which would compound over the test session run.
Doing the cleanup just once, at the end of the session, fixes the problem.
Fix#10896
By "empty traceback" I mean a traceback all of whose entries have been
filtered/cut/pruned out.
Currently, if an empty traceback needs to be repr'ed, the last entry
before the filtering is used instead (added in
accd962c9f).
Showing a hidden frame is not so good IMO. This commit does the
following instead:
1. Shows details of the exception.
2. Shows a message about how the full trace can be seen.
Example:
```
_____________ test _____________
E ZeroDivisionError: division by zero
All traceback entries are hidden. Pass `--full-trace` to see hidden and internal frames.
```
Also handles `--tb=native`, though there the `--full-trace` bit is not
shown.
This commit contains some pieces from
431ec6d34e (which has been reverted).
Helps towards fixing issue # 1904.
Co-authored-by: Felix Hofstätter <Felhof1@hotmail.com>
The old-style `sys.exc_info()` triplet is redundant nowadays with
`(type(exc), exc, exc.__traceback__)`, and is beginning to get
soft-deprecated in Python 3.12.
Add a nicer API to ExceptionInfo which takes just the exc instead of the
triplet. There are already a few internal uses which benefit.
Fix#10875
Without this, fails with
```
...
E AttributeError: 'TestCaseFunction' object has no attribute 'addDuration'
...
E RuntimeWarning: TestResult has no addDuration method
```
Updated the c fixture to be a little more consistent with other fixtures in the corresponding image. for example both e and g both have edges connected with the fixtures that they explicitly depend on.
Nowadays, the prepend and append import modes use importlib.import_module() instead of __import__().
There was a phrase “which avoids having to use `__import__`”, in which I couldn’t just replace `__import__` by `importlib.import_module` because the latter is used (in insert_missing_modules()) also when using importlib mode. Therefore I removed the part from the sentence.
This makes it possible to correlate pytest stages with external events, and also makes it readable when TestReports are exported externall (for example with pytest-reportlog).
Closes#10710
Instead of `SysCapture`/`FDCapture` inheriting from
`SysCaptureBinary`/`FDCaptureBinary`, have both inherit from a common
`SysCaptureBase`/`FDCaptureBase`. This fixes a Liskov substitution
violation.
Since pytest now requires Python>=3.7, we can use the stdlib attrs
clone, dataclasses, instead of the OG package.
attrs is still somewhat nicer than dataclasses and has some extra
functionality, but for pytest usage there's not really a justification
IMO to impose the extra dependency on users when a standard alternative
exists.
Makes `AbstractContextManager` the shared base class between "raises" and other context managers.
The motivation is for type checkers to narrow `pytest.raises(...) if x else nullcontext()` to a `ContextManager` rather than `object`.
Fixes https://github.com/pytest-dev/pytest/issues/10604 which could intermittently display unexpected behavior between checking if the path exists and requesting creation. This was fairly prevalent when pytest was being invoked in parallel by another test runner (CTest) and trying to create the same parent-folder for multiple XMLs. A modest amount of testing did not reproduce other filesystem race conditions.
This notably does not work around an edge case where the parent path of the XML could be created as a file instead of a folder or link. That vanishingly rare case should cause file creation to fail on the next line, with a fairly obvious exception message.
The latest 'packaging' release has dropped support for `LegacyVersion`, meaning
that version strings which do not conform to the standard now raise an error.
Packaging >=22 no longer supports 'latest' versions, which is the version that is assigned when building the docs.
https://github.com/pytest-dev/pytest/pull/10578#issuecomment-1348249045
Also first install pytest, then `docs/en/requirements.txt`:
Installing the requirements last will honor pins of libraries
shared between pytest and the docs build.
Closes#10473
Python <3.11 versions depend on `exceptiongroup>=1.0.0rc8`, and they released version `1.0.1`
6 days ago (2022/11/03) that as a side-effect changed the output of exceptions.
Otherwise we see this warning:
Warning: You are using "pypa/gh-action-pypi-publish@master". The "master" branch of this project has been sunset and will not receive any updates, not even security bug fixes. Please, make sure to use a supported version. If you want to pin to v1 major version, use "pypa/gh-action-pypi-publish@release/v1". If you feel adventurous, you may opt to use use "pypa/gh-action-pypi-publish@unstable/v1" instead. A more general recommendation is to pin to exact tags or commit shas.
@@ -272,24 +274,24 @@ Here is a simple overview, with pytest-specific bits:
#. Run all the tests
You need to have Python 3.7 available in your system. Now
You need to have Python 3.8 or later available in your system. Now
running tests is as simple as issuing this command::
$ tox -e linting,py37
$ tox -e linting,py39
This command will run tests via the "tox" tool against Python 3.7
This command will run tests via the "tox" tool against Python 3.9
and also perform "lint" coding-style checks.
#. You can now edit your local working copy and run the tests again as necessary. Please follow PEP-8 for naming.
#. You can now edit your local working copy and run the tests again as necessary. Please follow `PEP-8<https://www.python.org/dev/peps/pep-0008/>`_ for naming.
You can pass different options to ``tox``. For example, to run tests on Python 3.7 and pass options to pytest
You can pass different options to ``tox``. For example, to run tests on Python 3.9 and pass options to pytest
(e.g. enter pdb on failure) to pytest you can do::
$ tox -e py37 -- --pdb
$ tox -e py39 -- --pdb
Or to only run tests in a particular test module on Python 3.7::
Or to only run tests in a particular test module on Python 3.9::
$ tox -e py37 -- testing/test_config.py
$ tox -e py39 -- testing/test_config.py
When committing, ``pre-commit`` will re-format the files if necessary.
:class:`~pytest.PytestReturnNotNoneWarning` is now a subclass of :class:`~pytest.PytestRemovedIn8Warning`:the plan is to make returning non-``None`` from tests an error in the future.
pytest no longer depends on the ``py`` library. ``pytest`` provides a vendored copy of ``py.error`` and ``py.path`` modules but will use the ``py`` library if it is installed. If you need other ``py.*`` modules, continue to install the deprecated ``py`` library separately, otherwise it can usually be removed as a dependency.
Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expresed instead.
A warning is now emitted if a test function returns something other than `None`. This prevents a common mistake among beginners that expect that returning a `bool` (for example `return foo(a, b) == result`) would cause a test to pass or fail, instead of using `assert`.
Marks are now inherited according to the full MRO in test classes. Previously, if a test class inherited from two or more classes, only marks from the first super-class would apply.
When inheriting marks from super-classes, marks from the sub-classes are now ordered before marks from the super-classes, in MRO order. Previously it was the reverse.
When inheriting marks from super-classes, the `pytestmark` attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; use :func:`pytest.Node.iter_markers` instead.
Showing inner exceptions by forcing native display in ``ExceptionGroups`` even when using display options other than ``--tb=native``. A temporary step before full implementation of pytest-native display for inner exceptions in ``ExceptionGroups``.
The functionality for running tests written for ``nose`` has been officially deprecated.
This includes:
* Plain ``setup`` and ``teardown`` functions and methods: this might catch users by surprise, as ``setup()`` and ``teardown()`` are not pytest idioms, but part of the ``nose`` support.
* Setup/teardown using the `@with_setup <with-setup-nose>`_ decorator.
For more details, consult the :ref:`deprecation docs <nose-deprecation>`.
@@ -14,7 +14,7 @@ Each file should be named like ``<ISSUE>.<TYPE>.rst``, where
``<ISSUE>`` is an issue number, and ``<TYPE>`` is one of:
*``feature``: new user facing features, like new command-line options and new behavior.
*``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junitxml``, improved colors in terminal, etc).
*``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junit-xml``, improved colors in terminal, etc).
*``bugfix``: fixes a bug.
*``doc``: documentation improvement, like rewording an entire session or adding missing docs.
@@ -22,7 +22,7 @@ b) transitional: the old and new API don't conflict
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationwarning`).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationWarning`).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g. `PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
@@ -87,8 +87,11 @@ Released pytest versions support all Python versions that are actively maintaine
============== ===================
pytest version min. Python version
============== ===================
8.0+ 3.8+
7.1+ 3.7+
6.2 - 7.0 3.6+
5.0 - 6.1 3.5+
3.3 - 4.6 2.7, 3.4+
============== ===================
`Status of Python Versions <https://devguide.python.org/versions/>`__.
@@ -177,7 +177,7 @@ arguments they only pass on to the superclass.
resolved in future versions as we slowly get rid of the :pypi:`py`
dependency (see :issue:`9283` for a longer discussion).
Due to the ongoing migration of methods like :meth:`~_pytest.Item.reportinfo`
Due to the ongoing migration of methods like :meth:`~pytest.Item.reportinfo`
which still is expected to return a ``py.path.local`` object, nodes still have
both ``fspath`` (``py.path.local``) and ``path`` (``pathlib.Path``) attributes,
no matter what argument was used in the constructor. We expect to deprecate the
@@ -336,7 +336,7 @@ Diamond inheritance between :class:`pytest.Collector` and :class:`pytest.Item`
..deprecated:: 7.0
Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
Defining a custom pytest node type which is both an :class:`~pytest.Item` and a :class:`~pytest.Collector` (e.g. :class:`~pytest.File`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
Some plugins providing linting/code analysis have been using this as a hack.
@@ -348,8 +348,8 @@ Instead, a separate collector node should be used, which collects the item. See
.._uncooperative-constructors-deprecated:
Constructors of custom :class:`pytest.Node` subclasses should take ``**kwargs``
@@ -380,6 +380,25 @@ conflicts (such as :class:`pytest.File` now taking ``path`` instead of
``fspath``, as :ref:`outlined above <node-ctor-fspath-deprecation>`), a
deprecation warning is now raised.
Applying a mark to a fixture function
-------------------------------------
..deprecated:: 7.4
Applying a mark to a fixture function never had any effect, but it is a common user error.
..code-block::python
@pytest.mark.usefixtures("clean_database")
@pytest.fixture
defuser()->User:
...
Users expected in this case that the ``usefixtures`` mark would have its intended effect of using the ``clean_database`` fixture when ``user`` was invoked, when in fact it has no effect at all.
Now pytest will issue a warning when it encounters this problem, and will raise an error in the future versions.
Backward compatibilities in ``Parser.addoption``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -467,12 +486,127 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
Removed Features
----------------
Removed Features and Breaking Changes
-------------------------------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
Some breaking changes which could not be deprecated are also listed.
Collection changes in pytest 8
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Added a new :class:`pytest.Directory` base collection node, which all collector nodes for filesystem directories are expected to subclass.
This is analogous to the existing :class:`pytest.File` for file nodes.
Changed :class:`pytest.Package` to be a subclass of :class:`pytest.Directory`.
A ``Package`` represents a filesystem directory which is a Python package,
i.e. contains an ``__init__.py`` file.
:class:`pytest.Package` now only collects files in its own directory; previously it collected recursively.
Sub-directories are collected as sub-collector nodes, thus creating a collection tree which mirrors the filesystem hierarchy.
:attr:`session.name <pytest.Session.name>` is now ``""``; previously it was the rootdir directory name.
This matches :attr:`session.nodeid <_pytest.nodes.Node.nodeid>` which has always been `""`.
Added a new :class:`pytest.Dir` concrete collection node, a subclass of :class:`pytest.Directory`.
This node represents a filesystem directory, which is not a :class:`pytest.Package`,
i.e. does not contain an ``__init__.py`` file.
Similarly to ``Package``, it only collects the files in its own directory,
while collecting sub-directories as sub-collector nodes.
Files and directories are now collected in alphabetical order jointly, unless changed by a plugin.
Previously, files were collected before directories.
The collection tree now contains directories/packages up to the :ref:`rootdir <rootdir>`,
for initial arguments that are found within the rootdir.
For files outside the rootdir, only the immediate directory/package is collected --
note however that collecting from outside the rootdir is discouraged.
As an example, given the following filesystem tree::
myroot/
pytest.ini
top/
├── aaa
│ └── test_aaa.py
├── test_a.py
├── test_b
│ ├── __init__.py
│ └── test_b.py
├── test_c.py
└── zzz
├── __init__.py
└── test_zzz.py
the collection tree, as shown by `pytest --collect-only top/` but with the otherwise-hidden :class:`~pytest.Session` node added for clarity,
is now the following::
<Session>
<Dir myroot>
<Dir top>
<Dir aaa>
<Module test_aaa.py>
<Function test_it>
<Module test_a.py>
<Function test_it>
<Package test_b>
<Module test_b.py>
<Function test_it>
<Module test_c.py>
<Function test_it>
<Package zzz>
<Module test_zzz.py>
<Function test_it>
Previously, it was::
<Session>
<Module top/test_a.py>
<Function test_it>
<Module top/test_c.py>
<Function test_it>
<Module top/aaa/test_aaa.py>
<Function test_it>
<Package test_b>
<Module test_b.py>
<Function test_it>
<Package zzz>
<Module test_zzz.py>
<Function test_it>
Code/plugins which rely on a specific shape of the collection tree might need to update.
:class:`pytest.Package` is no longer a :class:`pytest.Module` or :class:`pytest.File`
@@ -246,9 +246,9 @@ You can ask which markers exist for your test suite - the list includes our just
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
For an example on how to add and work with markers from a plugin, see
@@ -397,7 +397,7 @@ the test needs:
$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
@@ -411,7 +411,7 @@ and here is one that specifies exactly the environment needed:
$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
@@ -438,9 +438,9 @@ The ``--markers`` option always gives you a list of available markers:
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
.._`passing callables to custom markers`:
@@ -604,14 +604,14 @@ then you will see two tests skipped and two executed tests as expected:
$ pytest -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items
test_plat.py s.s. [100%]
========================= short test summary info ==========================
SKIPPED [2] conftest.py:12: cannot run on platform linux
SKIPPED [2] conftest.py:13: cannot run on platform linux
======================= 2 passed, 2 skipped in 0.12s =======================
Note that if you specify a platform via the marker-command line option like this:
@@ -620,7 +620,7 @@ Note that if you specify a platform via the marker-command line option like this
$ pytest -m linux
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 3 deselected / 1 selected
@@ -683,7 +683,7 @@ We can now use the ``-m option`` to select one set:
$ pytest -m interface --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 2 deselected / 2 selected
@@ -709,7 +709,7 @@ or to select both "event" and "interface" tests:
$ pytest -m "interface or event" --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
@@ -24,8 +24,9 @@ The first few lines should look like this:
[project]
name="PACKAGENAME"
version="PACKAGEVERSION"
where ``PACKAGENAME``is the name of your package.
where ``PACKAGENAME``and ``PACKAGEVERSION`` are the name and version of your package respectively.
You can then install your package in "editable" mode by running from the same directory:
@@ -50,8 +51,8 @@ Conventions for Python test discovery
* In those directories, search for ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_.
* From those files, collect test items:
*``test`` prefixed test functions or methods outside of class
*``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method)
*``test`` prefixed test functions or methods outside of class.
*``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method). Methods decorated with ``@staticmethod`` and ``@classmethods`` are also considered.
For examples of how to customize your test discovery :doc:`/example/pythoncollection`.
@@ -270,8 +271,8 @@ tox
Once you are done with your work and want to make sure that your actual
package passes all tests you may want to look into :doc:`tox <tox:index>`, the
virtualenv test automation tool and its :doc:`pytest support <tox:example/pytest>`.
tox helps you to setup virtualenv environments with pre-defined
virtualenv test automation tool.
``tox`` helps you to setup virtualenv environments with pre-defined
dependencies and then executing a pre-configured test command with
options. It will run tests against the installed package and not
against your source code checkout, helping to detect packaging
@@ -293,3 +294,20 @@ See also `pypa/setuptools#1684 <https://github.com/pypa/setuptools/issues/1684>`
setuptools intends to
`remove the test command <https://github.com/pypa/setuptools/issues/931>`_.
Checking with flake8-pytest-style
---------------------------------
In order to ensure that pytest is being used correctly in your project,
it can be helpful to use the `flake8-pytest-style <https://github.com/m-burst/flake8-pytest-style>`_ flake8 plugin.
flake8-pytest-style checks for common mistakes and coding style violations in pytest code,
such as incorrect use of fixtures, test function names, and markers.
By using this plugin, you can catch these errors early in the development process
and ensure that your pytest code is consistent and easy to maintain.
A list of the lints detected by flake8-pytest-style can be found on its `PyPI page <https://pypi.org/project/flake8-pytest-style/>`_.
..note::
flake8-pytest-style is not an official pytest project. Some of the rules enforce certain style choices, such as using `@pytest.fixture()` over `@pytest.fixture`, but you can configure the plugin to fit your preferred style.
@@ -16,7 +16,7 @@ import process can be controlled through the ``--import-mode`` command-line flag
these values:
*``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of :py:data:`sys.path` if not already there, and then imported with the :func:`__import__ <__import__>` builtin.
of :py:data:`sys.path` if not already there, and then imported with the :func:`importlib.import_module <importlib.import_module>` function.
This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in :py:data:`sys.modules` after importing.
@@ -24,7 +24,7 @@ these values:
This is the classic mechanism, dating back from the time Python 2 was still supported.
*``append``: the directory containing each module is appended to the end of :py:data:`sys.path` if not already
there, and imported with ``__import__``.
there, and imported with :func:`importlib.import_module <importlib.import_module>`.
This better allows to run test modules against installed versions of a package even if the
package under test has the same import root. For example:
@@ -43,7 +43,7 @@ these values:
Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in :py:data:`sys.modules` after importing.
*``importlib``: new in pytest-6.0, this mode uses :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
*``importlib``: new in pytest-6.0, this mode uses more fine control mechanisms provided by :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
For this reason this doesn't require test module names to be unique.
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 3 items
@@ -167,7 +167,7 @@ Let's run this:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 3 items
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
