The check for short paths under Windows via os.path.samefile, introduced in #11936, also found similar tests in symlinked tests in the GH Actions CI.
Fixes#12039.
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Previously, if more than one fixture finalizer raised, only the first
was reported, and the other errors were lost.
Use an exception group to report them all. This is similar to the change
we made in node teardowns (in `SetupState`).
As detailed in
https://github.com/pytest-dev/pytest/issues/11475#issuecomment-1937043670,
currently with `--import-mode=importlib` pytest will try to import every
file by using a unique module name, regardless if that module could be
imported using the normal import mechanism without touching `sys.path`.
This has the consequence that non-test modules available in `sys.path`
(via other mechanism, such as being installed into a virtualenv,
PYTHONPATH, etc) would end up being imported as standalone modules,
instead of imported with their expected module names.
To illustrate:
```
.env/
lib/
site-packages/
anndata/
core.py
```
Given `anndata` is installed into the virtual environment, `python -c
"import anndata.core"` works, but pytest with `importlib` mode would
import that module as a standalone module named
`".env.lib.site-packages.anndata.core"`, because importlib module was
designed to import test files which are not reachable from `sys.path`,
but now it is clear that normal modules should be imported using the
standard mechanisms if possible.
Now `imporlib` mode will first try to import the module normally,
without changing `sys.path`, and if that fails it falls back to
importing the module as a standalone module.
This also makes `importlib` respect namespace packages.
This supersedes #11931.
Fix#11475Close#11931
(diff better viewed ignoring whitespace)
In pytest<8, the collection tree for `pyargs` arguments in an invocation
like this:
pytest --collect-only --pyargs pyflakes.test.test_undefined_names
looked like this:
```
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
The pytest 8 collection improvements changed it to this:
```
<Dir pytest>
<Dir .tox>
<Dir venv>
<Dir lib>
<Dir python3.11>
<Dir site-packages>
<Package pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Besides being egregious (and potentially even worse than the above,
going all the way to the root, for system-installed packages, as is
apparently common in CI), this also caused permission errors when trying
to probe some of those intermediate directories.
This change makes `--pyargs` arguments no longer try to add parent
directories to the collection tree according to the `--confcutdir` like
they're regular arguments. Instead, only add the parents that are in the
import path. This now looks like this:
```
<Package .tox/venv/lib/python3.11/site-packages/pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Fix#11904.
pytest allows a fixture to request its own name (directly or
indirectly), in which case the fixture with the same name but one level
up is used.
To know which fixture should be used next, pytest keeps a mutable
item-global dict `_arg2index` which maintains this state. This is not
great:
- Mutable state like this is hard to understand and reason about.
- It is conceptually buggy; the indexing is global (e.g. if requesting
`fix1` and `fix2`, the indexing is shared between them), but actually
different branches of the subrequest tree should not affect each
other.
This is not an issue in practice because pytest keeps a cache of the
fixturedefs it resolved anyway (`_fixture_defs`), but if the cache is
removed it becomes evident.
Instead of the `_arg2index` state, count how many `argname`s deep we are
in the subrequest tree ("the fixture stack") and use that for the index.
This way, no global mutable state and the logic is very localized and
easier to understand.
This is slower, however fixture stacks should not be so deep that this
matters much, I hope.
According to my understanding, this code, which handles obtaining the
relevant fixturedefs when a dynamic `getfixturevalue` is used, has an
optimization where it only grabs fixturedefs that are visible to the
*parent* of the item, instead of the item itself, under the assumption
that a fixturedef can't be visible to a single item, only to a
collector.
Remove this optimization for the following reasons:
- It doesn't save much (one loop iteration in `matchfactories`)
- It slightly complicates the complex fixtures code
- If some plugin wants to make a fixture visible only to a single item,
why not let it?
- In the static case (`getfixtureclosure`), this optimization is not
done (despite the confusing name `parentnode`, it is *not* the parent
node). This is inconsistent.
Allow for the output of test case execution to be controlled independently from the application verbosity level.
`verbosity_test_case` is the new ini setting to adjust this functionality.
Fix#11639
Passing a short path in the command line was causing the matchparts check to fail, because ``Path(short_path) != Path(long_path)``.
Using ``os.path.samefile`` as fallback ensures the comparsion works on Windows when comparing short/long paths.
Fix#11895
In `CallSpec2.setmulti` the `params` and `_arg2scope` dicts are always
set together.
Further, the `get_parametrized_fixture_keys` accesses `_arg2scope` for
all argnames without a check, which I think rules out that the code
protects against plugin shenanigans.
After removing the suppress, adjust the comment and code to make more
sense.
Fix#12021.
Reopens#11706.
This reverts commit 12b9bd5801.
This change caused a bad regression in pytest-xdist:
https://github.com/pytest-dev/pytest-xdist/issues/1024
pytest-xdist necessarily has special handling of `--maxfail` and session
fixture teardown get executed multiple times with the change.
Since I'm not sure how to adapt pytest-xdist myself, revert for now.
I kept the sticky `shouldstop`/`shouldfail` changes as they are good
ideas regardless I think.
Dicts these days preserve order, so the sort is no longer needed to
achieve determinism.
As shown by the `test_dynamic_parametrized_ordering` test, this can
change the ordering of items, but only in equivalent ways (same number
of setups/teardowns per scope), it will just respect the user's given
ordering better (hence `vxlan` items now ordered before `vlan` items
compared to the previous ordering).
Follow up to #12006, let's put some comments clarifying `is_in_confcutdir` semantics, as this is not the first time someone misunderstands it.
Also removed an obsolete comment in `_loadconftestmodules`: we already set the `confcutdir` based on `rootdir`/`initfile` if not explicitly given.
Co-authored-by: Ran Benita <ran@unusedvar.com>
As the comment above it states, we need to keep the comparison simple so mypy can understand it, otherwise it will fail to properly handle this on Windows and will flag ``os.getuid()`` missing.
Test:
`warnings.warn()` expects that its first argument is a `str` or a
`Warning`, but since 9454fc38d3
`pytest.warns()` no longer allows `Warning` instances unless the first
argument the `Warning` was initialized with is a `str`. Furthermore, if
the `Warning` was created without arguments then `pytest.warns()` raises
an unexpected `IndexError`. The new tests reveal the problem.
Fix:
`pytest.warns()` now allows using `warnings.warn()` with a `Warning`
instance, as is required by Python, with one exception. If the warning
used is a `UserWarning` that was created by passing it arguments and the
first argument was not a `str` then `pytest.raises()` still considers
that an error. This is because if an invalid type was used in
`warnings.warn()` then Python creates a `UserWarning` anyways and it
becomes impossible for `pytest` to figure out if that was done
automatically or not.
[ran: rebased on previous commit]
Today `pyproject.toml` is the standard for declaring a Python project root, so seems reasonable to consider it for the ini configuration (and specially `rootdir`) in case we do not find other suitable candidates.
Related to #11311
Previously this would trigger an `AssertionError`.
While it could be considered a bug-fix, but given it now can be relied upon, it is probably better to consider it an improvement.
Fix#11311
Should be around 40% faster according to this simplified small benchmark:
python -m timeit "a=[0, 1, 2, 3, 4];b=list((e if i in {0, len(a) -1} else str(e)) for i, e in enumerate(a))"
200000 loops, best of 5: 1.12 usec per loop
python -m timeit "a=[0, 1, 2, 3, 4];b=list((a[0], *(str(e) for e in a[1:-1]), a[-1]))"
500000 loops, best of 5: 651 nsec per loop
python -m timeit "a=[0, 1, 2, 3, 4,5,6,7,8,9,10,11,12,13,14,15,16];b=list((e if i in {0, len(a) -1} else str(e)) for i, e in enumerate(a))"
100000 loops, best of 5: 3.31 usec per loop
python -m timeit "a=[0, 1, 2, 3, 4,5,6,7,8,9,10,11,12,13,14,15,16];b=list((a[0], *(str(e) for e in a[1:-1]), a[-1]))"
200000 loops, best of 5: 1.72 usec per loop
* Improve error message when using @pytest.fixture twice
While obvious in hindsight, this error message confused me. I thought my fixture
function was used in a test function twice, since the wording is ambiguous.
Also, the error does not tell me *which* function is the culprit.
Finally, this adds a test, which wasn't done in
cfd16d0dac where this was originally implemented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
warning: The top-level linter settings are deprecated in favour of their counterparts in the `lint` section. Please update the following options in `pyproject.toml`:
- 'ignore' -> 'lint.ignore'
- 'select' -> 'lint.select'
Fix#11929.
Figured out what's going on. We have the following collection tree:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
```
And the `test_main.py` contains an autouse fixture (`fake_game_ui`) that
`doctest_main` needs in order to run properly. The fixture doesn't run!
It doesn't run because nothing collects the fixtures from (calls
`parsefactories()` on) the `test_main.py` `DoctestModule`.
How come it only started happening with commit
ab63ebb3dc07b89670b96ae97044f48406c44fa0? Turns out it mostly only
worked accidentally. Each `DoctestModule` is also collected as a normal
`Module`, with the `Module` collected after the `DoctestModule`. For
example, if we add a non-doctest test to `test_main.py`, the collection
tree looks like this:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
<Module test_main.py>
<Function test_it>
```
Now, `Module` *does* collect fixtures. When autouse fixtures are
collected, they are added to the `_nodeid_autousenames` dict.
Before ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *setup* time. At this point, the `Module` has
collected and so it ended up picking the autouse fixture (this relies on
another "accident", that the `DoctestModule` and `Module` have the same
node ID).
After ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *collection* time (= when it's created). At
this point, the `Module` hasn't collected yet, so the autouse fixture is
not picked out.
The fix is simple -- have `DoctestModule.collect()` call
`parsefactories`. From some testing I've done it shouldn't have negative
consequences (I hope).
Currently, `DoctestModule` does `import_path` on its own. This changes
it to use `importtestmodule` as `Module` does. The behavioral changes
are:
- Much better error messages on import errors.
- Handles a few more error cases (see `importtestmodule`). This
technically expands the cover of `--doctest-ignore-import-errors` but
I think it makes sense.
- Considers `pytest_plugins` in the module.
- Populates `self.obj` as properly (without double-imports) as is
expected from a `PyCollector`.
This is also needed for the next commit.
ruff is faster and handle everything we had prior.
isort configuration done based on the indication from
https://github.com/astral-sh/ruff/issues/4670, previousely based on
reorder-python-import (#11896)
flake8-docstrings was a wrapper around pydocstyle (now archived) that
explicitly asks to use ruff in https://github.com/PyCQA/pydocstyle/pull/658.
flake8-typing-import is useful mainly for project that support python 3.7
and the one useful check will be implemented in https://github.com/astral-sh/ruff/issues/2302
We need to keep blacken-doc because ruff does not handle detection
of python code inside .md and .rst. The direct link to the repo is
now used to avoid a redirection.
Manual fixes:
- Lines that became too long
- % formatting that was not done automatically
- type: ignore that were moved around
- noqa of hard to fix issues (UP031 generally)
- fmt: off and fmt: on that is not really identical
between black and ruff
- autofix re-order in pre-commit from faster to slower
Co-authored-by: Ran Benita <ran@unusedvar.com>
The current version (0.23.4) explicitly does not support pytest 8 yet, so we fallback to the previous release in the hope that at least our integration tests pass.
- Previously, `getpass.getuser()` would leak an ImportError if the
USERNAME environment variable was not set on Windows because the `pwd`
module cannot be imported.
- Starting in Python 3.13.0a3, it only raises `OSError`.
Fixes#11874
Change our mypy configuration to disallow untyped defs by default, which ensures *new* files added to the code base are fully typed.
To avoid having to type-annotate everything now, add `# mypy: allow-untyped-defs` to files which are not fully type annotated yet.
As we fully type annotate those modules, we can then just remove that directive from the top.
The function currently uses `find_suffixes` which iterates the entire
directory searching for files with the given suffix. In some cases
though, like in pytest's selftest, the directory can get big:
$ ls /tmp/pytest-of-ran/pytest-0/
7686
and iterating it many times can get slow.
This doesn't fix the underlying issue (iterating the directory) but at
least speeds it up a bit by using `os.scandir` instead of
`path.iterdir`. So `make_numbered_dir` is still slow for pytest's
selftests, but reduces ~10s for me.
Follow up to https://github.com/pytest-dev/pytest/pull/11754, noticed that the latest GitHub release does not contain the attached files.
Output log from the action:
```
🤔 Pattern 'dist/*' does not match any files.
```
We have a use case for this in the next commit.
The name can be obtained by using `manager.get_name(plugin)`, however
this is currently O(num plugins) in pluggy, which would be good to
avoid. Besides, it seems generally useful.
This removes one thing that directory collectors need to worry about.
This adds one hook dispatch per `__pycache__` file, but I think it's
worth it for consistency.
(Diff better viewed ignoring whitespace)
Since e1c66ab0ad, conftest loading is
handled at the directory level before sub-nodes are collected, so there
is no need for the doctest plugin to handle it specially.
This was probably the case even before
e1c66ab0ad, but I haven't verified this.
This is a useful addition to the existing `listchain`. While `listchain`
returns top-to-bottom, `iterparents` is bottom-to-top and doesn't require
an internal full iteration + `reverse`.
Since we're removing nose support, let's also drop support for this
attribute.
From doing a code search on github, this seems completely unused outside
of nose, except for some projects which used to use it, but no longer
do.
Refs #11662.
--- Problem
Each fixture definition has a "visibility", the `FixtureDef.baseid`
attribute. This is nodeid-like string. When a certain `node` requests a
certain fixture name, we match node's nodeid against the fixture
definitions with this name.
The matching currently happens on the *textual* representation of the
nodeid - we split `node.nodeid` to its "parent nodeids" and then check
if the fixture's `baseid` is in there.
While this has worked so far, we really should try to avoid textual
manipulation of nodeids as much as possible. It has also caused problem
in an odd case of a `Package` in the root directory: the `Package` gets
nodeid `.`, while a `Module` in it gets nodeid `test_module.py`. And
textually, `.` is not a parent of `test_module.py`.
--- Solution
Avoid this entirely by just checking the node hierarchy itself. This is
made possible by the fact that we now have proper `Directory` nodes
(`Dir` or `Package`) for the entire hierarchy.
Also do the same for `_getautousenames` which is a similar deal.
The `iterparentnodeids` function is no longer used and is removed.
Instead of modifying user objects like modules and classes that we
really shouldn't be touching, use the new `_register_fixture` internal
API to do it directly.
Add a function on the `FixtureManager` to register a fixture with
pytest. Currently this can only be done through `parsefactories`.
My aim is to eventually make something like this available to plugins,
as it's a pretty common need.
* Fix handling empty values of NO_COLOR and FORCE_COLOR
Fix handling NO_COLOR and FORCE_COLOR environment variables to correctly
be ignored when they are set to an empty value, as defined
in the specification:
> Command-line software which adds ANSI color to its output by default
> should check for a NO_COLOR environment variable that, when present
> *and not an empty string* (regardless of its value), prevents
> the addition of ANSI color.
(emphasis mine, https://no-color.org/)
The same is true of FORCE_COLOR, https://force-color.org/.
* Streamline testing for FORCE_COLOR and NO_COLOR
Streamline the tests for FORCE_COLOR and NO_COLOR variables, and cover
all possible cases (unset, set to empty, set to "1"). Combine the two
assert functions into one taking boolean parameters. Mock file.isatty
in all circumstances to ensure that the environment variables take
precedence over the fallback value resulting from isatty check (or that
the fallback is actually used, in the case of both FORCE_COLOR
and NO_COLOR being unset).
* Put a 'reset' color in front of the highlighting
When doing the highlighting, some lexers will not set the initial color
explicitly, which may lead to the red from the errors being propagated
to the start of the expression
* Add syntactic highlighting to the error explanations
This updates the various error reporting to highlight python code when
displayed, to increase readability and make it easier to understand
--- Current main
In current main (before pervious commit), calls to gethookproxy/ihook
are the trigger for loading non-initial conftests. This basically means
that conftest loading is done almost as a random side-effect,
uncontrolled and very non-obvious. And it also dashes any hope of making
gethookproxy faster (gethookproxy shows up prominently in pytest
profiles).
I've wanted to improve this for a while, #11268 was the latest step
towards that.
--- PR before change
In this PR, I ran into a problem.
Previously, Session and Package did all of the directory traversals
inside of their collect, which loaded the conftests as a side effect. If
the conftest loading failed, it will occur inside of the collect() and
cause it to be reported as a failure.
Now I've changed things such that Session.collect and Package.collect no
longer recurse, but just collect their immediate descendants, and
genitems does the recursive expansion work.
The problem though is that genitems() doesn't run inside of specific
collector's collect context. So when it loads a conftest, and the
conftest loading fails, the exception isn't handled by any CollectReport
and causes an internal error instead.
The way I've fixed this problem is by loading the conftests eagerly in a
pytest_collect_directory post-wrapper, but only during genitems to make
sure the directory is actually selected.
This solution in turn caused the conftests to be collected too early;
specifically, the plugins are loaded during the parent's collect(), one
after the other as the directory entries are collected. So when the
ihook is hoisted out of the loop, new plugins are loaded inside the
loop, and due to the way the hook proxy works, they are added to the
ihook even though they're supposed to be scoped to the child collectors.
So no hoisting.
--- PR after change
Now I've come up with a better solution: since now the collection tree
actually reflects the filesystem tree, what we really want is to load
the conftest of a directory right before we run its collect(). A
conftest can affect a directory's collect() (e.g. with a
pytest_ignore_collect hookimpl), but it cannot affect how the directory
node itself is collected. So I just moved the conftest loading to be
done right before calling collect, but still inside the CollectReport
context.
This allows the hoisting, and also removes conftest loading from
gethookproxy since it's no longer necessary. And it will probably enable
further cleanups. So I'm happy with it.
Previously the error report would have all sections glued together:
- The assertion representation
- The error explanation
- The full diff
This makes it hard to see at a glance where which one starts and ends.
One of the representation (dataclasses, tuples, attrs) does display a
newlines at the start already.
Let's add a newlines before the error explanation and before the full
diff, so we get an easier to read report.
This has one disadvantage: we get one line less in the least verbose
mode, where the output gets truncated.
The default for `_prepareconfig` is to use `sys.argv`, which in this
case are the flags passed to (top-level) `pytest`. This is not the
intention, the tests themselves should not be affected by it.
When running `tox -e py-lsof` I get a deluge of this warning:
```
src/pytest/.tox/py-lsof-numpy-pexpect/lib/python3.11/site-packages/_pytest/pytester.py:130: EncodingWarning: UTF-8 Mode affects locale.getpreferredencoding(). Consider locale.getencoding() instead.
```
Use `locale.getencoding` instead.
The normal default pretty printer is not great when objects are nested
and it can get hard to read the diff.
Instead, provide a pretty printer that behaves more like when json get
indented, which allows for smaller, more meaningful differences, at
the expense of a slightly longer diff.
This does not touch the other places where the pretty printer is used,
and only updated the full diff one.
Reset color-related environment variables in a fixture to prevent them
from affecting test results. Otherwise, some of the tests fail
e.g. if NO_COLOR is set in the calling environment.
We already have the AlwaysDispatchingPrettyPrinter override of the
default pretty printer. In order to make more in depth changes, we
need to copy the upstream version in, as it doesn't lend itself well to
being extended.
This does a verbatime copy, adding provenance information at the top.
if we were ever to change the `name:` field in `test.yml` this would suddenly break -- this references the workflow file directly (and is what github generates now for badges)
The tested py.path.local.make_numbered_dir function is *not*
multiprocess safe, because is uses os.listdir which itself is not.
The os.listdir documentation explicitly states that:
> If a file is removed from or added to the directory during the call
> of this function, whether a name for that file be included is unspecified.
This can lead to a race when:
1. process A attempts to create directory N
2. the creation fails, as another process already created it in the meantime
3. process A calls listdir to determine a more recent maxnum
4. processes B+ repeatedly create newer directories and they delete directory N
5. process A doesn't have directory N or any newer directory in listdir result
6. process A attempts to create directory N again and raises
For details, see https://github.com/pytest-dev/pytest/issues/11603#issuecomment-1805708144
and bellow.
Additionally, the test itself has a race in batch_make_numbered_dirs.
When this functions attempts to write to repro-N/foo,
repro-N may have already been removed by another process.
For details, see https://github.com/pytest-dev/pytest/issues/11603#issuecomment-1804714313
and bellow.
---
The tested py.path.local.make_numbered_dir function is not used in pytest.
There is a different implementation in _pytest.pathlib.
Closes#11603
Previously, the sidebar TOC had unlimited depth, making it useless and
interfering with the content. This seems to have regressed in pytest
7.2.x going by the RTD version selector.
Traditionally pytest publishes tags in the form `X.Y.Z`, however the deploy workflow (copied from somewhere else) published tags in the form `vX.Y.Z`.
This is the root cause of #11548, because it tried to publish the release notes for tag `X.Y.Z` (which did not exist).
Fix#11548
Logging has many global states, and we did foresee this by creating a ``cleanup_disabled_logging`` fixture,
however one might still forget to use it and failures leak later -- sometimes not even in the same PR, because the order
of the tests might change in the future, specially when running under xdist.
This problem surfaced during pytest-dev/pytest#11530, where tests unrelated to the change started to fail.
The current method as the following problem, described by Sadra
Barikbin:
The tests that request both `pytester` and `monkeypatch` and use
`monkeypatch.chdir` without context, relying on `monkeypatch`'s teardown
to restore cwd. This doesn't work because the following sequence of
actions take place:
- `monkeypatch` is set up.
- `pytester` is set up. It saves the original cwd and changes it to a
new one dedicated to the test function.
- Test function calls `monkeypatch.chdir()` without context.
`monkeypatch` saves cwd, which is not the original one, before
changing it.
- `pytester` is torn down. It restores the cwd to the original one.
- `monkeypatch` is torn down. It restores cwd to what it has saved.
The solution here is to have pytester use `monkeypatch.chdir()` itself,
then everything is handled correctly.
Tests if a captured exception group contains an expected exception.
Will raise `AssertionError` if the wrapped exception is not an exception group.
Supports recursive search into nested exception groups.
Starting with `resolve_package_path` and its associated tests,
this refactoring seeks to make path concatenation more
readable and consistent within tests/functions.
As discussed in #11413:
- code is free to use either `/` and `joinpath`
- consistency within a function is more important than consistency across the codebase
- it is nice to use `/` when it is more readable
- it is nice to use `joinpath` when there is little context
- be mindful that `joinpath` may be clearer when joining multiple segments
There used to be two callers to `_setup_fixtures()`, now there's only
one, so inline it and make `DoctestItem` more similar to `Function`.
(Eventually we may want to generalize `TopRequest` from taking
`Function` directly to some "fixture-supporting item", removing the
remaining `type: ignore` here and allowing plugins to do it in a stable
manner).
As discussed in #11408:
* Improve documentation for the release process.
* Fix the description for the PRs created by the `prepare release pr` workflow.
* Fix pushing tag in the `deploy` workflow.
For packages, `import_path` receives the path to the package's `__init__.py` file, however module names (as they live in `sys.modules`) should not include the `__init__` part.
For example, `app/core/__init__.py` should be imported as `app.core`, not as `app.core.__init__`.
Fix#11306
I could find 2 plugins that would be broken by this (pytest-play and
pytest-wdl), but they will be better served by just copying
`_fillfixtures` instead of use the private function.
`_pyfuncitem.fixturenames` is just an alias for
`_pyfuncitem._fixtureinfo.names_closure` (at least in core pytest), so
let's do the less abstraction-breaking thing.
Move handling of user_properties to `finalize()`.
Previously if a fixture failed during teardown, `pytest_runtest_logreport` would not be called with "teardown", resulting in the user properties not being saved on the JUnit XML file.
Fixes: #11367
* Build the package only once, and test on all platforms.
* Deploy is now triggered manually via an Action, which is then responsible for tagging the repository after the package has been uploaded successfully.
* Drop 'docs': we nowadays rely on readthedocs preview PR builds.
This reverts commit e938580257.
Revert "improve error msg and test"
This reverts commit c0cf822ca1.
Revert "error msg"
This reverts commit ec1053cc16.
Revert "changelog"
This reverts commit d2dc8a70b5.
Revert "simplify code / take out user-gen typeerror case"
This reverts commit b9cb87d862.
Setting `norecursedirs` overrides the default, so we end up scanning
dot-directories and such which slows down collection unnecessarily
(150ms on my working directory).
To remove fixtures.py::add_funcargs_pseudo_fixture_def and add its logic
i.e. registering funcargs as params and making corresponding fixturedefs,
right to Metafunc.parametrize in which parametrization takes place.
To remove funcargs from metafunc attributes as we populate metafunc
params and make pseudo fixturedefs simultaneously and there's no need to
keep funcargs separately.
Previously, when assigning a scope for a fully-indirect parameter set,
when there are multiple fixturedefs for a param (i.e. same-name fixture
chain), the highest scope was used, but it should be the lowest scope,
since that's the effective scope of the fixture.
What one typically actually wants in such a case is both, checking for some
resulting values *and* checking for some expected exception.
Since this is easily possible with the `nullcontext` context manager, adapt the
example accordingly (needlessly using a different name rather just confuses people).
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Previously, the `_getconftestmodules` function was used both to load
conftest modules for a path (during `pytest_load_initial_conftests`),
and to retrieve conftest modules for a path (during hook dispatch and
for fetching `collect_ignore`). This made things muddy - it is usually
nicer to have clear separation between "command" and "query" functions,
when they occur in separate phases.
So split into "load" and "get".
Currently, `gethookproxy` still loads conftest itself. I hope to change
this in the future.
this change
* switches from html index usage to json index usage
* adds using requests-cache as http cache to trim down runtime
* adds gh action caches for pip and the new http cache
This test makes clear the need for the `_check_scope()` call in the
`pytest_setup_fixture` impl in fixtures.py, which otherwise seems
redundant with the one in `_compute_fixture_value`.
FixtureDef is used in the `pytest_fixture_setup` hook so needs to be
public. However since its current internals are quite dubious (and not
all marked with `_` prefix) I've added an explicit note that only
documented fields/methods are considered public.
Refs #7469.
From understanding the code better I see this is the correct fix.
The fixturedefs can be None if `request.getfixturevalue("doesnotexist")`
is used.
In practice there is no change in behavior because this mapping is used
as `self._arg2fixturedefs.get(argname, None)` which ends up the same.
It makes for a more debuggable repr. Before:
<function FixtureRequest._schedule_finalizers.<locals>.<lambda> at 0x7fe4ae32d440>
After:
functools.partial(<bound method FixtureDef.finish of <FixtureDef argname='hello_package' scope='package' baseid=''>>, request=<SubRequest 'hello_package' for <Function test_hello>>)
Add explicit mentions of the default behavior to both API reference and
how-to section about ways to invoke pytest.
Co-authored-by: Ran Benita <ran@unusedvar.com>
Dict comparsion in the ApproxMapping class did not check if values were None before attempting to subtract for max_abs_diff stat, which was throwing an TypeError instead of being handled by pytest error assertion. Check for None has been added before these calculations, so that None will properly show as Obtained/Expected in pytest assert message
The initial implementation (in #7246) introduced the `importlib` mode, which
never added the imported module to `sys.modules`, so it included a test
to ensure calling `import_path` twice would yield different modules.
Not adding modules to `sys.modules` proved problematic, so we began to add the imported module to `sys.modules`
in #7870, but failed to realize that given we are now changing `sys.modules`, we might
as well avoid importing it more than once.
Then #10088 came along, passing `importlib` also when importing application modules
(as opposed to only test modules before), which caused problems due to imports
having side-effects and the expectation being that they are imported only once.
With this PR, `import_path` returns the module immediately if already in
`sys.modules`.
Fix#10811Fix#10341
I think it's helpful to separate the node classes from the other
objects, as they have their own unique usage.
I've chosen not to alphabetize the order, but to use a logical order
instead.
Also slightly improve the docstrings.
@@ -272,24 +274,24 @@ Here is a simple overview, with pytest-specific bits:
#. Run all the tests
You need to have Python 3.7 available in your system. Now
You need to have Python 3.8 or later available in your system. Now
running tests is as simple as issuing this command::
$ tox -e linting,py37
$ tox -e linting,py39
This command will run tests via the "tox" tool against Python 3.7
This command will run tests via the "tox" tool against Python 3.9
and also perform "lint" coding-style checks.
#. You can now edit your local working copy and run the tests again as necessary. Please follow PEP-8 for naming.
#. You can now edit your local working copy and run the tests again as necessary. Please follow `PEP-8<https://www.python.org/dev/peps/pep-0008/>`_ for naming.
You can pass different options to ``tox``. For example, to run tests on Python 3.7 and pass options to pytest
You can pass different options to ``tox``. For example, to run tests on Python 3.9 and pass options to pytest
(e.g. enter pdb on failure) to pytest you can do::
$ tox -e py37 -- --pdb
$ tox -e py39 -- --pdb
Or to only run tests in a particular test module on Python 3.7::
Or to only run tests in a particular test module on Python 3.9::
$ tox -e py37 -- testing/test_config.py
$ tox -e py39 -- testing/test_config.py
When committing, ``pre-commit`` will re-format the files if necessary.
Added :func:`ExceptionInfo.from_exception() <pytest.ExceptionInfo.from_exception>`, a simpler way to create an :class:`~pytest.ExceptionInfo` from an exception.
This can replace :func:`ExceptionInfo.from_exc_info() <pytest.ExceptionInfo.from_exc_info()>` for most uses.
When an exception traceback to be displayed is completely filtered out (by mechanisms such as ``__tracebackhide__``, internal frames, and similar), now only the exception string and the following message are shown:
"All traceback entries are hidden. Pass `--full-trace` to see hidden and internal frames.".
Previously, the last frame of the traceback was shown, even though it was hidden.
Added warning when :confval:`testpaths` is set, but paths are not found by glob. In this case, pytest will fall back to searching from the current directory.
When `--confcutdir` is not specified, and there is no config file present, the conftest cutoff directory (`--confcutdir`) is now set to the :ref:`rootdir`.
Previously in such cases, `conftest.py` files would be probed all the way to the root directory of the filesystem.
If you are badly affected by this change, consider adding an empty config file to your desired cutoff directory, or explicitly set `--confcutdir`.
Fixed traceback entries hidden with ``__tracebackhide__ = True`` still being shown for chained exceptions (parts after "... the above exception ..." message).
@@ -14,7 +14,7 @@ Each file should be named like ``<ISSUE>.<TYPE>.rst``, where
``<ISSUE>`` is an issue number, and ``<TYPE>`` is one of:
*``feature``: new user facing features, like new command-line options and new behavior.
*``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junitxml``, improved colors in terminal, etc).
*``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junit-xml``, improved colors in terminal, etc).
*``bugfix``: fixes a bug.
*``doc``: documentation improvement, like rewording an entire session or adding missing docs.
There can be many different definitions of "success". Pytest can run many nose_ and unittest_ tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right?
There can be many different definitions of "success". Pytest can run many unittest_ tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right?
It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies.
@@ -22,7 +22,7 @@ b) transitional: the old and new API don't conflict
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationwarning`).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationWarning`).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g. `PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
@@ -87,6 +87,7 @@ Released pytest versions support all Python versions that are actively maintaine
@@ -28,6 +28,731 @@ with advance notice in the **Deprecations** section of releases.
.. towncrier release notes start
pytest 8.1.0 (2024-03-03)
=========================
Features
--------
-`#11475 <https://github.com/pytest-dev/pytest/issues/11475>`_: Added the new :confval:`consider_namespace_packages` configuration option, defaulting to ``False``.
If set to ``True``, pytest will attempt to identify modules that are part of `namespace packages <https://packaging.python.org/en/latest/guides/packaging-namespace-packages>`__ when importing modules.
-`#11653 <https://github.com/pytest-dev/pytest/issues/11653>`_: Added the new :confval:`verbosity_test_cases` configuration option for fine-grained control of test execution verbosity.
See :ref:`Fine-grained verbosity <pytest.fine_grained_verbosity>` for more details.
Improvements
------------
-`#10865 <https://github.com/pytest-dev/pytest/issues/10865>`_: :func:`pytest.warns` now validates that :func:`warnings.warn` was called with a `str` or a `Warning`.
Currently in Python it is possible to use other types, however this causes an exception when :func:`warnings.filterwarnings` is used to filter those warnings (see `CPython #103577 <https://github.com/python/cpython/issues/103577>`__ for a discussion).
While this can be considered a bug in CPython, we decided to put guards in pytest as the error message produced without this check in place is confusing.
-`#11311 <https://github.com/pytest-dev/pytest/issues/11311>`_: When using ``--override-ini`` for paths in invocations without a configuration file defined, the current working directory is used
as the relative directory.
Previoulsy this would raise an :class:`AssertionError`.
-`#11475 <https://github.com/pytest-dev/pytest/issues/11475>`_: :ref:`--import-mode=importlib <import-mode-importlib>` now tries to import modules using the standard import mechanism (but still without changing :py:data:`sys.path`), falling back to importing modules directly only if that fails.
This means that installed packages will be imported under their canonical name if possible first, for example ``app.core.models``, instead of having the module name always be derived from their path (for example ``.env310.lib.site_packages.app.core.models``).
-`#11801 <https://github.com/pytest-dev/pytest/issues/11801>`_: Added the :func:`iter_parents() <_pytest.nodes.Node.iter_parents>` helper method on nodes.
It is similar to :func:`listchain <_pytest.nodes.Node.listchain>`, but goes from bottom to top, and returns an iterator, not a list.
-`#11850 <https://github.com/pytest-dev/pytest/issues/11850>`_: Added support for :data:`sys.last_exc` for post-mortem debugging on Python>=3.12.
-`#11962 <https://github.com/pytest-dev/pytest/issues/11962>`_: In case no other suitable candidates for configuration file are found, a ``pyproject.toml`` (even without a ``[tool.pytest.ini_options]`` table) will be considered as the configuration file and define the ``rootdir``.
-`#11978 <https://github.com/pytest-dev/pytest/issues/11978>`_: Add ``--log-file-mode`` option to the logging plugin, enabling appending to log-files. This option accepts either ``"w"`` or ``"a"`` and defaults to ``"w"``.
Previously, the mode was hard-coded to be ``"w"`` which truncates the file before logging.
-`#12047 <https://github.com/pytest-dev/pytest/issues/12047>`_: When multiple finalizers of a fixture raise an exception, now all exceptions are reported as an exception group.
Previously, only the first exception was reported.
Bug Fixes
---------
-`#11904 <https://github.com/pytest-dev/pytest/issues/11904>`_: Fixed a regression in pytest 8.0.0 that would cause test collection to fail due to permission errors when using ``--pyargs``.
This change improves the collection tree for tests specified using ``--pyargs``, see :pull:`12043` for a comparison with pytest 8.0 and <8.
-`#12011 <https://github.com/pytest-dev/pytest/issues/12011>`_: Fixed a regression in 8.0.1 whereby ``setup_module`` xunit-style fixtures are not executed when ``--doctest-modules`` is passed.
-`#12014 <https://github.com/pytest-dev/pytest/issues/12014>`_: Fix the ``stacklevel`` used when warning about marks used on fixtures.
-`#12039 <https://github.com/pytest-dev/pytest/issues/12039>`_: Fixed a regression in ``8.0.2`` where tests created using :fixture:`tmp_path` have been collected multiple times in CI under Windows.
Improved Documentation
----------------------
-`#11790 <https://github.com/pytest-dev/pytest/issues/11790>`_: Documented the retention of temporary directories created using the ``tmp_path`` fixture in more detail.
Trivial/Internal Changes
------------------------
-`#11785 <https://github.com/pytest-dev/pytest/issues/11785>`_: Some changes were made to private functions which may affect plugins which access them:
- ``FixtureManager._getautousenames()`` now takes a ``Node`` itself instead of the nodeid.
-``FixtureManager.getfixturedefs()`` now takes the ``Node`` itself instead of the nodeid.
- The ``_pytest.nodes.iterparentnodeids()`` function is removed without replacement.
Prefer to traverse the node hierarchy itself instead.
If you really need to, copy the function from the previous pytest release.
pytest 8.0.2 (2024-02-24)
=========================
Bug Fixes
---------
-`#11895 <https://github.com/pytest-dev/pytest/issues/11895>`_: Fix collection on Windows where initial paths contain the short version of a path (for example ``c:\PROGRA~1\tests``).
-`#11953 <https://github.com/pytest-dev/pytest/issues/11953>`_: Fix an ``IndexError`` crash raising from ``getstatementrange_ast``.
-`#12021 <https://github.com/pytest-dev/pytest/issues/12021>`_: Reverted a fix to `--maxfail` handling in pytest 8.0.0 because it caused a regression in pytest-xdist whereby session fixture teardowns may get executed multiple times when the max-fails is reached.
pytest 8.0.1 (2024-02-16)
=========================
Bug Fixes
---------
-`#11875 <https://github.com/pytest-dev/pytest/issues/11875>`_: Correctly handle errors from :func:`getpass.getuser` in Python 3.13.
-`#11879 <https://github.com/pytest-dev/pytest/issues/11879>`_: Fix an edge case where ``ExceptionInfo._stringify_exception`` could crash :func:`pytest.raises`.
-`#11906 <https://github.com/pytest-dev/pytest/issues/11906>`_: Fix regression with :func:`pytest.warns` using custom warning subclasses which have more than one parameter in their `__init__`.
-`#11907 <https://github.com/pytest-dev/pytest/issues/11907>`_: Fix a regression in pytest 8.0.0 whereby calling :func:`pytest.skip` and similar control-flow exceptions within a :func:`pytest.warns()` block would get suppressed instead of propagating.
-`#11929 <https://github.com/pytest-dev/pytest/issues/11929>`_: Fix a regression in pytest 8.0.0 whereby autouse fixtures defined in a module get ignored by the doctests in the module.
-`#11937 <https://github.com/pytest-dev/pytest/issues/11937>`_: Fix a regression in pytest 8.0.0 whereby items would be collected in reverse order in some circumstances.
pytest 8.0.0 (2024-01-27)
=========================
Bug Fixes
---------
-`#11842 <https://github.com/pytest-dev/pytest/issues/11842>`_: Properly escape the ``reason`` of a :ref:`skip <pytest.mark.skip ref>` mark when writing JUnit XML files.
-`#11861 <https://github.com/pytest-dev/pytest/issues/11861>`_: Avoid microsecond exceeds ``1_000_000`` when using ``log-date-format`` with ``%f`` specifier, which might cause the test suite to crash.
pytest 8.0.0rc2 (2024-01-17)
============================
Improvements
------------
-`#11233 <https://github.com/pytest-dev/pytest/issues/11233>`_: Improvements to ``-r`` for xfailures and xpasses:
* Report tracebacks for xfailures when ``-rx`` is set.
* Report captured output for xpasses when ``-rX`` is set.
* For xpasses, add ``-`` in summary between test name and reason, to match how xfail is displayed.
-`#11825 <https://github.com/pytest-dev/pytest/issues/11825>`_: The :hook:`pytest_plugin_registered` hook has a new ``plugin_name`` parameter containing the name by which ``plugin`` is registered.
Bug Fixes
---------
-`#11706 <https://github.com/pytest-dev/pytest/issues/11706>`_: Fix reporting of teardown errors in higher-scoped fixtures when using `--maxfail` or `--stepwise`.
NOTE: This change was reverted in pytest 8.0.2 to fix a `regression <https://github.com/pytest-dev/pytest-xdist/issues/1024>`_ it caused in pytest-xdist.
-`#11758 <https://github.com/pytest-dev/pytest/issues/11758>`_: Fixed ``IndexError: string index out of range`` crash in ``if highlighted[-1] == "\n" and source[-1] != "\n"``.
This bug was introduced in pytest 8.0.0rc1.
-`#9765 <https://github.com/pytest-dev/pytest/issues/9765>`_, `#11816 <https://github.com/pytest-dev/pytest/issues/11816>`_: Fixed a frustrating bug that afflicted some users with the only error being ``assert mod not in mods``. The issue was caused by the fact that ``str(Path(mod))`` and ``mod.__file__`` don't necessarily produce the same string, and was being erroneously used interchangably in some places in the code.
This fix also broke the internal API of ``PytestPluginManager.consider_conftest`` by introducing a new parameter -- we mention this in case it is being used by external code, even if marked as *private*.
pytest 8.0.0rc1 (2023-12-30)
============================
Breaking Changes
----------------
Old Deprecations Are Now Errors
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-`#7363 <https://github.com/pytest-dev/pytest/issues/7363>`_: **PytestRemovedIn8Warning deprecation warnings are now errors by default.**
Following our plan to remove deprecated features with as little disruption as
possible, all warnings of type ``PytestRemovedIn8Warning`` now generate errors
instead of warning messages by default.
**The affected features will be effectively removed in pytest 8.1**, so please consult the
:ref:`deprecations` section in the docs for directions on how to update existing code.
In the pytest ``8.0.X`` series, it is possible to change the errors back into warnings as a
stopgap measure by adding this to your ``pytest.ini`` file:
..code-block::ini
[pytest]
filterwarnings=
ignore::pytest.PytestRemovedIn8Warning
But this will stop working when pytest ``8.1`` is released.
**If you have concerns** about the removal of a specific feature, please add a
comment to :issue:`7363`.
Version Compatibility
^^^^^^^^^^^^^^^^^^^^^
-`#11151 <https://github.com/pytest-dev/pytest/issues/11151>`_: Dropped support for Python 3.7, which `reached end-of-life on 2023-06-27 <https://devguide.python.org/versions/>`__.
-``pluggy>=1.3.0`` is now required.
Collection Changes
^^^^^^^^^^^^^^^^^^
In this version we've made several breaking changes to pytest's collection phase,
particularly around how filesystem directories and Python packages are collected,
fixing deficiencies and allowing for cleanups and improvements to pytest's internals.
A deprecation period for these changes was not possible.
-`#7777 <https://github.com/pytest-dev/pytest/issues/7777>`_: Files and directories are now collected in alphabetical order jointly, unless changed by a plugin.
Previously, files were collected before directories.
See below for an example.
-`#8976 <https://github.com/pytest-dev/pytest/issues/8976>`_: Running `pytest pkg/__init__.py` now collects the `pkg/__init__.py` file (module) only.
Previously, it collected the entire `pkg` package, including other test files in the directory, but excluding tests in the `__init__.py` file itself
(unless :confval:`python_files` was changed to allow `__init__.py` file).
To collect the entire package, specify just the directory: `pytest pkg`.
-`#11137 <https://github.com/pytest-dev/pytest/issues/11137>`_: :class:`pytest.Package` is no longer a :class:`pytest.Module` or :class:`pytest.File`.
The ``Package`` collector node designates a Python package, that is, a directory with an `__init__.py` file.
Previously ``Package`` was a subtype of ``pytest.Module`` (which represents a single Python module),
the module being the `__init__.py` file.
This has been deemed a design mistake (see :issue:`11137` and :issue:`7777` for details).
The ``path`` property of ``Package`` nodes now points to the package directory instead of the ``__init__.py`` file.
Note that a ``Module`` node for ``__init__.py`` (which is not a ``Package``) may still exist,
if it is picked up during collection (e.g. if you configured :confval:`python_files` to include ``__init__.py`` files).
-`#7777 <https://github.com/pytest-dev/pytest/issues/7777>`_: Added a new :class:`pytest.Directory` base collection node, which all collector nodes for filesystem directories are expected to subclass.
This is analogous to the existing :class:`pytest.File` for file nodes.
Changed :class:`pytest.Package` to be a subclass of :class:`pytest.Directory`.
A ``Package`` represents a filesystem directory which is a Python package,
i.e. contains an ``__init__.py`` file.
:class:`pytest.Package` now only collects files in its own directory; previously it collected recursively.
Sub-directories are collected as their own collector nodes, which then collect themselves, thus creating a collection tree which mirrors the filesystem hierarchy.
Added a new :class:`pytest.Dir` concrete collection node, a subclass of :class:`pytest.Directory`.
This node represents a filesystem directory, which is not a :class:`pytest.Package`,
that is, does not contain an ``__init__.py`` file.
Similarly to ``Package``, it only collects the files in its own directory.
:class:`pytest.Session` now only collects the initial arguments, without recursing into directories.
This work is now done by the :func:`recursive expansion process <pytest.Collector.collect>` of directory collector nodes.
:attr:`session.name <pytest.Session.name>` is now ``""``; previously it was the rootdir directory name.
This matches :attr:`session.nodeid <_pytest.nodes.Node.nodeid>` which has always been `""`.
The collection tree now contains directories/packages up to the :ref:`rootdir <rootdir>`,
for initial arguments that are found within the rootdir.
For files outside the rootdir, only the immediate directory/package is collected --
note however that collecting from outside the rootdir is discouraged.
As an example, given the following filesystem tree::
myroot/
pytest.ini
top/
├── aaa
│ └── test_aaa.py
├── test_a.py
├── test_b
│ ├── __init__.py
│ └── test_b.py
├── test_c.py
└── zzz
├── __init__.py
└── test_zzz.py
the collection tree, as shown by `pytest --collect-only top/` but with the otherwise-hidden :class:`~pytest.Session` node added for clarity,
is now the following::
<Session>
<Dir myroot>
<Dir top>
<Dir aaa>
<Module test_aaa.py>
<Function test_it>
<Module test_a.py>
<Function test_it>
<Package test_b>
<Module test_b.py>
<Function test_it>
<Module test_c.py>
<Function test_it>
<Package zzz>
<Module test_zzz.py>
<Function test_it>
Previously, it was::
<Session>
<Module top/test_a.py>
<Function test_it>
<Module top/test_c.py>
<Function test_it>
<Module top/aaa/test_aaa.py>
<Function test_it>
<Package test_b>
<Module test_b.py>
<Function test_it>
<Package zzz>
<Module test_zzz.py>
<Function test_it>
Code/plugins which rely on a specific shape of the collection tree might need to update.
-`#11676 <https://github.com/pytest-dev/pytest/issues/11676>`_: The classes :class:`~_pytest.nodes.Node`, :class:`~pytest.Collector`, :class:`~pytest.Item`, :class:`~pytest.File`, :class:`~_pytest.nodes.FSCollector` are now marked abstract (see :mod:`abc`).
We do not expect this change to affect users and plugin authors, it will only cause errors when the code is already wrong or problematic.
Other breaking changes
^^^^^^^^^^^^^^^^^^^^^^
These are breaking changes where deprecation was not possible.
-`#11282 <https://github.com/pytest-dev/pytest/issues/11282>`_: Sanitized the handling of the ``default`` parameter when defining configuration options.
Previously if ``default`` was not supplied for :meth:`parser.addini <pytest.Parser.addini>` and the configuration option value was not defined in a test session, then calls to :func:`config.getini <pytest.Config.getini>` returned an *empty list* or an *empty string* depending on whether ``type`` was supplied or not respectively, which is clearly incorrect. Also, ``None`` was not honored even if ``default=None`` was used explicitly while defining the option.
Now the behavior of :meth:`parser.addini <pytest.Parser.addini>` is as follows:
* If ``default`` is NOT passed but ``type`` is provided, then a type-specific default will be returned. For example ``type=bool`` will return ``False``, ``type=str`` will return ``""``, etc.
* If ``default=None`` is passed and the option is not defined in a test session, then ``None`` will be returned, regardless of the ``type``.
* If neither ``default`` nor ``type`` are provided, assume ``type=str`` and return ``""`` as default (this is as per previous behavior).
The team decided to not introduce a deprecation period for this change, as doing so would be complicated both in terms of communicating this to the community as well as implementing it, and also because the team believes this change should not break existing plugins except in rare cases.
-`#11667 <https://github.com/pytest-dev/pytest/issues/11667>`_: pytest's ``setup.py`` file is removed.
If you relied on this file, e.g. to install pytest using ``setup.py install``,
please see `Why you shouldn't invoke setup.py directly <https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html#summary>`_ for alternatives.
-`#9288 <https://github.com/pytest-dev/pytest/issues/9288>`_: :func:`~pytest.warns` now re-emits unmatched warnings when the context
closes -- previously it would consume all warnings, hiding those that were not
matched by the function.
While this is a new feature, we announce it as a breaking change
because many test suites are configured to error-out on warnings, and will
therefore fail on the newly-re-emitted warnings.
- The internal ``FixtureManager.getfixtureclosure`` method has changed. Plugins which use this method or
which subclass ``FixtureManager`` and overwrite that method will need to adapt to the change.
Deprecations
------------
-`#10465 <https://github.com/pytest-dev/pytest/issues/10465>`_: Test functions returning a value other than ``None`` will now issue a :class:`pytest.PytestWarning` instead of ``pytest.PytestRemovedIn8Warning``, meaning this will stay a warning instead of becoming an error in the future.
-`#3664 <https://github.com/pytest-dev/pytest/issues/3664>`_: Applying a mark to a fixture function now issues a warning: marks in fixtures never had any effect, but it is a common user error to apply a mark to a fixture (for example ``usefixtures``) and expect it to work.
This will become an error in pytest 9.0.
Features and Improvements
-------------------------
Improved Diffs
^^^^^^^^^^^^^^
These changes improve the diffs that pytest prints when an assertion fails.
Note that syntax highlighting requires the ``pygments`` package.
-`#11520 <https://github.com/pytest-dev/pytest/issues/11520>`_: The very verbose (``-vv``) diff output is now colored as a diff instead of a big chunk of red.
Python code in error reports is now syntax-highlighted as Python.
The sections in the error reports are now better separated.
-`#1531 <https://github.com/pytest-dev/pytest/issues/1531>`_: The very verbose diff (``-vv``) for every standard library container type is improved. The indentation is now consistent and the markers are on their own separate lines, which should reduce the diffs shown to users.
Previously, the standard Python pretty printer was used to generate the output, which puts opening and closing
markers on the same line as the first/last entry, in addition to not having consistent indentation.
-`#10617 <https://github.com/pytest-dev/pytest/issues/10617>`_: Added more comprehensive set assertion rewrites for comparisons other than equality ``==``, with
the following operations now providing better failure messages: ``!=``, ``<=``, ``>=``, ``<``, and ``>``.
Separate Control For Assertion Verbosity
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-`#11387 <https://github.com/pytest-dev/pytest/issues/11387>`_: Added the new :confval:`verbosity_assertions` configuration option for fine-grained control of failed assertions verbosity.
If you've ever wished that pytest always show you full diffs, but without making everything else verbose, this is for you.
See :ref:`Fine-grained verbosity <pytest.fine_grained_verbosity>` for more details.
For plugin authors, :attr:`config.get_verbosity <pytest.Config.get_verbosity>` can be used to retrieve the verbosity level for a specific verbosity type.
Additional Support For Exception Groups and ``__notes__``
These changes improve pytest's support for exception groups.
-`#10441 <https://github.com/pytest-dev/pytest/issues/10441>`_: Added :func:`ExceptionInfo.group_contains() <pytest.ExceptionInfo.group_contains>`, an assertion helper that tests if an :class:`ExceptionGroup` contains a matching exception.
See :ref:`assert-matching-exception-groups` for an example.
-`#11227 <https://github.com/pytest-dev/pytest/issues/11227>`_: Allow :func:`pytest.raises```match`` argument to match against `PEP-678 <https://peps.python.org/pep-0678/>` ``__notes__``.
Custom Directory collectors
^^^^^^^^^^^^^^^^^^^^^^^^^^^
-`#7777 <https://github.com/pytest-dev/pytest/issues/7777>`_: Added a new hook :hook:`pytest_collect_directory`,
which is called by filesystem-traversing collector nodes,
such as :class:`pytest.Session`, :class:`pytest.Dir` and :class:`pytest.Package`,
to create a collector node for a sub-directory.
It is expected to return a subclass of :class:`pytest.Directory`.
This hook allows plugins to :ref:`customize the collection of directories <custom directory collectors>`.
"New-style" Hook Wrappers
^^^^^^^^^^^^^^^^^^^^^^^^^
-`#11122 <https://github.com/pytest-dev/pytest/issues/11122>`_: pytest now uses "new-style" hook wrappers internally, available since pluggy 1.2.0.
See `pluggy's 1.2.0 changelog <https://pluggy.readthedocs.io/en/latest/changelog.html#pluggy-1-2-0-2023-06-21>`_ and the :ref:`updated docs <hookwrapper>` for details.
Plugins which want to use new-style wrappers can do so if they require ``pytest>=8``.
Other Improvements
^^^^^^^^^^^^^^^^^^
-`#11216 <https://github.com/pytest-dev/pytest/issues/11216>`_: If a test is skipped from inside an :ref:`xunit setup fixture <classic xunit>`, the test summary now shows the test location instead of the fixture location.
-`#11314 <https://github.com/pytest-dev/pytest/issues/11314>`_: Logging to a file using the ``--log-file`` option will use ``--log-level``, ``--log-format`` and ``--log-date-format`` as fallback
if ``--log-file-level``, ``--log-file-format`` and ``--log-file-date-format`` are not provided respectively.
-`#11610 <https://github.com/pytest-dev/pytest/issues/11610>`_: Added the :func:`LogCaptureFixture.filtering() <pytest.LogCaptureFixture.filtering>` context manager which
adds a given :class:`logging.Filter` object to the :fixture:`caplog` fixture.
-`#11447 <https://github.com/pytest-dev/pytest/issues/11447>`_: :func:`pytest.deprecated_call` now also considers warnings of type :class:`FutureWarning`.
-`#11600 <https://github.com/pytest-dev/pytest/issues/11600>`_: Improved the documentation and type signature for :func:`pytest.mark.xfail <pytest.mark.xfail>`'s ``condition`` param to use ``False`` as the default value.
-`#7469 <https://github.com/pytest-dev/pytest/issues/7469>`_: :class:`~pytest.FixtureDef` is now exported as ``pytest.FixtureDef`` for typing purposes.
-`#11353 <https://github.com/pytest-dev/pytest/issues/11353>`_: Added typing to :class:`~pytest.PytestPluginManager`.
Bug Fixes
---------
-`#10701 <https://github.com/pytest-dev/pytest/issues/10701>`_: :meth:`pytest.WarningsRecorder.pop` will return the most-closely-matched warning in the list,
rather than the first warning which is an instance of the requested type.
-`#11255 <https://github.com/pytest-dev/pytest/issues/11255>`_: Fixed crash on `parametrize(..., scope="package")` without a package present.
-`#11277 <https://github.com/pytest-dev/pytest/issues/11277>`_: Fixed a bug that when there are multiple fixtures for an indirect parameter,
the scope of the highest-scope fixture is picked for the parameter set, instead of that of the one with the narrowest scope.
-`#11456 <https://github.com/pytest-dev/pytest/issues/11456>`_: Parametrized tests now *really do* ensure that the ids given to each input are unique - for
example, ``a, a, a0`` now results in ``a1, a2, a0`` instead of the previous (buggy) ``a0, a1, a0``.
This necessarily means changing nodeids where these were previously colliding, and for
readability adds an underscore when non-unique ids end in a number.
-`#11563 <https://github.com/pytest-dev/pytest/issues/11563>`_: Fixed a crash when using an empty string for the same parametrized value more than once.
-`#11712 <https://github.com/pytest-dev/pytest/issues/11712>`_: Fixed handling ``NO_COLOR`` and ``FORCE_COLOR`` to ignore an empty value.
-`#9036 <https://github.com/pytest-dev/pytest/issues/9036>`_: ``pytest.warns`` and similar functions now capture warnings when an exception is raised inside a ``with`` block.
Improved Documentation
----------------------
-`#11011 <https://github.com/pytest-dev/pytest/issues/11011>`_: Added a warning about modifying the root logger during tests when using ``caplog``.
-`#11065 <https://github.com/pytest-dev/pytest/issues/11065>`_: Use ``pytestconfig`` instead of ``request.config`` in cache example to be consistent with the API documentation.
Trivial/Internal Changes
------------------------
-`#11208 <https://github.com/pytest-dev/pytest/issues/11208>`_: The (internal) ``FixtureDef.cached_result`` type has changed.
Now the third item ``cached_result[2]``, when set, is an exception instance instead of an exception triplet.
-`#11218 <https://github.com/pytest-dev/pytest/issues/11218>`_: (This entry is meant to assist plugins which access private pytest internals to instantiate ``FixtureRequest`` objects.)
:class:`~pytest.FixtureRequest` is now an abstract class which can't be instantiated directly.
A new concrete ``TopRequest`` subclass of ``FixtureRequest`` has been added for the ``request`` fixture in test functions,
as counterpart to the existing ``SubRequest`` subclass for the ``request`` fixture in fixture functions.
-`#11315 <https://github.com/pytest-dev/pytest/issues/11315>`_: The :fixture:`pytester` fixture now uses the :fixture:`monkeypatch` fixture to manage the current working directory.
If you use ``pytester`` in combination with :func:`monkeypatch.undo() <pytest.MonkeyPatch.undo>`, the CWD might get restored.
Use :func:`monkeypatch.context() <pytest.MonkeyPatch.context>` instead.
-`#11333 <https://github.com/pytest-dev/pytest/issues/11333>`_: Corrected the spelling of ``Config.ArgsSource.INVOCATION_DIR``.
The previous spelling ``INCOVATION_DIR`` remains as an alias.
-`#11638 <https://github.com/pytest-dev/pytest/issues/11638>`_: Fixed the selftests to pass correctly if ``FORCE_COLOR``, ``NO_COLOR`` or ``PY_COLORS`` is set in the calling environment.
pytest 7.4.4 (2023-12-31)
=========================
Bug Fixes
---------
-`#11140 <https://github.com/pytest-dev/pytest/issues/11140>`_: Fix non-string constants at the top of file being detected as docstrings on Python>=3.8.
-`#11572 <https://github.com/pytest-dev/pytest/issues/11572>`_: Handle an edge case where :data:`sys.stderr` and :data:`sys.__stderr__` might already be closed when :ref:`faulthandler` is tearing down.
-`#11710 <https://github.com/pytest-dev/pytest/issues/11710>`_: Fixed tracebacks from collection errors not getting pruned.
-`#7966 <https://github.com/pytest-dev/pytest/issues/7966>`_: Removed unhelpful error message from assertion rewrite mechanism when exceptions are raised in ``__iter__`` methods. Now they are treated un-iterable instead.
Improved Documentation
----------------------
-`#11091 <https://github.com/pytest-dev/pytest/issues/11091>`_: Updated documentation to refer to hyphenated options: replaced ``--junitxml`` with ``--junit-xml`` and ``--collectonly`` with ``--collect-only``.
pytest 7.4.3 (2023-10-24)
=========================
Bug Fixes
---------
-`#10447 <https://github.com/pytest-dev/pytest/issues/10447>`_: Markers are now considered in the reverse mro order to ensure base class markers are considered first -- this resolves a regression.
-`#11239 <https://github.com/pytest-dev/pytest/issues/11239>`_: Fixed ``:=`` in asserts impacting unrelated test cases.
-`#11439 <https://github.com/pytest-dev/pytest/issues/11439>`_: Handled an edge case where :data:`sys.stderr` might already be closed when :ref:`faulthandler` is tearing down.
pytest 7.4.2 (2023-09-07)
=========================
Bug Fixes
---------
-`#11237 <https://github.com/pytest-dev/pytest/issues/11237>`_: Fix doctest collection of `functools.cached_property` objects.
-`#11306 <https://github.com/pytest-dev/pytest/issues/11306>`_: Fixed bug using ``--importmode=importlib`` which would cause package ``__init__.py`` files to be imported more than once in some cases.
-`#11367 <https://github.com/pytest-dev/pytest/issues/11367>`_: Fixed bug where `user_properties` where not being saved in the JUnit XML file if a fixture failed during teardown.
-`#11394 <https://github.com/pytest-dev/pytest/issues/11394>`_: Fixed crash when parsing long command line arguments that might be interpreted as files.
Improved Documentation
----------------------
-`#11391 <https://github.com/pytest-dev/pytest/issues/11391>`_: Improved disclaimer on pytest plugin reference page to better indicate this is an automated, non-curated listing.
pytest 7.4.1 (2023-09-02)
=========================
Bug Fixes
---------
-`#10337 <https://github.com/pytest-dev/pytest/issues/10337>`_: Fixed bug where fake intermediate modules generated by ``--import-mode=importlib`` would not include the
child modules as attributes of the parent modules.
-`#10702 <https://github.com/pytest-dev/pytest/issues/10702>`_: Fixed error assertion handling in :func:`pytest.approx` when ``None`` is an expected or received value when comparing dictionaries.
-`#10811 <https://github.com/pytest-dev/pytest/issues/10811>`_: Fixed issue when using ``--import-mode=importlib`` together with ``--doctest-modules`` that caused modules
to be imported more than once, causing problems with modules that have import side effects.
pytest 7.4.0 (2023-06-23)
=========================
Features
--------
-`#10901 <https://github.com/pytest-dev/pytest/issues/10901>`_: Added :func:`ExceptionInfo.from_exception() <pytest.ExceptionInfo.from_exception>`, a simpler way to create an :class:`~pytest.ExceptionInfo` from an exception.
This can replace :func:`ExceptionInfo.from_exc_info() <pytest.ExceptionInfo.from_exc_info()>` for most uses.
Improvements
------------
-`#10872 <https://github.com/pytest-dev/pytest/issues/10872>`_: Update test log report annotation to named tuple and fixed inconsistency in docs for :hook:`pytest_report_teststatus` hook.
-`#10907 <https://github.com/pytest-dev/pytest/issues/10907>`_: When an exception traceback to be displayed is completely filtered out (by mechanisms such as ``__tracebackhide__``, internal frames, and similar), now only the exception string and the following message are shown:
"All traceback entries are hidden. Pass `--full-trace` to see hidden and internal frames.".
Previously, the last frame of the traceback was shown, even though it was hidden.
-`#10940 <https://github.com/pytest-dev/pytest/issues/10940>`_: Improved verbose output (``-vv``) of ``skip`` and ``xfail`` reasons by performing text wrapping while leaving a clear margin for progress output.
Added ``TerminalReporter.wrap_write()`` as a helper for that.
-`#10991 <https://github.com/pytest-dev/pytest/issues/10991>`_: Added handling of ``%f`` directive to print microseconds in log format options, such as ``log-date-format``.
-`#11005 <https://github.com/pytest-dev/pytest/issues/11005>`_: Added the underlying exception to the cache provider's path creation and write warning messages.
-`#11013 <https://github.com/pytest-dev/pytest/issues/11013>`_: Added warning when :confval:`testpaths` is set, but paths are not found by glob. In this case, pytest will fall back to searching from the current directory.
-`#11043 <https://github.com/pytest-dev/pytest/issues/11043>`_: When `--confcutdir` is not specified, and there is no config file present, the conftest cutoff directory (`--confcutdir`) is now set to the :ref:`rootdir <rootdir>`.
Previously in such cases, `conftest.py` files would be probed all the way to the root directory of the filesystem.
If you are badly affected by this change, consider adding an empty config file to your desired cutoff directory, or explicitly set `--confcutdir`.
-`#11081 <https://github.com/pytest-dev/pytest/issues/11081>`_: The :confval:`norecursedirs` check is now performed in a :hook:`pytest_ignore_collect` implementation, so plugins can affect it.
If after updating to this version you see that your `norecursedirs` setting is not being respected,
it means that a conftest or a plugin you use has a bad `pytest_ignore_collect` implementation.
Most likely, your hook returns `False` for paths it does not want to ignore,
which ends the processing and doesn't allow other plugins, including pytest itself, to ignore the path.
The fix is to return `None` instead of `False` for paths your hook doesn't want to ignore.
-`#8711 <https://github.com/pytest-dev/pytest/issues/8711>`_: :func:`caplog.set_level() <pytest.LogCaptureFixture.set_level>` and :func:`caplog.at_level() <pytest.LogCaptureFixture.at_level>`
will temporarily enable the requested ``level`` if ``level`` was disabled globally via
``logging.disable(LEVEL)``.
Bug Fixes
---------
-`#10831 <https://github.com/pytest-dev/pytest/issues/10831>`_: Terminal Reporting: Fixed bug when running in ``--tb=line`` mode where ``pytest.fail(pytrace=False)`` tests report ``None``.
-`#11068 <https://github.com/pytest-dev/pytest/issues/11068>`_: Fixed the ``--last-failed`` whole-file skipping functionality ("skipped N files") for :ref:`non-python test files <non-python tests>`.
-`#11104 <https://github.com/pytest-dev/pytest/issues/11104>`_: Fixed a regression in pytest 7.3.2 which caused to :confval:`testpaths` to be considered for loading initial conftests,
even when it was not utilized (e.g. when explicit paths were given on the command line).
Now the ``testpaths`` are only considered when they are in use.
-`#1904 <https://github.com/pytest-dev/pytest/issues/1904>`_: Fixed traceback entries hidden with ``__tracebackhide__ = True`` still being shown for chained exceptions (parts after "... the above exception ..." message).
-`#7781 <https://github.com/pytest-dev/pytest/issues/7781>`_: Fix writing non-encodable text to log file when using ``--debug``.
Improved Documentation
----------------------
-`#9146 <https://github.com/pytest-dev/pytest/issues/9146>`_: Improved documentation for :func:`caplog.set_level() <pytest.LogCaptureFixture.set_level>`.
Trivial/Internal Changes
------------------------
-`#11031 <https://github.com/pytest-dev/pytest/issues/11031>`_: Enhanced the CLI flag for ``-c`` to now include ``--config-file`` to make it clear that this flag applies to the usage of a custom config file.
pytest 7.3.2 (2023-06-10)
=========================
@@ -264,7 +989,7 @@ Improvements
-`#8508 <https://github.com/pytest-dev/pytest/issues/8508>`_: Introduce multiline display for warning matching via :py:func:`pytest.warns` and
enhance match comparison for :py:func:`_pytest._code.ExceptionInfo.match` as returned by :py:func:`pytest.raises`.
enhance match comparison for :py:func:`pytest.ExceptionInfo.match` as returned by :py:func:`pytest.raises`.
-`#8646 <https://github.com/pytest-dev/pytest/issues/8646>`_: Improve :py:func:`pytest.raises`. Previously passing an empty tuple would give a confusing
@@ -273,7 +998,7 @@ Improvements
-`#9741 <https://github.com/pytest-dev/pytest/issues/9741>`_: On Python 3.11, use the standard library's :mod:`tomllib` to parse TOML.
:mod:`tomli` is no longer a dependency on Python 3.11.
`tomli` is no longer a dependency on Python 3.11.
-`#9742 <https://github.com/pytest-dev/pytest/issues/9742>`_: Display assertion message without escaped newline characters with ``-vv``.
@@ -308,7 +1033,7 @@ Bug Fixes
When inheriting marks from super-classes, marks from the sub-classes are now ordered before marks from the super-classes, in MRO order. Previously it was the reverse.
When inheriting marks from super-classes, the `pytestmark` attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; use :func:`pytest.Node.iter_markers` instead.
When inheriting marks from super-classes, the `pytestmark` attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; use :func:`Node.iter_markers <_pytest.nodes.Node.iter_markers>` instead.
-`#9159 <https://github.com/pytest-dev/pytest/issues/9159>`_: Showing inner exceptions by forcing native display in ``ExceptionGroups`` even when using display options other than ``--tb=native``. A temporary step before full implementation of pytest-native display for inner exceptions in ``ExceptionGroups``.
@@ -561,7 +1286,7 @@ Bug Fixes
-`#9355 <https://github.com/pytest-dev/pytest/issues/9355>`_: Fixed error message prints function decorators when using assert in Python 3.8 and above.
-`#9396 <https://github.com/pytest-dev/pytest/issues/9396>`_: Ensure :attr:`pytest.Config.inifile` is available during the :func:`pytest_cmdline_main <_pytest.hookspec.pytest_cmdline_main>` hook (regression during ``7.0.0rc1``).
-`#9396 <https://github.com/pytest-dev/pytest/issues/9396>`_: Ensure `pytest.Config.inifile` is available during the :hook:`pytest_cmdline_main` hook (regression during ``7.0.0rc1``).
@@ -706,13 +1431,13 @@ Deprecations
-``parser.addoption(..., type="int/string/float/complex")`` - use ``type=int`` etc. instead.
-`#8447 <https://github.com/pytest-dev/pytest/issues/8447>`_: Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
-`#8447 <https://github.com/pytest-dev/pytest/issues/8447>`_: Defining a custom pytest node type which is both an :class:`~pytest.Item` and a :class:`~pytest.Collector` (e.g. :class:`~pytest.File`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
See :ref:`the deprecation note <diamond-inheritance-deprecated>` for full details.
-`#8592 <https://github.com/pytest-dev/pytest/issues/8592>`_: :hook:`pytest_cmdline_preparse` has been officially deprecated. It will be removed in a future release. Use :hook:`pytest_load_initial_conftests` instead.
-`#8592 <https://github.com/pytest-dev/pytest/issues/8592>`_: ``pytest_cmdline_preparse`` has been officially deprecated. It will be removed in a future release. Use :hook:`pytest_load_initial_conftests` instead.
See :ref:`the deprecation note <cmdline-preparse-deprecated>` for full details.
@@ -748,7 +1473,7 @@ Features
-`#7132 <https://github.com/pytest-dev/pytest/issues/7132>`_: Added two environment variables :envvar:`PYTEST_THEME` and :envvar:`PYTEST_THEME_MODE` to let the users customize the pygments theme used.
-`#7259 <https://github.com/pytest-dev/pytest/issues/7259>`_: Added :meth:`cache.mkdir() <pytest.Cache.mkdir>`, which is similar to the existing :meth:`cache.makedir() <pytest.Cache.makedir>`,
-`#7259 <https://github.com/pytest-dev/pytest/issues/7259>`_: Added :meth:`cache.mkdir() <pytest.Cache.mkdir>`, which is similar to the existing ``cache.makedir()``,
but returns a :class:`pathlib.Path` instead of a legacy ``py.path.local``.
Added a ``paths`` type to :meth:`parser.addini() <pytest.Parser.addini>`,
@@ -774,7 +1499,7 @@ Features
-``pytest.HookRecorder`` for the :class:`HookRecorder <pytest.HookRecorder>` type returned from :class:`~pytest.Pytester`.
-``pytest.RecordedHookCall`` for the :class:`RecordedHookCall <pytest.HookRecorder>` type returned from :class:`~pytest.HookRecorder`.
-``pytest.RunResult`` for the :class:`RunResult <pytest.RunResult>` type returned from :class:`~pytest.Pytester`.
-``pytest.LineMatcher`` for the :class:`LineMatcher <pytest.RunResult>` type used in :class:`~pytest.RunResult` and others.
-``pytest.LineMatcher`` for the :class:`LineMatcher <pytest.LineMatcher>` type used in :class:`~pytest.RunResult` and others.
-``pytest.TestReport`` for the :class:`TestReport <pytest.TestReport>` type used in various hooks.
-``pytest.CollectReport`` for the :class:`CollectReport <pytest.CollectReport>` type used in various hooks.
@@ -807,7 +1532,7 @@ Features
-`#8251 <https://github.com/pytest-dev/pytest/issues/8251>`_: Implement ``Node.path`` as a ``pathlib.Path``. Both the old ``fspath`` and this new attribute gets set no matter whether ``path`` or ``fspath`` (deprecated) is passed to the constructor. It is a replacement for the ``fspath`` attribute (which represents the same path as ``py.path.local``). While ``fspath`` is not deprecated yet
due to the ongoing migration of methods like :meth:`~_pytest.Item.reportinfo`, we expect to deprecate it in a future release.
due to the ongoing migration of methods like :meth:`~pytest.Item.reportinfo`, we expect to deprecate it in a future release.
..note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes (the
@@ -839,7 +1564,7 @@ Features
See :ref:`plugin-stash` for details.
-`#8953 <https://github.com/pytest-dev/pytest/issues/8953>`_: :class:`RunResult <_pytest.pytester.RunResult>` method :meth:`assert_outcomes <_pytest.pytester.RunResult.assert_outcomes>` now accepts a
-`#8953 <https://github.com/pytest-dev/pytest/issues/8953>`_: :class:`~pytest.RunResult` method :meth:`~pytest.RunResult.assert_outcomes` now accepts a
``warnings`` argument to assert the total number of warnings captured.
@@ -851,7 +1576,7 @@ Features
used.
-`#9113 <https://github.com/pytest-dev/pytest/issues/9113>`_: :class:`RunResult <_pytest.pytester.RunResult>` method :meth:`assert_outcomes <_pytest.pytester.RunResult.assert_outcomes>` now accepts a
-`#9113 <https://github.com/pytest-dev/pytest/issues/9113>`_: :class:`~pytest.RunResult` method :meth:`~pytest.RunResult.assert_outcomes` now accepts a
``deselected`` argument to assert the total number of deselected tests.
@@ -864,7 +1589,7 @@ Improvements
-`#7480 <https://github.com/pytest-dev/pytest/issues/7480>`_: A deprecation scheduled to be removed in a major version X (e.g. pytest 7, 8, 9, ...) now uses warning category `PytestRemovedInXWarning`,
a subclass of :class:`~pytest.PytestDeprecationWarning`,
instead of :class:`PytestDeprecationWarning` directly.
instead of :class:`~pytest.PytestDeprecationWarning` directly.
See :ref:`backwards-compatibility` for more details.
@@ -903,7 +1628,7 @@ Improvements
-`#8803 <https://github.com/pytest-dev/pytest/issues/8803>`_: It is now possible to add colors to custom log levels on cli log.
By using :func:`add_color_level <_pytest.logging.add_color_level>` from a ``pytest_configure`` hook, colors can be added::
By using ``add_color_level`` from a :hook:`pytest_configure` hook, colors can be added::
-`#8503 <https://github.com/pytest-dev/pytest/issues/8503>`_: :meth:`pytest.MonkeyPatch.syspath_prepend` no longer fails when
``setuptools`` is not installed.
It now only calls :func:`pkg_resources.fixup_namespace_packages` if
It now only calls ``pkg_resources.fixup_namespace_packages`` if
``pkg_resources`` was previously imported, because it is not needed otherwise.
@@ -1195,7 +1920,7 @@ Features
This is part of the movement to use :class:`pathlib.Path` objects internally, in order to remove the dependency to ``py`` in the future.
Internally, the old :class:`Testdir <_pytest.pytester.Testdir>` is now a thin wrapper around :class:`Pytester <_pytest.pytester.Pytester>`, preserving the old interface.
Internally, the old ``pytest.Testdir`` is now a thin wrapper around :class:`~pytest.Pytester`, preserving the old interface.
-:issue:`7695`: A new hook was added, `pytest_markeval_namespace` which should return a dictionary.
@@ -1233,7 +1958,7 @@ Features
Improvements
------------
-:issue:`1265`: Added an ``__str__`` implementation to the :class:`~pytest.pytester.LineMatcher` class which is returned from ``pytester.run_pytest().stdout`` and similar. It returns the entire output, like the existing ``str()`` method.
-:issue:`1265`: Added an ``__str__`` implementation to the :class:`~pytest.LineMatcher` class which is returned from ``pytester.run_pytest().stdout`` and similar. It returns the entire output, like the existing ``str()`` method.
-:issue:`2044`: Verbose mode now shows the reason that a test was skipped in the test's terminal line after the "SKIPPED", "XFAIL" or "XPASS".
@@ -1297,7 +2022,7 @@ Bug Fixes
-:issue:`7911`: Directories created by by :fixture:`tmp_path` and :fixture:`tmpdir` are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.
-:issue:`7913`: Fixed a crash or hang in :meth:`pytester.spawn <_pytest.pytester.Pytester.spawn>` when the :mod:`readline` module is involved.
-:issue:`7913`: Fixed a crash or hang in :meth:`pytester.spawn <pytest.Pytester.spawn>` when the :mod:`readline` module is involved.
-:issue:`7951`: Fixed handling of recursive symlinks when collecting tests.
@@ -1414,7 +2139,7 @@ Deprecations
if you use this and want a replacement.
-:issue:`7255`: The :hook:`pytest_warning_captured` hook is deprecated in favor
-:issue:`7255`: The ``pytest_warning_captured`` hook is deprecated in favor
of :hook:`pytest_warning_recorded`, and will be removed in a future version.
@@ -1442,8 +2167,8 @@ Improvements
-:issue:`7572`: When a plugin listed in ``required_plugins`` is missing or an unknown config key is used with ``--strict-config``, a simple error message is now shown instead of a stacktrace.
-:issue:`7685`: Added two new attributes :attr:`rootpath <_pytest.config.Config.rootpath>` and :attr:`inipath <_pytest.config.Config.inipath>` to :class:`Config <_pytest.config.Config>`.
These attributes are :class:`pathlib.Path` versions of the existing :attr:`rootdir <_pytest.config.Config.rootdir>` and :attr:`inifile <_pytest.config.Config.inifile>` attributes,
-:issue:`7685`: Added two new attributes :attr:`rootpath <pytest.Config.rootpath>` and :attr:`inipath <pytest.Config.inipath>` to :class:`~pytest.Config`.
These attributes are :class:`pathlib.Path` versions of the existing ``rootdir`` and ``inifile`` attributes,
and should be preferred over them when possible.
@@ -1514,7 +2239,7 @@ Trivial/Internal Changes
-:issue:`7587`: The dependency on the ``more-itertools`` package has been removed.
-:issue:`7631`: The result type of :meth:`capfd.readouterr() <_pytest.capture.CaptureFixture.readouterr>` (and similar) is no longer a namedtuple,
-:issue:`7631`: The result type of :meth:`capfd.readouterr() <pytest.CaptureFixture.readouterr>` (and similar) is no longer a namedtuple,
but should behave like one in all respects. This was done for technical reasons.
@@ -1892,10 +2617,10 @@ Improvements
-:issue:`7128`: `pytest --version` now displays just the pytest version, while `pytest --version --version` displays more verbose information including plugins. This is more consistent with how other tools show `--version`.
-:issue:`7133`: :meth:`caplog.set_level() <_pytest.logging.LogCaptureFixture.set_level>` will now override any :confval:`log_level` set via the CLI or configuration file.
-:issue:`7133`: :meth:`caplog.set_level() <pytest.LogCaptureFixture.set_level>` will now override any :confval:`log_level` set via the CLI or configuration file.
-:issue:`7159`: :meth:`caplog.set_level() <_pytest.logging.LogCaptureFixture.set_level>` and :meth:`caplog.at_level() <_pytest.logging.LogCaptureFixture.at_level>` no longer affect
-:issue:`7159`: :meth:`caplog.set_level() <pytest.LogCaptureFixture.set_level>` and :meth:`caplog.at_level() <pytest.LogCaptureFixture.at_level>` no longer affect
the level of logs that are shown in the *Captured log report* report section.
@@ -1990,7 +2715,7 @@ Bug Fixes
parameter when Python is called with the ``-bb`` flag.
-:issue:`7143`: Fix :meth:`pytest.File.from_parent` so it forwards extra keyword arguments to the constructor.
-:issue:`7143`: Fix :meth:`pytest.File.from_parent <_pytest.nodes.Node.from_parent>` so it forwards extra keyword arguments to the constructor.
-:issue:`7145`: Classes with broken ``__getattribute__`` methods are displayed correctly during failures.
@@ -2241,7 +2966,7 @@ Improvements
-:issue:`6384`: Make `--showlocals` work also with `--tb=short`.
-:issue:`6653`: Add support for matching lines consecutively with :attr:`LineMatcher <_pytest.pytester.LineMatcher>`'s :func:`~_pytest.pytester.LineMatcher.fnmatch_lines` and :func:`~_pytest.pytester.LineMatcher.re_match_lines`.
-:issue:`6653`: Add support for matching lines consecutively with :class:`~pytest.LineMatcher`'s :func:`~pytest.LineMatcher.fnmatch_lines` and :func:`~pytest.LineMatcher.re_match_lines`.
-:issue:`6658`: Code is now highlighted in tracebacks when ``pygments`` is installed.
@@ -2309,7 +3034,7 @@ Bug Fixes
-:issue:`6597`: Fix node ids which contain a parametrized empty-string variable.
-:issue:`6646`: Assertion rewriting hooks are (re)stored for the current item, which fixes them being still used after e.g. pytester's :func:`testdir.runpytest <_pytest.pytester.Testdir.runpytest>` etc.
-:issue:`6646`: Assertion rewriting hooks are (re)stored for the current item, which fixes them being still used after e.g. pytester's ``testdir.runpytest`` etc.
-:issue:`6660`: :py:func:`pytest.exit` is handled when emitted from the :hook:`pytest_sessionfinish` hook. This includes quitting from a debugger.
@@ -2375,7 +3100,7 @@ Bug Fixes
``multiprocessing`` module.
-:issue:`6436`: :class:`FixtureDef <_pytest.fixtures.FixtureDef>` objects now properly register their finalizers with autouse and
-:issue:`6436`: :class:`~pytest.FixtureDef` objects now properly register their finalizers with autouse and
parameterized fixtures that execute before them in the fixture stack so they are torn
down at the right times, and in the right order.
@@ -2431,7 +3156,7 @@ Improvements
Bug Fixes
---------
-:issue:`5914`: pytester: fix :py:func:`~_pytest.pytester.LineMatcher.no_fnmatch_line` when used after positive matching.
-:issue:`5914`: pytester: fix :py:func:`~pytest.LineMatcher.no_fnmatch_line` when used after positive matching.
-:issue:`6082`: Fix line detection for doctest samples inside :py:class:`python:property` docstrings, as a workaround to :bpo:`17446`.
@@ -2495,8 +3220,8 @@ Features
rather than implicitly.
-:issue:`5914`: :fixture:`testdir` learned two new functions, :py:func:`~_pytest.pytester.LineMatcher.no_fnmatch_line` and
In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
*:hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
*:hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
*:hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
*:hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
*:hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
..note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
:ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
being ``path``) is **the opposite** of the situation for hooks (the old
argument being ``path``).
This is an unfortunate artifact due to historical reasons, which should be
resolved in future versions as we slowly get rid of the :pypi:`py`
dependency (see :issue:`9283` for a longer discussion).
Directly constructing internal classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -273,62 +81,6 @@ Directly constructing the following classes is now deprecated:
These constructors have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.
.._cmdline-preparse-deprecated:
Passing ``msg=`` to ``pytest.skip``, ``pytest.fail`` or ``pytest.exit``
Diamond inheritance between :class:`pytest.Collector` and :class:`pytest.Item`
@@ -336,7 +88,7 @@ Diamond inheritance between :class:`pytest.Collector` and :class:`pytest.Item`
..deprecated:: 7.0
Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
Defining a custom pytest node type which is both an :class:`~pytest.Item` and a :class:`~pytest.Collector` (e.g. :class:`~pytest.File`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
Some plugins providing linting/code analysis have been using this as a hack.
@@ -348,8 +100,8 @@ Instead, a separate collector node should be used, which collects the item. See
.._uncooperative-constructors-deprecated:
Constructors of custom :class:`pytest.Node` subclasses should take ``**kwargs``
@@ -380,28 +132,22 @@ conflicts (such as :class:`pytest.File` now taking ``path`` instead of
``fspath``, as :ref:`outlined above <node-ctor-fspath-deprecation>`), a
deprecation warning is now raised.
Backward compatibilities in ``Parser.addoption``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Applying a mark to a fixture function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated::2.4
..deprecated::7.4
Several behaviors of :meth:`Parser.addoption <pytest.Parser.addoption>` are now
scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):
Applying a mark to a fixture function never had any effect, but it is a common user error.
-``parser.addoption(..., help=".. %default ..")`` - use ``%(default)s`` instead.
-``parser.addoption(..., type="int/string/float/complex")`` - use ``type=int`` etc. instead.
..code-block::python
@pytest.mark.usefixtures("clean_database")
@pytest.fixture
defuser()->User:...
Using ``pytest.warns(None)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Users expected in this case that the ``usefixtures`` mark would have its intended effect of using the ``clean_database`` fixture when ``user`` was invoked, when in fact it has no effect at all.
..deprecated:: 7.0
:func:`pytest.warns(None) <pytest.warns>` is now deprecated because it was frequently misused.
Its correct usage was checking that the code emits at least one warning of any type - like ``pytest.warns()``
or ``pytest.warns(Warning)``.
See :ref:`warns use cases` for examples.
Now pytest will issue a warning when it encounters this problem, and will raise an error in the future versions.
Returning non-None value in test functions
@@ -444,19 +190,6 @@ The proper fix is to change the `return` to an `assert`:
assertfoo(a,b)==result
The ``--strict`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.2
The ``--strict`` command-line option has been deprecated in favor of ``--strict-markers``, which
better conveys what the option does.
We have plans to maybe in the future to reintroduce ``--strict`` and make it an encompassing
flag for all strictness related options (``--strict-markers`` and ``--strict-config``
at the moment, more might be introduced in the future).
The ``yield_fixture`` function/decorator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -467,12 +200,415 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
Removed Features
----------------
Removed Features and Breaking Changes
-------------------------------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
Some breaking changes which could not be deprecated are also listed.
.._node-ctor-fspath-deprecation:
``fspath`` argument for Node constructors replaced with ``pathlib.Path``
In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
*:hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
*:hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
*:hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
*:hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
*:hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
..note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
:ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
being ``path``) is **the opposite** of the situation for hooks (the old
argument being ``path``).
This is an unfortunate artifact due to historical reasons, which should be
resolved in future versions as we slowly get rid of the :pypi:`py`
dependency (see :issue:`9283` for a longer discussion).
.._nose-deprecation:
Support for tests written for nose
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.2
..versionremoved:: 8.0
Support for running tests written for `nose <https://nose.readthedocs.io/en/latest/>`__ is now deprecated.
``nose`` has been in maintenance mode-only for years, and maintaining the plugin is not trivial as it spills
over the code base (see :issue:`9886` for more details).
setup/teardown
^^^^^^^^^^^^^^
One thing that might catch users by surprise is that plain ``setup`` and ``teardown`` methods are not pytest native,
they are in fact part of the ``nose`` support.
..code-block::python
classTest:
defsetup(self):
self.resource=make_resource()
defteardown(self):
self.resource.close()
deftest_foo(self):...
deftest_bar(self):...
Native pytest support uses ``setup_method`` and ``teardown_method`` (see :ref:`xunit-method-setup`), so the above should be changed to:
..code-block::python
classTest:
defsetup_method(self):
self.resource=make_resource()
defteardown_method(self):
self.resource.close()
deftest_foo(self):...
deftest_bar(self):...
This is easy to do in an entire code base by doing a simple find/replace.
@with_setup
^^^^^^^^^^^
Code using `@with_setup <with-setup-nose>`_ such as this:
pytest as a testing framework needs to import test modules and ``conftest.py`` files for execution.
Importing files in Python (at least until recently) is a non-trivial processes, often requiring
changing :data:`sys.path`. Some aspects of the
Importing files in Python is a non-trivial processes, so aspects of the
import process can be controlled through the ``--import-mode`` command-line flag, which can assume
these values:
*``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of :py:data:`sys.path` if not already there, and then imported with the :func:`importlib.import_module <importlib.import_module>` function.
.._`import-mode-prepend`:
This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in :py:data:`sys.modules` after importing.
*``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of :py:data:`sys.path` if not already there, and then imported with
the :func:`importlib.import_module <importlib.import_module>` function.
It is highly recommended to arrange your test modules as packages by adding ``__init__.py`` files to your directories
containing tests. This will make the tests part of a proper Python package, allowing pytest to resolve their full
name (for example ``tests.core.test_core`` for ``test_core.py`` inside the ``tests.core`` package).
If the test directory tree is not arranged as packages, then each test file needs to have a unique name
compared to the other test files, otherwise pytest will raise an error if it finds two tests with the same name.
This is the classic mechanism, dating back from the time Python 2 was still supported.
.._`import-mode-append`:
*``append``: the directory containing each module is appended to the end of :py:data:`sys.path` if not already
there, and imported with :func:`importlib.import_module <importlib.import_module>`.
@@ -38,32 +46,78 @@ these values:
the tests will run against the installed version
of ``pkg_under_test`` when ``--import-mode=append`` is used whereas
with ``prepend`` they would pick up the local version. This kind of confusion is why
we advocate for using :ref:`src <src-layout>` layouts.
we advocate for using :ref:`src-layouts <src-layout>`.
Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in :py:data:`sys.modules` after importing.
*``importlib``: new in pytest-6.0, this mode uses more fine control mechanisms provided by :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
.._`import-mode-importlib`:
For this reason this doesn't require test module names to be unique.
*``importlib``: this mode uses more fine control mechanisms provided by :mod:`importlib` to import test modules, without changing :py:data:`sys.path`.
One drawback however is that test modules are non-importable by each other. Also, utility
modules in the tests directories are not automatically importable because the tests directory is no longer
added to :py:data:`sys.path`.
Advantages of this mode:
Initially we intended to make ``importlib`` the default in future releases, however it is clear now that
it has its own set of drawbacks so the default will remain ``prepend`` for the foreseeable future.
* pytest will not change :py:data:`sys.path` at all.
* Test module names do not need to be unique -- pytest will generate a unique name automatically based on the ``rootdir``.
Disadvantages:
* Test modules can't import each other.
* Testing utility modules in the tests directories (for example a ``tests.helpers`` module containing test-related functions/classes)
are not importable. The recommendation in this case it to place testing utility modules together with the application/library
code, for example ``app.testing.helpers``.
Important: by "test utility modules" we mean functions/classes which are imported by
other tests directly; this does not include fixtures, which should be placed in ``conftest.py`` files, along
with the test modules, and are discovered automatically by pytest.
It works like this:
1. Given a certain module path, for example ``tests/core/test_models.py``, derives a canonical name
like ``tests.core.test_models`` and tries to import it.
For non-test modules this will work if they are accessible via :py:data:`sys.path`, so
for example ``.env/lib/site-packages/app/core.py`` will be importable as ``app.core``.
This is happens when plugins import non-test modules (for example doctesting).
If this step succeeds, the module is returned.
For test modules, unless they are reachable from :py:data:`sys.path`, this step will fail.
2. If the previous step fails, we import the module directly using ``importlib`` facilities, which lets us import it without
changing :py:data:`sys.path`.
Because Python requires the module to also be available in :py:data:`sys.modules`, pytest derives a unique name for it based
on its relative location from the ``rootdir``, and adds the module to :py:data:`sys.modules`.
For example, ``tests/core/test_models.py`` will end up being imported as the module ``tests.core.test_models``.
..versionadded:: 6.0
..note::
Initially we intended to make ``importlib`` the default in future releases, however it is clear now that
it has its own set of drawbacks so the default will remain ``prepend`` for the foreseeable future.
..note::
By default, pytest will not attempt to resolve namespace packages automatically, but that can
be changed via the :confval:`consider_namespace_packages` configuration variable.
..seealso::
The :confval:`pythonpath` configuration variable.
The :confval:`consider_namespace_packages` configuration variable.
:ref:`test layout`.
``prepend`` and ``append`` import modes scenarios
-------------------------------------------------
Here's a list of scenarios when using ``prepend`` or ``append`` import modes where pytest needs to
change ``sys.path`` in order to import test modules or ``conftest.py`` files, and the issues users
change :py:data:`sys.path` in order to import test modules or ``conftest.py`` files, and the issues users
might encounter because of that.
Test modules / ``conftest.py`` files inside packages
@@ -92,7 +146,7 @@ pytest will find ``foo/bar/tests/test_foo.py`` and realize it is part of a packa
there's an ``__init__.py`` file in the same folder. It will then search upwards until it can find the
last folder which still contains an ``__init__.py`` file in order to find the package *root* (in
this case ``foo/``). To load the module, it will insert ``root/`` to the front of
``sys.path`` (if not there already) in order to load
:py:data:`sys.path` (if not there already) in order to load
``test_foo.py`` as the *module*``foo.bar.tests.test_foo``.
The same logic applies to the ``conftest.py`` file: it will be imported as ``foo.conftest`` module.
@@ -122,8 +176,8 @@ When executing:
pytest will find ``foo/bar/tests/test_foo.py`` and realize it is NOT part of a package given that
there's no ``__init__.py`` file in the same folder. It will then add ``root/foo/bar/tests`` to
``sys.path`` in order to import ``test_foo.py`` as the *module*``test_foo``. The same is done
with the ``conftest.py`` file by adding ``root/foo`` to ``sys.path`` to import it as ``conftest``.
:py:data:`sys.path` in order to import ``test_foo.py`` as the *module*``test_foo``. The same is done
with the ``conftest.py`` file by adding ``root/foo`` to :py:data:`sys.path` to import it as ``conftest``.
For this reason this layout cannot have test modules with the same name, as they all will be
imported in the global import namespace.
@@ -136,7 +190,7 @@ Invoking ``pytest`` versus ``python -m pytest``
-----------------------------------------------
Running pytest with ``pytest [...]`` instead of ``python -m pytest [...]`` yields nearly
equivalent behaviour, except that the latter will add the current directory to ``sys.path``, which
equivalent behaviour, except that the latter will add the current directory to :py:data:`sys.path`, which
@@ -433,7 +433,7 @@ marked ``smtp_connection`` fixture function. Running the test looks like this:
$ pytest test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 2 items
@@ -494,7 +494,7 @@ Fixtures are created when first requested by a test, and are destroyed based on
*``function``: the default scope, the fixture is destroyed at the end of the test.
*``class``: the fixture is destroyed during teardown of the last test in the class.
*``module``: the fixture is destroyed during teardown of the last test in the module.
*``package``: the fixture is destroyed during teardown of the last test in the package.
*``package``: the fixture is destroyed during teardown of the last test in the package where the fixture is defined, including sub-packages and sub-directories within it.
*``session``: the fixture is destroyed at the end of the test session.
..note::
@@ -771,7 +771,7 @@ For yield fixtures, the first teardown code to run is from the right-most fixtur
$ pytest -s test_finalizers.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
@@ -805,7 +805,7 @@ For finalizers, the first fixture to run is last call to `request.addfinalizer`.
$ pytest -s test_finalizers.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
@@ -1271,7 +1271,7 @@ configured in multiple ways.
Extending the previous example, we can flag the fixture to create two
``smtp_connection`` fixture instances which will cause all tests using the fixture
to run twice. The fixture function gets access to each parameter
through the special :py:class:`request <FixtureRequest>` object:
through the special :py:class:`request <pytest.FixtureRequest>` object:
..code-block::python
@@ -1414,27 +1414,28 @@ Running the above tests results in the following test IDs being used:
$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 12 items
<Module test_anothersmtp.py>
<Function test_showhelo[smtp.gmail.com]>
<Function test_showhelo[mail.python.org]>
<Module test_emaillib.py>
<Function test_email_received>
<Module test_finalizers.py>
<Function test_bar>
<Module test_ids.py>
<Function test_a[spam]>
<Function test_a[ham]>
<Function test_b[eggs]>
<Function test_b[1]>
<Module test_module.py>
<Function test_ehlo[smtp.gmail.com]>
<Function test_noop[smtp.gmail.com]>
<Function test_ehlo[mail.python.org]>
<Function test_noop[mail.python.org]>
<Dir fixtures.rst-214>
<Module test_anothersmtp.py>
<Function test_showhelo[smtp.gmail.com]>
<Function test_showhelo[mail.python.org]>
<Module test_emaillib.py>
<Function test_email_received>
<Module test_finalizers.py>
<Function test_bar>
<Module test_ids.py>
<Function test_a[spam]>
<Function test_a[ham]>
<Function test_b[eggs]>
<Function test_b[1]>
<Module test_module.py>
<Function test_ehlo[smtp.gmail.com]>
<Function test_noop[smtp.gmail.com]>
<Function test_ehlo[mail.python.org]>
<Function test_noop[mail.python.org]>
======================= 12 tests collected in 0.12s ========================
@@ -1468,7 +1469,7 @@ Running this test will *skip* the invocation of ``data_set`` with value ``2``:
$ pytest test_fixture_marks.py -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 3 items
@@ -1518,7 +1519,7 @@ Here we declare an ``app`` fixture which receives the previously defined
$ pytest -v test_appsetup.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 2 items
@@ -1598,7 +1599,7 @@ Let's run the tests in verbose mode and with looking at the print-output:
$ pytest -v -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-8.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collecting ... collected 8 items
@@ -1698,7 +1699,7 @@ and declare its use in a test module via a ``usefixtures`` marker:
classTestDirectoryInit:
deftest_cwd_starts_empty(self):
assertos.listdir(os.getcwd())==[]
withopen("myfile","w")asf:
withopen("myfile","w",encoding="utf-8")asf:
f.write("hello")
deftest_cwd_again_starts_empty(self):
@@ -1720,8 +1721,7 @@ You can specify multiple fixtures like this:
``pytest`` has basic support for running tests written for nose_.
..warning::
This functionality has been deprecated and is likely to be removed in ``pytest 8.x``.
.._nosestyle:
Usage
-------------
After :ref:`installation` type:
..code-block::bash
python setup.py develop # make sure tests can import our package
pytest # instead of 'nosetests'
and you should be able to run your nose style tests and
make use of pytest's capabilities.
Supported nose Idioms
----------------------
*``setup()`` and ``teardown()`` at module/class/method level: any function or method called ``setup`` will be called during the setup phase for each test, same for ``teardown``.
*``SkipTest`` exceptions and markers
* setup/teardown decorators
*``__test__`` attribute on modules/classes/functions
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.