The following snippet would have resulted in crash on multiple places since
`_get_verbose_word` expects only string, not a tuple.
```python
@pytest.hookimpl(tryfirst=True)
def pytest_report_teststatus(report: pytest.CollectReport | pytest.TestReport, config: pytest.Config):
if report.when == "call":
return ("error", "A", ("AVC", {"bold": True, "red": True}))
return None
```
```
Traceback (most recent call last):
File "/home/pbrezina/workspace/sssd/.venv/bin/pytest", line 8, in <module>
sys.exit(console_main())
^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/config/__init__.py", line 207, in console_main
code = main()
^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/config/__init__.py", line 179, in main
ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 103, in _multicall
res = hook_impl.function(*args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/main.py", line 333, in pytest_cmdline_main
return wrap_session(config, _main)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/main.py", line 321, in wrap_session
config.hook.pytest_sessionfinish(
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 122, in _multicall
teardown.throw(exception) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/logging.py", line 872, in pytest_sessionfinish
return (yield)
^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 124, in _multicall
teardown.send(result) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/terminal.py", line 899, in pytest_sessionfinish
self.config.hook.pytest_terminal_summary(
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_hooks.py", line 513, in __call__
return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_manager.py", line 120, in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 139, in _multicall
raise exception.with_traceback(exception.__traceback__)
File "/home/pbrezina/workspace/sssd/.venv/lib64/python3.11/site-packages/pluggy/_callers.py", line 124, in _multicall
teardown.send(result) # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/terminal.py", line 923, in pytest_terminal_summary
self.short_test_summary()
File "/home/pbrezina/workspace/pytest/src/_pytest/terminal.py", line 1272, in short_test_summary
action(lines)
File "/home/pbrezina/workspace/pytest/src/_pytest/terminal.py", line 1205, in show_simple
line = _get_line_with_reprcrash_message(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/terminal.py", line 1429, in _get_line_with_reprcrash_message
word = tw.markup(verbose_word, **word_markup)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/pbrezina/workspace/pytest/src/_pytest/_io/terminalwriter.py", line 114, in markup
text = "".join(f"\x1b[{cod}m" for cod in esc) + text + "\x1b[0m"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
TypeError: can only concatenate str (not "tuple") to str
```
Signed-off-by: Pavel Březina <pbrezina@redhat.com>
BPO is in read-only mode and all issues have been migrated to GitHub.
This patch replaces the use of that role with `:issue:`, recently
integrated via #12522. It also disables the built-in `extlinks` Sphinx
extension, as it's no longer in use.
These tests are known to only be executed partially or not at all. So
we always get incomplete, missing, and sometimes flaky, coverage in
the test functions that are expected to fail.
This change updates the ``coverage.py`` config to prevent said tests
from influencing the coverage level measurement.
This extension implements more generic roles that can also be used
more flexibly. Relying on an external extension allows to stop
maintaining an in-repo copy of the commonly used behavior.
It's necessary since it seems that the currently used Codecov uploader
doesn't read the token from config sometimes.
This is a follow-up for #12508 which wasn't enough.
The #11921 update broke uploading coverage of the `main` branch (or
any in-repo pushes for that matter) to Codecov 4 months ago.
Version 4 requires an upload token to be provided and since there was
no configuration for it, the upload was failing. But the step itself
was showing up as successful due to `fail_ci_if_error: true` being
set. The error is visible in the console output, though.
This patch flips the setting to `fail_ci_if_error: false` and sets the
Codecov upload token in the config in clear text. The non-secret part
allows the PRs uploads to be more stable.
Co-authored-by: Ronny Pfannschmidt <opensource@ronnypfannschmidt.de>
Previously, a part of the environments weren't documented in the
config, making it difficult for the newbies to figure out what
their purposes are. This patch sets the descriptions for all the
envs listed with the `tox -av` command, leveraging the dynamic
factor-dependent explanation fragments.
```console
default environments:
linting -> run pre-commit-defined linters under `python3`
py38 -> run the tests under `py38`
py39 -> run the tests under `py39`
py310 -> run the tests under `py310`
py311 -> run the tests under `py311`
py312 -> run the tests under `py312`
py313 -> run the tests under `py313`
pypy3 -> run the tests under `pypy3`
py38-pexpect -> run the tests against `pexpect` under `py38`
py38-xdist -> run the tests with pytest in parallel mode under `py38`
py38-unittestextras -> run the tests against the unit test extras under `py38`
py38-numpy -> run the tests against `numpy` under `py38`
py38-pluggymain -> run the tests against the bleeding edge `pluggy` from Git under `py38`
py38-pylib -> run the tests against `py` lib under `py38`
doctesting -> run the tests under `~/.pyenv/versions/3.12.3/envs/pytest-pyenv-py3.12.3/bin/python` including doctests
doctesting-coverage -> run the tests collecting coverage under `~/.pyenv/versions/3.12.3/envs/pytest-pyenv-py3.12.3/bin/python` including doctests
plugins -> run reverse dependency testing against pytest plugins under `~/.pyenv/versions/3.12.3/envs/pytest-pyenv-py3.12.3/bin/python`
py38-freeze -> test pytest frozen with `pyinstaller` under `py38`
docs -> build the documentation site under `~/src/github/pytest-dev/pytest/doc/en/_build/html` with `python3`
docs-checklinks -> check the links in the documentation with `python3`
py311-exceptiongroup -> run the tests against `exceptiongroup` under `py311`
additional environments:
regen -> regenerate documentation examples under `python3`
release -> do a release, required posarg of the version number
prepare-release-pr -> prepare a release PR from a manual trigger in GitHub actions
generate-gh-release-notes -> generate release notes that can be published as GitHub Release
nobyte -> run the tests in no-bytecode mode under `~/.pyenv/versions/3.12.3/envs/pytest-pyenv-py3.12.3/bin/python`
lsof -> run the tests with `--lsof` pytest CLI option under `~/.pyenv/versions/3.12.3/envs/pytest-pyenv-py3.12.3/bin/python`
```
The new change note types are `packaging`, `contrib` and `misc`.
`packaging` is intended for the audience of downstream redistributors.
The `contrib` notes are meant to be documenting news affecting the
project contributors, their development, and processes.
Finally, `misc` is for things that don't fit anywhere but are still
desired to be documented for some reason.
It is uncapped in RTD. Apparently, `tox -e docs` is not invoked in CI,
neither is it called in RTD, resulting in the regression having been
caught only in local development environments.
This is a follow-up for #12493.
It is called when building the docs. Apparently, `tox -e docs` is not
invoked in CI, neither is it called in RTD, resulting in the
regression having been caught only in local development environments.
This is a follow-up for #12493.
The earlier implementation was generating a temporary file, when
the docs site was being built with `tox`. However, this was not
enabled in RTD and is hackish.
This patch integrates the `sphinxcontrib-towncrier` extension to
make it work in any environment where Sphinx docs are being built.
Previously, it was possible to use several different config file name
variants. However, there is one canonical name now and the others are
being deprecated [[1]].
[1]: https://docs.rtfd.io/en/stable/config-file/index.html
Previously, this workflow was being used to cherry-pick PRs made
against `main` into older stable branches. Now that #12475 integrated
the Patchback GitHub App, it's no longer needed as it's making
duplicate pull requests which don't even trigger CI runs automatically
due to #10354. So this patch removes said workflow to address the
problem.
There was a regression caused by #12431 that excluded the patch on
broken CPython patch version ranges 3.11.0–3.11.8 and 3.12.0–3.12.2.
This change fixes that by adjusting the patching conditional clause
to include said versions.
Closes#12430.
Co-authored-by: Florian Bruhin <me@the-compiler.org>
* 🚑 Explicitly set encoding @ `subprocess.run()`
This invocation is present in the `test_issue_9765` regression test
that is always skipped unconditionally, resulting in the
`EncodingWarning` never manifesting itself in CI or development
environments of the contributors.
* Change `setup.py` call w/ `pip install` @ tests
Using this CLI interface has been deprecated in `setuptools` [[1]].
[1]: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html
* Change Contribution doc title to match sidebar
* Rearrange sentence for clarity
* Update backwards-compatibility.rst
some minor grammar changes
* Update pythonpath.rst
fixed some gramatical errors
* Update AUTHORS
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update doc/en/explanation/pythonpath.rst
From a quick overview it looks like lowercase is more consistent, although some pages do use `pytest` in code blocks
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
---------
Co-authored-by: Mackerello <82668740+Mackerello@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
This fixes unittest test reruns when using plugins like
pytest-rerunfailures.
The `instance` property uses AttributeError to check if the instance
needs to be initialized, so `del` is the correct way to clear it, not
setting to `None`.
Regressed in 8.2.2.
* Only rely on _find_lineno on Python 3.10 and earlier.
* Update docstring to reflect current expectation.
* Only rely on _find on Python 3.9 and earlier.
* Mark line as uncovered.
* Remove empty else block (implicit is better than explicit).
Closes#12430Closes#12432
Launchable with ``pre-commit run --hook-stage manual pyupgrade -a``
---------
Signed-off-by: Tomasz Kłoczko <kloczek@github.com>
Co-authored-by: Pierre Sassoulas <pierre.sassoulas@gmail.com>
Fix#12355.
In the issue, it was reported that the `reorder_items` has quadratic (or
worse...) behavior with certain simple parametrizations. After some
debugging I found that the problem happens because the "Fix
items_by_argkey order" loop keeps adding the same item to the deque,
and it reaches epic sizes which causes the slowdown.
I don't claim to understand how the `reorder_items` algorithm works, but
if as far as I understand, if an item already exists in the deque, the
correct thing to do is to move it to the front. Since a deque doesn't
have such an (efficient) operation, this switches to `OrderedDict` which
can efficiently append from both sides, deduplicate and move to front.
Fix#12381
Test plan:
It's possible to write a deterministic test case for this, but somewhat
of a hassle so I tested it manually. I reproduced by removing existing
`.pytest_cache`, adding a sleep before the rename and running two
pytests. I verified that it doesn't reproduce after the fix.
* [pylint 'use-maxsplit-arg'] Do not split more than necessary when using the first element
* [pylint 'consider-using-enumerate'] Use zip when iterating on two iterators
* [pylint] 'cell-var-from-loop' and 'disallowed-name' permanent disable
* [pylint] Disable 'possibly-used-before-assignment' following 3.2.0 release
This is used to set the <link rel="canonical" href="X"> tag that points to the canonical version of the webpage. Including this indicates to search engines which version to include in their indexes, and should prevent older versions showing up.
Fixes#12363
* furo theme for docs site
* removing duplicate tocs from deprecations and reference pages
* removing pallets references in code and config
* reverting trainings to sidebar
* removed sphinx style and unpinned packaging version
* updated styles
The `.parent` was incorrectly removed in
a21fb87a90, but it was actually correct
(well, it assumes that Module.path.parent == Package.path, which I'm not
sure is always correct, but that's a different matter). Restore it.
Fix#12328.
Closes#11706.
Originally fixed in #11721, but then reverted in #12022 due to a regression in pytest-xdist.
The regression was fixed on the pytest-xdist side in pytest-dev/pytest-xdist#1026.
In these two cases which re-raise an immediately-caught exception, do
the re-raising with `raise` instead of `raise exc`, because the `raise
exc` adds an uninteresting entry to the exception's traceback (that of
itself), while `raise` doesn't.
* added test case in testing/python/approx.py based on test case provided by reporter in issue #12114
* test cases pass for pytest testing/python/approx.py
* expanded the type annotation to include objects which may cast to a array and renamed other_side to other_side_as_array and asserted that it is not none
Now `importlib` mode will correctly set the imported modules as an attribute of their parent modules.
As helpfully posted on #12194, that's how the Python import module works so we should follow suit.
In addition, we also try to import the parent modules as part of the process of importing a child module, again mirroring how Python importing works.
Fix#12194
Previously we used a hand crafted approach to detect namespace packages, however we should rely on ``importlib`` to detect them for us.
Fix#12112
---------
Co-authored-by: Ran Benita <ran@unusedvar.com>
Creating and initializing the cache directory is interruptible; this
avoids a pathological case where interrupting a cache write can cause
the cache directory to never be properly initialized with its supporting
files.
Unify `Cache.mkdir` with `Cache.set` while I'm here so the former also
properly initializes the cache directory.
Closes#12167.
It makes more sense, also, we have a long term idea of generalizing
fixture support to items defined by other plugins, not just python, in
which case `--fixtures` would definitely not be python-plugin
specific.
Based on pylint's message ``consider-iterating-dictionary`` suggestion.
Surprisingly using a dict or set comprehension instead of a new temp var is
actually consistently slower here, which was not intuitive for me.
```python
from timeit import timeit
families = {1: {"testcase": [1, 2, 3, 5, 8]}}
attrs = {1: "a", 2: "b", 3: "c", 4: "d", 5: "e", 6: "f", 7: "g", 8: "h"}
class Old:
def old(self):
self.attrs = attrs
temp_attrs = {}
for key in self.attrs.keys():
if key in families[1]["testcase"]:
temp_attrs[key] = self.attrs[key]
self.attrs = temp_attrs
class OldBis:
def old(self):
self.attrs = attrs
temp_attrs = {}
for key in self.attrs:
if key in families[1]["testcase"]:
temp_attrs[key] = self.attrs[key]
self.attrs = temp_attrs
class New:
def new(self):
self.attrs = attrs
self.attrs = { # Even worse with k: v for k in self.attrs.items()
k: self.attrs[k] for k in self.attrs if k in families[1]["testcase"]
}
if __name__ == "__main__":
n = 1000000
print(f"Old: {timeit(Old().old, number=n)}")
print(f"Just removing the keys(): {timeit(OldBis().old, number=n)}")
print(f"List comp, no temp var: {timeit(New().new, number=n)}")
```
Result:
Old: 0.9493889989680611
Just removing the keys(): 0.9042672360083088
List comp, no temp var: 0.9916125109884888
It's also true for the other example with similar benchmark, but the exact
code probably does not need to be in the commit message.
The plugin is abandoned and no longer working with new pytest versions.
I also reordered a bit to put pytest-rerunfailures first since it seems most maintained and is under pytest-dev.
and also fixes a regression in pytest 8.0.0 where `setup_method` crashes
if the class has static or class method tests.
It is allowed to have a test class with static/class methods which
request non-static/class method fixtures (including `setup_method`
xunit-fixture). I take it as a given that we need to support this
somewhat odd scenario (stdlib unittest also supports it).
This raises a question -- when a staticmethod test requests a bound
fixture, what is that fixture's `self`?
stdlib unittest says - a fresh instance for the test.
Previously, pytest said - some instance that is shared by all
static/class methods. This is definitely broken since it breaks test
isolation.
Change pytest to behave like stdlib unittest here.
In practice, this means stopping to rely on `self.obj.__self__` to get
to the instance from the test function's binding. This doesn't work
because staticmethods are not bound to anything.
Instead, keep the instance explicitly and use that.
BTW, I think this will allow us to change `Class`'s fixture collection
(`parsefactories`) to happen on the class itself instead of a class
instance, allowing us to avoid one class instantiation. But needs more
work.
Fixes#12065.
- Separate the requesting from the requested.
- Avoid the term "factory", I think most people don't distinguish
between "fixture" and "fixture function" (i.e. "factory") and would
find the term "factory" unfamiliar.
There are two non-optimal things in the current way scope checking is
done:
- It runs on `SubRequest`, but doesn't use the `SubRequest's scope,
which is confusing. Instead it takes `invoking_scope` and
`requested_scope`.
- Because `_check_scope` is only defined on `SubRequest` and not
`TopRequest`, `_compute_fixture_value` first creates the `SubRequest`
only then checks the scope (hence the need for the previous point).
Instead, also define `_check_scope` on `TopRequest` (always valid), and
remove `invoking_scope`, using `self._scope` instead.
Previously, the `obj` of a `TestCaseFunction` (the unittest plugin item
type) was the unbound method. This is unlike regular `Class` where the
`obj` is a bound method to a fresh instance.
This difference necessitated several special cases in in places outside
of the unittest plugin, such as `FixtureDef` and `FixtureRequest`, and
made things a bit harder to understand.
Instead, match how the python plugin does it, including collecting
fixtures from a fresh instance.
The downside is that now this instance for fixture-collection is kept
around in memory, but it's the same as `Class` so nothing new. Users
should only initialize stuff in `setUp`/`setUpClass` and similar
methods, and not in `__init__` which is generally off-limits in
`TestCase` subclasses.
I am not sure why there was a difference in the first place, though I
will say the previous unittest approach is probably the preferable one,
but first let's get consistency.
The check for short paths under Windows via os.path.samefile, introduced in #11936, also found similar tests in symlinked tests in the GH Actions CI.
Fixes#12039.
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Previously, if more than one fixture finalizer raised, only the first
was reported, and the other errors were lost.
Use an exception group to report them all. This is similar to the change
we made in node teardowns (in `SetupState`).
As detailed in
https://github.com/pytest-dev/pytest/issues/11475#issuecomment-1937043670,
currently with `--import-mode=importlib` pytest will try to import every
file by using a unique module name, regardless if that module could be
imported using the normal import mechanism without touching `sys.path`.
This has the consequence that non-test modules available in `sys.path`
(via other mechanism, such as being installed into a virtualenv,
PYTHONPATH, etc) would end up being imported as standalone modules,
instead of imported with their expected module names.
To illustrate:
```
.env/
lib/
site-packages/
anndata/
core.py
```
Given `anndata` is installed into the virtual environment, `python -c
"import anndata.core"` works, but pytest with `importlib` mode would
import that module as a standalone module named
`".env.lib.site-packages.anndata.core"`, because importlib module was
designed to import test files which are not reachable from `sys.path`,
but now it is clear that normal modules should be imported using the
standard mechanisms if possible.
Now `imporlib` mode will first try to import the module normally,
without changing `sys.path`, and if that fails it falls back to
importing the module as a standalone module.
This also makes `importlib` respect namespace packages.
This supersedes #11931.
Fix#11475Close#11931
(diff better viewed ignoring whitespace)
In pytest<8, the collection tree for `pyargs` arguments in an invocation
like this:
pytest --collect-only --pyargs pyflakes.test.test_undefined_names
looked like this:
```
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
The pytest 8 collection improvements changed it to this:
```
<Dir pytest>
<Dir .tox>
<Dir venv>
<Dir lib>
<Dir python3.11>
<Dir site-packages>
<Package pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Besides being egregious (and potentially even worse than the above,
going all the way to the root, for system-installed packages, as is
apparently common in CI), this also caused permission errors when trying
to probe some of those intermediate directories.
This change makes `--pyargs` arguments no longer try to add parent
directories to the collection tree according to the `--confcutdir` like
they're regular arguments. Instead, only add the parents that are in the
import path. This now looks like this:
```
<Package .tox/venv/lib/python3.11/site-packages/pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Fix#11904.
pytest allows a fixture to request its own name (directly or
indirectly), in which case the fixture with the same name but one level
up is used.
To know which fixture should be used next, pytest keeps a mutable
item-global dict `_arg2index` which maintains this state. This is not
great:
- Mutable state like this is hard to understand and reason about.
- It is conceptually buggy; the indexing is global (e.g. if requesting
`fix1` and `fix2`, the indexing is shared between them), but actually
different branches of the subrequest tree should not affect each
other.
This is not an issue in practice because pytest keeps a cache of the
fixturedefs it resolved anyway (`_fixture_defs`), but if the cache is
removed it becomes evident.
Instead of the `_arg2index` state, count how many `argname`s deep we are
in the subrequest tree ("the fixture stack") and use that for the index.
This way, no global mutable state and the logic is very localized and
easier to understand.
This is slower, however fixture stacks should not be so deep that this
matters much, I hope.
According to my understanding, this code, which handles obtaining the
relevant fixturedefs when a dynamic `getfixturevalue` is used, has an
optimization where it only grabs fixturedefs that are visible to the
*parent* of the item, instead of the item itself, under the assumption
that a fixturedef can't be visible to a single item, only to a
collector.
Remove this optimization for the following reasons:
- It doesn't save much (one loop iteration in `matchfactories`)
- It slightly complicates the complex fixtures code
- If some plugin wants to make a fixture visible only to a single item,
why not let it?
- In the static case (`getfixtureclosure`), this optimization is not
done (despite the confusing name `parentnode`, it is *not* the parent
node). This is inconsistent.
Allow for the output of test case execution to be controlled independently from the application verbosity level.
`verbosity_test_case` is the new ini setting to adjust this functionality.
Fix#11639
Passing a short path in the command line was causing the matchparts check to fail, because ``Path(short_path) != Path(long_path)``.
Using ``os.path.samefile`` as fallback ensures the comparsion works on Windows when comparing short/long paths.
Fix#11895
In `CallSpec2.setmulti` the `params` and `_arg2scope` dicts are always
set together.
Further, the `get_parametrized_fixture_keys` accesses `_arg2scope` for
all argnames without a check, which I think rules out that the code
protects against plugin shenanigans.
After removing the suppress, adjust the comment and code to make more
sense.
Fix#12021.
Reopens#11706.
This reverts commit 12b9bd5801.
This change caused a bad regression in pytest-xdist:
https://github.com/pytest-dev/pytest-xdist/issues/1024
pytest-xdist necessarily has special handling of `--maxfail` and session
fixture teardown get executed multiple times with the change.
Since I'm not sure how to adapt pytest-xdist myself, revert for now.
I kept the sticky `shouldstop`/`shouldfail` changes as they are good
ideas regardless I think.
Dicts these days preserve order, so the sort is no longer needed to
achieve determinism.
As shown by the `test_dynamic_parametrized_ordering` test, this can
change the ordering of items, but only in equivalent ways (same number
of setups/teardowns per scope), it will just respect the user's given
ordering better (hence `vxlan` items now ordered before `vlan` items
compared to the previous ordering).
Follow up to #12006, let's put some comments clarifying `is_in_confcutdir` semantics, as this is not the first time someone misunderstands it.
Also removed an obsolete comment in `_loadconftestmodules`: we already set the `confcutdir` based on `rootdir`/`initfile` if not explicitly given.
Co-authored-by: Ran Benita <ran@unusedvar.com>
As the comment above it states, we need to keep the comparison simple so mypy can understand it, otherwise it will fail to properly handle this on Windows and will flag ``os.getuid()`` missing.
Test:
`warnings.warn()` expects that its first argument is a `str` or a
`Warning`, but since 9454fc38d3
`pytest.warns()` no longer allows `Warning` instances unless the first
argument the `Warning` was initialized with is a `str`. Furthermore, if
the `Warning` was created without arguments then `pytest.warns()` raises
an unexpected `IndexError`. The new tests reveal the problem.
Fix:
`pytest.warns()` now allows using `warnings.warn()` with a `Warning`
instance, as is required by Python, with one exception. If the warning
used is a `UserWarning` that was created by passing it arguments and the
first argument was not a `str` then `pytest.raises()` still considers
that an error. This is because if an invalid type was used in
`warnings.warn()` then Python creates a `UserWarning` anyways and it
becomes impossible for `pytest` to figure out if that was done
automatically or not.
[ran: rebased on previous commit]
Today `pyproject.toml` is the standard for declaring a Python project root, so seems reasonable to consider it for the ini configuration (and specially `rootdir`) in case we do not find other suitable candidates.
Related to #11311
Previously this would trigger an `AssertionError`.
While it could be considered a bug-fix, but given it now can be relied upon, it is probably better to consider it an improvement.
Fix#11311
Should be around 40% faster according to this simplified small benchmark:
python -m timeit "a=[0, 1, 2, 3, 4];b=list((e if i in {0, len(a) -1} else str(e)) for i, e in enumerate(a))"
200000 loops, best of 5: 1.12 usec per loop
python -m timeit "a=[0, 1, 2, 3, 4];b=list((a[0], *(str(e) for e in a[1:-1]), a[-1]))"
500000 loops, best of 5: 651 nsec per loop
python -m timeit "a=[0, 1, 2, 3, 4,5,6,7,8,9,10,11,12,13,14,15,16];b=list((e if i in {0, len(a) -1} else str(e)) for i, e in enumerate(a))"
100000 loops, best of 5: 3.31 usec per loop
python -m timeit "a=[0, 1, 2, 3, 4,5,6,7,8,9,10,11,12,13,14,15,16];b=list((a[0], *(str(e) for e in a[1:-1]), a[-1]))"
200000 loops, best of 5: 1.72 usec per loop
* Improve error message when using @pytest.fixture twice
While obvious in hindsight, this error message confused me. I thought my fixture
function was used in a test function twice, since the wording is ambiguous.
Also, the error does not tell me *which* function is the culprit.
Finally, this adds a test, which wasn't done in
cfd16d0dac where this was originally implemented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
warning: The top-level linter settings are deprecated in favour of their counterparts in the `lint` section. Please update the following options in `pyproject.toml`:
- 'ignore' -> 'lint.ignore'
- 'select' -> 'lint.select'
Fix#11929.
Figured out what's going on. We have the following collection tree:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
```
And the `test_main.py` contains an autouse fixture (`fake_game_ui`) that
`doctest_main` needs in order to run properly. The fixture doesn't run!
It doesn't run because nothing collects the fixtures from (calls
`parsefactories()` on) the `test_main.py` `DoctestModule`.
How come it only started happening with commit
ab63ebb3dc07b89670b96ae97044f48406c44fa0? Turns out it mostly only
worked accidentally. Each `DoctestModule` is also collected as a normal
`Module`, with the `Module` collected after the `DoctestModule`. For
example, if we add a non-doctest test to `test_main.py`, the collection
tree looks like this:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
<Module test_main.py>
<Function test_it>
```
Now, `Module` *does* collect fixtures. When autouse fixtures are
collected, they are added to the `_nodeid_autousenames` dict.
Before ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *setup* time. At this point, the `Module` has
collected and so it ended up picking the autouse fixture (this relies on
another "accident", that the `DoctestModule` and `Module` have the same
node ID).
After ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *collection* time (= when it's created). At
this point, the `Module` hasn't collected yet, so the autouse fixture is
not picked out.
The fix is simple -- have `DoctestModule.collect()` call
`parsefactories`. From some testing I've done it shouldn't have negative
consequences (I hope).
Currently, `DoctestModule` does `import_path` on its own. This changes
it to use `importtestmodule` as `Module` does. The behavioral changes
are:
- Much better error messages on import errors.
- Handles a few more error cases (see `importtestmodule`). This
technically expands the cover of `--doctest-ignore-import-errors` but
I think it makes sense.
- Considers `pytest_plugins` in the module.
- Populates `self.obj` as properly (without double-imports) as is
expected from a `PyCollector`.
This is also needed for the next commit.
ruff is faster and handle everything we had prior.
isort configuration done based on the indication from
https://github.com/astral-sh/ruff/issues/4670, previousely based on
reorder-python-import (#11896)
flake8-docstrings was a wrapper around pydocstyle (now archived) that
explicitly asks to use ruff in https://github.com/PyCQA/pydocstyle/pull/658.
flake8-typing-import is useful mainly for project that support python 3.7
and the one useful check will be implemented in https://github.com/astral-sh/ruff/issues/2302
We need to keep blacken-doc because ruff does not handle detection
of python code inside .md and .rst. The direct link to the repo is
now used to avoid a redirection.
Manual fixes:
- Lines that became too long
- % formatting that was not done automatically
- type: ignore that were moved around
- noqa of hard to fix issues (UP031 generally)
- fmt: off and fmt: on that is not really identical
between black and ruff
- autofix re-order in pre-commit from faster to slower
Co-authored-by: Ran Benita <ran@unusedvar.com>
The current version (0.23.4) explicitly does not support pytest 8 yet, so we fallback to the previous release in the hope that at least our integration tests pass.
- Previously, `getpass.getuser()` would leak an ImportError if the
USERNAME environment variable was not set on Windows because the `pwd`
module cannot be imported.
- Starting in Python 3.13.0a3, it only raises `OSError`.
Fixes#11874
Change our mypy configuration to disallow untyped defs by default, which ensures *new* files added to the code base are fully typed.
To avoid having to type-annotate everything now, add `# mypy: allow-untyped-defs` to files which are not fully type annotated yet.
As we fully type annotate those modules, we can then just remove that directive from the top.
The function currently uses `find_suffixes` which iterates the entire
directory searching for files with the given suffix. In some cases
though, like in pytest's selftest, the directory can get big:
$ ls /tmp/pytest-of-ran/pytest-0/
7686
and iterating it many times can get slow.
This doesn't fix the underlying issue (iterating the directory) but at
least speeds it up a bit by using `os.scandir` instead of
`path.iterdir`. So `make_numbered_dir` is still slow for pytest's
selftests, but reduces ~10s for me.
Follow up to https://github.com/pytest-dev/pytest/pull/11754, noticed that the latest GitHub release does not contain the attached files.
Output log from the action:
```
🤔 Pattern 'dist/*' does not match any files.
```
We have a use case for this in the next commit.
The name can be obtained by using `manager.get_name(plugin)`, however
this is currently O(num plugins) in pluggy, which would be good to
avoid. Besides, it seems generally useful.
This removes one thing that directory collectors need to worry about.
This adds one hook dispatch per `__pycache__` file, but I think it's
worth it for consistency.
(Diff better viewed ignoring whitespace)
Since e1c66ab0ad, conftest loading is
handled at the directory level before sub-nodes are collected, so there
is no need for the doctest plugin to handle it specially.
This was probably the case even before
e1c66ab0ad, but I haven't verified this.
This is a useful addition to the existing `listchain`. While `listchain`
returns top-to-bottom, `iterparents` is bottom-to-top and doesn't require
an internal full iteration + `reverse`.
Since we're removing nose support, let's also drop support for this
attribute.
From doing a code search on github, this seems completely unused outside
of nose, except for some projects which used to use it, but no longer
do.
Refs #11662.
--- Problem
Each fixture definition has a "visibility", the `FixtureDef.baseid`
attribute. This is nodeid-like string. When a certain `node` requests a
certain fixture name, we match node's nodeid against the fixture
definitions with this name.
The matching currently happens on the *textual* representation of the
nodeid - we split `node.nodeid` to its "parent nodeids" and then check
if the fixture's `baseid` is in there.
While this has worked so far, we really should try to avoid textual
manipulation of nodeids as much as possible. It has also caused problem
in an odd case of a `Package` in the root directory: the `Package` gets
nodeid `.`, while a `Module` in it gets nodeid `test_module.py`. And
textually, `.` is not a parent of `test_module.py`.
--- Solution
Avoid this entirely by just checking the node hierarchy itself. This is
made possible by the fact that we now have proper `Directory` nodes
(`Dir` or `Package`) for the entire hierarchy.
Also do the same for `_getautousenames` which is a similar deal.
The `iterparentnodeids` function is no longer used and is removed.
Instead of modifying user objects like modules and classes that we
really shouldn't be touching, use the new `_register_fixture` internal
API to do it directly.
Add a function on the `FixtureManager` to register a fixture with
pytest. Currently this can only be done through `parsefactories`.
My aim is to eventually make something like this available to plugins,
as it's a pretty common need.
* Fix handling empty values of NO_COLOR and FORCE_COLOR
Fix handling NO_COLOR and FORCE_COLOR environment variables to correctly
be ignored when they are set to an empty value, as defined
in the specification:
> Command-line software which adds ANSI color to its output by default
> should check for a NO_COLOR environment variable that, when present
> *and not an empty string* (regardless of its value), prevents
> the addition of ANSI color.
(emphasis mine, https://no-color.org/)
The same is true of FORCE_COLOR, https://force-color.org/.
* Streamline testing for FORCE_COLOR and NO_COLOR
Streamline the tests for FORCE_COLOR and NO_COLOR variables, and cover
all possible cases (unset, set to empty, set to "1"). Combine the two
assert functions into one taking boolean parameters. Mock file.isatty
in all circumstances to ensure that the environment variables take
precedence over the fallback value resulting from isatty check (or that
the fallback is actually used, in the case of both FORCE_COLOR
and NO_COLOR being unset).
* Put a 'reset' color in front of the highlighting
When doing the highlighting, some lexers will not set the initial color
explicitly, which may lead to the red from the errors being propagated
to the start of the expression
* Add syntactic highlighting to the error explanations
This updates the various error reporting to highlight python code when
displayed, to increase readability and make it easier to understand
--- Current main
In current main (before pervious commit), calls to gethookproxy/ihook
are the trigger for loading non-initial conftests. This basically means
that conftest loading is done almost as a random side-effect,
uncontrolled and very non-obvious. And it also dashes any hope of making
gethookproxy faster (gethookproxy shows up prominently in pytest
profiles).
I've wanted to improve this for a while, #11268 was the latest step
towards that.
--- PR before change
In this PR, I ran into a problem.
Previously, Session and Package did all of the directory traversals
inside of their collect, which loaded the conftests as a side effect. If
the conftest loading failed, it will occur inside of the collect() and
cause it to be reported as a failure.
Now I've changed things such that Session.collect and Package.collect no
longer recurse, but just collect their immediate descendants, and
genitems does the recursive expansion work.
The problem though is that genitems() doesn't run inside of specific
collector's collect context. So when it loads a conftest, and the
conftest loading fails, the exception isn't handled by any CollectReport
and causes an internal error instead.
The way I've fixed this problem is by loading the conftests eagerly in a
pytest_collect_directory post-wrapper, but only during genitems to make
sure the directory is actually selected.
This solution in turn caused the conftests to be collected too early;
specifically, the plugins are loaded during the parent's collect(), one
after the other as the directory entries are collected. So when the
ihook is hoisted out of the loop, new plugins are loaded inside the
loop, and due to the way the hook proxy works, they are added to the
ihook even though they're supposed to be scoped to the child collectors.
So no hoisting.
--- PR after change
Now I've come up with a better solution: since now the collection tree
actually reflects the filesystem tree, what we really want is to load
the conftest of a directory right before we run its collect(). A
conftest can affect a directory's collect() (e.g. with a
pytest_ignore_collect hookimpl), but it cannot affect how the directory
node itself is collected. So I just moved the conftest loading to be
done right before calling collect, but still inside the CollectReport
context.
This allows the hoisting, and also removes conftest loading from
gethookproxy since it's no longer necessary. And it will probably enable
further cleanups. So I'm happy with it.
This adds a GHA job that reliably determines if all the required
dependencies have succeeded or not.
It also allows to reduce the list of required branch protection CI
statuses to just one — `check`. This reduces the maintenance burden
by a lot and have been battle-tested across a small bunch of projects
in its action form and in-house implementations of other people.
I was curious about the spread of use. And I've just discovered that
it is now in use in aiohttp (and other aio-libs projects), CherryPy,
some of the Ansible repositories, all of the jaraco's projects (like
`setuptools`, `importlib_metadata`), some PyCQA, PyCA and pytest
projects, a few AWS Labs projects. Admittedly, I maintain a few of
these but it seems to address some of the pain folks have:
https://github.com/jaraco/skeleton/pull/55#issuecomment-1106638475.
The story behind this is explained in more detail at
https://github.com/marketplace/actions/alls-green#why.
Previously the error report would have all sections glued together:
- The assertion representation
- The error explanation
- The full diff
This makes it hard to see at a glance where which one starts and ends.
One of the representation (dataclasses, tuples, attrs) does display a
newlines at the start already.
Let's add a newlines before the error explanation and before the full
diff, so we get an easier to read report.
This has one disadvantage: we get one line less in the least verbose
mode, where the output gets truncated.
The default for `_prepareconfig` is to use `sys.argv`, which in this
case are the flags passed to (top-level) `pytest`. This is not the
intention, the tests themselves should not be affected by it.
When running `tox -e py-lsof` I get a deluge of this warning:
```
src/pytest/.tox/py-lsof-numpy-pexpect/lib/python3.11/site-packages/_pytest/pytester.py:130: EncodingWarning: UTF-8 Mode affects locale.getpreferredencoding(). Consider locale.getencoding() instead.
```
Use `locale.getencoding` instead.
The normal default pretty printer is not great when objects are nested
and it can get hard to read the diff.
Instead, provide a pretty printer that behaves more like when json get
indented, which allows for smaller, more meaningful differences, at
the expense of a slightly longer diff.
This does not touch the other places where the pretty printer is used,
and only updated the full diff one.
Reset color-related environment variables in a fixture to prevent them
from affecting test results. Otherwise, some of the tests fail
e.g. if NO_COLOR is set in the calling environment.
We already have the AlwaysDispatchingPrettyPrinter override of the
default pretty printer. In order to make more in depth changes, we
need to copy the upstream version in, as it doesn't lend itself well to
being extended.
This does a verbatime copy, adding provenance information at the top.
if we were ever to change the `name:` field in `test.yml` this would suddenly break -- this references the workflow file directly (and is what github generates now for badges)
The tested py.path.local.make_numbered_dir function is *not*
multiprocess safe, because is uses os.listdir which itself is not.
The os.listdir documentation explicitly states that:
> If a file is removed from or added to the directory during the call
> of this function, whether a name for that file be included is unspecified.
This can lead to a race when:
1. process A attempts to create directory N
2. the creation fails, as another process already created it in the meantime
3. process A calls listdir to determine a more recent maxnum
4. processes B+ repeatedly create newer directories and they delete directory N
5. process A doesn't have directory N or any newer directory in listdir result
6. process A attempts to create directory N again and raises
For details, see https://github.com/pytest-dev/pytest/issues/11603#issuecomment-1805708144
and bellow.
Additionally, the test itself has a race in batch_make_numbered_dirs.
When this functions attempts to write to repro-N/foo,
repro-N may have already been removed by another process.
For details, see https://github.com/pytest-dev/pytest/issues/11603#issuecomment-1804714313
and bellow.
---
The tested py.path.local.make_numbered_dir function is not used in pytest.
There is a different implementation in _pytest.pathlib.
Closes#11603
Previously, the sidebar TOC had unlimited depth, making it useless and
interfering with the content. This seems to have regressed in pytest
7.2.x going by the RTD version selector.
Traditionally pytest publishes tags in the form `X.Y.Z`, however the deploy workflow (copied from somewhere else) published tags in the form `vX.Y.Z`.
This is the root cause of #11548, because it tried to publish the release notes for tag `X.Y.Z` (which did not exist).
Fix#11548
Logging has many global states, and we did foresee this by creating a ``cleanup_disabled_logging`` fixture,
however one might still forget to use it and failures leak later -- sometimes not even in the same PR, because the order
of the tests might change in the future, specially when running under xdist.
This problem surfaced during pytest-dev/pytest#11530, where tests unrelated to the change started to fail.
The current method as the following problem, described by Sadra
Barikbin:
The tests that request both `pytester` and `monkeypatch` and use
`monkeypatch.chdir` without context, relying on `monkeypatch`'s teardown
to restore cwd. This doesn't work because the following sequence of
actions take place:
- `monkeypatch` is set up.
- `pytester` is set up. It saves the original cwd and changes it to a
new one dedicated to the test function.
- Test function calls `monkeypatch.chdir()` without context.
`monkeypatch` saves cwd, which is not the original one, before
changing it.
- `pytester` is torn down. It restores the cwd to the original one.
- `monkeypatch` is torn down. It restores cwd to what it has saved.
The solution here is to have pytester use `monkeypatch.chdir()` itself,
then everything is handled correctly.
Tests if a captured exception group contains an expected exception.
Will raise `AssertionError` if the wrapped exception is not an exception group.
Supports recursive search into nested exception groups.
Starting with `resolve_package_path` and its associated tests,
this refactoring seeks to make path concatenation more
readable and consistent within tests/functions.
As discussed in #11413:
- code is free to use either `/` and `joinpath`
- consistency within a function is more important than consistency across the codebase
- it is nice to use `/` when it is more readable
- it is nice to use `joinpath` when there is little context
- be mindful that `joinpath` may be clearer when joining multiple segments
There used to be two callers to `_setup_fixtures()`, now there's only
one, so inline it and make `DoctestItem` more similar to `Function`.
(Eventually we may want to generalize `TopRequest` from taking
`Function` directly to some "fixture-supporting item", removing the
remaining `type: ignore` here and allowing plugins to do it in a stable
manner).
As discussed in #11408:
* Improve documentation for the release process.
* Fix the description for the PRs created by the `prepare release pr` workflow.
* Fix pushing tag in the `deploy` workflow.
For packages, `import_path` receives the path to the package's `__init__.py` file, however module names (as they live in `sys.modules`) should not include the `__init__` part.
For example, `app/core/__init__.py` should be imported as `app.core`, not as `app.core.__init__`.
Fix#11306
I could find 2 plugins that would be broken by this (pytest-play and
pytest-wdl), but they will be better served by just copying
`_fillfixtures` instead of use the private function.
`_pyfuncitem.fixturenames` is just an alias for
`_pyfuncitem._fixtureinfo.names_closure` (at least in core pytest), so
let's do the less abstraction-breaking thing.
Move handling of user_properties to `finalize()`.
Previously if a fixture failed during teardown, `pytest_runtest_logreport` would not be called with "teardown", resulting in the user properties not being saved on the JUnit XML file.
Fixes: #11367
* Build the package only once, and test on all platforms.
* Deploy is now triggered manually via an Action, which is then responsible for tagging the repository after the package has been uploaded successfully.
* Drop 'docs': we nowadays rely on readthedocs preview PR builds.
This reverts commit e938580257.
Revert "improve error msg and test"
This reverts commit c0cf822ca1.
Revert "error msg"
This reverts commit ec1053cc16.
Revert "changelog"
This reverts commit d2dc8a70b5.
Revert "simplify code / take out user-gen typeerror case"
This reverts commit b9cb87d862.
Setting `norecursedirs` overrides the default, so we end up scanning
dot-directories and such which slows down collection unnecessarily
(150ms on my working directory).
To remove fixtures.py::add_funcargs_pseudo_fixture_def and add its logic
i.e. registering funcargs as params and making corresponding fixturedefs,
right to Metafunc.parametrize in which parametrization takes place.
To remove funcargs from metafunc attributes as we populate metafunc
params and make pseudo fixturedefs simultaneously and there's no need to
keep funcargs separately.
Previously, when assigning a scope for a fully-indirect parameter set,
when there are multiple fixturedefs for a param (i.e. same-name fixture
chain), the highest scope was used, but it should be the lowest scope,
since that's the effective scope of the fixture.
What one typically actually wants in such a case is both, checking for some
resulting values *and* checking for some expected exception.
Since this is easily possible with the `nullcontext` context manager, adapt the
example accordingly (needlessly using a different name rather just confuses people).
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
Previously, the `_getconftestmodules` function was used both to load
conftest modules for a path (during `pytest_load_initial_conftests`),
and to retrieve conftest modules for a path (during hook dispatch and
for fetching `collect_ignore`). This made things muddy - it is usually
nicer to have clear separation between "command" and "query" functions,
when they occur in separate phases.
So split into "load" and "get".
Currently, `gethookproxy` still loads conftest itself. I hope to change
this in the future.
this change
* switches from html index usage to json index usage
* adds using requests-cache as http cache to trim down runtime
* adds gh action caches for pip and the new http cache
This test makes clear the need for the `_check_scope()` call in the
`pytest_setup_fixture` impl in fixtures.py, which otherwise seems
redundant with the one in `_compute_fixture_value`.
FixtureDef is used in the `pytest_fixture_setup` hook so needs to be
public. However since its current internals are quite dubious (and not
all marked with `_` prefix) I've added an explicit note that only
documented fields/methods are considered public.
Refs #7469.
From understanding the code better I see this is the correct fix.
The fixturedefs can be None if `request.getfixturevalue("doesnotexist")`
is used.
In practice there is no change in behavior because this mapping is used
as `self._arg2fixturedefs.get(argname, None)` which ends up the same.
It makes for a more debuggable repr. Before:
<function FixtureRequest._schedule_finalizers.<locals>.<lambda> at 0x7fe4ae32d440>
After:
functools.partial(<bound method FixtureDef.finish of <FixtureDef argname='hello_package' scope='package' baseid=''>>, request=<SubRequest 'hello_package' for <Function test_hello>>)
Add explicit mentions of the default behavior to both API reference and
how-to section about ways to invoke pytest.
Co-authored-by: Ran Benita <ran@unusedvar.com>
Dict comparsion in the ApproxMapping class did not check if values were None before attempting to subtract for max_abs_diff stat, which was throwing an TypeError instead of being handled by pytest error assertion. Check for None has been added before these calculations, so that None will properly show as Obtained/Expected in pytest assert message
The initial implementation (in #7246) introduced the `importlib` mode, which
never added the imported module to `sys.modules`, so it included a test
to ensure calling `import_path` twice would yield different modules.
Not adding modules to `sys.modules` proved problematic, so we began to add the imported module to `sys.modules`
in #7870, but failed to realize that given we are now changing `sys.modules`, we might
as well avoid importing it more than once.
Then #10088 came along, passing `importlib` also when importing application modules
(as opposed to only test modules before), which caused problems due to imports
having side-effects and the expectation being that they are imported only once.
With this PR, `import_path` returns the module immediately if already in
`sys.modules`.
Fix#10811Fix#10341
I think it's helpful to separate the node classes from the other
objects, as they have their own unique usage.
I've chosen not to alphabetize the order, but to use a logical order
instead.
Also slightly improve the docstrings.
stale-issue-message:"This issue is stale because it has been open for 14 days with no activity."
close-issue-message:"This issue was closed because it has been inactive for 7 days since being marked as stale."
stale-issue-message:"This issue is stale because it has the `status: needs information` label and requested follow-up information was not provided for 14 days."
close-issue-message:"This issue was closed because it has the `status: needs information` label and follow-up information has not been provided for 7 days since being marked as stale."
@@ -272,35 +270,35 @@ Here is a simple overview, with pytest-specific bits:
#. Run all the tests
You need to have Python 3.7 available in your system. Now
You need to have Python 3.8 or later available in your system. Now
running tests is as simple as issuing this command::
$ tox -e linting,py37
$ tox -e linting,py39
This command will run tests via the "tox" tool against Python 3.7
This command will run tests via the "tox" tool against Python 3.9
and also perform "lint" coding-style checks.
#. You can now edit your local working copy and run the tests again as necessary. Please follow PEP-8 for naming.
#. You can now edit your local working copy and run the tests again as necessary. Please follow `PEP-8<https://www.python.org/dev/peps/pep-0008/>`_ for naming.
You can pass different options to ``tox``. For example, to run tests on Python 3.7 and pass options to pytest
You can pass different options to ``tox``. For example, to run tests on Python 3.9 and pass options to pytest
(e.g. enter pdb on failure) to pytest you can do::
$ tox -e py37 -- --pdb
$ tox -e py39 -- --pdb
Or to only run tests in a particular test module on Python 3.7::
Or to only run tests in a particular test module on Python 3.9::
$ tox -e py37 -- testing/test_config.py
$ tox -e py39 -- testing/test_config.py
When committing, ``pre-commit`` will re-format the files if necessary.
#. If instead of using ``tox`` you prefer to run the tests directly, then we suggest to create a virtual environment and use
an editable install with the ``testing`` extra::
an editable install with the ``dev`` extra::
$ python3 -m venv .venv
$ source .venv/bin/activate # Linux
$ .venv/Scripts/activate.bat # Windows
$ pip install -e ".[testing]"
$ pip install -e ".[dev]"
Afterwards, you can edit the files and run pytest normally::
Added `--xfail-tb` flag, which turns on traceback output for XFAIL results.
* If the `--xfail-tb` flag is not sent, tracebacks for XFAIL results are NOT shown.
* The style of traceback for XFAIL is set with `--tb`, and can be `auto|long|short|line|native|no`.
* Note: Even if you have `--xfail-tb` set, you won't see them if `--tb=no`.
Some history:
With pytest 8.0, `-rx` or `-ra` would not only turn on summary reports for xfail, but also report the tracebacks for xfail results. This caused issues with some projects that utilize xfail, but don't want to see all of the xfail tracebacks.
This change detaches xfail tracebacks from `-rx`, and now we turn on xfail tracebacks with `--xfail-tb`. With this, the default `-rx`/ `-ra` behavior is identical to pre-8.0 with respect to xfail tracebacks. While this is a behavior change, it brings default behavior back to pre-8.0.0 behavior, which ultimately was considered the better course of action.
Fix crash with `assert testcase is not None` assertion failure when re-running unittest tests using plugins like pytest-rerunfailures. Regressed in 8.2.2.
Fixed a crash when returning category ``"error"`` or ``"failed"`` with a custom test status from :hook:`pytest_report_teststatus` hook -- :user:`pbrezina`.
There can be many different definitions of "success". Pytest can run many nose_ and unittest_ tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right?
There can be many different definitions of "success". Pytest can run many unittest_ tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right?
It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies.
pytest is actively evolving and is a project that has been decades in the making,
we keep learning about new and better structures to express different details about testing.
Pytest is an actively evolving project that has been decades in the making.
We keep learning about new and better structures to express different details about testing.
While we implement those modifications we try to ensure an easy transition and don't want to impose unnecessary churn on our users and community/plugin authors.
While we implement those modifications, we try to ensure an easy transition and don't want to impose unnecessary churn on our users and community/plugin authors.
As of now, pytest considers multiple types of backward compatibility transitions:
a) trivial: APIs which trivially translate to the new mechanism,
and do not cause problematic changes.
a) trivial: APIs that trivially translate to the new mechanism and do not cause problematic changes.
We try to support those indefinitely while encouraging users to switch to newer/better mechanisms through documentation.
We try to support those indefinitely while encouraging users to switch to newer or better mechanisms through documentation.
b) transitional: the old and new API don't conflict
and we can help users transition by using warnings, while supporting both for a prolonged time.
b) transitional: the old and new APIs don't conflict, and we can help users transition by using warnings while supporting both for a prolonged period of time.
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
We will only start the removal of deprecated functionality in major releases (e.g., if we deprecate something in 3.0, we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g., if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationwarning`).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationWarning`).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g. `PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
When the deprecation expires (e.g., 4.0 is released), we won't remove the deprecated functionality immediately but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g.,`PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g., 4.1), the feature will be effectively removed.
c) true breakage: should only be considered when normal transition is unreasonably unsustainable and would offset important development/features by years.
In addition, they should be limited to APIs where the number of actual users is very small (for example only impacting some plugins), and can be coordinated with the community in advance.
c) True breakage should only be considered when a normal transition is unreasonably unsustainable and would offset important developments or features by years. In addition, they should be limited to APIs where the number of actual users is very small (for example, only impacting some plugins) and can be coordinated with the community in advance.
Keeping backwards compatibility has a very high priority in the pytest project. Although we have deprecated functionality over the years, most of it is still supported. All deprecations in pytest were done because simpler or more efficient ways of accomplishing the same tasks have emerged, making the old way of doing things unnecessary.
With the pytest 3.0 release we introduced a clear communication scheme for when we will actually remove the old busted joint and politely ask you to use the new hotness instead, while giving you enough time to adjust your tests or raise concerns if there are valid reasons to keep deprecated functionality around.
With the pytest 3.0 release, we introduced a clear communication scheme for when we will actually remove the old busted joint and politely ask you to use the new hotness instead, while giving you enough time to adjust your tests or raise concerns if there are valid reasons to keep deprecated functionality around.
To communicate changes we issue deprecation warnings using a custom warning hierarchy (see :ref:`internal-warnings`). These warnings may be suppressed using the standard means: ``-W`` command-line flag or ``filterwarnings`` ini options (see :ref:`warnings`), but we suggest to use these sparingly and temporarily, and heed the warnings when possible.
To communicate changes, we issue deprecation warnings using a custom warning hierarchy (see :ref:`internal-warnings`). These warnings may be suppressed using the standard means: ``-W`` command-line flag or ``filterwarnings`` ini options (see :ref:`warnings`), but we suggest to use these sparingly and temporarily, and heed the warnings when possible.
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0, we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn them into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
@@ -87,6 +83,7 @@ Released pytest versions support all Python versions that are actively maintaine
Diamond inheritance between :class:`pytest.Collector` and :class:`pytest.Item`
@@ -336,7 +184,7 @@ Diamond inheritance between :class:`pytest.Collector` and :class:`pytest.Item`
..deprecated:: 7.0
Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
Defining a custom pytest node type which is both an :class:`~pytest.Item` and a :class:`~pytest.Collector` (e.g. :class:`~pytest.File`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
Some plugins providing linting/code analysis have been using this as a hack.
@@ -348,8 +196,8 @@ Instead, a separate collector node should be used, which collects the item. See
.._uncooperative-constructors-deprecated:
Constructors of custom :class:`pytest.Node` subclasses should take ``**kwargs``
@@ -380,28 +228,22 @@ conflicts (such as :class:`pytest.File` now taking ``path`` instead of
``fspath``, as :ref:`outlined above <node-ctor-fspath-deprecation>`), a
deprecation warning is now raised.
Backward compatibilities in ``Parser.addoption``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Applying a mark to a fixture function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated::2.4
..deprecated::7.4
Several behaviors of :meth:`Parser.addoption <pytest.Parser.addoption>` are now
scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):
Applying a mark to a fixture function never had any effect, but it is a common user error.
-``parser.addoption(..., help=".. %default ..")`` - use ``%(default)s`` instead.
-``parser.addoption(..., type="int/string/float/complex")`` - use ``type=int`` etc. instead.
..code-block::python
@pytest.mark.usefixtures("clean_database")
@pytest.fixture
defuser()->User:...
Using ``pytest.warns(None)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Users expected in this case that the ``usefixtures`` mark would have its intended effect of using the ``clean_database`` fixture when ``user`` was invoked, when in fact it has no effect at all.
..deprecated:: 7.0
:func:`pytest.warns(None) <pytest.warns>` is now deprecated because it was frequently misused.
Its correct usage was checking that the code emits at least one warning of any type - like ``pytest.warns()``
or ``pytest.warns(Warning)``.
See :ref:`warns use cases` for examples.
Now pytest will issue a warning when it encounters this problem, and will raise an error in the future versions.
Returning non-None value in test functions
@@ -444,19 +286,6 @@ The proper fix is to change the `return` to an `assert`:
assertfoo(a,b)==result
The ``--strict`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.2
The ``--strict`` command-line option has been deprecated in favor of ``--strict-markers``, which
better conveys what the option does.
We have plans to maybe in the future to reintroduce ``--strict`` and make it an encompassing
flag for all strictness related options (``--strict-markers`` and ``--strict-config``
at the moment, more might be introduced in the future).
The ``yield_fixture`` function/decorator
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -467,12 +296,348 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
Removed Features
----------------
Removed Features and Breaking Changes
-------------------------------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
Some breaking changes which could not be deprecated are also listed.
.._nose-deprecation:
Support for tests written for nose
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.2
..versionremoved:: 8.0
Support for running tests written for `nose <https://nose.readthedocs.io/en/latest/>`__ is now deprecated.
``nose`` has been in maintenance mode-only for years, and maintaining the plugin is not trivial as it spills
over the code base (see :issue:`9886` for more details).
setup/teardown
^^^^^^^^^^^^^^
One thing that might catch users by surprise is that plain ``setup`` and ``teardown`` methods are not pytest native,
they are in fact part of the ``nose`` support.
..code-block::python
classTest:
defsetup(self):
self.resource=make_resource()
defteardown(self):
self.resource.close()
deftest_foo(self):...
deftest_bar(self):...
Native pytest support uses ``setup_method`` and ``teardown_method`` (see :ref:`xunit-method-setup`), so the above should be changed to:
..code-block::python
classTest:
defsetup_method(self):
self.resource=make_resource()
defteardown_method(self):
self.resource.close()
deftest_foo(self):...
deftest_bar(self):...
This is easy to do in an entire code base by doing a simple find/replace.
@with_setup
^^^^^^^^^^^
Code using `@with_setup <with-setup-nose>`_ such as this:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
