Tags from the main repository are required for testing to work properly, but the current documentation does not reflect this, and at least one new contributor has been tripped up by this. Update the documentation with a short explanation and directions.
mypy stubs recently changed warnings.catch_warnings to a Generic, in
order to have proper overloads depending on the parameters passed to it, whihc
triggers this mypy error now when we subclass it:
src/_pytest/recwarn.py:170: error: Missing type parameters for generic type "catch_warnings" [type-arg]
For our porpuses the parameter is not relevant (we always use record=True), so
decided to just ignore the type error.
The builtin actions token is not allowed to trigger further workflows.
So when we create a PR using that token, the PR checks don't run, and a
maintainer must close & open the PR to make them run. This is quite
annoying, so switch back to the "machine user" approach.
Fixes#9554.
* Recommend importlib import mode for new projects
* Recommend src layout more strongly
* Switch to hatchling as the packaging tool in the example (following PyPA)
* Add explanation about the different import modes
It is unnecessary and has been deleted from the setuptools' docs too.
The setuptools' PEP 517 build backend implementation has been
auto-adding the `wheel` dependency since it's first been implemented.
* Squashed commit of the following:
commit 41d339c46763bbe26123e1e6504b6e32290e33e1
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 17:01:04 2022 +0800
test in all py versions
commit b3572a5a12672228c3276fc8c8e05980dfb7888a
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 16:41:06 2022 +0800
add test
commit 7166a2a51e4f99046b028b663c193d8b558c7fd4
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 16:00:07 2022 +0800
update changelog
commit b958c73d489157f0c0d4e46425083a5e2e2bc851
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date: Thu Jun 23 07:50:52 2022 +0000
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
commit ea7f376c6ca37c40c83df0e4a1cfaaedb34bae91
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 15:48:21 2022 +0800
Fix MyPy
commit 97469beb1da40257e9a061a5e19548546c9312c4
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 15:03:48 2022 +0800
fix if ExceptionGroup not exist
commit 84e553642cd69b4d499231d733df91ebfa84c7ad
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date: Thu Jun 23 03:43:27 2022 +0000
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
commit 76bbef449b88bbd74fb5cca3b5293337a624ef03
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 11:40:41 2022 +0800
adding changelog
commit db82bebc5a4969e2083adcd97bdfd2a63bb17d98
Author: Cheukting <cheukting.ho@gmail.com>
Date: Thu Jun 23 11:33:10 2022 +0800
fall back to native when handeling to exception groups
* Typed ExceptionGroupTypes and changed to BaseExceptionGroup, fixed exceptionchain (excinfo->excinfo_, set reprcrash. Extended tests, though they're wip.
* added exceptiongroup to pre-commit-config, moved away from tuple to directly defining BaseExceptionGroup, added block comment, added match line for inner exception, changked mark.skipif to importorskip to not need top-level import, changed tox.ini a bit - only uncovered should now be py37 without exceptiongroup, due to hypothesis
* added py311-exceptiongroup to github CI, exceptiongroup is now a hard dependency on py<3.11, renamed bad variable names
* added use_coverage to ubuntu-py311
* import BaseExceptionGroup with explicit version check instead of try/catch
* removed from CI, added comments to tox and pre-commit
While taking a look at the docs I noticed they were not properly sorted.
This just moves the sections around so they are sorted alphabetically, without any other changes.
This should help users with the common issue of patching the wrong place.
Also took the opportunity of using proper links in the monkeypatch introduction.
Related to #10216
Co-authored-by: Ran Benita <ran@unusedvar.com>
The main motivation for this change is to simplify the type shown in
code editors -- `Sequence[str]` is easier to follow than
`Union[list[str], tuple[str, ...]]`.
It also permits using other types if desired. It might lead to problems
if someone uses some oddball sequence type, but hopefully they won't do
that.
New Sphinx added support for overloads and always displays them all with
full type annotations etc. This regresses the API reference for
overloaded functions like `fixture()`, `warns()`, `raises()` and friends
to become impossible to read.
I tried various workarounds but none worked except this one.
New versions of sphinx starting showing `__init__` parameters even when
we don't want them to show because they are private (have `_ispytest`
argument).
The only working solution I found was to switch to
`autodoc_typehints_description_target = "documented"` and explicitly
document parameters for which we want to show the types. It's a little
tedious and repetitive in some simple cases, but overall it results in
nicer API docs.
This started to fail recently with:
```
mypy.....................................................................Failed
- hook id: mypy
- exit code: 1
src/_pytest/assertion/rewrite.py:284: error: Returning Any from function declared to return "TraversableResources" [no-any-return]
Found 1 error in 1 file (checked 219 source files)
```
Not sure why that started failing, but seems like ignoring that error specifically is OK.
Given we are already creating a `PurePath`, just pass the parts directly to it.
This avoids using `os.path.sep`, that although is an official API, seems not to be available in all systems.
Fix#9791
`usage.rst` has been moved from `doc/en` to `doc/en/how-to`, so the `man_pages` configuration
value needs to be updated to the new location, so that we dont get this warning:
writing... WARNING: "man_pages" config value references unknown document usage
`usage.rst` has been moved from `doc/en` to `doc/en/how-to`, so the `man_pages` configuration
value needs to be updated to the new location, so that we dont get this warning:
writing... WARNING: "man_pages" config value references unknown document usage
- Remove the index page https://docs.pytest.org/en/7.1.x/genindex.html
Such an index is reasonable in books but not so much in HTML pages.
IMHO the integrated search is much more user-friendly and sufficient.
- Remove the PDF link from the entry page. The PDF is already linked
in the sidebar. I anticipate that getting the PDF documentation is
not a major concern for users.
- Remove the sections "Changelog" and "License" from the entry page.
These section only contain links to the respective pages.
These topics are purely technical. They should be easily accessible
(which they still are through sidebar section "About the project"),
but they don't need additional advertisement through a dedicated
section on the entry page.
This is especially useful for large repositories (e.g. monorepos) that
use a hierarchical file system organization for nested test paths.
src/*/tests
The implementation uses the standard `glob` module to perform wildcard
expansion in Config.parse().
The related logic that determines whether or not to include 'testpaths'
in the terminal header was previously relying on a weak heuristic: if
Config.args matched 'testpaths', then its value was printed. That
generally worked, but it could also print when the user explicitly used
the same arguments on the command-line as listed in 'testpaths'. Not a
big deal, but it shows that the check was logically incorrect.
Now that 'testpaths' can contain wildcards, it's no longer possible to
perform this simple comparison, so this change also introduces a public
Config.ArgSource enum and Config.args_source attribute that explicitly
names the "source" of the arguments: the command line, the invocation
directory, or the 'testdata' configuration value.
This changes the order to:
- API Reference
- Fixtures reference
- Configuration
- Exit codes
- Plugin List
which is approximately sorted from general to specific and often used to less used.
Plugin List ist at the end because it points to further external resources.
This extensive explanation is not need anymore because they are now
unsupported for a long time.
Instead add as short section on Python version compatibility in the
backward compatibility docs.
It's better to have the documentation in one place, instead
of having some in the docstring and some additional
information added to the reference documentation in
`reference.rst`.
Up to now, some had the prefix, some didn't. I think it's good to have
this prefix to give more context, otherwise the links sometimes blend
into the description.
In one case, the link goes to an example, so I used `**Example**:` there.
Up to now, some had the prefix, some didn't. I think it's good to have
this prefix to give more context, otherwise the links sometimes blend
into the description.
In one case, the link goes to an example, so I used `**Example**:` there.
These pytester utility methods were annotated to only receive `str`
names, but they naturally support os.PathLike values, as well.
This makes writing some pytester calls a little nicer, such as when
creating a directory based on a `.joinpath()` call. We previously needed
to cast that intermediate value to a `str`.
As discussed in https://github.com/pytest-dev/pytest/pull/9800, this uses
monkeypatch to change directories for two tests in `test_collection.py`, to preserve
their original purpose (even if just removing it doesn't seem to affect the outcome).
Getting the current working directory has no side-effects, so these context managers were no-ops.
Discovered because Path.__enter__ is deprecated in Python 3.11, for removal in 3.13.
This will trigger a deploy directly when a tag is pushed, avoiding
having to wait for another lenghty build.
This is safe since we only push release tags after the release PR has passed all its tests.
See comments in #9793.
The left/right operands produced when `verbose > 1` should not contain newlines, because they are used to
build the `summary` string. The `assertrepr_compare` function returns a list of lines, and the summary is one of those lines and should not contain newlines itself.
Fix#9742
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
The change from `path not in confuctdir.parents` to the `relative_to`
check in 0c98f19231 broke picking up
conftest files when running against an installed package/site-packages.
See the issue for more details.
Fix#9767.
Including the file name is enough to let the user know what the problem is.
The same is not needed for `.ini` files because the error message includes the path to the file by default.
Fix#9730
It is tempting to use `monkeypatch` to replace the other mechanisms in pytester which change global
state: `CwdSnapshot`, `SysModulesSnapshot`, `SysPathsSnapshot`, however those are more delicate
than they look at first glance so leaving those alone for now.
Close#9708
The dummy modules we introduce in `insert_missing_modules` (due to #7856 and #7859)
would cause problems if the dummy modules actually end up replacing modules
which could be imported normally because they are available in `PYTHONPATH`.
Now we attempt to first import the module via normal mechanisms, and only
introduce the dummy modules if the intermediary modules don't actually exist.
Close#9645
In the following
@pytest.mark.parametrize(..., ids=[val])
the ID values are only allowed to be `str`, `float`, `int` or `bool`.
In the following
@pytest.mark.parametrize(..., [val])
@pytest.mark.parametrize(..., [pytest.param(..., id=val])
a different code path is used, which also allows `bytes`, `complex`,
`re.Pattern`, `Enum` and anything with a `__name__`.
In the interest of consistency, use the latter code path for all cases.
originally reproduced with this pollution set:
```
testing/test_assertrewrite.py::TestEarlyRewriteBailout::test_pattern_contains_subdirectories
testing/test_assertrewrite.py::TestRewriteOnImport::test_remember_rewritten_modules
```
This fixes#9610.
pytest 7.0.0 (unintentionally) changed `UnitTestFunction.obj`'s' behavior
to match `Function.obj`. That is probably a good thing to have, however
it evidently causes some regressions as described in the issue, so
restore the previous behavior for now. In the future we might want to
make this change again, but with proper consideration.
re: review from @asottile that this should only get imported in the function
modify the else/if logic since inside the function we already know the python version is >= 3.10, and just have to know if it is 3.11 or greater
I still think it makes sense to always serve docs from the release branch (rather than the latest tagged release), but that means we need to update it on new releases.
The docs already assume an 'upstream' remote, so we can only fetch from there instead of fetching all remotes. We also don't need to hardcode the remote URL.
alinsa_vix in Discord noticed that we are not testing Python 3.9 and 3.10 on macOS, which seems strange. Maybe this is due how to macOS CI resources were quite scarce for a while, but I believe this has improved since.
* releasing: Add template for major releases
With pytest 6.0.0, we still used a manual releasing workflow (at least if I
remember correctly), and apparently we never wrote a release announcement
template for major releases. Instead, the minor release template claimed that
the release would contain "breaking changes", which doesn't seem reasonable.
Thus, this adds a new major template based on the former minor template, and
adjusts the latter to only mention fixes and new features instead.
* Update scripts/release.minor.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* Add additional docs for uncooperative ctor deprecation
Fixes#9488
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Break up long line
* Recommend kwonly args
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit only refactors, it does not change or add functionality yet. Public
API is retained. Reason or refactoring:
User provided parameter IDs (e.g. Metafunc.parametrize(ids=...)) had so far
only been used to calculate a unique test ID for each test invocation. That
test ID was a joined string where each parameter contributed some partial ID.
We're soon going to reuse functionality to generate parameter keys for
reorder_items and FixtureDef cache. We will be interested in the partial
IDs, and only if they originate from explicit user information. Refactoring
makes logic and data accessible for reuse, and increases cohesion in general.
The docstring (and function name itself) described things as if IDs are
being assigned to the argnames, but actually they're assigned to the
parameter sets.
If we do the `update`s in the right order, we can avoid the `mark.name
not in self.keywords` check, since `self.keywords` starts out clean and
`update` will override previously set keywords.
By my analysis, this deleted code block has no effect:
1. `self.keywords` is `update`d with `callspec.marks`.
2. `self.own_markers` is `update`d with `callspec.marks`.
3. `self.keywords` is `update`d with `self.own_markers`.
So together steps 2+3 completely undo step 1.
This does have a slight semantic change: in a node hierarchy parent ->
child, if parent has a marker applied, then child is constructed, then
`parent.themarker = "overridden"`, previously
`child.keywords['themarker']` would return `True`, now it returns
`"overridden"`. But that's actually what I would have expected so I see
it as more of a bugfix.
(except `Instance`)
Currently, `Function` does this manually, but other node types don't get
their markers added to their `keywords`, but they should, if only for
consistency.
Marks are added to keywords in three places:
- `Node.add_marker`: name -> `Mark`
- `Function.__init__(callspec)`: name -> `Mark`
- `Function.__init__ iter_markers`: name -> True
I think it should be consistent, which will also help with some upcoming
code cleaning. The `Mark` seems more useful than just a `True`, so
switch to that.
* Add docs on pytest.warns(None) deprecation
* Add new section for common warnings use cases
* Fix references for warnings use cases
* Fix reference link
I think this named function makes the code a bit easier to understand.
Also change the check to explicitly check for "is a sub-path of" instead
of the previous check which only worked assuming that path is within
confcutdir or a direct parent of it.
This made the cache not work as intended, causing a major slowdown.
See #9478 for discussion and context.
Authored-by: Anthony Sottile <asottile@umich.edu>
Before 7.0.0rc1, the function accepted `Union[str, py.path.local]`, and
`py.path.local` compares equal to the string path, so a user was able to
pass the path as a string and it would work. In 7.0.0rc1 we changed the
`py.path.local` to `Path` which doesn't compare equal to the string
path, which breaks compatibility (e.g. the `sybil` package).
This restores compatibility for this function by accepting any
`os.PathLike[str]` and only comparing the string representations.
Since commit 89f0b5b5a2 cases as in the
added test started to fail, like they do for the standard pytest names
(`setup_module` etc). But the name `setup` in particular is way too
common for us to start taking it over more aggressively, so restore the
previous behavior which required the object to be callable.
Fix#9391.
To backport a PR, e.g. 1000, to another branch, e.g. `7.0.x`, add a
label `backport 7.0.x` to the PR. This will trigger a workflow which
will create a branch `backport-1000-to-7.0.x` based on the `7.0.x`
branch with a cherry-pick of the PR's merge commit, and create a new PR
for it against the `7.0.x` branch.
It is very simplistic, for instance it doesn't handle cherry-pick
failure gracefully, doesn't validate the state of the PR, doesn't check
if the branch already exists, etc. But we can improve on it later as
needed.
Finally, PRs created by github actions do not themselves trigger further
actions, i.e. the PR isn't checked. You need to close & reopen the PR
for the checks to trigger. There are workarounds for this but they are
either less secure or require more setup.
The `pytest_pycollector_makeitem` argument `collector` is currently
annotated with type `PyCollector`. As part of #7469, that would have
required us to expose it in the public API. But really it's an
implementation detail, not something we want to expose. So replace the
annotation with the concrete python collector types that are passed.
Strictly speaking, `pytest_pycollector_makeitem` is called from
`PyCollector.collect()`, so the new type annotation is incorrect if
another type subclasses `PyCollector`. But the set of python collectors
is closed (mapping to language constructs), and the type is private, so
there shouldn't be any other deriving classes, and we can consider it
effectively sealed (unfortunately Python does not provide a way to
express this - yet?).
* scripts: Use release branch for changelog URL
With a prerelease, /stable won't show the correct changelog.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This reverts commit 15989ddc8f.
Reverts #8945, thus reintroducing #8926 temporarily, but we should be close to
the 7.0.0 release now, where this is the correct URL to use.
Closes#8831 as follow-up to #8858.
* Improve reference and path/fspath docs
Closes#9283
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixups
* Add explanation
* Update wording after #9363
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Rename pytest_ignore_collect fspath parameter to collection_path
* Rename pytest_collect_file fspath parameter to file_path
* Rename pytest_pycollect_makemodule fspath parameter to module_path
* Rename pytest_report_header startpath parameter to start_path
* Rename pytest_report_collectionfinish startpath parameter to start_path
* Update docs with the renamed parameters
* Use pytest-flakes fork temporarily to prove it works
* Use pytest-flakes 4.0.5
Follow-up to #9309.
The issue in the changelog broke the docs build with:
doc/en/_changelog_towncrier_draft.rst:47: WARNING: Inline literal start-string
without end-string.
The second change isn't as critical, but caused the text to be rendered as
monospace including the tilde (i.e. `~pytest.PytestDeprecationwarning`).
Closes#7480.
This allows us to more easily follow our deprecation policy of turning
warnings into errors for the X.0 releases before complete removal in
X.1.
It also makes the deprecation timeline clear to both the users and
pytest developers -- it can be hard to keep track.
Note that the designation is not meant to be a binding contract - if the
time comes for removal of a specific deprecation but we decide it's too
soon, can just bump it to the next major.
Inspired by Django:
https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/submitting-patches/#deprecating-a-feature
* Remove changelog entry for #8251
Reverted in #8903
* Move #9202 changelog to to trivial
This won't concern users of pytest
* Streamline deprecation changelogs/docs
* Remove #8994 changelog
This is an impovement for a warning introduced in this release, so including it in a changelog against the last release seems confusing.
* Remove #9241 changelog
This is an impovement for a doc update introduced in this release, so including it in a changelog against the last release seems confusing. The issue number also seems about something different.
* Remove #8897 changelog
Empty file...
* Various minor changelog fixes
* nodes: keep plugins which subclass Item, File working for a bit more
Fix#8435.
* Update src/_pytest/nodes.py
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Florian Bruhin <me@the-compiler.org>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* porting pytest.skip() to use reason=, adding tests
* avoid adding **kwargs, it breaks other functionality, use optional msg= instead
* deprecation of `pytest.fail(msg=...)`
* fix bug with not capturing the returned reason value
* pass reason= in acceptance async tests instead of msg=
* finalising deprecations of `msg` in `pytest.skip()` and `pytest.fail()`
* Update doc/en/deprecations.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* Update doc/en/deprecations.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* fix failing test after upstream merge
* adding deprecation to `pytest.exit(msg=...)`
* add docs for pytest.exit deprecations
* finalising deprecation of msg for pytest.skip, pytest.exit and pytest.fail
* hold a reference to the Scope instance to please mypy
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Fixes#9272.
Fixing the issue directly in the plugin is somewhat hard, so do it in
core. Since the plugin is going to be deprecated, I figure it's OK to
cheat a bit.
The current PDF docs attempt to format the list of all plugins as a
table, without any word-wrapping of the plugin description. That results
in almost all the information getting cut off. This PR formats the same
information into more of a paragraph format for the PDF, with nothing
cut off.
Fixes#451
This is unfortunately a dependency on `py.path` which cannot be moved to
an external plugins or eased in any way, so has to be deprecated in
order for pytest to be able to eventually remove the dependency on `py`.
Export `HookRecorder`, `RecordedHookCall` (originally `ParsedCall`),
`RunResult`, `LineMatcher`.
These types are reachable through `Pytester` and so should be public
themselves for typing and other purposes.
The name `ParsedCall` I think is too generic under the `pytest`
namespace, so rename it to `RecordedHookCall`.
The `HookRecorder`'s constructor is made private -- it should only be
constructed by `Pytester`.
`LineMatcher` and `RunResult` are exported as is - no private and no
rename, since they're being used.
All of the classes are made final as they are not designed for
subclassing.
This causes `Session` documentation to be rendered again in full under
`pytester` and `testdir` in the API Reference. I tried but couldn't get
sphinx to hide it.
Since it's a pretty odd thing to have (should just use
`pytest.Session`), and I couldn't find any plugin which uses this, let's
just remove it.
`reportinfo()` is the last remaining py.path-only code path in pytest,
i.e. the last piece holding back py.path deprecation. The problem with
it is that plugins/users use it from both sides -- implementing it
(returning the value) and using it (using the return value). Dealing
with implementers is easy enough -- allow to return `os.PathLike[str]`.
But for callers who expect strictly `py.path` this will break and
there's not really a good way to provide backward compat for this.
From analyzing a corpus of 680 pytest plugins, the vast majority of
`reportinfo` appearances are implementations, and the few callers don't
actually access the path part of the return tuple.
As for test suites that might access `reportinfo` (e.g. using
`request.node.reportinfo()` or other ways), that is much harder to
survey, but from the ones I searched, I only found case
(`pytest_teamcity`, but even then it uses `str(fspath)` so is unlikely
to be affected in practice). They are better served with using
`node.location` or `node.path` directly.
Therefore, just break it and change the return type to
`str|os.PathLike[str]`.
Refs #7259.
This type is semi-private; not documented but many plugins access it
through `item.callspec`. However, plugins access the public fields and
almost none try to construct or monkeypatch it. So we should be allowed
to clean it up some.
- Convert to attrs, add slots and frozen
- Instead of doing `new = old.copy(); new.setmulti2()`, do `new =
old.setmulti()`. This is cleaner and faster.
- Remove the `metafunc` attribute. This causes a reference cycle
(multifunc._calls -> callspec -> multifunc) for no good reason --
neither pytest itself or plugins access this attribute, so let's not
keep the Metafunc objects alive past their due.
- Some comments.
I would have also like to make the dicts and lists themselves immutable,
however some plugins mess with those so that should be done separately,
if at all.
this issues is less likely to hit due to the recent regendoc release
which includes a wheel
* migrate to setuptools_scm 6.3.2
* use SETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYTEST
Now that it's no longer using `@lru_cache`, use another check to avoid
re-computation. Although `@lru_cache` is faster than the full function
call + checks, this approach also has the advantage that the caching
works for more than 128 entries.
* Fix non-sensical error message
Introduced in 12de92cd2b / #7698
* Add a test
* Put the unit back into unittest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
A new plugin has this summary:
Continiously runs pytest on changes in *.py files
The `*` is interpreted as a special character and fails the CI.
Add some rudimentary escaping to hopefully prevent this.
The projects page unfortunately contains a lot of old projects, and some information there appears to be stale.
It was created at a time pytest was starting to be used, but seems pointless now as pytest is the most popular testing framework, so listing a few dozen projects which use it seems pointless.
Rather than updating it, I think we should remove it because this has the chance to become stale again soon, and not sure how valuable it is anyway.
The PDF documentation on readthedocs was missing the figures in the
fixtures reference chapter. This PR uses a Sphinx plugin that
automatically converts the checked-in SVG files to the PDF input files
that LaTeX requires.
The SVG-to-PDF conversion is done by inkscape, which gave the best
conversion among the tools I tried. However, it [does not yet
understand][href-bug] that you can write a plain `href` instead of
`xlink:href` in svg files, so I’ve had to edit the SVG files
accordingly.
[href-bug]: https://github.com/TeX-Live/luatex.git
Some of the top search-engine hits for pytest.approx use the function without actually comparing it to anything.
This PR will cause these tests to fail by implementing approx.__bool__() to raise an AssertionError that briefly explains how to correctly use approx.
* expose `warnings=` to pytester `assert_outcomes()`
* fix test fallout from adding warnings= to assert_outcomes()
* #closes 8593 - Improve test and add a `changelog` entry for the change
It is not clear yet how we should proceed with this deprecation
because `pytest.Item.reportinfo` is public API and returns a `py.path` object,
and is not clear how plugins and our examples should handle that.
Reverting just the deprecation aspect of #8251 so we can get a 7.0.0 release out.
We will reintroduce the deprecation later once we have a clear path moving forward with replacing `reportinfo`.
Closes#8445Closes#8821
the best practice is to only `<` when you know something is broken, and even then prefer `!=` assuming it will be fixed upstream for maximum compatibility
* fix-contributors-list
* remove-bots
* delete-extraneous-file
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* issue a warning when Items and Collector form a diamond
addresses #8435
* Apply suggestions from code review
Co-authored-by: Ran Benita <ran@unusedvar.com>
* Return support for the broken File/Item hybrids
* adds deprecation
* ads necessary support code in node construction
* fix incorrect mypy based assertions
* add docs for deprecation of Item/File inheritance
* warn when a non-cooperative ctor is encountered
* use getattr instead of cast to get the class __init__ for legacy ctors
* update documentation references for node inheritance
* clean up file+item inheritance test
enhance docs
move import upwards
Co-authored-by: Ran Benita <ran@unusedvar.com>
* cache: Move repetitive code to fixture
* cache: Explicitly test for chmod result
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
In the "Getting Started" doc, the test class instance example for
instance sharing doesn't actually demonstrate anything about the
reinstantiation of the class. This change shows clearly how the instance
data isn't retained between test runs.
* [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/pyupgrade: v2.16.0 → v2.18.2](https://github.com/asottile/pyupgrade/compare/v2.16.0...v2.18.2)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
`Parser` is used by many plugins and custom hooks. `OptionGroup` is
exposed by the `parser.addgroup` API.
The constructors of both are marked private, they are not meant to be
constructed directly.
Given a `RecursionError` traceback, the `Traceback.recursionindex()`
method returns the index of the frame which started the recursion
(repeated set of frames). To do so it attempts to check whether two
frames are equivalent. Just checking the function/line is not enough
because the recursion variable(s) might differ (e.g. imagine the numeric
value in a recursive factorial implementation). So it also compares the
`f_locals` (local variables) of each frame for equivalence.
For some reason, the locals comparison is wrapped in an `eval` whose
purpose is to evaluate the comparison in one of the compared frame's
context (locals + globals in scope). However, I can not think of any way
in which the global scope could affect the evaluation. It would have an
affect when the locals are bound but that's already done. So this seems
unnecessary - remove it.
* add feature to view fixture source location in invocations with --fixtures-per-test option
* remove unrelated changes to show_fixtures_per_test::test_doctest_items
* eshew the extraneous else in _show_fixtures_per_test.write_fixture
* enable the accommodation of multi-line docstring with --fixtures-per-test option
* add feature to view fixture source location in invocations with --fixtures
* add colour encoding to fixture location paths
* add changelog for #8606 fixing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Replace for loop using the operator
* Replace for loop with a generator expression inside any()
* Replace for loop with a dictionary comprehension
* Use list comprehension
* Simplify arguments for range()
* Change newfuncargs variable to in-line dictionary comprehension
* is_ancestor: return base.is_relative_to(query)
* Remove unneeded import of pathlib
* Try using PurePath
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Import PurePath on new line
* Revert and remove is_relative_to
Co-authored-by: Zachary Kneupper <zacharykneupper@Zacharys-MBP.lan>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Remove unnecessary else clause in repr_failure()
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Zachary Kneupper <zacharykneupper@Zacharys-MBP.lan>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Expand command line argument examples with validation examples
This shows how to pass more detailed error messages back to the user when
they've provided an incorrect value for a custom command line option.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix issue where TestCase.setUpClass is not called for test methods with a / in its name by checking if there is :: before the selected / or any :: after. Also added a test case for this.
* removed print statement that was added
* Change iterparentnodeids to consume / parts until the first ::. Then consider ::. Tests were changed to reflect this.
* Update changelog/8509.improvement.rst
Co-authored-by: Ran Benita <ran@unusedvar.com>
2021-04-29 17:02:43 +03:00
283 changed files with 25748 additions and 8904 deletions
Look through the `GitHub issues for bugs <https://github.com/pytest-dev/pytest/labels/type:%20bug>`_.
See also the `"status: easy" issues <https://github.com/pytest-dev/pytest/labels/status%3A%20easy>`_
that are friendly to new contributors.
:ref:`Talk <contact>` to developers to find out how you can fix specific bugs. To indicate that you are going
to work on a particular issue, add a comment to that effect on the specific issue.
@@ -221,7 +223,7 @@ changes you want to review and merge. Pull requests are stored on
Once you send a pull request, we can discuss its potential modifications and
even add more commits to it later on. There's an excellent tutorial on how Pull
Requests work in the
`GitHub Help Center <https://help.github.com/articles/using-pull-requests/>`_.
`GitHub Help Center <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>`_.
Here is a simple overview, with pytest-specific bits:
@@ -242,6 +244,11 @@ Here is a simple overview, with pytest-specific bits:
be released in micro releases whereas features will be released in
minor releases and incompatible changes in major releases.
You will need the tags to test locally, so be sure you have the tags from the main repository. If you suspect you don't, set the main repository as upstream and fetch the tags::
@@ -324,7 +331,7 @@ Here is a simple overview, with pytest-specific bits:
Writing Tests
~~~~~~~~~~~~~
Writing tests for plugins or for pytest itself is often done using the `pytester fixture <https://docs.pytest.org/en/stable/reference.html#pytester>`_, as a "black-box" test.
Writing tests for plugins or for pytest itself is often done using the `pytester fixture <https://docs.pytest.org/en/stable/reference/reference.html#pytester>`_, as a "black-box" test.
For example, to ensure a simple test passes you can write:
@@ -378,7 +385,7 @@ them.
Backporting bug fixes for the next patch release
------------------------------------------------
Pytest makes feature release every few weeks or months. In between, patch releases
Pytest makes a feature release every few weeks or months. In between, patch releases
are made to the previous feature release, containing bug fixes only. The bug fixes
usually fix regressions, but may be any change that should reach users before the
next feature release.
@@ -387,10 +394,17 @@ Suppose for example that the latest release was 1.2.3, and you want to include
a bug fix in 1.2.4 (check https://github.com/pytest-dev/pytest/releases for the
actual latest release). The procedure for this is:
#. First, make sure the bug is fixed the ``main`` branch, with a regular pull
#. First, make sure the bug is fixed in the ``main`` branch, with a regular pull
request, as described above. An exception to this is if the bug fix is not
applicable to ``main`` anymore.
Automatic method:
Add a ``backport 1.2.x`` label to the PR you want to backport. This will create
a backport PR against the ``1.2.x`` branch.
Manual method:
#.``git checkout origin/1.2.x -b backport-XXXX`` # use the main PR number here
#. Locate the merge commit on the PR, in the *merged* message, for example:
#. For major and minor releases, change the default version in the `Read the Docs Settings <https://readthedocs.org/dashboard/pytest/advanced/>`_ to the new branch.
#. Send an email announcement with the contents from::
The types of objects used in pytest's API are now exported so they may be used in type annotations.
The newly-exported types are:
-``pytest.Mark`` for :class:`marks <pytest.Mark>`.
-``pytest.MarkDecorator`` for :class:`mark decorators <pytest.MarkDecorator>`.
-``pytest.MarkGenerator`` for the :class:`pytest.mark <pytest.MarkGenerator>` singleton.
-``pytest.Metafunc`` for the :class:`metafunc <pytest.MarkGenerator>` argument to the :func:`pytest_generate_tests <pytest.hookspec.pytest_generate_tests>` hook.
-``pytest.CallInfo`` for the :class:`CallInfo <pytest.CallInfo>` type passed to various hooks.
-``pytest.ExceptionInfo`` for the :class:`ExceptionInfo <pytest.ExceptionInfo>` type returned from :func:`pytest.raises` and passed to various hooks.
Constructing them directly is not supported; they are only meant for use in type annotations.
Doing so will emit a deprecation warning, and may become a hard-error in pytest 7.0.
Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.
The ``@pytest.mark.skip`` decorator now correctly handles its arguments. When the ``reason`` argument is accidentally given both positional and as a keyword (e.g. because it was confused with ``skipif``), a ``TypeError`` now occurs. Before, such tests were silently skipped, and the positional argument ignored. Additionally, ``reason`` is now documented correctly as positional or keyword (rather than keyword-only).
Use private names for internal fixtures that handle classic setup/teardown so that they don't show up with the default ``--fixtures`` invocation (but they still show up with ``--fixtures -v``).
The :confval:`required_plugins` config option now works correctly when pre-releases of plugins are installed, rather than falsely claiming that those plugins aren't installed at all.
@@ -10,10 +10,9 @@ Are you an enthusiastic pytest user, the local testing guru in your workplace? O
We will pair experienced pytest users with open source projects, for a month's effort of getting new development teams started with pytest.
In 2015 we are trying this for the first time. In February and March 2015 we will gather volunteers on both sides, in April we will do the work, and in May we will evaluate how it went. This effort is being coordinated by Brianna Laugher. If you have any questions or comments, you can raise them on the `@pytestdotorg twitter account <https://twitter.com/pytestdotorg>`_ the `issue tracker`_ or the `pytest-dev mailing list`_.
In 2015 we are trying this for the first time. In February and March 2015 we will gather volunteers on both sides, in April we will do the work, and in May we will evaluate how it went. This effort is being coordinated by Brianna Laugher. If you have any questions or comments, you can raise them on the `@pytestdotorg twitter account <https://twitter.com/pytestdotorg>`_\, the :issue:`issue tracker <676>` or the `pytest-dev mailing list`_.
@@ -22,7 +22,9 @@ b) transitional: the old and new API don't conflict
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn them into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationwarning`).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g. `PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
c) true breakage: should only be considered when normal transition is unreasonably unsustainable and would offset important development/features by years.
@@ -30,15 +32,15 @@ c) true breakage: should only be considered when normal transition is unreasonab
Examples for such upcoming changes:
* removal of ``pytest_runtest_protocol/nextitem`` - `#895`_
* removal of ``pytest_runtest_protocol/nextitem`` - :issue:`895`
* rearranging of the node tree to include ``FunctionDefinition``
* rearranging of ``SetupState```#895`_
* rearranging of ``SetupState``:issue:`895`
True breakages must be announced first in an issue containing:
* Detailed description of the change
* Rationale
* Expected impact on users and plugin authors (example in `#895`_)
* Expected impact on users and plugin authors (example in :issue:`895`)
After there's no hard *-1* on the issue it should be followed up by an initial proof-of-concept Pull Request.
@@ -77,4 +79,16 @@ Features currently deprecated and removed in previous releases can be found in :
We track future deprecation and removal of features using milestones and the `deprecation <https://github.com/pytest-dev/pytest/issues?q=label%3A%22type%3A+deprecation%22>`_ and `removal <https://github.com/pytest-dev/pytest/labels/type%3A%20removal>`_ labels on GitHub.
In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
*:hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
*:hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
*:hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
*:hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
*:hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
..note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
:ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
being ``path``) is **the opposite** of the situation for hooks (the old
argument being ``path``).
``Node.fspath`` in favor of ``pathlib`` and ``Node.path``
Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
Some plugins providing linting/code analysis have been using this as a hack.
Instead, a separate collector node should be used, which collects the item. See
:ref:`non-python tests` for an example, as well as an `example pr fixing inheritance`_.
:func:`unittest.skip` in unittest test cases is fully supported.
See :ref:`warns use cases` for examples.
Returning non-None value in test functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.2
A :class:`pytest.PytestReturnNotNoneWarning` is now emitted if a test function returns something other than `None`.
This prevents a common mistake among beginners that expect that returning a `bool` would cause a test to pass or fail, for example:
..code-block::python
@pytest.mark.parametrize(
["a","b","result"],
[
[1,2,5],
[2,3,8],
[5,3,18],
],
)
deftest_foo(a,b,result):
returnfoo(a,b)==result
Given that pytest ignores the return value, this might be surprising that it will never fail.
The proper fix is to change the `return` to an `assert`:
..code-block::python
@pytest.mark.parametrize(
["a","b","result"],
[
[1,2,5],
[2,3,8],
[5,3,18],
],
)
deftest_foo(a,b,result):
assertfoo(a,b)==result
The ``--strict`` command-line option
@@ -92,29 +467,42 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Removed Features
----------------
..deprecated:: 6.0
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recored`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest.collect`` module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
The ``pytest.collect`` module is no longer part of the public API, all its names
should now be imported from ``pytest`` directly instead.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recorded`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest._fillfuncargs`` function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
This function was kept for backward compatibility with an older plugin.
@@ -123,12 +511,6 @@ it, use `function._request._fillfixtures()` instead, though note this is not
a public API and may break in the future.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
``--no-print-logs`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -143,6 +525,7 @@ A ``--show-capture`` command-line option was added in ``pytest 3.5.0`` which all
display captured output when tests fail: ``no``, ``stdout``, ``stderr``, ``log`` or ``all`` (the default).
.._resultlog deprecated:
Result log (``--result-log``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -165,8 +548,8 @@ at some point, depending on the plans for the plugins and number of users using
..versionremoved:: 6.0
The ``pytest_collect_directory`` has not worked properly for years (it was called
but the results were ignored). Users may consider using :func:`pytest_collection_modifyitems <_pytest.hookspec.pytest_collection_modifyitems>` instead.
The ``pytest_collect_directory`` hook has not worked properly for years (it was called
but the results were ignored). Users may consider using :hook:`pytest_collection_modifyitems` instead.
TerminalReporter.writer
~~~~~~~~~~~~~~~~~~~~~~~
@@ -180,6 +563,8 @@ with ``py.io.TerminalWriter``.
Plugins that used ``TerminalReporter.writer`` directly should instead use ``TerminalReporter``
methods that provide the same functionality.
.._junit-family changed default value:
``junit_family`` default value change to "xunit2"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -267,6 +652,8 @@ in places where we or plugin authors must distinguish between fixture names and
names supplied by non-fixture things such as ``pytest.mark.parametrize``.
.._pytest.config global deprecated:
``pytest.config`` global
~~~~~~~~~~~~~~~~~~~~~~~~
@@ -311,7 +698,7 @@ Becomes:
If you still have concerns about this deprecation and future removal, please comment on
@@ -234,21 +234,21 @@ You can ask which markers exist for your test suite - the list includes our just
@pytest.mark.slow: mark test as slow.
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/warnings.html#pytest-mark-filterwarnings
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-skipif
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-skipif
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-xfail
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-xfail
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/parametrize.html for more info and examples.
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/how-to/parametrize.html for more info and examples.
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/fixture.html#usefixtures
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
For an example on how to add and work with markers from a plugin, see
@@ -346,7 +346,7 @@ Custom marker and command line option to control test runs
Plugins can provide custom markers and implement specific behaviour
based on it. This is a self-contained example which adds a command
line option and a parametrized test function marker to run tests
specifies via named environments:
specified via named environments:
..code-block::python
@@ -375,7 +375,7 @@ specifies via named environments:
pytest.skip("test requires env in {!r}".format(envnames))
pytest.skip(f"test requires env in {envnames!r}")
A test file using this local plugin:
@@ -397,9 +397,8 @@ the test needs:
$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_someenv.py s [100%]
@@ -412,9 +411,8 @@ and here is one that specifies exactly the environment needed:
$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_someenv.py . [100%]
@@ -428,21 +426,21 @@ The ``--markers`` option always gives you a list of available markers:
$ pytest --markers
@pytest.mark.env(name): mark test to run only on named environment
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/warnings.html#pytest-mark-filterwarnings
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-skipif
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-skipif
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-xfail
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-xfail
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/parametrize.html for more info and examples.
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/how-to/parametrize.html for more info and examples.
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/fixture.html#usefixtures
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
.._`passing callables to custom markers`:
@@ -488,7 +486,7 @@ The output is as follows:
..code-block::pytest
$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef>,), kwargs={})
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef0001>,), kwargs={})
.
1 passed in 0.12s
@@ -530,7 +528,7 @@ test function. From a conftest file we can read it like this:
Here is an example ``conftest.py`` (extracted from Ali Afshar's special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yaml`` files and will execute the yaml-formatted content as custom tests:
@@ -22,16 +21,15 @@ You can create a simple example file:
..include:: nonpython/test_simple.yaml
:literal:
and if you installed `PyYAML`_ or a compatible YAML-parser you can
and if you installed :pypi:`PyYAML` or a compatible YAML-parser you can
now execute the test specification:
..code-block::pytest
nonpython $ pytest test_simple.yaml
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project/nonpython
collected 2 items
test_simple.yaml F. [100%]
@@ -66,9 +64,9 @@ consulted when reporting in ``verbose`` mode:
nonpython $ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project/nonpython
collecting ... collected 2 items
test_simple.yaml::hello FAILED [ 50%]
@@ -92,9 +90,8 @@ interesting to just look at the collection tree:
nonpython $ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
@@ -28,7 +28,7 @@ Flaky tests sometimes appear when a test suite is run in parallel (such as use o
Overly strict assertion
~~~~~~~~~~~~~~~~~~~~~~~
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/stable/reference.html#pytest-approx>`_ is useful here.
Overly strict assertions can cause problems with floating point comparison as well as timing issues. :func:`pytest.approx` is useful here.
Pytest features
@@ -94,7 +94,7 @@ Mark Lapierre discusses the `Pros and Cons of Quarantined Tests <https://dev.to/
CI tools that rerun on failure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/azure/devops/release-notes/2017/dec-11-vsts#identify-flaky-tests>`_ and rerun failed tests.
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/previous-versions/azure/devops/2017/dec-11-vsts?view=tfs-2017#identify-flaky-tests>`_ and rerun failed tests.
pytest as a testing framework needs to import test modules and ``conftest.py`` files for execution.
Importing files in Python (at least until recently) is a non-trivial processes, often requiring
changing `sys.path <https://docs.python.org/3/library/sys.html#sys.path>`__. Some aspects of the
changing :data:`sys.path`. Some aspects of the
import process can be controlled through the ``--import-mode`` command-line flag, which can assume
these values:
*``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of :py:data:`sys.path` if not already there, and then imported with the `__import__ <https://docs.python.org/3/library/functions.html#__import__>`__ builtin.
of :py:data:`sys.path` if not already there, and then imported with the :func:`importlib.import_module <importlib.import_module>` function.
This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in :py:data:`sys.modules` after importing.
@@ -24,7 +24,7 @@ these values:
This is the classic mechanism, dating back from the time Python 2 was still supported.
*``append``: the directory containing each module is appended to the end of :py:data:`sys.path` if not already
there, and imported with ``__import__``.
there, and imported with :func:`importlib.import_module <importlib.import_module>`.
This better allows to run test modules against installed versions of a package even if the
package under test has the same import root. For example:
@@ -43,12 +43,21 @@ these values:
Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in :py:data:`sys.modules` after importing.
*``importlib``: new in pytest-6.0, this mode uses `importlib <https://docs.python.org/3/library/importlib.html>`__ to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
*``importlib``: new in pytest-6.0, this mode uses more fine control mechanisms provided by :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
For this reason this doesn't require test module names to be unique, but also makes test
modules non-importable by each other.
For this reason this doesn't require test module names to be unique.
One drawback however is that test modules are non-importable by each other. Also, utility
modules in the tests directories are not automatically importable because the tests directory is no longer
added to :py:data:`sys.path`.
Initially we intended to make ``importlib`` the default in future releases, however it is clear now that
it has its own set of drawbacks so the default will remain ``prepend`` for the foreseeable future.
..seealso::
The :confval:`pythonpath` configuration variable.
We intend to make ``importlib`` the default in future releases, depending on feedback.
``pytest`` requires: Python 3.6, 3.7, 3.8, 3.9, or PyPy3.
``pytest`` requires: Python 3.7+ or PyPy3.
1. Run the following command in your command line:
@@ -22,7 +22,7 @@ Install ``pytest``
..code-block::bash
$ pytest --version
pytest 6.2.3
pytest 7.2.2
.._`simpletest`:
@@ -47,9 +47,8 @@ The test
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_sample.py F [100%]
@@ -71,7 +70,7 @@ The ``[100%]`` refers to the overall progress of running all test cases. After i
..note::
You can use the ``assert`` statement to verify test expectations. pytest’s `Advanced assertion introspection <https://docs.python.org/reference/simple_stmts.html>`_ will intelligently report intermediate values of the assert expression so you can avoid the many names `of JUnit legacy methods <https://docs.python.org/library/unittest.html>`_.
You can use the ``assert`` statement to verify test expectations. pytest’s :ref:`Advanced assertion introspection <python:assert>` will intelligently report intermediate values of the assert expression so you can avoid the many names :ref:`of JUnit legacy methods <testcase-objects>`.
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/stable/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
``pytest`` provides :std:doc:`Builtin fixtures/function arguments <builtin>` to request arbitrary resources, like a unique temporary directory:
..code-block::python
@@ -227,21 +220,21 @@ List the name ``tmp_path`` in the test function signature and ``pytest`` will lo
More info on temporary directory handling is available at :ref:`Temporary directories and files <tmpdir handling>`.
More info on temporary directory handling is available at :ref:`Temporary directories and files <tmp_path handling>`.
Find out what kind of builtin :ref:`pytest fixtures <fixtures>` exist with the command:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
