This is used to set the <link rel="canonical" href="X"> tag that points to the canonical version of the webpage. Including this indicates to search engines which version to include in their indexes, and should prevent older versions showing up.
Fixes#12363
* Recommend importlib import mode for new projects
* Recommend src layout more strongly
* Switch to hatchling as the packaging tool in the example (following PyPA)
* Add explanation about the different import modes
(cherry picked from commit 245a8c23dd)
Including the file name is enough to let the user know what the problem is.
The same is not needed for `.ini` files because the error message includes the path to the file by default.
Fix#9730
It is tempting to use `monkeypatch` to replace the other mechanisms in pytester which change global
state: `CwdSnapshot`, `SysModulesSnapshot`, `SysPathsSnapshot`, however those are more delicate
than they look at first glance so leaving those alone for now.
Close#9708
The dummy modules we introduce in `insert_missing_modules` (due to #7856 and #7859)
would cause problems if the dummy modules actually end up replacing modules
which could be imported normally because they are available in `PYTHONPATH`.
Now we attempt to first import the module via normal mechanisms, and only
introduce the dummy modules if the intermediary modules don't actually exist.
Close#9645
In the following
@pytest.mark.parametrize(..., ids=[val])
the ID values are only allowed to be `str`, `float`, `int` or `bool`.
In the following
@pytest.mark.parametrize(..., [val])
@pytest.mark.parametrize(..., [pytest.param(..., id=val])
a different code path is used, which also allows `bytes`, `complex`,
`re.Pattern`, `Enum` and anything with a `__name__`.
In the interest of consistency, use the latter code path for all cases.
originally reproduced with this pollution set:
```
testing/test_assertrewrite.py::TestEarlyRewriteBailout::test_pattern_contains_subdirectories
testing/test_assertrewrite.py::TestRewriteOnImport::test_remember_rewritten_modules
```
This fixes#9610.
pytest 7.0.0 (unintentionally) changed `UnitTestFunction.obj`'s' behavior
to match `Function.obj`. That is probably a good thing to have, however
it evidently causes some regressions as described in the issue, so
restore the previous behavior for now. In the future we might want to
make this change again, but with proper consideration.
re: review from @asottile that this should only get imported in the function
modify the else/if logic since inside the function we already know the python version is >= 3.10, and just have to know if it is 3.11 or greater
I still think it makes sense to always serve docs from the release branch (rather than the latest tagged release), but that means we need to update it on new releases.
The docs already assume an 'upstream' remote, so we can only fetch from there instead of fetching all remotes. We also don't need to hardcode the remote URL.
alinsa_vix in Discord noticed that we are not testing Python 3.9 and 3.10 on macOS, which seems strange. Maybe this is due how to macOS CI resources were quite scarce for a while, but I believe this has improved since.
* releasing: Add template for major releases
With pytest 6.0.0, we still used a manual releasing workflow (at least if I
remember correctly), and apparently we never wrote a release announcement
template for major releases. Instead, the minor release template claimed that
the release would contain "breaking changes", which doesn't seem reasonable.
Thus, this adds a new major template based on the former minor template, and
adjusts the latter to only mention fixes and new features instead.
* Update scripts/release.minor.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* Add additional docs for uncooperative ctor deprecation
Fixes#9488
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Break up long line
* Recommend kwonly args
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit only refactors, it does not change or add functionality yet. Public
API is retained. Reason or refactoring:
User provided parameter IDs (e.g. Metafunc.parametrize(ids=...)) had so far
only been used to calculate a unique test ID for each test invocation. That
test ID was a joined string where each parameter contributed some partial ID.
We're soon going to reuse functionality to generate parameter keys for
reorder_items and FixtureDef cache. We will be interested in the partial
IDs, and only if they originate from explicit user information. Refactoring
makes logic and data accessible for reuse, and increases cohesion in general.
The docstring (and function name itself) described things as if IDs are
being assigned to the argnames, but actually they're assigned to the
parameter sets.
If we do the `update`s in the right order, we can avoid the `mark.name
not in self.keywords` check, since `self.keywords` starts out clean and
`update` will override previously set keywords.
By my analysis, this deleted code block has no effect:
1. `self.keywords` is `update`d with `callspec.marks`.
2. `self.own_markers` is `update`d with `callspec.marks`.
3. `self.keywords` is `update`d with `self.own_markers`.
So together steps 2+3 completely undo step 1.
This does have a slight semantic change: in a node hierarchy parent ->
child, if parent has a marker applied, then child is constructed, then
`parent.themarker = "overridden"`, previously
`child.keywords['themarker']` would return `True`, now it returns
`"overridden"`. But that's actually what I would have expected so I see
it as more of a bugfix.
(except `Instance`)
Currently, `Function` does this manually, but other node types don't get
their markers added to their `keywords`, but they should, if only for
consistency.
Marks are added to keywords in three places:
- `Node.add_marker`: name -> `Mark`
- `Function.__init__(callspec)`: name -> `Mark`
- `Function.__init__ iter_markers`: name -> True
I think it should be consistent, which will also help with some upcoming
code cleaning. The `Mark` seems more useful than just a `True`, so
switch to that.
* Add docs on pytest.warns(None) deprecation
* Add new section for common warnings use cases
* Fix references for warnings use cases
* Fix reference link
I think this named function makes the code a bit easier to understand.
Also change the check to explicitly check for "is a sub-path of" instead
of the previous check which only worked assuming that path is within
confcutdir or a direct parent of it.
This made the cache not work as intended, causing a major slowdown.
See #9478 for discussion and context.
Authored-by: Anthony Sottile <asottile@umich.edu>
Before 7.0.0rc1, the function accepted `Union[str, py.path.local]`, and
`py.path.local` compares equal to the string path, so a user was able to
pass the path as a string and it would work. In 7.0.0rc1 we changed the
`py.path.local` to `Path` which doesn't compare equal to the string
path, which breaks compatibility (e.g. the `sybil` package).
This restores compatibility for this function by accepting any
`os.PathLike[str]` and only comparing the string representations.
Since commit 89f0b5b5a2 cases as in the
added test started to fail, like they do for the standard pytest names
(`setup_module` etc). But the name `setup` in particular is way too
common for us to start taking it over more aggressively, so restore the
previous behavior which required the object to be callable.
Fix#9391.
To backport a PR, e.g. 1000, to another branch, e.g. `7.0.x`, add a
label `backport 7.0.x` to the PR. This will trigger a workflow which
will create a branch `backport-1000-to-7.0.x` based on the `7.0.x`
branch with a cherry-pick of the PR's merge commit, and create a new PR
for it against the `7.0.x` branch.
It is very simplistic, for instance it doesn't handle cherry-pick
failure gracefully, doesn't validate the state of the PR, doesn't check
if the branch already exists, etc. But we can improve on it later as
needed.
Finally, PRs created by github actions do not themselves trigger further
actions, i.e. the PR isn't checked. You need to close & reopen the PR
for the checks to trigger. There are workarounds for this but they are
either less secure or require more setup.
The `pytest_pycollector_makeitem` argument `collector` is currently
annotated with type `PyCollector`. As part of #7469, that would have
required us to expose it in the public API. But really it's an
implementation detail, not something we want to expose. So replace the
annotation with the concrete python collector types that are passed.
Strictly speaking, `pytest_pycollector_makeitem` is called from
`PyCollector.collect()`, so the new type annotation is incorrect if
another type subclasses `PyCollector`. But the set of python collectors
is closed (mapping to language constructs), and the type is private, so
there shouldn't be any other deriving classes, and we can consider it
effectively sealed (unfortunately Python does not provide a way to
express this - yet?).
* scripts: Use release branch for changelog URL
With a prerelease, /stable won't show the correct changelog.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This reverts commit 15989ddc8f.
Reverts #8945, thus reintroducing #8926 temporarily, but we should be close to
the 7.0.0 release now, where this is the correct URL to use.
Closes#8831 as follow-up to #8858.
* Improve reference and path/fspath docs
Closes#9283
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fixups
* Add explanation
* Update wording after #9363
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Rename pytest_ignore_collect fspath parameter to collection_path
* Rename pytest_collect_file fspath parameter to file_path
* Rename pytest_pycollect_makemodule fspath parameter to module_path
* Rename pytest_report_header startpath parameter to start_path
* Rename pytest_report_collectionfinish startpath parameter to start_path
* Update docs with the renamed parameters
* Use pytest-flakes fork temporarily to prove it works
* Use pytest-flakes 4.0.5
Follow-up to #9309.
The issue in the changelog broke the docs build with:
doc/en/_changelog_towncrier_draft.rst:47: WARNING: Inline literal start-string
without end-string.
The second change isn't as critical, but caused the text to be rendered as
monospace including the tilde (i.e. `~pytest.PytestDeprecationwarning`).
Closes#7480.
This allows us to more easily follow our deprecation policy of turning
warnings into errors for the X.0 releases before complete removal in
X.1.
It also makes the deprecation timeline clear to both the users and
pytest developers -- it can be hard to keep track.
Note that the designation is not meant to be a binding contract - if the
time comes for removal of a specific deprecation but we decide it's too
soon, can just bump it to the next major.
Inspired by Django:
https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/submitting-patches/#deprecating-a-feature
* Remove changelog entry for #8251
Reverted in #8903
* Move #9202 changelog to to trivial
This won't concern users of pytest
* Streamline deprecation changelogs/docs
* Remove #8994 changelog
This is an impovement for a warning introduced in this release, so including it in a changelog against the last release seems confusing.
* Remove #9241 changelog
This is an impovement for a doc update introduced in this release, so including it in a changelog against the last release seems confusing. The issue number also seems about something different.
* Remove #8897 changelog
Empty file...
* Various minor changelog fixes
* nodes: keep plugins which subclass Item, File working for a bit more
Fix#8435.
* Update src/_pytest/nodes.py
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Florian Bruhin <me@the-compiler.org>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* porting pytest.skip() to use reason=, adding tests
* avoid adding **kwargs, it breaks other functionality, use optional msg= instead
* deprecation of `pytest.fail(msg=...)`
* fix bug with not capturing the returned reason value
* pass reason= in acceptance async tests instead of msg=
* finalising deprecations of `msg` in `pytest.skip()` and `pytest.fail()`
* Update doc/en/deprecations.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* Update doc/en/deprecations.rst
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
* fix failing test after upstream merge
* adding deprecation to `pytest.exit(msg=...)`
* add docs for pytest.exit deprecations
* finalising deprecation of msg for pytest.skip, pytest.exit and pytest.fail
* hold a reference to the Scope instance to please mypy
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Fixes#9272.
Fixing the issue directly in the plugin is somewhat hard, so do it in
core. Since the plugin is going to be deprecated, I figure it's OK to
cheat a bit.
The current PDF docs attempt to format the list of all plugins as a
table, without any word-wrapping of the plugin description. That results
in almost all the information getting cut off. This PR formats the same
information into more of a paragraph format for the PDF, with nothing
cut off.
Fixes#451
This is unfortunately a dependency on `py.path` which cannot be moved to
an external plugins or eased in any way, so has to be deprecated in
order for pytest to be able to eventually remove the dependency on `py`.
Export `HookRecorder`, `RecordedHookCall` (originally `ParsedCall`),
`RunResult`, `LineMatcher`.
These types are reachable through `Pytester` and so should be public
themselves for typing and other purposes.
The name `ParsedCall` I think is too generic under the `pytest`
namespace, so rename it to `RecordedHookCall`.
The `HookRecorder`'s constructor is made private -- it should only be
constructed by `Pytester`.
`LineMatcher` and `RunResult` are exported as is - no private and no
rename, since they're being used.
All of the classes are made final as they are not designed for
subclassing.
This causes `Session` documentation to be rendered again in full under
`pytester` and `testdir` in the API Reference. I tried but couldn't get
sphinx to hide it.
Since it's a pretty odd thing to have (should just use
`pytest.Session`), and I couldn't find any plugin which uses this, let's
just remove it.
`reportinfo()` is the last remaining py.path-only code path in pytest,
i.e. the last piece holding back py.path deprecation. The problem with
it is that plugins/users use it from both sides -- implementing it
(returning the value) and using it (using the return value). Dealing
with implementers is easy enough -- allow to return `os.PathLike[str]`.
But for callers who expect strictly `py.path` this will break and
there's not really a good way to provide backward compat for this.
From analyzing a corpus of 680 pytest plugins, the vast majority of
`reportinfo` appearances are implementations, and the few callers don't
actually access the path part of the return tuple.
As for test suites that might access `reportinfo` (e.g. using
`request.node.reportinfo()` or other ways), that is much harder to
survey, but from the ones I searched, I only found case
(`pytest_teamcity`, but even then it uses `str(fspath)` so is unlikely
to be affected in practice). They are better served with using
`node.location` or `node.path` directly.
Therefore, just break it and change the return type to
`str|os.PathLike[str]`.
Refs #7259.
This type is semi-private; not documented but many plugins access it
through `item.callspec`. However, plugins access the public fields and
almost none try to construct or monkeypatch it. So we should be allowed
to clean it up some.
- Convert to attrs, add slots and frozen
- Instead of doing `new = old.copy(); new.setmulti2()`, do `new =
old.setmulti()`. This is cleaner and faster.
- Remove the `metafunc` attribute. This causes a reference cycle
(multifunc._calls -> callspec -> multifunc) for no good reason --
neither pytest itself or plugins access this attribute, so let's not
keep the Metafunc objects alive past their due.
- Some comments.
I would have also like to make the dicts and lists themselves immutable,
however some plugins mess with those so that should be done separately,
if at all.
this issues is less likely to hit due to the recent regendoc release
which includes a wheel
* migrate to setuptools_scm 6.3.2
* use SETUPTOOLS_SCM_PRETEND_VERSION_FOR_PYTEST
Now that it's no longer using `@lru_cache`, use another check to avoid
re-computation. Although `@lru_cache` is faster than the full function
call + checks, this approach also has the advantage that the caching
works for more than 128 entries.
* Fix non-sensical error message
Introduced in 12de92cd2b / #7698
* Add a test
* Put the unit back into unittest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
A new plugin has this summary:
Continiously runs pytest on changes in *.py files
The `*` is interpreted as a special character and fails the CI.
Add some rudimentary escaping to hopefully prevent this.
The projects page unfortunately contains a lot of old projects, and some information there appears to be stale.
It was created at a time pytest was starting to be used, but seems pointless now as pytest is the most popular testing framework, so listing a few dozen projects which use it seems pointless.
Rather than updating it, I think we should remove it because this has the chance to become stale again soon, and not sure how valuable it is anyway.
The PDF documentation on readthedocs was missing the figures in the
fixtures reference chapter. This PR uses a Sphinx plugin that
automatically converts the checked-in SVG files to the PDF input files
that LaTeX requires.
The SVG-to-PDF conversion is done by inkscape, which gave the best
conversion among the tools I tried. However, it [does not yet
understand][href-bug] that you can write a plain `href` instead of
`xlink:href` in svg files, so I’ve had to edit the SVG files
accordingly.
[href-bug]: https://github.com/TeX-Live/luatex.git
Some of the top search-engine hits for pytest.approx use the function without actually comparing it to anything.
This PR will cause these tests to fail by implementing approx.__bool__() to raise an AssertionError that briefly explains how to correctly use approx.
* expose `warnings=` to pytester `assert_outcomes()`
* fix test fallout from adding warnings= to assert_outcomes()
* #closes 8593 - Improve test and add a `changelog` entry for the change
It is not clear yet how we should proceed with this deprecation
because `pytest.Item.reportinfo` is public API and returns a `py.path` object,
and is not clear how plugins and our examples should handle that.
Reverting just the deprecation aspect of #8251 so we can get a 7.0.0 release out.
We will reintroduce the deprecation later once we have a clear path moving forward with replacing `reportinfo`.
Closes#8445Closes#8821
the best practice is to only `<` when you know something is broken, and even then prefer `!=` assuming it will be fixed upstream for maximum compatibility
* fix-contributors-list
* remove-bots
* delete-extraneous-file
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* issue a warning when Items and Collector form a diamond
addresses #8435
* Apply suggestions from code review
Co-authored-by: Ran Benita <ran@unusedvar.com>
* Return support for the broken File/Item hybrids
* adds deprecation
* ads necessary support code in node construction
* fix incorrect mypy based assertions
* add docs for deprecation of Item/File inheritance
* warn when a non-cooperative ctor is encountered
* use getattr instead of cast to get the class __init__ for legacy ctors
* update documentation references for node inheritance
* clean up file+item inheritance test
enhance docs
move import upwards
Co-authored-by: Ran Benita <ran@unusedvar.com>
* cache: Move repetitive code to fixture
* cache: Explicitly test for chmod result
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
In the "Getting Started" doc, the test class instance example for
instance sharing doesn't actually demonstrate anything about the
reinstantiation of the class. This change shows clearly how the instance
data isn't retained between test runs.
* [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/asottile/pyupgrade: v2.16.0 → v2.18.2](https://github.com/asottile/pyupgrade/compare/v2.16.0...v2.18.2)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
`Parser` is used by many plugins and custom hooks. `OptionGroup` is
exposed by the `parser.addgroup` API.
The constructors of both are marked private, they are not meant to be
constructed directly.
Given a `RecursionError` traceback, the `Traceback.recursionindex()`
method returns the index of the frame which started the recursion
(repeated set of frames). To do so it attempts to check whether two
frames are equivalent. Just checking the function/line is not enough
because the recursion variable(s) might differ (e.g. imagine the numeric
value in a recursive factorial implementation). So it also compares the
`f_locals` (local variables) of each frame for equivalence.
For some reason, the locals comparison is wrapped in an `eval` whose
purpose is to evaluate the comparison in one of the compared frame's
context (locals + globals in scope). However, I can not think of any way
in which the global scope could affect the evaluation. It would have an
affect when the locals are bound but that's already done. So this seems
unnecessary - remove it.
* add feature to view fixture source location in invocations with --fixtures-per-test option
* remove unrelated changes to show_fixtures_per_test::test_doctest_items
* eshew the extraneous else in _show_fixtures_per_test.write_fixture
* enable the accommodation of multi-line docstring with --fixtures-per-test option
* add feature to view fixture source location in invocations with --fixtures
* add colour encoding to fixture location paths
* add changelog for #8606 fixing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Replace for loop using the operator
* Replace for loop with a generator expression inside any()
* Replace for loop with a dictionary comprehension
* Use list comprehension
* Simplify arguments for range()
* Change newfuncargs variable to in-line dictionary comprehension
* is_ancestor: return base.is_relative_to(query)
* Remove unneeded import of pathlib
* Try using PurePath
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Import PurePath on new line
* Revert and remove is_relative_to
Co-authored-by: Zachary Kneupper <zacharykneupper@Zacharys-MBP.lan>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Remove unnecessary else clause in repr_failure()
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Zachary Kneupper <zacharykneupper@Zacharys-MBP.lan>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Expand command line argument examples with validation examples
This shows how to pass more detailed error messages back to the user when
they've provided an incorrect value for a custom command line option.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix issue where TestCase.setUpClass is not called for test methods with a / in its name by checking if there is :: before the selected / or any :: after. Also added a test case for this.
* removed print statement that was added
* Change iterparentnodeids to consume / parts until the first ::. Then consider ::. Tests were changed to reflect this.
* Update changelog/8509.improvement.rst
Co-authored-by: Ran Benita <ran@unusedvar.com>
The `Sphinx-PyPI-upload` tool the section corresponds to hasn't been updated since 2009, and the new (c. 2018) PyPI doesn't support documentation uploads anyway.
* [pre-commit.ci] pre-commit autoupdate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
* Add tmp_path_factory methods to the docs
Add the methods of tmp_path_factory to the docs.
Currently, only the class description appears in the docs.
* Add tmpdir_factory methods to the docs
pytest uses a root temp directory named `/tmp/pytest-of-<username>`. The
name is predictable, and the directory might already exists from a
previous run, so that's allowed.
This makes it possible for my_user to pre-create
`/tmp/pytest-of-another_user`, thus giving my_user control of
another_user's tempdir.
Prevent this scenario by adding a couple of safety checks. I believe
they are sufficient.
Testing the first check requires changing the owner, which requires
root permissions, so can't be unit-tested easily, but I checked it
manually.
(Written for a Unix system, but might be applicable to Windows as well).
pytest creates a root temporary directory under /tmp, named
`pytest-of-<username>`, and creates tmp_path's and other under it.
/tmp is shared between all users of the system.
This root temporary directory was created with 0o777&~umask permissions,
which usually becomes 0o755, meaning any user in the system could list
and read the files, which is undesirable.
Use 0o700 permissions instead. Also for subdirectories, because the root
dir is adjustable.
Calling pkg_resources.fixup_namespace_packages() is only needed for packages
that use pkg_resources.declare_namespace() and hence they already imported
pkg_resources. When pkg_resources is not imported, we don't need to use it.
This avoids an unneeded runtime dependency on setuptools.
The code is tested by test_syspath_prepend_with_namespace_packages,
behavior should remain unchanged, hence no new test was added.
When people drop pkg_resources from sys.modules, they are on their own.
If someone has a actual use case making this valid to support,
they can come in and provide a test, a reference and a fix.
Last time I "fixed" this I left a `\` at the start of the string
to avoid an initial newline, but didn't realize it was a raw string.
This should fix it now for good.
* Changed "Table Of Contents" link to a plain heading.
* Removed Customize, API Reference, 3rd party plugins links altogether.
* Added an "About the project" heading
* re-ordered and grouped how-to guides in how-to landing page
* retitled moved documents appropriately
* separated writing plugins/writing hook functions into two documents
Warnings like this:
python3.9/site-packages/setuptools/dist.py:634: UserWarning: Usage of
dash-separated 'upload-dir' will not be supported in future versions.
Please use the underscore name 'upload_dir' instead
This type is most prominent in `pytest.raises` and we should allow to
refer to it by a public name.
The type is not in a perfectly "exposable" state. In particular:
- The `traceback` property with type `Traceback` which is derived from
the `py.code` API and exposes a bunch more types transitively. This
stuff is *not* exported and probably won't be.
- The `getrepr` method which probably should be private.
But they're already used in the wild so no point in just hiding them
now.
The __init__ API is hidden -- the public API for this are the `from_*`
classmethods.
The prefixes make the API Reference docs (for e.g. `pytest.raises`,
`pytest.fixture`) uglier.
Being under `_pytest` is sufficient from a privacy perspective, so let's
drop them.
Sections have been moved to:
* reference/fixtures.rst
* how-to/fixtures.rst
* fixtures.rst
according to their function. Further refinement and rewriting will be required.
Some material has been moved to a new "Anatomy of a test" document, in
anticipation that material from other sections will also find a natural
home there later.
Removed several unneeded reference targets from fixtures documentation.
Moved various documents into subdirectories, how-to and
reference.
Updated multiple links to use `:ref:` instead of `:doc:`,
meaning that files can henceforth be moved around without
breaking references.
This configures Sphinx autodoc to include the type annotations
along with the description of the function/method, instead of including
it into the signature.
Fix#8405
* Fix test_strict_and_skip
The `--strict` argument was removed in #2552, but the removal wasn't
actually correct - see #1472.
* Fix argument handling in pytest.mark.skip
See #8384
* Raise from None
* Fix test name
* retry writing pytest-of dir when invalid chars are in directory name
* add unit tests for getbasetemp() and changelog
* patch _basetemp & _given_basetemp for testing basetemp()
* Tweak changelog for #8317, tidy up comments
Similarly to #7143, at work we have a project with a custom pytest.Class
subclass, adding an additional argument to the constructor.
All from_parent implementations in pytest accept and forward *kw, except
Class (before this change) and DoctestItem - since I'm not familiar with
doctest support, I've left the latter as-is.
doc This paragraph looks like it is a more verbose version of the sentence right above it. Removing it doesn't reduce the amount of information here but does make the section flow a little better.
In assert.rst at line 175 and further there is an example of an assert encountering comparisons. The code-block for this example starts with a comment (line 177) and then it has 2 empty lines. Comparing this example code (test_assert2.py) with the previously mentioned example code on the same page (i.e. test_assert1.py) you can see that there should not be 2 empty lines after the comment. These 2 empty lines are removed.
* Type annotation polishing for symbols around Pytester.run
Hopefully these will help document readers understand pertinent methods
and constants better.
Following up #8294
* Use NOTSET instead of object
Also update these tests ensure they pass, and be explicit about the test
file called in an existing test to avoid unintentional calls to the
added tests
SetupState maintains its own state, so it can store the exception
itself, instead of using the node's store, which is better avoided when
possible.
This also reduces the lifetime of the reference-cycle-inducing exception
objects which is never a bad thing.
The assertion ensures that when `addfinalizer(finalizer, node)` is
called, the node is in the stack. This then would ensure that the
finalization is actually properly executed properly during the node's
teardown. Anything else indicates something is wrong.
Previous commits fixed all of the tests which previously failed this, so
can be reenabeld now.
Previously the tests (probably unintentionally) mixed a fresh SetupState
and the generated item Session's SetupState, which led to some serious
head scratching when prodding it a bit.
When the stack is empty, the finalizers which are supposed to be
attached to nodes in the stack really ought to be empty as well. So the
code here is dead. If this doesn't happen, the assert will trigger.
It seems the code that would not install pytest's faulthandler support
if it was already enabled is not really needed at all, and even detrimental
when using `python -X dev -m pytest` to run Python in "dev" mode.
Also simplified the plugin by removing the hook class, now the hooks
will always be active so there's no need to delay the hook definitions anymore.
Fix#8258
The Twisted Logger will return an invalid file descriptor since it is
not backed by an FD. So, let's also forward this to the same code path
as with `pytest-xdist`.
as PyObjMixin is always supposed to be mixed in the mro
before nodes.Node the behavior doesn't change,
but all the typing information carry over to help mypy.
extracted from #8037
It is not very clear why this code exists -- we are not running any
unittest or nose code during collection, and really these frameworks
don't have the concept of collection at all, and just raising these
exceptions at e.g. the module level would cause an error. So unless I'm
missing something, I don't think anyone is using this.
Deprecate it so we can eventually clear up this code and keep unittest
more tightly restricted to its plugin.
When `pytest.skip()` is called inside a test function, the skip location
should be reported as the line that made the call, however when
`pytest.skip()` is called by the `pytest.mark.skip` and similar
mechanisms, the location should be reported at the item's location,
because the exact location is some irrelevant internal code.
Currently the item-location case is implemented by the caller setting a
boolean key on the item's store and the `skipping` plugin checking it
and fixing up the location if needed. This is really roundabout IMO and
breaks encapsulation.
Instead, allow the caller to specify directly on the skip exception
whether to use the item's location or not. For now, this is entirely
private.
When line_index was a large negative number, get_source failed
on `source.lines[line_index]`.
Use the same dummy Source as with a large positive line_index.
Pytest document currently does not index the top-level package name
`pytest`, which causes some trouble when building documentation that
cross-refers to the pytest package via ``:mod:`pytest` ``.
In https://docs.pytest.org/en/stable/reference.html#testdir, it is
suggested:
> New code should avoid using testdir in favor of pytester.
Multiple spots in the documents still use testdir and they can be quite
confusing (especially the plugin writing guide).
* [pre-commit.ci] pre-commit autoupdate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* manual fixes after configuration update
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Anthony Sottile <asottile@umich.edu>
The type cannot be constructed directly, but is exported for use in type
annotations, since it is reachable through existing public API.
This also documents `from_call` as public, because at least
pytest-forked uses it, so we must treat it as public already anyway.
On Windows, os.path.samefile returns false for paths mounted in UNC paths which
point to the same location.
I couldn't reproduce the actual case reported, but looking at the code it seems
this commit should fix the issue.
Fix#7678Fix#8076
2020-12-12 08:54:49 -03:00
289 changed files with 29061 additions and 12281 deletions
@@ -13,7 +13,7 @@ If this change fixes an issue, please:
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/master/changelog/README.rst) for details.
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/main/changelog/README.rst) for details.
Write sentences in the **past or present tense**, examples:
Look through the `GitHub issues for bugs <https://github.com/pytest-dev/pytest/labels/type:%20bug>`_.
See also the `"status: easy" issues <https://github.com/pytest-dev/pytest/labels/status%3A%20easy>`_
that are friendly to new contributors.
:ref:`Talk <contact>` to developers to find out how you can fix specific bugs. To indicate that you are going
to work on a particular issue, add a comment to that effect on the specific issue.
@@ -160,7 +162,7 @@ the following:
- an issue tracker for bug reports and enhancement requests.
- a `changelog <http://keepachangelog.com/>`_.
- a `changelog <https://keepachangelog.com/>`_.
If no contributor strongly objects and two agree, the repository can then be
transferred to the ``pytest-dev`` organisation.
@@ -221,7 +223,7 @@ changes you want to review and merge. Pull requests are stored on
Once you send a pull request, we can discuss its potential modifications and
even add more commits to it later on. There's an excellent tutorial on how Pull
Requests work in the
`GitHub Help Center <https://help.github.com/articles/using-pull-requests/>`_.
`GitHub Help Center <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>`_.
Here is a simple overview, with pytest-specific bits:
@@ -234,9 +236,9 @@ Here is a simple overview, with pytest-specific bits:
Given we have "major.minor.micro" version numbers, bug fixes will usually
be released in micro releases whereas features will be released in
@@ -259,7 +261,7 @@ Here is a simple overview, with pytest-specific bits:
Tox is used to run all the tests and will automatically setup virtualenvs
to run the tests in.
(will implicitly use http://www.virtualenv.org/en/latest/)::
(will implicitly use https://virtualenv.pypa.io/en/latest/)::
$ pip install tox
@@ -318,26 +320,26 @@ Here is a simple overview, with pytest-specific bits:
compare: your-branch-name
base-fork: pytest-dev/pytest
base: master
base: main
Writing Tests
~~~~~~~~~~~~~
Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/stable/reference.html#testdir>`_, as a "black-box" test.
Writing tests for plugins or for pytest itself is often done using the `pytester fixture <https://docs.pytest.org/en/stable/reference/reference.html#pytester>`_, as a "black-box" test.
For example, to ensure a simple test passes you can write:
..code-block::python
deftest_true_assertion(testdir):
testdir.makepyfile(
deftest_true_assertion(pytester):
pytester.makepyfile(
"""
def test_foo():
assert True
"""
)
result=testdir.runpytest()
result=pytester.runpytest()
result.assert_outcomes(failed=0,passed=1)
@@ -346,14 +348,14 @@ Alternatively, it is possible to make checks based on the actual output of the t
When choosing a file where to write a new test, take a look at the existing files and see if there's
@@ -387,15 +389,22 @@ Suppose for example that the latest release was 1.2.3, and you want to include
a bug fix in 1.2.4 (check https://github.com/pytest-dev/pytest/releases for the
actual latest release). The procedure for this is:
#. First, make sure the bug is fixed the ``master`` branch, with a regular pull
#. First, make sure the bug is fixed the ``main`` branch, with a regular pull
request, as described above. An exception to this is if the bug fix is not
applicable to ``master`` anymore.
applicable to ``main`` anymore.
#.``git checkout origin/1.2.x -b backport-XXXX`` # use the master PR number here
Automatic method:
Add a ``backport 1.2.x`` label to the PR you want to backport. This will create
a backport PR against the ``1.2.x`` branch.
Manual method:
#.``git checkout origin/1.2.x -b backport-XXXX`` # use the main PR number here
#. Locate the merge commit on the PR, in the *merged* message, for example:
nicoddemus merged commit 0f8b462 into pytest-dev:master
nicoddemus merged commit 0f8b462 into pytest-dev:main
#.``git cherry-pick -x -m1 REVISION`` # use the revision you found above (``0f8b462``).
@@ -408,8 +417,8 @@ actual latest release). The procedure for this is:
Who does the backporting
~~~~~~~~~~~~~~~~~~~~~~~~
As mentioned above, bugs should first be fixed on ``master`` (except in rare occasions
that a bug only happens in a previous release). So who should do the backport procedure described
As mentioned above, bugs should first be fixed on ``main`` (except in rare occasions
that a bug only happens in a previous release). So, who should do the backport procedure described
above?
1. If the bug was fixed by a core developer, it is the main responsibility of that core developer
@@ -417,8 +426,8 @@ above?
2. However, often the merge is done by another maintainer, in which case it is nice of them to
do the backport procedure if they have the time.
3. For bugs submitted by non-maintainers, it is expected that a core developer will to do
the backport, normally the one that merged the PR on ``master``.
4. If a non-maintainers notices a bug which is fixed on ``master`` but has not been backported
the backport, normally the one that merged the PR on ``main``.
4. If a non-maintainers notices a bug which is fixed on ``main`` but has not been backported
(due to maintainers forgetting to apply the *needs backport* label, or just plain missing it),
they are also welcome to open a PR with the backport. The procedure is simple and really
helps with the maintenance of the project.
@@ -447,7 +456,7 @@ can always reopen the issue/pull request in their own time later if it makes sen
When to close
~~~~~~~~~~~~~
Here are a few general rules the maintainers use to decide when to close issues/PRs because
Here are a few general rules the maintainers use deciding when to close issues/PRs because
of lack of inactivity:
* Issues labeled ``question`` or ``needs information``: closed after 14 days inactive.
@@ -459,15 +468,15 @@ The above are **not hard rules**, but merely **guidelines**, and can be (and oft
Closing pull requests
~~~~~~~~~~~~~~~~~~~~~
When closing a Pull Request, it needs to be acknowledge the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
When closing a Pull Request, it needs to be acknowledging the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss a stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
Hi <contributor>,
First of all we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
First of all, we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
We noticed it has been awhile since you have updated this PR, however. pytest is a high activity project, with many issues/PRs being opened daily, so it is hard for us maintainers to track which PRs are ready for merging, for review, or need more attention.
So for those reasons we think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
So for those reasons we, think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
Again we appreciate your time for working on this, and hope you might get back to this at a later time!
#. For major and minor releases, change the default version in the `Read the Docs Settings <https://readthedocs.org/dashboard/pytest/advanced/>`_ to the new branch.
#. Send an email announcement with the contents from::
Showing inner exceptions by forcing native display in ``ExceptionGroups`` even when using display options other than ``--tb=native``. A temporary step before full implementation of pytest-native display for inner exceptions in ``ExceptionGroups``.
@@ -10,10 +10,9 @@ Are you an enthusiastic pytest user, the local testing guru in your workplace? O
We will pair experienced pytest users with open source projects, for a month's effort of getting new development teams started with pytest.
In 2015 we are trying this for the first time. In February and March 2015 we will gather volunteers on both sides, in April we will do the work, and in May we will evaluate how it went. This effort is being coordinated by Brianna Laugher. If you have any questions or comments, you can raise them on the `@pytestdotorg twitter account <https://twitter.com/pytestdotorg>`_ the `issue tracker`_ or the `pytest-dev mailing list`_.
In 2015 we are trying this for the first time. In February and March 2015 we will gather volunteers on both sides, in April we will do the work, and in May we will evaluate how it went. This effort is being coordinated by Brianna Laugher. If you have any questions or comments, you can raise them on the `@pytestdotorg twitter account <https://twitter.com/pytestdotorg>`_\, the :issue:`issue tracker <676>` or the `pytest-dev mailing list`_.
@@ -22,7 +22,9 @@ b) transitional: the old and new API don't conflict
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn them into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationwarning`).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g. `PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
c) true breakage: should only be considered when normal transition is unreasonably unsustainable and would offset important development/features by years.
@@ -30,15 +32,15 @@ c) true breakage: should only be considered when normal transition is unreasonab
Examples for such upcoming changes:
* removal of ``pytest_runtest_protocol/nextitem`` - `#895`_
* removal of ``pytest_runtest_protocol/nextitem`` - :issue:`895`
* rearranging of the node tree to include ``FunctionDefinition``
* rearranging of ``SetupState```#895`_
* rearranging of ``SetupState``:issue:`895`
True breakages must be announced first in an issue containing:
* Detailed description of the change
* Rationale
* Expected impact on users and plugin authors (example in `#895`_)
* Expected impact on users and plugin authors (example in :issue:`895`)
After there's no hard *-1* on the issue it should be followed up by an initial proof-of-concept Pull Request.
@@ -77,4 +79,16 @@ Features currently deprecated and removed in previous releases can be found in :
We track future deprecation and removal of features using milestones and the `deprecation <https://github.com/pytest-dev/pytest/issues?q=label%3A%22type%3A+deprecation%22>`_ and `removal <https://github.com/pytest-dev/pytest/labels/type%3A%20removal>`_ labels on GitHub.
In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
*:hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
*:hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
*:hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
*:hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
*:hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
..note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
:ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
being ``path``) is **the opposite** of the situation for hooks (the old
argument being ``path``).
This is an unfortunate artifact due to historical reasons, which should be
resolved in future versions as we slowly get rid of the :pypi:`py`
dependency (see :issue:`9283` for a longer discussion).
Directly constructing internal classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.0
Directly constructing the following classes is now deprecated:
-``_pytest.mark.structures.Mark``
-``_pytest.mark.structures.MarkDecorator``
-``_pytest.mark.structures.MarkGenerator``
-``_pytest.python.Metafunc``
-``_pytest.runner.CallInfo``
-``_pytest._code.ExceptionInfo``
-``_pytest.config.argparsing.Parser``
-``_pytest.config.argparsing.OptionGroup``
-``_pytest.pytester.HookRecorder``
These constructors have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.
.._cmdline-preparse-deprecated:
Passing ``msg=`` to ``pytest.skip``, ``pytest.fail`` or ``pytest.exit``
Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
Some plugins providing linting/code analysis have been using this as a hack.
Instead, a separate collector node should be used, which collects the item. See
:ref:`non-python tests` for an example, as well as an `example pr fixing inheritance`_.
If custom subclasses of nodes like :class:`pytest.Item` override the
``__init__`` method, they should take ``**kwargs``. Thus,
..code-block::python
classCustomItem(pytest.Item):
def__init__(self,name,parent,additional_arg):
super().__init__(name,parent)
self.additional_arg=additional_arg
should be turned into:
..code-block::python
classCustomItem(pytest.Item):
def__init__(self,*,additional_arg,**kwargs):
super().__init__(**kwargs)
self.additional_arg=additional_arg
to avoid hard-coding the arguments pytest can pass to the superclass.
See :ref:`non-python tests` for a full example.
For cases without conflicts, no deprecation warning is emitted. For cases with
conflicts (such as :class:`pytest.File` now taking ``path`` instead of
``fspath``, as :ref:`outlined above <node-ctor-fspath-deprecation>`), a
deprecation warning is now raised.
Backward compatibilities in ``Parser.addoption``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 2.4
Several behaviors of :meth:`Parser.addoption <pytest.Parser.addoption>` are now
scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):
-``parser.addoption(..., help=".. %default ..")`` - use ``%(default)s`` instead.
-``parser.addoption(..., type="int/string/float/complex")`` - use ``type=int`` etc. instead.
Using ``pytest.warns(None)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 7.0
:func:`pytest.warns(None) <pytest.warns>` is now deprecated because it was frequently misused.
Its correct usage was checking that the code emits at least one warning of any type - like ``pytest.warns()``
or ``pytest.warns(Warning)``.
See :ref:`warns use cases` for examples.
The ``--strict`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -41,29 +275,42 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Removed Features
----------------
..deprecated:: 6.0
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recored`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest.collect`` module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
The ``pytest.collect`` module is no longer part of the public API, all its names
should now be imported from ``pytest`` directly instead.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recorded`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest._fillfuncargs`` function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 6.0
..versionremoved:: 7.0
This function was kept for backward compatibility with an older plugin.
@@ -72,12 +319,6 @@ it, use `function._request._fillfixtures()` instead, though note this is not
a public API and may break in the future.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
``--no-print-logs`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -92,6 +333,7 @@ A ``--show-capture`` command-line option was added in ``pytest 3.5.0`` which all
display captured output when tests fail: ``no``, ``stdout``, ``stderr``, ``log`` or ``all`` (the default).
.._resultlog deprecated:
Result log (``--result-log``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -114,8 +356,8 @@ at some point, depending on the plans for the plugins and number of users using
..versionremoved:: 6.0
The ``pytest_collect_directory`` has not worked properly for years (it was called
but the results were ignored). Users may consider using :func:`pytest_collection_modifyitems <_pytest.hookspec.pytest_collection_modifyitems>` instead.
The ``pytest_collect_directory`` hook has not worked properly for years (it was called
but the results were ignored). Users may consider using :hook:`pytest_collection_modifyitems` instead.
TerminalReporter.writer
~~~~~~~~~~~~~~~~~~~~~~~
@@ -129,6 +371,8 @@ with ``py.io.TerminalWriter``.
Plugins that used ``TerminalReporter.writer`` directly should instead use ``TerminalReporter``
methods that provide the same functionality.
.._junit-family changed default value:
``junit_family`` default value change to "xunit2"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -216,6 +460,8 @@ in places where we or plugin authors must distinguish between fixture names and
names supplied by non-fixture things such as ``pytest.mark.parametrize``.
.._pytest.config global deprecated:
``pytest.config`` global
~~~~~~~~~~~~~~~~~~~~~~~~
@@ -260,7 +506,7 @@ Becomes:
If you still have concerns about this deprecation and future removal, please comment on
@@ -234,17 +234,17 @@ You can ask which markers exist for your test suite - the list includes our just
@pytest.mark.slow: mark test as slow.
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/warnings.html#pytest-mark-filterwarnings
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-skipif
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-skipif
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-xfail
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-xfail
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/parametrize.html for more info and examples.
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/how-to/parametrize.html for more info and examples.
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/fixture.html#usefixtures
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@@ -397,9 +397,8 @@ the test needs:
$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_someenv.py s [100%]
@@ -412,9 +411,8 @@ and here is one that specifies exactly the environment needed:
$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_someenv.py . [100%]
@@ -428,17 +426,17 @@ The ``--markers`` option always gives you a list of available markers:
$ pytest --markers
@pytest.mark.env(name): mark test to run only on named environment
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/warnings.html#pytest-mark-filterwarnings
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-skipif
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-skipif
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-xfail
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-xfail
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/parametrize.html for more info and examples.
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/how-to/parametrize.html for more info and examples.
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/fixture.html#usefixtures
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@@ -488,7 +486,7 @@ The output is as follows:
..code-block::pytest
$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef>,), kwargs={})
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef0001>,), kwargs={})
.
1 passed in 0.12s
@@ -605,9 +603,8 @@ then you will see two tests skipped and two executed tests as expected:
$ pytest -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items
test_plat.py s.s. [100%]
@@ -622,9 +619,8 @@ Note that if you specify a platform via the marker-command line option like this
$ pytest -m linux
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 3 deselected / 1 selected
test_plat.py . [100%]
@@ -686,9 +682,8 @@ We can now use the ``-m option`` to select one set:
$ pytest -m interface --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 4 items / 2 deselected / 2 selected
test_module.py FF [100%]
@@ -713,9 +708,8 @@ or to select both "event" and "interface" tests:
$ pytest -m "interface or event" --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
Here is an example ``conftest.py`` (extracted from Ali Afshar's special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yaml`` files and will execute the yaml-formatted content as custom tests:
@@ -22,16 +21,15 @@ You can create a simple example file:
..include:: nonpython/test_simple.yaml
:literal:
and if you installed `PyYAML`_ or a compatible YAML-parser you can
and if you installed :pypi:`PyYAML` or a compatible YAML-parser you can
now execute the test specification:
..code-block::pytest
nonpython $ pytest test_simple.yaml
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project/nonpython
collected 2 items
test_simple.yaml F. [100%]
@@ -66,9 +64,9 @@ consulted when reporting in ``verbose`` mode:
nonpython $ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project/nonpython
collecting ... collected 2 items
test_simple.yaml::hello FAILED [ 50%]
@@ -92,9 +90,8 @@ interesting to just look at the collection tree:
nonpython $ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
@@ -28,7 +28,7 @@ Flaky tests sometimes appear when a test suite is run in parallel (such as use o
Overly strict assertion
~~~~~~~~~~~~~~~~~~~~~~~
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/stable/reference.html#pytest-approx>`_ is useful here.
Overly strict assertions can cause problems with floating point comparison as well as timing issues. :func:`pytest.approx` is useful here.
Pytest features
@@ -94,7 +94,7 @@ Mark Lapierre discusses the `Pros and Cons of Quarantined Tests <https://dev.to/
CI tools that rerun on failure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/azure/devops/release-notes/2017/dec-11-vsts#identify-flaky-tests>`_ and rerun failed tests.
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/previous-versions/azure/devops/2017/dec-11-vsts?view=tfs-2017#identify-flaky-tests>`_ and rerun failed tests.
@@ -113,7 +113,7 @@ Resources
* `Eradicating Non-Determinism in Tests <https://martinfowler.com/articles/nonDeterminism.html>`_ by Martin Fowler, 2011
* `No more flaky tests on the Go team <https://www.thoughtworks.com/insights/blog/no-more-flaky-tests-go-team>`_ by Pavan Sudarshan, 2012
* `The Build That Cried Broken: Building Trust in your Continuous Integration Tests <https://www.youtube.com/embed/VotJqV4n8ig>`_ talk (video) by `Angie Jones <http://angiejones.tech/>`_ at SeleniumConf Austin 2017
* `The Build That Cried Broken: Building Trust in your Continuous Integration Tests <https://www.youtube.com/embed/VotJqV4n8ig>`_ talk (video) by `Angie Jones <https://angiejones.tech/>`_ at SeleniumConf Austin 2017
* `Test and Code Podcast: Flaky Tests and How to Deal with Them <https://testandcode.com/50>`_ by Brian Okken and Anthony Shaw, 2018
pytest as a testing framework needs to import test modules and ``conftest.py`` files for execution.
Importing files in Python (at least until recently) is a non-trivial processes, often requiring
changing `sys.path <https://docs.python.org/3/library/sys.html#sys.path>`__. Some aspects of the
changing :data:`sys.path`. Some aspects of the
import process can be controlled through the ``--import-mode`` command-line flag, which can assume
these values:
* ``prepend`` (default): the directory path containing each module will be inserted into the *beginning*
of ``sys.path`` if not already there, and then imported with the `__import__ <https://docs.python.org/3/library/functions.html#__import__>`__ builtin.
of :py:data:`sys.path` if not already there, and then imported with the :func:`__import__ <__import__>` builtin.
This requires test module names to be unique when the test directory tree is not arranged in
packages, because the modules will put in ``sys.modules`` after importing.
packages, because the modules will put in :py:data:`sys.modules` after importing.
This is the classic mechanism, dating back from the time Python 2 was still supported.
* ``append``: the directory containing each module is appended to the end of ``sys.path`` if not already
* ``append``: the directory containing each module is appended to the end of :py:data:`sys.path` if not already
there, and imported with ``__import__``.
This better allows to run test modules against installed versions of a package even if the
@@ -41,17 +41,23 @@ these values:
we advocate for using :ref:`src <src-layout>` layouts.
Same as ``prepend``, requires test module names to be unique when the test directory tree is
not arranged in packages, because the modules will put in ``sys.modules`` after importing.
not arranged in packages, because the modules will put in :py:data:`sys.modules` after importing.
* ``importlib``: new in pytest-6.0, this mode uses `importlib <https://docs.python.org/3/library/importlib.html>`__ to import test modules. This gives full control over the import process, and doesn't require
changing ``sys.path`` or ``sys.modules`` at all.
* ``importlib``: new in pytest-6.0, this mode uses :mod:`importlib` to import test modules. This gives full control over the import process, and doesn't require changing :py:data:`sys.path`.
For this reason this doesn't require test module names to be unique at all, but also makes test
modules non-importable by each other. This was made possible in previous modes, for tests not residing
in Python packages, because of the side-effects of changing ``sys.path`` and ``sys.modules``
mentioned above. Users which require this should turn their tests into proper packages instead.
For this reason this doesn't require test module names to be unique.
One drawback however is that test modules are non-importable by each other. Also, utility
modules in the tests directories are not automatically importable because the tests directory is no longer
added to :py:data:`sys.path`.
Initially we intended to make ``importlib`` the default in future releases, however it is clear now that
it has its own set of drawbacks so the default will remain ``prepend`` for the foreseeable future.
..seealso::
The :confval:`pythonpath` configuration variable.
We intend to make ``importlib`` the default in future releases.
``prepend`` and ``append`` import modes scenarios
-------------------------------------------------
@@ -133,4 +139,4 @@ Running pytest with ``pytest [...]`` instead of ``python -m pytest [...]`` yield
equivalent behaviour, except that the latter will add the current directory to ``sys.path``, which
**Documentation as PDF**: `download latest <https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf>`_
``pytest`` is a framework that makes building simple and scalable tests easy. Tests are expressive and readable—no boilerplate code required. Get started in minutes with a small unit test or complex functional test for your application or library.
.._`getstarted`:
.._`installation`:
Install ``pytest``
----------------------------------------
``pytest`` requires: Python 3.7+ or PyPy3.
1. Run the following command in your command line:
Create a simple test function with just four lines of code:
Create a new file called ``test_sample.py``, containing a function, and a test:
..code-block::python
@@ -47,15 +41,14 @@ Create a simple test function with just four lines of code:
deftest_answer():
assertfunc(3)==5
That’s it. You can now execute the test function:
The test
..code-block::pytest
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 1 item
test_sample.py F [100%]
@@ -77,7 +70,7 @@ The ``[100%]`` refers to the overall progress of running all test cases. After i
..note::
You can use the ``assert`` statement to verify test expectations. pytest’s `Advanced assertion introspection <http://docs.python.org/reference/simple_stmts.html#the-assert-statement>`_ will intelligently report intermediate values of the assert expression so you can avoid the many names `of JUnit legacy methods <http://docs.python.org/library/unittest.html#test-cases>`_.
You can use the ``assert`` statement to verify test expectations. pytest’s :ref:`Advanced assertion introspection <python:assert>` will intelligently report intermediate values of the assert expression so you can avoid the many names :ref:`of JUnit legacy methods <testcase-objects>`.
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/stable/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
``pytest`` provides :std:doc:`Builtin fixtures/function arguments <builtin>` to request arbitrary resources, like a unique temporary directory:
..code-block::python
# content of test_tmpdir.py
deftest_needsfiles(tmpdir):
print(tmpdir)
# content of test_tmp_path.py
deftest_needsfiles(tmp_path):
print(tmp_path)
assert0
List the name ``tmpdir`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory:
List the name ``tmp_path`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory:
before you import it (a good place to do that is in your root ``conftest.py``).
For further information, Benjamin Peterson wrote up `Behind the scenes of pytest's new assertion rewriting <http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html>`_.
@@ -86,9 +86,8 @@ If you then run it with ``--lf``:
$ pytest --lf
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 2 items
run-last-failure: rerun previous 2 failures
@@ -133,9 +132,8 @@ of ``FF`` and dots):
$ pytest --ff
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 50 items
run-last-failure: rerun previous 2 failures first
@@ -277,73 +275,14 @@ You can always peek at the content of the cache using the
$ pytest --cache-show
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
cachedir: /home/sweet/project/.pytest_cache
--------------------------- cache values for '*' ---------------------------
cache/lastfailed contains:
{'test_50.py::test_num[17]': True,
'test_50.py::test_num[25]': True,
'test_assert1.py::test_function': True,
'test_assert2.py::test_set_comparison': True,
'test_caching.py::test_function': True,
'test_foocompare.py::test_compare': True}
{'test_caching.py::test_function': True}
cache/nodeids contains:
['test_50.py::test_num[0]',
'test_50.py::test_num[10]',
'test_50.py::test_num[11]',
'test_50.py::test_num[12]',
'test_50.py::test_num[13]',
'test_50.py::test_num[14]',
'test_50.py::test_num[15]',
'test_50.py::test_num[16]',
'test_50.py::test_num[17]',
'test_50.py::test_num[18]',
'test_50.py::test_num[19]',
'test_50.py::test_num[1]',
'test_50.py::test_num[20]',
'test_50.py::test_num[21]',
'test_50.py::test_num[22]',
'test_50.py::test_num[23]',
'test_50.py::test_num[24]',
'test_50.py::test_num[25]',
'test_50.py::test_num[26]',
'test_50.py::test_num[27]',
'test_50.py::test_num[28]',
'test_50.py::test_num[29]',
'test_50.py::test_num[2]',
'test_50.py::test_num[30]',
'test_50.py::test_num[31]',
'test_50.py::test_num[32]',
'test_50.py::test_num[33]',
'test_50.py::test_num[34]',
'test_50.py::test_num[35]',
'test_50.py::test_num[36]',
'test_50.py::test_num[37]',
'test_50.py::test_num[38]',
'test_50.py::test_num[39]',
'test_50.py::test_num[3]',
'test_50.py::test_num[40]',
'test_50.py::test_num[41]',
'test_50.py::test_num[42]',
'test_50.py::test_num[43]',
'test_50.py::test_num[44]',
'test_50.py::test_num[45]',
'test_50.py::test_num[46]',
'test_50.py::test_num[47]',
'test_50.py::test_num[48]',
'test_50.py::test_num[49]',
'test_50.py::test_num[4]',
'test_50.py::test_num[5]',
'test_50.py::test_num[6]',
'test_50.py::test_num[7]',
'test_50.py::test_num[8]',
'test_50.py::test_num[9]',
'test_assert1.py::test_function',
'test_assert2.py::test_set_comparison',
'test_caching.py::test_function',
'test_foocompare.py::test_compare']
['test_caching.py::test_function']
cache/stepwise contains:
[]
example/value contains:
@@ -358,10 +297,9 @@ filtering:
$ pytest --cache-show example/*
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
cachedir: /home/sweet/project/.pytest_cache
----------------------- cache values for 'example/*' -----------------------
example/value contains:
42
@@ -383,7 +321,9 @@ servers where isolation and correctness is more important
than speed.
.._cache stepwise:
Stepwise
--------
As an alternative to ``--lf -x``, especially for cases where you expect a large part of the test suite will fail, ``--sw``, ``--stepwise`` allows you to fix them one at a time. The test suite will run until the first failure and then stop. At the next invocation, tests will continue from the last failing test and then run until the next failing test. You may use the ``--stepwise-skip`` option to ignore one failing test and stop the test execution on the second failing test instead. This is useful if you get stuck on a failing test and just want to ignore it until later.
As an alternative to ``--lf -x``, especially for cases where you expect a large part of the test suite will fail, ``--sw``, ``--stepwise`` allows you to fix them one at a time. The test suite will run until the first failure and then stop. At the next invocation, tests will continue from the last failing test and then run until the next failing test. You may use the ``--stepwise-skip`` option to ignore one failing test and stop the test execution on the second failing test instead. This is useful if you get stuck on a failing test and just want to ignore it until later. Providing ``--stepwise-skip`` will also enable ``--stepwise`` implicitly.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.