The env var effects all of the pip installs, including regendoc which
also uses setuptools-scm, so it gets the wrong version, and fails to
install with the new pip resolver:
ERROR: Requested regendoc from 206e495142/regendoc-0.6.1.tar.gz (sha256)=db1e8c9ae02c1af559eae105bfd77ba41ed07fc8ca7030ea59db5f3f161236a4 has different version in metadata: '6.2.0'
Add a new hook , `pytest_markeval_namespace` which should return a dictionary.
This dictionary will be used to augment the "global" variables available to evaluate skipif/xfail/xpass markers.
Pseudo example
``conftest.py``:
.. code-block:: python
def pytest_markeval_namespace():
return {"color": "red"}
``test_func.py``:
.. code-block:: python
@pytest.mark.skipif("color == 'blue'", reason="Color is not red")
def test_func():
assert False
* Custom multiple marker execution order
https://github.com/pytest-dev/pytest/issues/8020 issue stated that ordering of multiple custom markers is from inside - out. I have added example for the same in the documentation. Please let me know for further changes / concerns.
* remove trailing spaces
The last commit was failing due to extra spaces
* Ran tox tests locally to debug white space trimming issues
* Resolve: ERROR: docs: commands failed for tox -e docs
* Update doc/en/reference.rst
Committed PR suggestions.
Co-authored-by: Florian Bruhin <me@the-compiler.org>
* Added reference to Node.iter_markers_with_node in documentation
* Add myself to Authors
Co-authored-by: Shubham <shubham.adep@wsu.edu>
Co-authored-by: Florian Bruhin <me@the-compiler.org>
Explanation: The default handling of these lifetimes is done in
`tmpdir.TempPathFactory.getbasetemp`, which passes `keep=3` to
`pathlib.make_numbered_dir_with_cleanup`.
GH Issue: #8036
* Prefix contextmanagers with module name in doc examples
* Import pytest explicitly for doctests
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Python 3.7 changes the pyc format by adding a flags byte. Even though it
is not necessary for us to match it, it is nice to be able to read pyc
files we emit for debugging the rewriter.
Update our custom pyc files to use that format. We write flags==0
meaning we still use the mtime+size format rather the newer hash format.
In order to allow users to type annotate fixtures they request, the
types need to be imported from the `pytest` namespace. They are/were
always available to import from the `_pytest` namespace, but that is
not guaranteed to be stable.
These types are only exported for the purpose of typing. Specifically,
the following are *not* public:
- Construction (`__init__`)
- Subclassing
- staticmethods and classmethods
We try to combat them being used anyway by:
- Marking the classes as `@final` when possible (already done).
- Not documenting private stuff in the API Reference.
- Using `_`-prefixed names or marking as `:meta private:` for private
stuff.
- Adding a keyword-only `_ispytest=False` to private constructors,
warning if False, and changing pytest itself to pass True. In the
future it will (hopefully) become a hard error.
Hopefully that will be enough.
An upcoming commit wants to import from `_pytest.pytester` in the public
`pytest` module. This means that `_pytest.pytester` would start to get
imported during import time, which it hasn't up to now -- it was
imported by the plugin loader (if requested). When a plugin is loaded,
it is subjected to assertion rewriting, but only if the module isn't
imported yet, it issues a warning "Module already imported so cannot be
rewritten" and skips the rewriting. So we'd end up with the pytester
plugin not being rewritten, but it wants to be.
Absent better ideas, the solution here is to split the pytester
assertions to their own plugin (which will always only be imported by
the plugin loader) and exclude pytester itself from plugin rewriting.
When --doctest-modules is used, an `__init__.py` file is not a `Package`
but a `DoctestModule`, but some collection code assumed that
`__init__.py` implies a `Package`. That code caused only a single test
to be collected in the scenario in the subject.
Tighten up this check to explicitly check for `Package`. There are
better solutions, but for another time.
Report & test by Nick Gates <nickgatzgates@gmail.com>.
We want to export `pytest.MonkeyPatch` for the purpose of
type-annotating the `monkeypatch` fixture. For other fixtures we export
in this way, we also make direct construction of them (e.g.
`MonkeyPatch()`) private. But unlike the others, `MonkeyPatch` is also
widely used directly already, mostly because the `monkeypatch` fixture
only works in `function` scope (issue #363), but also in other cases. So
making it private will be annoying and we don't offer a decent
replacement yet.
So, let's just make direct construction public & documented.
Don't import `pytest` from within some `_pytest` modules since an
upcoming commit will import from them into `pytest`.
It would have been nice not to have to do it, so that internal plugins
look more like external plugins, but with the existing layout this seems
unavoidable.
When pytest was run on a directory containing a recursive symlink it failed
with ELOOP as the library was not able to determine the type of the
direntry:
src/_pytest/main.py:685: in collect
if not direntry.is_file():
E OSError: [Errno 40] Too many levels of symbolic links: '/home/florian/proj/pytest/tests/recursive'
This is fixed by handling ELOOP and other errors in the visit function in
pathlib.py, so the entries whose is_file() call raises an OSError with the
pre-defined list of error numbers will be exluded from the result.
The _ignore_errors function was copied from Lib/pathlib.py of cpython 3.9.
Fixes#7951
* adding --sw-skip shorthand for stepwise skip
* be explicit rather than implicit with default args for stepwise
* add constant for sw cache dir; only register plugin if necessary rather check check activity always;
* use str format; remove unused args in hooks
* assert cache upfront, allow stepwise to have a reference to the cache
* type hinting lf, skip, move literal strings into module constants
* convert parametrized option into a list
* add a sessionfinish hook for stepwise to keep backwards behaviour the same
* add changelog for #7938
* Improve performance of stepwise modifyitems & address PR feedback
* add test for stepwise deselected based on performance enhancements
* Apply suggestions from code review
* delete from items, account for edge case where failed_index = 0
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
We run mypy through pre-commit, and we don't keep duplicate targets in
tox for all of the other linters. Since this adds some (small)
maintenance overhead, remove it.
In pytester tests, pytest stashes & restores the sys.modules for each
test. So if the test imports a new module, it is initialized anew each
time.
Turns out the readline module isn't multi-init safe, which causes
pytester.spawn to crash or hang. So preserve it as a workaround.
It turns out all autouse fixtures are kept in a global list, and thinned
out for a particular node using a linear scan of the entire list each
time.
Change the list to a dict, and only take the nodes we need.
ischildnode can be quite hot in some cases involving many fixtures.
However it is always used in a way that the nodeid is constant and the
baseid is iterated. So we can save work by pre-computing the parents of
the nodeid and use a simple containment test.
The `_getautousenames` function has the same stuff open-coded, so change
it to use the new function as well.
Since commit 0f918b1a9d pytest uses auto-generated autouse
pytest fixtures for the xunit fixtures
{setup,teardown}_{module,class,method,function}. All of these fixtures
were given the same name.
Unfortunately, pytest fixture lookup for a name works by grabbing all of
the fixtures globally declared with a name and filtering to only those
which match the specific node. So each xunit-using item iterates over a
list (of fixturedefs) of a size of all previous same-xunit-using items,
i.e. quadratic.
Fixing this properly to use a better data structure is likely to take
some effort, but we can avoid the immediate problem by just using
a different name for each item's autouse fixture, so it only matches
itself.
A benchmark is added to demonstrate the issue. It is still way too slow
after the fix and possibly still quadratic, but for a different reason
which is another matter.
Running --collect-only, before (snipped):
202533232 function calls (201902604 primitive calls) in 86.379 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 85.688 85.688 main.py:320(pytest_collection)
1 0.000 0.000 85.688 85.688 main.py:567(perform_collect)
80557/556 0.021 0.000 85.050 0.153 {method 'extend' of 'list' objects}
85001/15001 0.166 0.000 85.045 0.006 main.py:785(genitems)
10002 0.050 0.000 84.219 0.008 runner.py:455(collect_one_node)
10002 0.049 0.000 83.763 0.008 runner.py:340(pytest_make_collect_report)
10002 0.079 0.000 83.668 0.008 runner.py:298(from_call)
10002 0.019 0.000 83.541 0.008 runner.py:341(<lambda>)
5001 0.184 0.000 81.922 0.016 python.py:412(collect)
5000 0.020 0.000 81.072 0.016 python.py:842(collect)
30003 0.118 0.000 78.478 0.003 python.py:218(pytest_pycollect_makeitem)
30000 0.190 0.000 77.957 0.003 python.py:450(_genfunctions)
40001 0.081 0.000 76.664 0.002 nodes.py:183(from_parent)
30000 0.087 0.000 76.629 0.003 python.py:1595(from_parent)
40002 0.092 0.000 76.583 0.002 nodes.py:102(_create)
30000 0.305 0.000 76.404 0.003 python.py:1533(__init__)
15000 0.132 0.000 74.765 0.005 fixtures.py:1439(getfixtureinfo)
15000 0.165 0.000 73.664 0.005 fixtures.py:1492(getfixtureclosure)
15000 0.044 0.000 57.584 0.004 fixtures.py:1653(getfixturedefs)
30000 18.840 0.001 57.540 0.002 fixtures.py:1668(_matchfactories)
37507500 31.352 0.000 38.700 0.000 nodes.py:76(ischildnode)
15000 10.464 0.001 15.806 0.001 fixtures.py:1479(_getautousenames)
112930587/112910019 7.333 0.000 7.339 0.000 {built-in method builtins.len}
After:
51890333 function calls (51259706 primitive calls) in 27.306 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 26.783 26.783 main.py:320(pytest_collection)
1 0.000 0.000 26.783 26.783 main.py:567(perform_collect)
80557/556 0.020 0.000 26.108 0.047 {method 'extend' of 'list' objects}
85001/15001 0.151 0.000 26.103 0.002 main.py:785(genitems)
10002 0.047 0.000 25.324 0.003 runner.py:455(collect_one_node)
10002 0.045 0.000 24.888 0.002 runner.py:340(pytest_make_collect_report)
10002 0.069 0.000 24.805 0.002 runner.py:298(from_call)
10002 0.017 0.000 24.690 0.002 runner.py:341(<lambda>)
5001 0.168 0.000 23.150 0.005 python.py:412(collect)
5000 0.019 0.000 22.223 0.004 python.py:858(collect)
30003 0.101 0.000 19.818 0.001 python.py:218(pytest_pycollect_makeitem)
30000 0.161 0.000 19.368 0.001 python.py:450(_genfunctions)
30000 0.302 0.000 18.236 0.001 python.py:1611(from_parent)
40001 0.084 0.000 18.051 0.000 nodes.py:183(from_parent)
40002 0.116 0.000 17.967 0.000 nodes.py:102(_create)
30000 0.308 0.000 17.770 0.001 python.py:1549(__init__)
15000 0.117 0.000 16.111 0.001 fixtures.py:1439(getfixtureinfo)
15000 0.134 0.000 15.135 0.001 fixtures.py:1492(getfixtureclosure)
15000 9.320 0.001 14.738 0.001 fixtures.py:1479(_getautousenames)
--lf has an optimization where it skips collecting Modules (python
files) which don't contain failing tests. The optimization works by
getting the paths of all cached failed tests and skipping the collection
of Modules whose path is not included in that list.
In pytest, Package nodes are Module nodes with the fspath being the file
`<package dir>/__init__.py`. Since it's a Module the logic above
triggered for it, and because it's an `__init__.py` file which is
unlikely to have any failing tests in it, it is skipped, which causes
its entire directory to be skipped, including any Modules inside it with
failing tests.
Fix by special-casing Packages to never filter. This means entire
Packages are never filtered, the Modules themselves are always checked.
It is reasonable to consider an optimization which does filter entire
packages bases on parent paths etc. but this wouldn't actually save any
real work so is really not worth it.
Regressed in 6.1.0 in 62e249a1f9.
The `x` is an `str` but is expected to be a `pathlib.Path`. Not caught
by mypy because `config.getini()` returns `Any`.
Fix by just removing the `bestrelpath` call:
- testpaths are always relative to the rootdir, it thus would be very
unusual to specify an absolute path there.
- The code was wrong even before the regression: `py.path.local`'s
`bestrelpath` function expects a `py.path.local`, not an `str`. But it
had some weird `try ... except AttributeError` fallback which just
returns the argument, i.e. it was a no-op. So there is no behavior
change.
- It seems reasonable to me to just print the full path if that's what
the ini specifies.
When switching from py.path.local to pathlib (70f3ad1c1f),
`local.parts(reverse=True)` was translated incorrectly, leading to the
wrong rootdir being determined in some non-trivial cases where parent
directories have config files as well.
This indicates at least for people using type checkers that these
classes are not designed for inheritance and we make no stability
guarantees regarding inheritance of them.
Currently this doesn't show up in the docs. Sphinx does actually support
`@final`, however it only works when imported directly from `typing`,
while we import from `_pytest.compat`.
In the future there might also be a `@sealed` decorator which would
cover some more cases.
Usually, we use semi-colon punctuation mark to connect closely related
ideas. Sentences which are after semicolon, begins with small letter,
and last sentence always ends with a period mark, see "Garner's Modern
American Usage" book for more information about usage of punctuation
mark. So removing punctuation mark altogether is a good idea, as
@gnikonorov suggested [1].
[1]: https://github.com/pytest-dev/pytest/pull/7760#pullrequestreview-489232607
For decorated functions, the lineno of the FunctionDef AST node points
to the `def` line, not to the first decorator line. On the other hand,
in code objects, the `co_firstlineno` points to the first decorator
line.
Assertion rewriting inserts some imports to code it rewrites. The
imports are inserted at the lineno of the first statement in the AST. In
turn, the code object compiled from the rewritten AST uses the lineno of
the first statement (which is the first inserted import).
This means that given a module like this,
```py
@foo
@bar
def baz(): pass
```
the lineno of the code object without assertion rewriting
(`--assertion=plain`) is 1, but with assertion rewriting it is 3.
And *this* causes some issues for the exception repr when e.g. the
decorator line is invalid and raises during collection. The code becomes
confused and crashes with
INTERNALERROR> File "_pytest/_code/code.py", line 638, in get_source
INTERNALERROR> lines.append(space_prefix + source.lines[line_index].strip())
INTERNALERROR> IndexError: list index out of range
Fix it by special casing decorators. Maybe there are other cases like
this but off hand I can't think of another Python construct where the
lineno of the item would be after its first line, and this is the only
such issue we have had reported.
- be more vague about "packaging metadata" over explicitly mentioning `setup.py`
(such that `pyproject.toml`-based distributions are allowed)
- drop extensions on `README.txt` / `LICENSE.txt` (it's more common to have `.md`
/ `.rst` / no extension)
- remove duplicate mention of license packaging metadata
The release-on-comment script is always executed on *master*, so we should
execute the `release.py` script using tox to ensure we create the
right environment.
Also fixed errors in the error handling code.
When a name is exported from `pytest`, prefer to refer to it by that
rather than its `_pytest` import path. It is shorter and more
appropriate in user-facing documentation (although that's not really
visible).
Our plan is to expose more names for typing purposes, in which can this
could be more comprehensive.
Before, `PluginManager` was a copy of the pluggy doc, and
`PytestPluginManager` was documented separately.
pytest users only really need to know about `PytestPluginManager`, so
instead of splitting have the `PytestPluginManager` documentation
include all of `PluginManager` members and remove `PluginManager` from
the reference (it is still shown as the base class).
Warnings are a central part of Python, so much that Python itself has
command-line and environtment variables to handle warnings.
By moving the concept of warning handling into Config, it becomes natural to
filter warnings issued as early as possible, even before the "_pytest.warnings"
plugin is given a chance to spring into action. This also avoids the weird
coupling between config and the warnings plugin that was required before.
Fix#6681Fix#2891Fix#7620Fix#7626Close#7649
Co-authored-by: Ran Benita <ran@unusedvar.com>
Inline `_makeitem()` so that `self.ihook` (which is moderately
expensive) can be called only once.
Note: the removed test "test_makeitem_non_underscore" comes from an old
behavior of skipping names that start with `_` which has since been
generalized, making the test no longer relevant.
When a Python object (module/class/instance) is collected, for each name
in `obj.__dict__` (and up its MRO) the pytest_pycollect_makeitem hook is
called for potentially creating a node for it.
These Python objects have a bunch of builtin attributes that are
extremely unlikely to be collected. But due to their pervasiveness,
dispatching the hook for them ends up being mildly expensive and also
pollutes PYTEST_DEBUG=1 output and such.
Let's just ignore these attributes.
On the pandas test suite commit 04e9e0afd476b1b8bed930e47bf60e,
collect only, irrelevant lines snipped, about 5% improvement:
Before:
```
51195095 function calls (48844352 primitive calls) in 39.089 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
226602/54 0.145 0.000 38.940 0.721 manager.py:90(_hookexec)
72227 0.285 0.000 20.146 0.000 python.py:424(_makeitem)
72227 0.171 0.000 16.678 0.000 python.py:218(pytest_pycollect_makeitem)
```
After:
```
48410921 function calls (46240870 primitive calls) in 36.950 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
181429/54 0.113 0.000 36.777 0.681 manager.py:90(_hookexec)
27054 0.130 0.000 17.755 0.001 python.py:465(_makeitem)
27054 0.121 0.000 16.219 0.001 python.py:218(pytest_pycollect_makeitem)
```
In ff8b7884e8 NOTSET was changed to a
singleton enum, which ended up unexpectedly triggering a code path in ID
generation which checks for `isinstance(Enum)`.
Add an explicit case for it, which is not too bad anyway.
* Update talks/trainings page
- Remove past webinar
- Add new open training
- Add some talks/webinars by Oliver Bestwalter and by me
- Remove some stale link targets
* Move sidebar to index
This is a more sensible interface for matchnodes.
This also fixes a sort-of bug where a recursive call to matchnodes
raises NoMatch which would terminate the entire tree, even if other
branches may find a match. Though I don't think it's actually possible.
This reverts commit f10ab021e2.
The commit was good in that it removed a non-trivial amount of code
duplication. However it was done in the wrong layer (nodes.py) and split
up a major part of the collection (the filesystem traversal) to a
separate class making it harder to understand.
We should try to reduce the duplication, but in a more appropriate
manner.
The path part of a `<path>::part1::part2` style collection argument must
be a file, not a directory.
Previously this crashed with an uncool assert "invalid arg".
The `CaptureManager.global_and_fixture_disabled()` context manager (and
`CaptureFixture.disabled()` which calls it) did `suspend(); ...;
resume()` but if the capturing was already suspended, the `resume()`
would resume it when it shouldn't.
This caused caused some messages to be swallowed when `--log-cli` is
used because it uses `global_and_fixture_disabled` when capturing is not
necessarily resumed.
This fixes an issue where pylint complains about missing implementations
of abstract methods in subclasses of `File` which only override
`collect()` (as they should).
It is also cleaner and makes sense, these methods really don't need to
be overridden.
The previous methods defined directly on `FSCollector` and `Package` are
deprecated, to be removed in pytest 7.
See commits e2934c3f8c and
f10ab021e2 for reference.
and include a demo of toml 'literal strings'
Update doc/en/warnings.rst
Apply suggestion by Ran
Fix linting
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Didn't call it absolute or absolute_path to avoid conflicts with
possible variable names.
Didn't call it abspath to avoid confusion with os.path.abspath.
This makes mypy raise an error whenever it detects code which is
statically unreachable, e.g.
x: int
if isinstance(x, str):
... # Statement is unreachable [unreachable]
This is really neat and finds quite a few logic and typing bugs.
Sometimes the code is intentionally unreachable in terms of types, e.g.
raising TypeError when a function is given an argument with a wrong
type. In these cases a `type: ignore[unreachable]` is needed, but I
think it's a nice code hint.
This is a follow up to 3f8200676f which didn't
make it clear that log_print is also removed in the changelog and didn't remove
it from the reference docs.
There are some ones we *would* like to enforce, like
D401 First line should be in imperative mood
but have too many false positives, so I left them out.
This prevents referring to a generic type without filling in its generic
type parameters.
The FixtureDef typing might need some more refining in the future.
In Python, if module A defines a name `name`, and module B does `import
name from A`, then another module C can `import name from B`.
Sometimes it is intentional -- module B is meant to "reexport" `name`.
But sometimes it is just confusion/inconsistency on where `name` should
be imported from.
mypy has a flag `--no-implicit-reexport` which puts some order into
this. A name can only be imported from a module if
1. The module defines the name
2. The module's `__all__` includes the name
3. The module imports the name as `from ... import .. as name`.
This flag is included in mypy's `--strict` flag.
I like this flag, but I realize it is a bit controversial, and in
particular item 3 above is a bit unfriendly to contributors who don't
know about it. So I didn't intend to add it to pytest.
But while investigating issue 7589 I came upon mypy issue 8754 which
causes `--no-implicit-reexport` to leak into installed libraries and
causes some unexpected typing differences *in pytest* if the user uses
this flag.
Since the diff mostly makes sense, let's just conform to it.
We barely use it; the couple places that do are not really worth the
extra dependency, I think the code is clearer without it.
Also simplifies one (regular) itertools usage.
Also improves a check and an error message in `pytest.raises`.
Part of the effort to reduce dependency on the py library.
Besides that, py.xml implements its own XML serialization which is
pretty scary.
I tried to keep the code with minimal changes (though it could use some
cleanups). The differences in behavior I have noticed are:
- Attributes in the output are not sorted.
- Some unneeded escaping is no longer performed, for example escaping
`"` to `"` in a text node.
1. Remove sys.maxunicode check & comment. Nowadays it is always a
constant 0x10ffff.
2. Pre-generate the pattern. Possible due to 1.
3. Compile the regex lazily. No reason to pay startup cost for it.
4. Add docstring in particular to explain a subtle point.
`os.scandir()`, introduced in Python 3.5, is much faster than
`os.listdir()`. See https://www.python.org/dev/peps/pep-0471/.
It also has a `DirEntry` which can be used to further reduce syscalls in
some cases.
Part of reducing dependency on `py`. Also enables upcoming improvements.
In cases where there are simpler alternatives (in tests), I used those.
What's left are a couple of uses in `_pytest.main` and `_pytest.python`
and they only have modest requirements, so all of the featureful code
from py is not needed.
This fixes CI on Windows since GitHub Actions started installing WSL on
their images which apparently installs some wrapper `bash` which does
not run actual bash.
Previously was restricted to >=1.8.2,<2.1, but newer versions have some
nice improvements I'd like to be able to use in upcoming changes.
Changelog: https://www.sphinx-doc.org/en/master/changes.html#release-3-1-0-released-jun-08-2020
There are two issues that came up:
1. `highlightlang` is deprecated for `highlight`.
2. Doesn't like having two `automethod` generated for the same
`Metafunc.parametrize` method. Gives this warning:
`pytest/doc/en/reference.rst:846: WARNING: duplicate object description of _pytest.python.Metafunc.parametrize, other instance in reference, use :noindex: for one of them`
To work around this I make `pytest.mark.parametrize` link to
`Metafunc.parametrize` instead of repeating it.
The previous typing had an object passed to the user, which they can't
do anything with without asserting, which is inconvenient. Change it to
Any instead.
Note that what comes *back* to pytest (the return value) should be an
`object`, because we want to handle arbitrary objects without assuming
anything about them.
If a test runtest phase (not setup) dynamically adds a pytest.mark.xfail
mark to the item, it should be respected, but it wasn't. This regressed
in 3e6fe92b7e (not released).
Fix it by just always refreshing the mark if needed. This is mostly what
was done before but in a more roundabout way.
This introduces a new role, `:globalvar:`, so we can mark/reference
variables like `pytest_plugins`, `pytestmark`, etc. This besides being useful
also makes the documentation look more consistent.
eval() is used for evaluating string conditions in skipif/xfail e.g.
@pytest.mark.skipif("1 == 0")
This is the only code that uses `_pytest._code.compile()`, so removing
its last use enables us to remove it entirely.
In this case it doesn't add much. Plain compile() gives a good enough
error message.
For regular exceptions, the message is the same.
For SyntaxError exceptions, e.g. "1 ==", the previous code adds a little
bit of useful context:
```
invalid syntax (skipping.py:108>, line 1)
The above exception was the direct cause of the following exception:
1 ==
^
(code was compiled probably from here: <0-codegen /pytest/src/_pytest/skipping.py:108>) (line 1)
During handling of the above exception, another exception occurred:
Error evaluating 'skipif' condition
1 ==
^
SyntaxError: invalid syntax
```
The new code loses it:
```
unexpected EOF while parsing (<skipif condition>, line 1)
During handling of the above exception, another exception occurred:
Error evaluating 'skipif' condition
1 ==
^
SyntaxError: invalid syntax
```
Since the old message is a minor improvement to an unlikely error
condition in a deprecated feature, I think it is not worth all the code
that it requires.
This has been there since as far as the git history goes (2007), is not
covered by any test, and says "Buggy python version consider upgrading".
Hopefully everyone have upgraded...
Setting log_level via the CLI or .ini will control the log level of the
report that is dumped upon failure of a test.
If caplog modified the log level during the execution of that test, it
should not impact the level that is displayed upon failure in the
"captured log report" section.
[
ran:
- rebased
- reused handler
- changed store keys also to "caplog_handler_*"
- added changelog
all bugs are mine :)
]
There is no need to do the XPASS check here, pytest_runtest_makereport
already handled that (the current handling there is dead code).
All the hook needs to do is refresh the xfail evaluation if needed, and
check the NOTRUN condition again.
Previously, skipif/xfail marks were evaluated using a `MarkEvaluator`
class. I found this class very difficult to understand.
Instead of `MarkEvaluator`, rewrite using straight functions which are
hopefully easier to follow.
I tried to keep the semantics exactly as before, except improving a few
error messages.
This type was actually in `_pytest.skipping` previously, but was moved to
`_pytest.mark.evaluate` in cf40c0743c.
I think the previous location was more appropriate, because the
`MarkEvaluator` is not a generic mark facility, it is explicitly and
exclusively used by the `skipif` and `xfail` marks to evaluate their
particular set of arguments. So it is better to put it in the plugin
code.
Putting `skipping` related functionality into the core `_pytest.mark`
module also causes some import cycles which we can avoid.
`@pytest.mark.xfail` is meant to work with arbitrary items, and there is
a test `test_mark_xfail_item` which verifies this.
However, the code for some reason uses `pytest_pyfunc_call` for the
call phase check, which only works for Function items. The test
mentioned above only passed "accidentally" because the
`pytest_runtest_makereport` hook also runs a `evalxfail.istrue()` which
triggers and evaluation, but conceptually it shouldn't do that.
Change to `pytest_runtest_call` to make the xfail checking properly
generic.
While working on improving the documentation of the
`pytest_runtest_setup` hook, I came up with this text:
> Called to perform the setup phase of the test item.
>
> The default implementation runs ``setup()`` on item and all of its
> parents (which haven't been setup yet). This includes obtaining the
> values of fixtures required by the item (which haven't been obtained
> yet).
But upon closer inspection I noticed this line at the start of
`SetupState.prepare` (which is what does the actual work for
`pytest_runtest_setup`):
self._teardown_towards(needed_collectors)
which implies that the setup phase of one item might trigger teardowns
of *previous* items. This complicates the simple explanation. It also
seems like a completely undesirable thing to do, because it breaks
isolation between tests -- e.g. a failed teardown of one item shouldn't
cause the failure of some other items just because it happens to run
after it.
So the first thing I tried was to remove that line and see if anything
breaks -- nothing did. At least pytest's own test suite runs fine. So
maybe it's just dead code?
This option checks even functions which are not annotated. It's a good
step to ensure that existing type annotation are correct.
In a Pareto fashion, the last few holdouts are always the ugliest,
beware.
The FixtureFunctionMarker attrs class already converts the params
itself.
When adding types, the previous converter composition causes some type
error, but extracting it to a standalone function fixes the issue (a
lambda is not supported by the mypy plugin, currently).
* Fix typo in capture.rst documentation
Rename ``capfsysbinary`` to ``capsysbinary`` as the former does not exist as far as i can see.
* Make Python uppercase in doc/en/capture.rst
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Remove the sentence entirely
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
* enable tox -e docs natively on windows using a wrapper
* rename the towncrier script; run the towncrier command in a safer manner
* use subprocess.call; call exit() around main on towncrier wrapper
* change to sys.exit() instead of builtin exit()
There are state transitions start/done/suspend/resume and two additional
operations snap/writeorg.
Previously it was not well defined in what order they can be called, and
which operations are idempotent.
Formalize this and enforce using assert checks with informative error
messages if they fail (rather than random AttributeErrors).
Make it easier to read the file in progression, and avoid forward
references for upcoming type annotations.
There is one cycle, CaptureManager <-> CaptureFixture, which is hard to
untangle.
(This commit should be added to `.gitblameignore`).
`TerminalWriter`, imported recently from `py`, contains its own
incomplete wcwidth (`char_with`/`get_line_width`) implementation. The
`TerminalReporter` also needs this, but uses the external `wcwidth`
package.
This commit brings the `TerminalWriter` implementation up-to-par with
`wcwidth`, moves to implementation to a new file `_pytest._io.wcwidth`
which is used everywhere, and removes the dependency.
The differences compared to the `wcwidth` package are:
- Normalizes the string before counting.
- Uses Python's `unicodedata` instead of vendored Unicode tables. This
means the data corresponds to the Python's version Unicode version
instead of the `wcwidth`'s package version.
- Apply some optimizations.
pytest.fixture() can be used either as
@pytest.fixture
def func(): ...
or as
@pytest.fixture()
def func(): ...
or (while maybe not intended)
func = pytest.fixture(func)
so it needs to inspect internally whether it got a function in the first
positional argument or not.
Previously, there were was oddity. In the following,
func = pytest.fixture(func, autouse=True)
# OR
func = pytest.fixture(func, parms=['a', 'b'])
The result is as if `func` wasn't passed.
There isn't any reason for this special that I can understand, so remove
it.
_pytest.timing is an indirection to 'time' functions, which pytest production
code should use instead of 'time' directly.
'mock_timing' is a new fixture which then mocks those functions, allowing us
to write time-reliable tests which run instantly and are not flaky.
This was triggered by recent flaky junitxml tests on Windows related to timing
issues.
This is already done in pytest_runtest_logstart, so the fspath is
already guaranteed to have been printed (for xdist, it is disabled
anyway).
write_fspath_result is mildly expensive so it is worth avoiding calling
it twice.
The `FDCapture`/`FDCaptureBinary` classes, used by `capfd`/`capfdbinary`
fixtures and the `--capture=fd` option (set by default), redirect FDs
1/2 (stdout/stderr) to a temporary file. To do this, they need to save
the old file by duplicating the FD before redirecting it, to be restored
once finished.
Previously, if this duplicating (`os.dup()`) failed, most likely due to
that FD being invalid, the FD redirection would silently not be done. The
FD capturing also performs python-level redirection (monkeypatching
`sys.stdout`/`sys.stderr`) which would still be done, but direct writes
to the FDs would fail.
This is not great. If pytest is run with `--capture=fd`, or a test is
using `capfd`, it expects writes to the FD to work and be captured,
regardless of external circumstances.
So, instead of disabling FD capturing, keep the redirection to a
temporary file, just don't restore it after closing, because there is
nothing to restore to.
This makes a difference for e.g. pytest-xdist:
Before:
```
--dist=distmode set mode for distributing tests to exec environments. each: …
available environment. loadscope: …
grouped by file to any available environment. (default) no: …
```
After:
```
--dist=distmode set mode for distributing tests to exec environments.
each: send each test to all available environments.
load: load balance by sending any pending test to any available environment.
…
(default) no: run tests inprocess, don't distribute.
```
This might also result in unexpected changes (hard wrapping), when line
endings where used unintentionally, e.g. with:
```
help="""
some long
help text
"""
```
But the benefits from that are worth it, and it is easy to fix, as will
be done for the internal `assertmode` option.
Currently, a bad logging call, e.g.
logger.info('oops', 'first', 2)
triggers the default logging handling, which is printing an error to
stderr but otherwise continuing.
For regular programs this behavior makes sense, a bad log message
shouldn't take down the program. But during tests, it is better not to
skip over such mistakes, but propagate them to the user.
Previously, a LoggingCaptureHandler was instantiated for each test's
setup/call/teardown which turns out to be expensive.
Instead, only keep one instance and reset it between runs.
The tests came via c629f6b18 and c61ff31ffa.
The fixes from there are kind of obsoleted by 4cd08f9 (moving to importlib),
but it makes sense to keep them as integration tests in general.
The logstart/logreport/logfinish hooks don't need the stuff in
_runtest_for. The test capturing catching_logs call is irrelevant for
them, and the item-conditional sections are gone.
- Instead of making it optional, always set up a handler, but possibly
going to /dev/null. This simplifies the code by removing a lot of
conditionals. It also can replace the NullHandler() we already add.
- Change `set_log_path` to just change the stream, instead of recreating
one. Besides plugging a resource leak, it enables the next item.
- Remove the capturing_logs from _runtest_for, since it sufficiently
covered by the one in pytest_runtestloop now, which wraps all other
_runtest_for calls.
The first item alone would have had an adverse performance impact, but
the last item removes it.
Remove usage of `@contextmanager` as it is a bit slower than
hand-rolling, and also disallows re-entry which we want to use.
Removing protections around addHandler()/removeHandler(), because
logging already checks that internally.
Conceptually it doesn't check per catching_logs (and catching_logs
doesn't restore the older one either). It is just something that is
defined for each handler once.
The default message is often hard to read:
E _pytest.config.ConftestImportFailure: (local('D:\\projects\\pytest\\.tmp\\root\\foo\\conftest.py'), (<class 'RuntimeError'>, RuntimeError('some error',), <traceback object at 0x000001CCC3E39348>))
Using a shorter message is better:
E _pytest.config.ConftestImportFailure: RuntimeError: some error (from D:\projects\pytest\.tmp\root\foo\conftest.py)
And we don't really lose any information due to exception chaining.
This removes the KeyError from the traceback chain when an
conftest fails to import:
return self._conftestpath2mod[key]
E KeyError: WindowsPath('D:/projects/pytest/.tmp/root/foo/conftest.py')
During handling of the above exception, another exception occurred:
...
raise RuntimeError("some error")
E RuntimeError: some error
During handling of the above exception, another exception occurred:
...
E _pytest.config.ConftestImportFailure: (...)
By slightly changing the code, we can remove the first chain, which is often
very confusing to users and doesn't help with anything.
Fix#7223
Only filter with known failures, and explicitly keep paths of passed
arguments.
This also displays the "run-last-failure" status before collected files,
and does not update the cache with "--collect-only".
Fixes https://github.com/pytest-dev/pytest/issues/6968.
The previous commit made this possible, so utilize it.
Since legacy.py becomes pretty bare, I inlined it into __init__.py. I'm
not sure it's really "legacy" anyway!
Using a simple 50000 items benchmark with `--collect-only -k nomatch`:
Before (two commits ago):
======================== 50000 deselected in 10.31s =====================
19129345 function calls (18275596 primitive calls) in 10.634 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.001 0.001 2.270 2.270 __init__.py:149(pytest_collection_modifyitems)
1 0.036 0.036 2.270 2.270 __init__.py:104(deselect_by_keyword)
50000 0.055 0.000 2.226 0.000 legacy.py:87(matchkeyword)
After:
======================== 50000 deselected in 9.37s =========================
18029363 function calls (17175972 primitive calls) in 9.701 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 1.394 1.394 __init__.py:239(pytest_collection_modifyitems)
1 0.057 0.057 1.393 1.393 __init__.py:162(deselect_by_keyword)
The matching itself can be optimized more but that's a different story.
In current pytest, the same expression is matched against all items. But
it is re-parsed for every match.
Add support for "compiling" an expression and reusing the result. Errors
may only occur during compilation.
This is done by parsing the expression into a Python `ast.Expression`,
then `compile()`ing it into a code object. Evaluation is then done using
`eval()`.
Note: historically we used to use `eval` directly on the user input --
this is not the case here, the expression is entirely under our control
according to our grammar, we just JIT-compile it to Python as a
(completely safe) optimization.
New errors:
testing/test_setupplan.py:104:15: E741 ambiguous variable name 'l'
testing/test_setupplan.py:107:15: E741 ambiguous variable name 'l'
extra/get_issues.py:48:29: E741 ambiguous variable name 'l'
testing/test_error_diffs.py:270:32: E741 ambiguous variable name 'l'
Not so sure about it but easier to just fix.
But more importantly, is a large amount of typing-related issues there
were fixed which necessitated noqa's which can now be removed.
The `-k '-expr'` syntax is an old alias to `-k 'not expr'`. It's also
not a very convenient to have syntax that start with `-` on the CLI.
Deprecate it and suggest replacing with `not`.
---
The `-k 'expr:'` syntax discards all items until the first match and
keeps all subsequent, e.g. `-k foo` with
test_bar
test_foo
test_baz
results in `test_foo`, `test_baz`. That's a bit weird, so deprecate it
without a replacement. If someone complains we can reconsider or devise
a better alternative.
Running `pytest | head -1` and similar causes an annoying error to be
printed to stderr:
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
(or possibly even a propagating exception in older/other Python versions).
The standard UNIX behavior is to handle the EPIPE silently. To
recommended method to do this in Python is described here:
https://docs.python.org/3/library/signal.html#note-on-sigpipe
It is not appropriate to apply this recommendation to `pytest.main()`,
which is used programmatically for in-process runs. Hence, change
pytest's entrypoint to a new `pytest.console_main()` function, to be
used exclusively by pytest's CLI, and add the SIGPIPE code there.
Fixes#4375.
When setting up the warnings capture, filter strings (with the general
form `action:message:category:module:line`) are collected from the
cmdline, ini and item and applied. This happens for every test and other
cases.
To apply a string it needs to be parsed into a tuple, and it turns out
this is slow. Since we already vendor the parsing code from Python's
warnings.py, we can speed it up by caching the result. After splitting
the parsing part from the applying part, the parsing is pure and is
straightforward to cache.
An alternative is to parse ahead of time and reuse the result, however
the caching solution turns out cleaner and more general in this case.
On this benchmark:
import pytest
@pytest.mark.parametrize("x", range(5000))
def test_foo(x): pass
Before:
============================ 5000 passed in 14.11s =============================
14365646 function calls (13450775 primitive calls) in 14.536 seconds
After:
============================ 5000 passed in 13.61s =============================
13290372 function calls (12375498 primitive calls) in 14.034 seconds
- replace "tests with warnings" with just warnings: they a) might not
come from a test in the first place, and b) a single test might have
multiple warnings.
- fix usage of (unused) argument to `collapsed_location_report`
Co-authored-by: Ran Benita <ran@unusedvar.com>
Move {Passthrough,CaptureIO} to capture module, and rename Passthrough
-> Tee to match the existing terminology.
Co-authored-by: Ran Benita <ran@unusedvar.com>
Also delay calling tearDown() when --pdb is given, so users still have
access to the instance variables (which are usually cleaned up during tearDown())
when debugging.
Fix#6947
Instead of trying to handle unittest-async functions in pytest_pyfunc_call,
let the unittest framework handle them instead.
This lets us remove the hack in pytest_pyfunc_call, with the upside that
we should support any unittest-async based framework.
Also included 'asynctest' as test dependency for py37-twisted, and renamed
'twisted' to 'unittestextras' to better reflect that we install 'twisted' and
'asynctest' now.
This also fixes the problem of cleanUp functions not being properly called
for async functions.
Fix#7110Fix#6924
Previously, the expressions given to the `-m` and `-k` options were
evaluated with `eval`. This causes a few issues:
- Python keywords cannot be used.
- Constants like numbers, None, True, False are not handled correctly.
- Various syntax like numeric operators and `X if Y else Z` is supported
unintentionally.
- `eval()` is somewhat dangerous for arbitrary input.
- Can fail in many ways so requires `except Exception`.
The format we want to support is quite simple, so change to a custom
parser. This fixes the issues above, and gives us full control of the
format, so can be documented comprehensively and even be extended in the
future if we wish.
Currently this property is computed eagerly, which means
get_line_width() is computed on everything written, but that is a slow
function.
Compute it lazily, so that get_line_width() only runs when needed.
Flushing on every write is somewhat expensive.
Rely on line buffering instead (if line buffering for stdout is
disabled, there must be some reason...), and add explicit flushes when
not outputting lines.
This is how regular `print()` e.g. work so should be familiar.
The code used an O(n^2) loop. Replace list with set to make it O(n).
For backward compatibility the filesystem cache still remains a list.
On this test:
import pytest
@pytest.mark.parametrize("x", range(5000))
def test_foo(x): pass
run with `pytest --collect-only`:
Before: 0m1.251s
After: 0m0.921s
Remove this message which was shown in the warning summary of pytest's
own testsuite:
testing/test_cacheprovider.py::TestNewAPI::test_cache_writefail_cachfile_silent
testing/test_cacheprovider.py:40: PytestCacheWarning: could not create cache path /tmp/pytest-of-ran/pytest-2/test_cache_writefail_cachfile_silent0/.pytest_cache/v/test/broken
cache.set("test/broken", [])
Currently this test issues a warning which is displayed in the warning
summary (of pytest's own test suite):
testing/acceptance_test.py::TestGeneralUsage::test_early_skip
/tmp/pytest-of-ran/pytest-396/test_early_skip0/conftest.py:2: PytestDeprecationWarning: The pytest_collect_directory hook is not working.
Please use collect_ignore in conftests or pytest_collection_modifyitems.
def pytest_collect_directory():
I think the filter was meant to be `ignore` in the first place, and not
`always` which is not a valid action AFAIK.
This function is exposed and kept alive for the oejskit plugin which is
abandoned and no longer works with recent plugins, so let's prepare to
completely remove it.
Earlier pytest-bot would only print out the exception in cases of
failure but did not provide context on failing command and error from
command. This patch adds the errors from the command to the exception
message.
`Command` provides abstraction over the command to run and helps
in collecting errors from the first failing command only.
With this, we don't need to check `returncode` from each command that
we run, we can run all the commands and will have access to the error
from the first command that failed. This pattern was taken from Go.
Please refer: https://blog.golang.org/errors-are-values
The `_pytest._code._reprcompare` that was referred to previously doesn't
exist -- it was moved to other places but wasn't updated. This regressed
in f423ce9c01. Now we don't want it
anymore, so keep the status quo by explicitly removing them.
* tests: fix TypeError with test_mark_closest
It fails when trying to run it actually:
> TypeError: test_has_inherited() takes 0 positional arguments but 1 was given
* Fix testing/test_collection.py::TestCollector::test_getparent
Mostly I wanted to get rid of mentions of "MarkItem" which is something
that no longer exists, but I improved a little beyond that and annotated
some simple types.
It was surprising that `tryfirst=True` would not result in lines being
added to the beginning with `pytest_report_header`.
This is due to lines being reversed, and therefore the same applies to
`pytest_report_collectionfinish`.
TestDurations tests the `--durations=N` functionality which reports N
slowest tests, with durations <= 0.005s not shown by default.
The test relies on real time.sleep() (in addition to the code which uses
time.perf_counter()) which makes it flaky and inconsistent between
platforms.
Instead of trying to tweak it more, make it use fake time instead. The
way it is done is a little hacky but seems to work.
This hook has some functionality to provide explicit markup for the test
status. It seemed unused and wasn't tested, so I was tempted to remove
it, but I found that the pytest-rerunfailures plugin uses it, so
document it and add a test instead.
Use `testdir.syspathinsert()` with multiprocessing tests:
- test_chained_exceptions_no_reprcrash
- test_exception_handling_no_traceback
This only works currently because `_importtestmodule` changes `sys.path`
as a side-effect.
It appears to be only required on Windows though - likely due to the
multiprocessing method used there.
The dynamic scope feature added in 10bf6aac76
necessitated some wrangling of arguments in pytest.fixture(). In
particular, it deprecated positional arguments in favor of keyword-only
arguments, while keeping backward compatibility.
The way it did this avoided some code duplication but ended up being
quite hard to follow and to annotate with types.
Replace it with some straightforward code, which is not very DRY but is
simple and easy to remove when the time comes.
This changes the link anchors in "reference.html", from e.g.
`reference.html#pytest-current-test` to
`reference.html#envvar-PYTEST_CURRENT_TEST`, but I think that is OK, and
not worth adding labels for the old anchors.
twisted started to use `attr.s(eq)` argument which was added recently,
so it fails with oldattrs. One of the CI jobs ran twisted and oldattrs
together, so it started to fail.
Move the twisted code to be covered by another job, and remove it from
the job with the oldattrs.
Previously, writing to sys.stdout/stderr in text-mode (e.g.
`print('foo')`) while a `capsysbinary` fixture is active, would crash
with:
/usr/lib/python3.7/contextlib.py:119: in __exit__
next(self.gen)
E TypeError: write() argument must be str, not bytes
This is due to some confusion in the types. The relevant functions are
`snap()` and `writeorg()`. The function `snap()` returns what was
captured, and the return type should be `bytes` for the binary captures
and `str` for the regular ones. The `snap()` return value is eventually
passed to `writeorg()` to be written to the original file, so it's input
type should correspond to `snap()`. But this was incorrect for
`SysCaptureBinary`, which handled it like `str`.
To fix this, be explicit in the `snap()` and `writeorg()`
implementations, also of the other Capture types.
We can't add type annotations yet, because the current inheritance
scheme breaks Liskov Substitution and mypy would complain. To be
refactored later.
Fixes: https://github.com/pytest-dev/pytest/issues/6871
Co-authored-by: Ran Benita (some modifications & commit message)
I tried to understand what the `safe_text_dupfile()` function and
`EncodedFile` class do. Outside tests, `EncodedFile` is only used by
`safe_text_dupfile`, and `safe_text_dupfile` is only used by
`FDCaptureBinary.__init__()`. I then started to eliminate always-true
conditions based on the single call site, and in the end nothing was
left except of a couple workarounds that are still needed.
The commit which added the checks for os.dup a15afb5e48
suggests it was done for Jython. But pytest doesn't support Jython
anymore (Jython is Python 2 only).
Furthermore, it looks like the faulthandler plugin (bundled in pytest
and enabled by default) uses os.dup() unprotected and there have not
been any complaints.
So seems better to just remove these checks, and only add if someone
with a legitimate use case complains.
`KeywordMapping` returns a bool on lookup which when passed to eval
fail on certain operations such as index access and attribute access.
We catch all exceptions and raise a `UsageError`.
- it should not document the deprecated `result`; used the same as
pluggy documents itself
- add a "hookwrapper" label, that could be used by pluggy (currently it
links to the section)
- use pluggy's `hookwrappers` label for linking to its documentation
When using `(i)var` in the class docstring it would link `duration` to
`_pytest.runner.TestReport.duration`.
This moves the docstrings to the attributes properly.
This was changed unintentionally in 45c4a8fb3 (pytest 5.3.0), but only
EnvironmentErrors might have `errno`.
Since that is not really guaranteed and it is good to have more
information this uses the string representation of the exc in the trace
message.
<!-- What are you trying to do, that you are unable to achieve with pytest as it currently stands? -->
#### Describe the solution you'd like
<!-- A clear and concise description of what you want to happen. -->
<!-- Provide examples of real-world use cases that this would enable and how it solves the problem described above. -->
#### Alternative Solutions
<!-- Have you tried to workaround the problem using a pytest plugin or other tools? Or a different approach to solving this issue? Please elaborate here. -->
#### Additional context
<!-- Add any other context, links, etc. about the feature here. -->
@@ -7,6 +7,10 @@ Here is a quick checklist that should be present in PRs.
- [ ] Include new tests or update existing tests when applicable.
- [X] Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.
If this change fixes an issue, please:
- [ ] Add text like ``closes #XYZW`` to the PR description and/or commits (where ``XYZW`` is the issue number). See the [github docs](https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more information.
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/master/changelog/README.rst) for details.
or ``trivial``. You may skip creating the changelog entry if the change doesn't affect the
documented behaviour of pytest.
#. Add yourself to ``AUTHORS`` file if not there yet, in alphabetical order.
#. Commit and push once your tests pass and you are happy with your change(s)::
$ git commit -a -m "<commit message>"
$ git push -u
#. Create a new changelog entry in ``changelog``. The file should be named ``<issueid>.<type>.rst``,
where *issueid* is the number of the issue related to the change and *type* is one of
``bugfix``, ``removal``, ``feature``, ``vendor``, ``doc`` or ``trivial``. You may not create a
changelog entry if the change doesn't affect the documented behaviour of Pytest.
#. Add yourself to ``AUTHORS`` file if not there yet, in alphabetical order.
#. Finally, submit a pull request through the GitHub website using this data::
head-fork: YOUR_GITHUB_USERNAME/pytest
@@ -292,9 +322,9 @@ Here is a simple overview, with pytest-specific bits:
Writing Tests
----------------------------
~~~~~~~~~~~~~
Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/latest/reference.html#testdir>`_, as a "black-box" test.
Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/stable/reference.html#testdir>`_, as a "black-box" test.
For example, to ensure a simple test passes you can write:
@@ -331,16 +361,121 @@ one file which looks like a good fit. For example, a regression test about a bug
should go into ``test_cacheprovider.py``, given that this option is implemented in ``cacheprovider.py``.
If in doubt, go ahead and open a PR with your best guess and we can discuss this over the code.
Joining the Development Team
----------------------------
Anyone who has successfully seen through a pull request which did not
require any extra work from the development team to merge will
themselves gain commit access if they so wish (if we forget to ask please send a friendly
reminder). This does not mean your workflow to contribute changes,
reminder). This does not mean there is any change in your contribution workflow:
everyone goes through the same pull-request-and-review process and
no-one merges their own pull requests unless already approved. It does however mean you can
participate in the development process more fully since you can merge
pull requests from other contributors yourself after having reviewed
them.
Backporting bug fixes for the next patch release
------------------------------------------------
Pytest makes feature release every few weeks or months. In between, patch releases
are made to the previous feature release, containing bug fixes only. The bug fixes
usually fix regressions, but may be any change that should reach users before the
next feature release.
Suppose for example that the latest release was 1.2.3, and you want to include
a bug fix in 1.2.4 (check https://github.com/pytest-dev/pytest/releases for the
actual latest release). The procedure for this is:
#. First, make sure the bug is fixed the ``master`` branch, with a regular pull
request, as described above. An exception to this is if the bug fix is not
applicable to ``master`` anymore.
#.``git checkout origin/1.2.x -b backport-XXXX`` # use the master PR number here
#. Locate the merge commit on the PR, in the *merged* message, for example:
nicoddemus merged commit 0f8b462 into pytest-dev:master
#.``git cherry-pick -x -m1 REVISION`` # use the revision you found above (``0f8b462``).
#. Open a PR targeting ``1.2.x``:
* Prefix the message with ``[1.2.x]``.
* Delete the PR body, it usually contains a duplicate commit message.
Who does the backporting
~~~~~~~~~~~~~~~~~~~~~~~~
As mentioned above, bugs should first be fixed on ``master`` (except in rare occasions
that a bug only happens in a previous release). So who should do the backport procedure described
above?
1. If the bug was fixed by a core developer, it is the main responsibility of that core developer
to do the backport.
2. However, often the merge is done by another maintainer, in which case it is nice of them to
do the backport procedure if they have the time.
3. For bugs submitted by non-maintainers, it is expected that a core developer will to do
the backport, normally the one that merged the PR on ``master``.
4. If a non-maintainers notices a bug which is fixed on ``master`` but has not been backported
(due to maintainers forgetting to apply the *needs backport* label, or just plain missing it),
they are also welcome to open a PR with the backport. The procedure is simple and really
helps with the maintenance of the project.
All the above are not rules, but merely some guidelines/suggestions on what we should expect
about backports.
Handling stale issues/PRs
-------------------------
Stale issues/PRs are those where pytest contributors have asked for questions/changes
and the authors didn't get around to answer/implement them yet after a somewhat long time, or
the discussion simply died because people seemed to lose interest.
There are many reasons why people don't answer questions or implement requested changes:
they might get busy, lose interest, or just forget about it,
but the fact is that this is very common in open source software.
The pytest team really appreciates every issue and pull request, but being a high-volume project
with many issues and pull requests being submitted daily, we try to reduce the number of stale
issues and PRs by regularly closing them. When an issue/pull request is closed in this manner,
it is by no means a dismissal of the topic being tackled by the issue/pull request, but it
is just a way for us to clear up the queue and make the maintainers' work more manageable. Submitters
can always reopen the issue/pull request in their own time later if it makes sense.
When to close
~~~~~~~~~~~~~
Here are a few general rules the maintainers use to decide when to close issues/PRs because
of lack of inactivity:
* Issues labeled ``question`` or ``needs information``: closed after 14 days inactive.
* Issues labeled ``proposal``: closed after six months inactive.
* Pull requests: after one month, consider pinging the author, update linked issue, or consider closing. For pull requests which are nearly finished, the team should consider finishing it up and merging it.
The above are **not hard rules**, but merely **guidelines**, and can be (and often are!) reviewed on a case-by-case basis.
Closing pull requests
~~~~~~~~~~~~~~~~~~~~~
When closing a Pull Request, it needs to be acknowledge the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
Hi <contributor>,
First of all we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
We noticed it has been awhile since you have updated this PR, however. pytest is a high activity project, with many issues/PRs being opened daily, so it is hard for us maintainers to track which PRs are ready for merging, for review, or need more attention.
So for those reasons we think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
Again we appreciate your time for working on this, and hope you might get back to this at a later time!
<bye>
Closing Issues
--------------
When a pull request is submitted to fix an issue, add text like ``closes #XYZW`` to the PR description and/or commits (where ``XYZW`` is the issue number). See the `GitHub docs <https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword>`_ for more information.
When an issue is due to user error (e.g. misunderstanding of a functionality), please politely explain to the user why the issue raised is really a non-issue and ask them to close the issue if they have no further questions. If the original requestor is unresponsive, the issue will be handled as described in the section `Handling stale issues/PRs`_ above.
========================== 1 failed in 0.04 seconds ===========================
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started <https://docs.pytest.org/en/latest/getting-started.html#our-first-test-run>`_ for more examples.
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started <https://docs.pytest.org/en/stable/getting-started.html#our-first-test-run>`_ for more examples.
Features
--------
- Detailed info on failing `assert statements <https://docs.pytest.org/en/latest/assert.html>`_ (no need to remember ``self.assert*`` names);
- Detailed info on failing `assert statements <https://docs.pytest.org/en/stable/assert.html>`_ (no need to remember ``self.assert*`` names)
There can be many different definitions of "success". Pytest can run many `nose and unittest`_ tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right?
There can be many different definitions of "success". Pytest can run many nose_ and unittest_ tests by default, so using pytest as your testrunner may be possible from day 1. Job done, right?
It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies.
.._`nose and unittest`: faq.html#how-does-pytest-relate-to-nose-and-unittest
@@ -7,7 +7,7 @@ This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all who contributed to this release, among them:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
