Test:
`warnings.warn()` expects that its first argument is a `str` or a
`Warning`, but since 9454fc38d3
`pytest.warns()` no longer allows `Warning` instances unless the first
argument the `Warning` was initialized with is a `str`. Furthermore, if
the `Warning` was created without arguments then `pytest.warns()` raises
an unexpected `IndexError`. The new tests reveal the problem.
Fix:
`pytest.warns()` now allows using `warnings.warn()` with a `Warning`
instance, as is required by Python, with one exception. If the warning
used is a `UserWarning` that was created by passing it arguments and the
first argument was not a `str` then `pytest.raises()` still considers
that an error. This is because if an invalid type was used in
`warnings.warn()` then Python creates a `UserWarning` anyways and it
becomes impossible for `pytest` to figure out if that was done
automatically or not.
[ran: rebased on previous commit]
Today `pyproject.toml` is the standard for declaring a Python project root, so seems reasonable to consider it for the ini configuration (and specially `rootdir`) in case we do not find other suitable candidates.
Related to #11311
Previously this would trigger an `AssertionError`.
While it could be considered a bug-fix, but given it now can be relied upon, it is probably better to consider it an improvement.
Fix#11311
Fix#11929.
Figured out what's going on. We have the following collection tree:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
```
And the `test_main.py` contains an autouse fixture (`fake_game_ui`) that
`doctest_main` needs in order to run properly. The fixture doesn't run!
It doesn't run because nothing collects the fixtures from (calls
`parsefactories()` on) the `test_main.py` `DoctestModule`.
How come it only started happening with commit
ab63ebb3dc07b89670b96ae97044f48406c44fa0? Turns out it mostly only
worked accidentally. Each `DoctestModule` is also collected as a normal
`Module`, with the `Module` collected after the `DoctestModule`. For
example, if we add a non-doctest test to `test_main.py`, the collection
tree looks like this:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
<Module test_main.py>
<Function test_it>
```
Now, `Module` *does* collect fixtures. When autouse fixtures are
collected, they are added to the `_nodeid_autousenames` dict.
Before ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *setup* time. At this point, the `Module` has
collected and so it ended up picking the autouse fixture (this relies on
another "accident", that the `DoctestModule` and `Module` have the same
node ID).
After ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *collection* time (= when it's created). At
this point, the `Module` hasn't collected yet, so the autouse fixture is
not picked out.
The fix is simple -- have `DoctestModule.collect()` call
`parsefactories`. From some testing I've done it shouldn't have negative
consequences (I hope).
- Previously, `getpass.getuser()` would leak an ImportError if the
USERNAME environment variable was not set on Windows because the `pwd`
module cannot be imported.
- Starting in Python 3.13.0a3, it only raises `OSError`.
Fixes#11874
We have a use case for this in the next commit.
The name can be obtained by using `manager.get_name(plugin)`, however
this is currently O(num plugins) in pluggy, which would be good to
avoid. Besides, it seems generally useful.
This is a useful addition to the existing `listchain`. While `listchain`
returns top-to-bottom, `iterparents` is bottom-to-top and doesn't require
an internal full iteration + `reverse`.
Refs #11662.
--- Problem
Each fixture definition has a "visibility", the `FixtureDef.baseid`
attribute. This is nodeid-like string. When a certain `node` requests a
certain fixture name, we match node's nodeid against the fixture
definitions with this name.
The matching currently happens on the *textual* representation of the
nodeid - we split `node.nodeid` to its "parent nodeids" and then check
if the fixture's `baseid` is in there.
While this has worked so far, we really should try to avoid textual
manipulation of nodeids as much as possible. It has also caused problem
in an odd case of a `Package` in the root directory: the `Package` gets
nodeid `.`, while a `Module` in it gets nodeid `test_module.py`. And
textually, `.` is not a parent of `test_module.py`.
--- Solution
Avoid this entirely by just checking the node hierarchy itself. This is
made possible by the fact that we now have proper `Directory` nodes
(`Dir` or `Package`) for the entire hierarchy.
Also do the same for `_getautousenames` which is a similar deal.
The `iterparentnodeids` function is no longer used and is removed.
* Fix handling empty values of NO_COLOR and FORCE_COLOR
Fix handling NO_COLOR and FORCE_COLOR environment variables to correctly
be ignored when they are set to an empty value, as defined
in the specification:
> Command-line software which adds ANSI color to its output by default
> should check for a NO_COLOR environment variable that, when present
> *and not an empty string* (regardless of its value), prevents
> the addition of ANSI color.
(emphasis mine, https://no-color.org/)
The same is true of FORCE_COLOR, https://force-color.org/.
* Streamline testing for FORCE_COLOR and NO_COLOR
Streamline the tests for FORCE_COLOR and NO_COLOR variables, and cover
all possible cases (unset, set to empty, set to "1"). Combine the two
assert functions into one taking boolean parameters. Mock file.isatty
in all circumstances to ensure that the environment variables take
precedence over the fallback value resulting from isatty check (or that
the fallback is actually used, in the case of both FORCE_COLOR
and NO_COLOR being unset).
* Put a 'reset' color in front of the highlighting
When doing the highlighting, some lexers will not set the initial color
explicitly, which may lead to the red from the errors being propagated
to the start of the expression
* Add syntactic highlighting to the error explanations
This updates the various error reporting to highlight python code when
displayed, to increase readability and make it easier to understand
Previously the error report would have all sections glued together:
- The assertion representation
- The error explanation
- The full diff
This makes it hard to see at a glance where which one starts and ends.
One of the representation (dataclasses, tuples, attrs) does display a
newlines at the start already.
Let's add a newlines before the error explanation and before the full
diff, so we get an easier to read report.
This has one disadvantage: we get one line less in the least verbose
mode, where the output gets truncated.
The normal default pretty printer is not great when objects are nested
and it can get hard to read the diff.
Instead, provide a pretty printer that behaves more like when json get
indented, which allows for smaller, more meaningful differences, at
the expense of a slightly longer diff.
This does not touch the other places where the pretty printer is used,
and only updated the full diff one.
Reset color-related environment variables in a fixture to prevent them
from affecting test results. Otherwise, some of the tests fail
e.g. if NO_COLOR is set in the calling environment.
The current method as the following problem, described by Sadra
Barikbin:
The tests that request both `pytester` and `monkeypatch` and use
`monkeypatch.chdir` without context, relying on `monkeypatch`'s teardown
to restore cwd. This doesn't work because the following sequence of
actions take place:
- `monkeypatch` is set up.
- `pytester` is set up. It saves the original cwd and changes it to a
new one dedicated to the test function.
- Test function calls `monkeypatch.chdir()` without context.
`monkeypatch` saves cwd, which is not the original one, before
changing it.
- `pytester` is torn down. It restores the cwd to the original one.
- `monkeypatch` is torn down. It restores cwd to what it has saved.
The solution here is to have pytester use `monkeypatch.chdir()` itself,
then everything is handled correctly.
Tests if a captured exception group contains an expected exception.
Will raise `AssertionError` if the wrapped exception is not an exception group.
Supports recursive search into nested exception groups.
For packages, `import_path` receives the path to the package's `__init__.py` file, however module names (as they live in `sys.modules`) should not include the `__init__` part.
For example, `app/core/__init__.py` should be imported as `app.core`, not as `app.core.__init__`.
Fix#11306
Move handling of user_properties to `finalize()`.
Previously if a fixture failed during teardown, `pytest_runtest_logreport` would not be called with "teardown", resulting in the user properties not being saved on the JUnit XML file.
Fixes: #11367
This reverts commit e938580257.
Revert "improve error msg and test"
This reverts commit c0cf822ca1.
Revert "error msg"
This reverts commit ec1053cc16.
Revert "changelog"
This reverts commit d2dc8a70b5.
Revert "simplify code / take out user-gen typeerror case"
This reverts commit b9cb87d862.
Previously, when assigning a scope for a fully-indirect parameter set,
when there are multiple fixturedefs for a param (i.e. same-name fixture
chain), the highest scope was used, but it should be the lowest scope,
since that's the effective scope of the fixture.
FixtureDef is used in the `pytest_fixture_setup` hook so needs to be
public. However since its current internals are quite dubious (and not
all marked with `_` prefix) I've added an explicit note that only
documented fields/methods are considered public.
Refs #7469.
Dict comparsion in the ApproxMapping class did not check if values were None before attempting to subtract for max_abs_diff stat, which was throwing an TypeError instead of being handled by pytest error assertion. Check for None has been added before these calculations, so that None will properly show as Obtained/Expected in pytest assert message
The initial implementation (in #7246) introduced the `importlib` mode, which
never added the imported module to `sys.modules`, so it included a test
to ensure calling `import_path` twice would yield different modules.
Not adding modules to `sys.modules` proved problematic, so we began to add the imported module to `sys.modules`
in #7870, but failed to realize that given we are now changing `sys.modules`, we might
as well avoid importing it more than once.
Then #10088 came along, passing `importlib` also when importing application modules
(as opposed to only test modules before), which caused problems due to imports
having side-effects and the expectation being that they are imported only once.
With this PR, `import_path` returns the module immediately if already in
`sys.modules`.
Fix#10811Fix#10341
Fixes#11104.
See the issue for a description of the problem.
Now, we use the same logic for initial conftest paths as we do for
deciding the initial args, which was the idea behind checking
`namespace.file_or_dir` and `testpaths` previously.
This fixes the issue of `testpaths` being considered for initial
conftests even when it's not used for the args.
(Another issue in faeb16146b was that the
`testpaths` were not glob-expanded, this is also fixed.)
Currently, if `--confcutdir` is not set, `inipath.parent` is used, and
if `initpath` is not set, then `confcutdir` is None, which means there
is no cutoff.
Having no cutoff is not great, it means we potentially start probing
stuff all the way up to the filesystem root directory. So let's add
another fallback, to `rootpath`, which is always something reasonable.
In #10758 we introduced the support for the use of the walrus operator in the test cases. There was a case which was not handled that caused a bug report #11028. This PR aims to fix the issue and also to improve how the walrus operator is handled in the AssertionRewriter class.
Closes#11028
Added handling of %f directive to print microseconds in log format options, such as log-date-format. It is impossible to do with a standard logging.Formatter because it uses time.strftime which doesn't know about milliseconds and %f. In this PR I added a custom Formatter which converts LogRecord to a datetime.datetime object and formats it with %f flag. This behaviour is enabled only if a microsecond flag is specified in a format string.
Also added a few tests to check the standard and changed behavior.
Closes#10991
`--lf` has a feature where if a certain `Module` (python file) does not
contain any failed tests, it is skipped entirely at the collector level
instead of being collected and each item skipped individually. When this
happens the collection summary looks like this:
run-last-failure: rerun previous 1 failure (skipped 1 file)
However, this feature didn't work for `Module`s inside of `Package`s,
only for those directly beneath the `Session`.
Fix#11054.
Forces requested `caplog` logging levels to be enabled if they were disabled via `logging.disable()`
`[attr-defined]` mypy error ignored in `logging.py` because there were existing errors with the imports
and `loggin.Logger.manager` is an attr set at runtime. Since it's in the standard lib I can't really fix that.
Ignored an attr-defined error in `src/_pytest/config/__init__.py` because the re-export is necessary.
Fixes#8711
`_set_initial_conftests` could break on some systems if a very long
option was passed, because the `Path.exists()` call raises an
`OSError` instead of returning `False`.
Fix#10169
Closes#10831.
This fixes a small bug where running tests that contained
`pytest.fail(pytrace=False)` with the `--tb=line` flag set results in
an output of "None" in the Failures section of the output, and adds
a test to ensure the behavior is correct.
The problem is that we would loop over all directories of the basetemp directory searching for dead symlinks, for each test, which would compound over the test session run.
Doing the cleanup just once, at the end of the session, fixes the problem.
Fix#10896
The old-style `sys.exc_info()` triplet is redundant nowadays with
`(type(exc), exc, exc.__traceback__)`, and is beginning to get
soft-deprecated in Python 3.12.
Add a nicer API to ExceptionInfo which takes just the exc instead of the
triplet. There are already a few internal uses which benefit.
Fix#10875
Without this, fails with
```
...
E AttributeError: 'TestCaseFunction' object has no attribute 'addDuration'
...
E RuntimeWarning: TestResult has no addDuration method
```
pytest could crash given pathological AST position attributes, which shouldn't happen when testing real Python code, but could happen when testing AST produced by e.g. Hylang.
Another example of the failure is in the nightly CI for the JAX project: https://github.com/google/jax/actions/runs/4607513902/jobs/8142126075
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Jake VanderPlas <jakevdp@google.com>
Ran Benita
Reagan Lee
Eero Vaher
Bruno Oliveira
whysage
Clément Robert
Russell Martin
Dương Quốc Khánh
clee2000
faph
Franck Charras
Fabian Sturm
Ben Brown
Michał Górny
Benjamin Schubert
Jens Tröger
Simon Blanchard
Patrick Lannigan
Avasam
Sharad Nair
Bruno Oliveira
Benjamin Schubert
Carsten Grohmann
ryanpudd
Tanya Agarwal
Chris Mahoney
Mihail Milushev
Marc Mueller
Fraser Stark
Ronny Pfannschmidt
Stefaan Lippens
Israel Fruchter
Jon Parise
Zac Hatfield-Dodds
Tyler Smart
Sadra Barikbin
Isaac Virshup
Kenny Y
Milan Lesnek
Lesnek
Akhilesh Ramakrishnan
Cheukting
pre-commit-ci[bot]
Tushar Sadhwani
Thomas Grainger
Alessio Izzo
theirix
Roberto Aldera
Erik Hasse
Alex Lambson
Ville Skyttä
Adam J. Stewart
Bruno Oliveira
Brian Larsen
Alex
Kodi Arfer
Felix Hofstätter
Paul Kehrer
bluthej