(diff better viewed ignoring whitespace)
In pytest<8, the collection tree for `pyargs` arguments in an invocation
like this:
pytest --collect-only --pyargs pyflakes.test.test_undefined_names
looked like this:
```
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
The pytest 8 collection improvements changed it to this:
```
<Dir pytest>
<Dir .tox>
<Dir venv>
<Dir lib>
<Dir python3.11>
<Dir site-packages>
<Package pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Besides being egregious (and potentially even worse than the above,
going all the way to the root, for system-installed packages, as is
apparently common in CI), this also caused permission errors when trying
to probe some of those intermediate directories.
This change makes `--pyargs` arguments no longer try to add parent
directories to the collection tree according to the `--confcutdir` like
they're regular arguments. Instead, only add the parents that are in the
import path. This now looks like this:
```
<Package .tox/venv/lib/python3.11/site-packages/pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Fix#11904.
Passing a short path in the command line was causing the matchparts check to fail, because ``Path(short_path) != Path(long_path)``.
Using ``os.path.samefile`` as fallback ensures the comparsion works on Windows when comparing short/long paths.
Fix#11895
ruff is faster and handle everything we had prior.
isort configuration done based on the indication from
https://github.com/astral-sh/ruff/issues/4670, previousely based on
reorder-python-import (#11896)
flake8-docstrings was a wrapper around pydocstyle (now archived) that
explicitly asks to use ruff in https://github.com/PyCQA/pydocstyle/pull/658.
flake8-typing-import is useful mainly for project that support python 3.7
and the one useful check will be implemented in https://github.com/astral-sh/ruff/issues/2302
We need to keep blacken-doc because ruff does not handle detection
of python code inside .md and .rst. The direct link to the repo is
now used to avoid a redirection.
Manual fixes:
- Lines that became too long
- % formatting that was not done automatically
- type: ignore that were moved around
- noqa of hard to fix issues (UP031 generally)
- fmt: off and fmt: on that is not really identical
between black and ruff
- autofix re-order in pre-commit from faster to slower
Co-authored-by: Ran Benita <ran@unusedvar.com>
This removes one thing that directory collectors need to worry about.
This adds one hook dispatch per `__pycache__` file, but I think it's
worth it for consistency.
--- Current main
In current main (before pervious commit), calls to gethookproxy/ihook
are the trigger for loading non-initial conftests. This basically means
that conftest loading is done almost as a random side-effect,
uncontrolled and very non-obvious. And it also dashes any hope of making
gethookproxy faster (gethookproxy shows up prominently in pytest
profiles).
I've wanted to improve this for a while, #11268 was the latest step
towards that.
--- PR before change
In this PR, I ran into a problem.
Previously, Session and Package did all of the directory traversals
inside of their collect, which loaded the conftests as a side effect. If
the conftest loading failed, it will occur inside of the collect() and
cause it to be reported as a failure.
Now I've changed things such that Session.collect and Package.collect no
longer recurse, but just collect their immediate descendants, and
genitems does the recursive expansion work.
The problem though is that genitems() doesn't run inside of specific
collector's collect context. So when it loads a conftest, and the
conftest loading fails, the exception isn't handled by any CollectReport
and causes an internal error instead.
The way I've fixed this problem is by loading the conftests eagerly in a
pytest_collect_directory post-wrapper, but only during genitems to make
sure the directory is actually selected.
This solution in turn caused the conftests to be collected too early;
specifically, the plugins are loaded during the parent's collect(), one
after the other as the directory entries are collected. So when the
ihook is hoisted out of the loop, new plugins are loaded inside the
loop, and due to the way the hook proxy works, they are added to the
ihook even though they're supposed to be scoped to the child collectors.
So no hoisting.
--- PR after change
Now I've come up with a better solution: since now the collection tree
actually reflects the filesystem tree, what we really want is to load
the conftest of a directory right before we run its collect(). A
conftest can affect a directory's collect() (e.g. with a
pytest_ignore_collect hookimpl), but it cannot affect how the directory
node itself is collected. So I just moved the conftest loading to be
done right before calling collect, but still inside the CollectReport
context.
This allows the hoisting, and also removes conftest loading from
gethookproxy since it's no longer necessary. And it will probably enable
further cleanups. So I'm happy with it.
Previously, the `_getconftestmodules` function was used both to load
conftest modules for a path (during `pytest_load_initial_conftests`),
and to retrieve conftest modules for a path (during hook dispatch and
for fetching `collect_ignore`). This made things muddy - it is usually
nicer to have clear separation between "command" and "query" functions,
when they occur in separate phases.
So split into "load" and "get".
Currently, `gethookproxy` still loads conftest itself. I hope to change
this in the future.
I think it's helpful to separate the node classes from the other
objects, as they have their own unique usage.
I've chosen not to alphabetize the order, but to use a logical order
instead.
Also slightly improve the docstrings.
Since pytest now requires Python>=3.7, we can use the stdlib attrs
clone, dataclasses, instead of the OG package.
attrs is still somewhat nicer than dataclasses and has some extra
functionality, but for pytest usage there's not really a justification
IMO to impose the extra dependency on users when a standard alternative
exists.
New Sphinx added support for overloads and always displays them all with
full type annotations etc. This regresses the API reference for
overloaded functions like `fixture()`, `warns()`, `raises()` and friends
to become impossible to read.
I tried various workarounds but none worked except this one.
I think this named function makes the code a bit easier to understand.
Also change the check to explicitly check for "is a sub-path of" instead
of the previous check which only worked assuming that path is within
confcutdir or a direct parent of it.
* Rename pytest_ignore_collect fspath parameter to collection_path
* Rename pytest_collect_file fspath parameter to file_path
* Rename pytest_pycollect_makemodule fspath parameter to module_path
* Rename pytest_report_header startpath parameter to start_path
* Rename pytest_report_collectionfinish startpath parameter to start_path
* Update docs with the renamed parameters
* Use pytest-flakes fork temporarily to prove it works
* Use pytest-flakes 4.0.5
* issue a warning when Items and Collector form a diamond
addresses #8435
* Apply suggestions from code review
Co-authored-by: Ran Benita <ran@unusedvar.com>
* Return support for the broken File/Item hybrids
* adds deprecation
* ads necessary support code in node construction
* fix incorrect mypy based assertions
* add docs for deprecation of Item/File inheritance
* warn when a non-cooperative ctor is encountered
* use getattr instead of cast to get the class __init__ for legacy ctors
* update documentation references for node inheritance
* clean up file+item inheritance test
enhance docs
move import upwards
Co-authored-by: Ran Benita <ran@unusedvar.com>
* Replace for loop using the operator
* Replace for loop with a generator expression inside any()
* Replace for loop with a dictionary comprehension
* Use list comprehension
* Simplify arguments for range()
* Change newfuncargs variable to in-line dictionary comprehension
* is_ancestor: return base.is_relative_to(query)
* Remove unneeded import of pathlib
* Try using PurePath
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Import PurePath on new line
* Revert and remove is_relative_to
Co-authored-by: Zachary Kneupper <zacharykneupper@Zacharys-MBP.lan>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit autoupdate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
Ran Benita
Bruno Oliveira
Pierre Sassoulas
Bruno Oliveira
Ben Brown
Zac Hatfield-Dodds
Chris Mahoney
Hugo van Kemenade
Tobias Diez
pre-commit-ci[bot]
Anthony Sottile
Ronny Pfannschmidt
Zack Kneupper
Tadeu Manoel