Commit Graph

249 Commits

Author SHA1 Message Date
antonblr 4ed9a38519 tests: Migrate testing/python to pytester fixture 2020-12-13 07:42:12 -08:00
Ran Benita e398c93884
Merge pull request #8055 from bluetech/unraisable
Add unraisableexception and threadexception plugins
2020-12-05 21:52:17 +02:00
Ran Benita 760a73c08c
Merge pull request #8017 from bluetech/typing-public-fixtures
Export types of builtin fixtures for type annotations
2020-12-05 21:51:20 +02:00
Maximilian Cosmo Sitter 0cef530d10
Add str() support to LineMatcher (#8050) 2020-11-21 20:45:20 +02:00
Ran Benita 148e3c582a pytester: always close stdin pipe in pytester.run()
If the user passed stdin=PIPE for some reason, they have no way to close
it themselves since it is not exposed.
2020-11-20 17:55:30 +02:00
symonk 3b677f79f4 stop assigning nextline if its potentially not used 2020-11-17 11:30:09 +00:00
Ran Benita f1e6fdcddb Export types of builtin fixture for type annotations
In order to allow users to type annotate fixtures they request, the
types need to be imported from the `pytest` namespace. They are/were
always available to import from the `_pytest` namespace, but that is
not guaranteed to be stable.

These types are only exported for the purpose of typing. Specifically,
the following are *not* public:

- Construction (`__init__`)
- Subclassing
- staticmethods and classmethods

We try to combat them being used anyway by:

- Marking the classes as `@final` when possible (already done).

- Not documenting private stuff in the API Reference.

- Using `_`-prefixed names or marking as `:meta private:` for private
  stuff.

- Adding a keyword-only `_ispytest=False` to private constructors,
  warning if False, and changing pytest itself to pass True. In the
  future it will (hopefully) become a hard error.

Hopefully that will be enough.
2020-11-13 11:25:09 +02:00
Ran Benita b050578882 pytester: split asserts to a separate plugin, don't rewrite pytester itself
An upcoming commit wants to import from `_pytest.pytester` in the public
`pytest` module. This means that `_pytest.pytester` would start to get
imported during import time, which it hasn't up to now -- it was
imported by the plugin loader (if requested). When a plugin is loaded,
it is subjected to assertion rewriting, but only if the module isn't
imported yet, it issues a warning "Module already imported so cannot be
rewritten" and skips the rewriting. So we'd end up with the pytester
plugin not being rewritten, but it wants to be.

Absent better ideas, the solution here is to split the pytester
assertions to their own plugin (which will always only be imported by
the plugin loader) and exclude pytester itself from plugin rewriting.
2020-11-13 11:25:09 +02:00
Ran Benita 1cbb0c3554 Stop importing `pytest` to avoid upcoming import cycles
Don't import `pytest` from within some `_pytest` modules since an
upcoming commit will import from them into `pytest`.

It would have been nice not to have to do it, so that internal plugins
look more like external plugins, but with the existing layout this seems
unavoidable.
2020-11-07 18:08:30 +02:00
Sanket Duthade 3bcd316f07
test_collection.py migrate from testdir to Pytester (#8003) 2020-11-07 16:56:00 +02:00
Ran Benita ca82214444 pytester: workaround issue causing spawn to crash or hang
In pytester tests, pytest stashes & restores the sys.modules for each
test. So if the test imports a new module, it is initialized anew each
time.

Turns out the readline module isn't multi-init safe, which causes
pytester.spawn to crash or hang. So preserve it as a workaround.
2020-10-25 16:31:47 +02:00
Ran Benita 1c0c56dfb9 pytester: minor doc fixes 2020-10-23 21:01:31 +03:00
Ran Benita 1b23a111d2 Update mypy 0.782 -> 0.790 2020-10-17 19:25:45 +03:00
Ran Benita e5e47c1097 Fix typing related to iniconfig
iniconfig now has typing stubs which reveal a couple issues.
2020-10-17 19:25:45 +03:00
Bruno Oliveira 3cae145e41 List Testdir members in the docs
Also include docstrings pointing to the counterparts in Pytester.

Fix #7892
2020-10-13 12:01:11 -03:00
Bruno Oliveira 69419cb700
New pytester fixture (#7854) 2020-10-12 12:13:06 -03:00
Anthony Sottile 33d119f71a py36+: com2ann 2020-10-05 18:33:17 -07:00
Anthony Sottile 66bd44c13a py36+: pyupgrade: py36+ 2020-10-03 12:46:54 -07:00
Anthony Sottile fb1d550aac py36+: remove rexport of Path and PurePath 2020-10-03 12:16:52 -07:00
Anthony Sottile 022ac9b9e8
Merge pull request #7846 from asottile/py36_black
py36+: update the target version of black to py36
2020-10-03 11:45:45 -07:00
Anthony Sottile f295b0267d py36+: update the target version of black to py36 2020-10-03 08:17:22 -07:00
Anthony Sottile 7f0d2beb50 py36+: remove _pytest.compat.overload 2020-10-03 08:01:22 -07:00
Anthony Sottile bfadd4060e py36+: from typing import Type: no longer need guard 2020-10-02 19:50:10 -07:00
Anthony Sottile a238d1f37d py36+: remove TYPE_CHECKING from _pytest.compat
automated with:

```bash
git grep -l 'from .* import TYPE_CHECKING' |
    xargs reorder-python-imports \
        --application-directories .:src \
        --remove-import 'from _pytest.compat import TYPE_CHECKING' \
        --add-import 'from typing import TYPE_CHECKING'
```
2020-10-02 15:03:24 -07:00
Ran Benita a99ca879e7 Mark some public and to-be-public classes as `@final`
This indicates at least for people using type checkers that these
classes are not designed for inheritance and we make no stability
guarantees regarding inheritance of them.

Currently this doesn't show up in the docs. Sphinx does actually support
`@final`, however it only works when imported directly from `typing`,
while we import from `_pytest.compat`.

In the future there might also be a `@sealed` decorator which would
cover some more cases.
2020-09-22 12:40:40 +03:00
Ran Benita f28af14457 Don't use NotImplementedError in `@overload`s
We used it as a shortcut for avoiding coverage, but pylint has a special
interpretation of it as an abstract method which we don't want.
2020-08-14 13:54:46 +03:00
Ran Benita f0eb82f7d4 pytester: improve type annotations 2020-08-04 22:46:30 +03:00
Ran Benita 0242de4f56 Format docstrings in a consistent style 2020-08-01 17:14:37 +03:00
Ran Benita bcff02c4c6 pytester: some type annotations 2020-07-10 13:08:56 +03:00
Ran Benita 7b1ba7c0db pytester: slightly clean up LsofFdLeakChecker 2020-07-01 19:39:48 +03:00
Ran Benita 4655b79985 config: improve typing 2020-06-22 16:39:14 +03:00
Bruno Oliveira 3151219785 assertoutcomes() only accepts plural forms
Fix #6505
2020-06-13 10:29:22 -03:00
Bruno Oliveira c17d50829f
Add pyproject.toml support (#7247) 2020-06-08 10:03:10 -03:00
Ran Benita 54ad048be7 Enable check_untyped_defs mypy option for testing/ too 2020-06-05 11:34:20 +03:00
Ran Benita 71dfdca4df Enable check_untyped_defs mypy option for src/
This option checks even functions which are not annotated. It's a good
step to ensure that existing type annotation are correct.

In a Pareto fashion, the last few holdouts are always the ugliest,
beware.
2020-06-05 11:34:20 +03:00
Ran Benita e68a26199c Type annotate misc functions 2020-06-05 11:34:19 +03:00
Ran Benita 247c4c0482 Type annotate some more hooks & impls 2020-06-05 11:34:19 +03:00
Ran Benita 0fb081aec6 Type annotate some hookspecs & impls
Annotate some "easy" arguments of hooks that repeat in a lot of internal
plugins.

Not all of the arguments are annotated fully for now.
2020-06-05 11:34:19 +03:00
Bruno Oliveira cf640a9b67
Merge pull request #7292 from kerizane/7259_iniconfig 2020-06-03 19:00:48 -03:00
Bruno Oliveira 7a8b8a5c9f
Merge branch 'master' into fix-flaky-test 2020-06-02 11:33:15 -03:00
Keri Volans 2748feed38 7291: Replace py.iniconfig with iniconfig 2020-06-01 18:09:57 +01:00
Ran Benita 491239d9b2 capture: remove some indirection in MultiCapture
Removing this indirection enables some further clean ups.
2020-05-26 00:25:49 +03:00
Bruno Oliveira 1780924b27 Introduce _pytest.timing as a way to control timing during tests
_pytest.timing is an indirection to 'time' functions, which pytest production
code should use instead of 'time' directly.

'mock_timing' is a new fixture which then mocks those functions, allowing us
to write time-reliable tests which run instantly and are not flaky.

This was triggered by recent flaky junitxml tests on Windows related to timing
issues.
2020-05-23 11:50:33 -03:00
Simon K 05c22ff823
7154-Improve-testdir-documentation-on-makefiles (#7239) 2020-05-23 11:27:06 -03:00
Ran Benita 46d768503e
Merge pull request #6839 from blueyed/typing-get_dirs_from_args-parseconfig
typing: get_dirs_from_args, parseconfig
2020-05-05 21:11:26 +03:00
Keri Volans 678440e46d 7018: Use internal version of make_numbered_dir 2020-05-03 10:45:06 +01:00
Ran Benita 3cd97d50f9 pre-commit: update pyupgrade 1.18.0 -> 2.2.1 2020-04-24 21:57:38 +03:00
Simon K f479cbce10
Document pytester fixtures so --fixtures makes more sense (#7080) 2020-04-13 20:58:50 +03:00
Daniel Hahler a42e85ed54 Fix documentation for _pytest.pytester.RunResult
When using `(i)var` in the class docstring it would link `duration` to
`_pytest.runner.TestReport.duration`.

This moves the docstrings to the attributes properly.
2020-03-03 18:12:12 +01:00
Daniel Hahler acec0b688f
Fix usage of pytester with doctests (#6802)
Use `request.node.name` instead of `request.function.__name__`:
`request.function` is `None` with `DoctestItem`s.
2020-03-04 05:33:50 +01:00
Daniel Hahler 37d074efc8
Fix (internal) documentation for pytester's LineComp (#6850) 2020-03-04 05:25:32 +01:00
Daniel Hahler 6092d3c6e1 typing: parseconfig 2020-02-28 20:40:39 +01:00
Daniel Hahler be23aeb989
Remove unused `pytester.getdecoded` (#6805)
Last usage was removed in 22dc47d9f.
2020-02-24 14:33:26 +01:00
Daniel Hahler 4021770688
pytester: LineMatcher: use `_fail` with `_match_lines_random` (#6747) 2020-02-19 13:16:57 +01:00
Daniel Hahler bc976dca3b
pytester: add __tracebackhide__ for matching randomly (#6746) 2020-02-16 10:29:59 +01:00
Bruno Oliveira 78baa7b575 Merge remote-tracking branch 'upstream/master' into mm
Conflicts:
	src/_pytest/main.py
	src/_pytest/mark/structures.py
	src/_pytest/python.py
	testing/test_main.py
	testing/test_parseopt.py
2020-02-11 19:22:28 -03:00
Ran Benita d33da078a8 Move ExitCode's definition from _pytest.main to _pytest.config
ExitCode is used in several internal modules and hooks and so with type
annotations added, needs to be imported a lot.

_pytest.main, being the entry point, generally sits at the top of the
import tree.

So, it's not great to have ExitCode defined in _pytest.main, because it
will cause a lot of import cycles once type annotations are added (in
fact there is already one, which this change removes).

Move it to _pytest.config instead.

_pytest.main still imports ExitCode, so importing from there still
works, although external users should really be importing from `pytest`.
2020-02-10 23:55:06 +02:00
Daniel Hahler cdc7e13067
pytester: clarify _makefile signature (#6675) 2020-02-05 20:42:57 +01:00
Daniel Hahler 39d9f7cff5
pytester: LineMatcher: typing, docs, consecutive line matching (#6653) 2020-02-04 22:47:18 +01:00
Daniel Hahler 5256542ea4 pytester.LineMatcher: add support for matching lines consecutively 2020-02-01 23:48:58 +01:00
Daniel Hahler 50f81db817 revisit/improve docstrings 2020-02-01 23:48:58 +01:00
Daniel Hahler 2681b0aed7 typing: pytester: LineMatcher 2020-02-01 23:48:57 +01:00
Daniel Hahler b10ab0211c Use TypeError instead of AssertionError for no sequence
Improve/extends tests.
2020-02-01 23:48:57 +01:00
Daniel Hahler 30922ee694 Merge master into features 2020-01-28 01:40:14 +01:00
Daniel Hahler 9c716e4d74 typing: Testdir.plugins 2020-01-28 00:41:46 +01:00
Daniel Hahler 94ac0f7e6b typing: self._mod_collections, collect_by_name 2020-01-28 00:41:46 +01:00
Daniel Hahler 440881d63a typing: Testdir.__init__ 2020-01-28 00:41:46 +01:00
Daniel Hahler aca1723d45 Merge master into features 2020-01-22 19:18:13 +01:00
Bruno Oliveira 93b74d28d2 Merge remote-tracking branch 'upstream/master' into mm
Conflicts:
 * 	src/_pytest/_code/code.py
 * 	src/_pytest/main.py
 * 	testing/python/metafunc.py
 * 	testing/test_parseopt.py
 * 	testing/test_pytester.py
2020-01-22 11:03:45 -03:00
Daniel Hahler b8787b8732 tests: fix test_cwd_snapshot
Without restoring the cwd, successive tests might fail to parse the
config (via `_pytest.config._prepareconfig()`, for when `--lsof` is
used).

And it is good practice to restore the cwd in any case anyway.
2020-01-22 14:34:11 +01:00
Daniel Hahler 36944157f8 pytester: typing for `spawn`/`spawn_pytest` 2020-01-17 05:58:25 +01:00
Daniel Hahler 83813bf515 Merge master into features
Conflicts:
	.github/workflows/main.yml
2020-01-16 19:45:52 +01:00
Daniel Hahler 4630e2725e Use `TYPE_CHECKING` instead of `False`
This allows for e.g. Jedi to infer types (it checks the name).

It was only used to support Python 3.5.0/3.5.1, where this is is not
available in the `typing` module.

Ref: https://github.com/davidhalter/jedi/issues/1472

Uses `TYPE_CHECKING = False` in `_pytest.outcomes` to avoid having to
work around circular import.
2020-01-16 16:11:39 +01:00
Ran Benita 090e260517 master: update mypy 0.740 -> 0.761
(cherry picked from commit 16ff9f591e)
(cherry picked from commit 4848bbdf9a)
2020-01-14 06:31:41 +01:00
Daniel Hahler 1356d20e90 Merge master into features 2020-01-10 05:20:41 +01:00
Ran Benita 4848bbdf9a Update mypy 0.750 -> 0.761
This fixes some type: ignores due to typeshed update.

Newer mypy seem to ignore unannotated functions better, so add a few
minor annotations so that existing correct type:ignores make sense.
2020-01-01 15:22:16 +02:00
Daniel Hahler 1c0242dec1 Fix `RunResult.parseoutcomes` (follow-up to #6353) 2019-12-30 17:08:52 +01:00
Alexandre Mulatinho dc7bf518b3 pytester: quick fix error introduced in #5990
- added a test to check this condition

Signed-off-by: Alexandre Mulatinho <alex@mulatinho.net>
2019-12-18 16:36:24 -03:00
Daniel Hahler a1219ab8fc pytester: use no colors with inline runs by default
Sets `PY_COLORS=0` in the environment by default, which is used by pylib.

Via https://github.com/blueyed/pytest/pull/58

(initially cherry picked from commit f153ad33d10)
2019-11-25 23:48:11 +01:00
Ronny Pfannschmidt c99c7d0f95 deprecate direct node construction and introduce Node.from_parent 2019-11-23 21:54:11 +01:00
Daniel Hahler b0ebcfb785 pytester: remove special handling of env during inner runs
Closes https://github.com/pytest-dev/pytest/issues/6213.
2019-11-22 21:50:31 +01:00
Daniel Hahler b96e0a71a6 pytester: LineMatcher: __tracebackhide__ with _fail
Follow-up to 2228ccb (gone lost in resolving the conflict).
2019-11-21 01:23:36 +01:00
Daniel Hahler 2228ccbfb4 pytester: reset log output in _match_lines (#70)
This is necessary for when using e.g. `no_fnmatch_line` after it.
Factor it out into `_fail`.

(cherry picked from commit aade7ed0045ba32557ef8565cbab28a2c91053a7)

Ref: https://github.com/pytest-dev/pytest/pull/5914#issuecomment-549182242
2019-11-20 05:24:18 +01:00
Daniel Hahler 54a954514b re-run black 2019-11-16 18:55:32 +01:00
Daniel Hahler 6ddf7c3d42 pytester: Hookrecorder: improve assertoutcome
Before:

        def assertoutcome(self, passed: int = 0, skipped: int = 0, failed: int = 0) -> None:
            realpassed, realskipped, realfailed = self.listoutcomes()
            assert passed == len(realpassed)
    >       assert skipped == len(realskipped)
    E       assert 1 == 0
    E        +  where 0 = len([])

After:

    >       reprec = testdir.inline_run(testpath, "-s")
    E       AssertionError: ([], [], [<TestReport 'nodeid' when='call' outcome='failed'>])
    E       assert {'failed': 1, 'passed': 0, 'skipped': 0} == {'failed': 0, 'passed': 0, 'skipped': 1}
2019-11-13 13:48:20 +01:00
Daniel Hahler 86e9ae39f0 pytester: assert_outcomes: use/set __tracebackhide__ 2019-11-12 15:28:36 +01:00
Ran Benita 265a9eb6a2 Add type annotations to some of _pytest.pytester 2019-11-07 14:13:47 +02:00
Daniel Hahler ec27363748
Merge pull request #6025 from blueyed/pytester-typing
pytester: typing
2019-10-28 05:31:31 +01:00
Daniel Hahler 716f532a38
Merge pull request #6070 from blueyed/pexpect-freebsd
pytester: spawn: do not skip FreeBSD
2019-10-28 05:13:31 +01:00
Daniel Hahler 886a3ad609 pytester: typing 2019-10-27 12:32:14 +01:00
Daniel Hahler 7f1af84f47 Merge master into features
Conflicts:
	src/_pytest/logging.py
2019-10-27 02:06:36 +01:00
Daniel Hahler 8e8a8fa4b9 pytester: spawn: do not skip FreeBSD
Fixes https://github.com/pytest-dev/pytest/issues/6069
2019-10-27 00:44:25 +02:00
Ran Benita 0b8c35516f Replace py.io.TextIO with io.StringIO
In Python3, py.io.TextIO is just an alias to io.StringIO. Remove the
indirection.
2019-10-26 16:33:57 +03:00
Bruno Oliveira d1bc2601e4
pytester: align prefixes (#6026)
pytester: align prefixes
2019-10-24 18:50:29 -03:00
Daniel Hahler 8ef4287bf0 pytester: align prefixes
This is important for using another match_nickname, e.g. "re.match".

TODO:

- [ ] changelog
- [ ] test
2019-10-24 23:20:12 +02:00
Daniel Hahler 6242777818
Merge pull request #6041 from blueyed/pytester-splitlines
pytester: runpytest_inprocess: use splitlines()
2019-10-23 21:42:21 +02:00
Bruno Oliveira 20ee883b5f
Show the mnemonic of pytest.ExitCode in RunResult's repr (#6043)
Show the mnemonic of pytest.ExitCode in RunResult's repr
2019-10-23 10:46:58 -03:00
Bruno Oliveira 7beb520555 Show the mnemonic of pytest.ExitCode in RunResult's repr
Fix #4901
2019-10-23 09:16:02 -03:00
Daniel Hahler 2f589a9769 pytester: runpytest_inprocess: use splitlines()
This avoids having a trailing empty lines always.
2019-10-23 04:34:29 +02:00
Daniel Hahler b47f57a08a pytester: parseconfigure: remove duplicate config._ensure_unconfigure
This gets done in `parseconfig` already.
2019-10-21 01:15:27 +02:00