This fixes CI on Windows since GitHub Actions started installing WSL on
their images which apparently installs some wrapper `bash` which does
not run actual bash.
Previously was restricted to >=1.8.2,<2.1, but newer versions have some
nice improvements I'd like to be able to use in upcoming changes.
Changelog: https://www.sphinx-doc.org/en/master/changes.html#release-3-1-0-released-jun-08-2020
There are two issues that came up:
1. `highlightlang` is deprecated for `highlight`.
2. Doesn't like having two `automethod` generated for the same
`Metafunc.parametrize` method. Gives this warning:
`pytest/doc/en/reference.rst:846: WARNING: duplicate object description of _pytest.python.Metafunc.parametrize, other instance in reference, use :noindex: for one of them`
To work around this I make `pytest.mark.parametrize` link to
`Metafunc.parametrize` instead of repeating it.
The previous typing had an object passed to the user, which they can't
do anything with without asserting, which is inconvenient. Change it to
Any instead.
Note that what comes *back* to pytest (the return value) should be an
`object`, because we want to handle arbitrary objects without assuming
anything about them.
If a test runtest phase (not setup) dynamically adds a pytest.mark.xfail
mark to the item, it should be respected, but it wasn't. This regressed
in 3e6fe92b7e (not released).
Fix it by just always refreshing the mark if needed. This is mostly what
was done before but in a more roundabout way.
This introduces a new role, `:globalvar:`, so we can mark/reference
variables like `pytest_plugins`, `pytestmark`, etc. This besides being useful
also makes the documentation look more consistent.
eval() is used for evaluating string conditions in skipif/xfail e.g.
@pytest.mark.skipif("1 == 0")
This is the only code that uses `_pytest._code.compile()`, so removing
its last use enables us to remove it entirely.
In this case it doesn't add much. Plain compile() gives a good enough
error message.
For regular exceptions, the message is the same.
For SyntaxError exceptions, e.g. "1 ==", the previous code adds a little
bit of useful context:
```
invalid syntax (skipping.py:108>, line 1)
The above exception was the direct cause of the following exception:
1 ==
^
(code was compiled probably from here: <0-codegen /pytest/src/_pytest/skipping.py:108>) (line 1)
During handling of the above exception, another exception occurred:
Error evaluating 'skipif' condition
1 ==
^
SyntaxError: invalid syntax
```
The new code loses it:
```
unexpected EOF while parsing (<skipif condition>, line 1)
During handling of the above exception, another exception occurred:
Error evaluating 'skipif' condition
1 ==
^
SyntaxError: invalid syntax
```
Since the old message is a minor improvement to an unlikely error
condition in a deprecated feature, I think it is not worth all the code
that it requires.
This has been there since as far as the git history goes (2007), is not
covered by any test, and says "Buggy python version consider upgrading".
Hopefully everyone have upgraded...
Setting log_level via the CLI or .ini will control the log level of the
report that is dumped upon failure of a test.
If caplog modified the log level during the execution of that test, it
should not impact the level that is displayed upon failure in the
"captured log report" section.
[
ran:
- rebased
- reused handler
- changed store keys also to "caplog_handler_*"
- added changelog
all bugs are mine :)
]
There is no need to do the XPASS check here, pytest_runtest_makereport
already handled that (the current handling there is dead code).
All the hook needs to do is refresh the xfail evaluation if needed, and
check the NOTRUN condition again.
Previously, skipif/xfail marks were evaluated using a `MarkEvaluator`
class. I found this class very difficult to understand.
Instead of `MarkEvaluator`, rewrite using straight functions which are
hopefully easier to follow.
I tried to keep the semantics exactly as before, except improving a few
error messages.
This type was actually in `_pytest.skipping` previously, but was moved to
`_pytest.mark.evaluate` in cf40c0743c.
I think the previous location was more appropriate, because the
`MarkEvaluator` is not a generic mark facility, it is explicitly and
exclusively used by the `skipif` and `xfail` marks to evaluate their
particular set of arguments. So it is better to put it in the plugin
code.
Putting `skipping` related functionality into the core `_pytest.mark`
module also causes some import cycles which we can avoid.
`@pytest.mark.xfail` is meant to work with arbitrary items, and there is
a test `test_mark_xfail_item` which verifies this.
However, the code for some reason uses `pytest_pyfunc_call` for the
call phase check, which only works for Function items. The test
mentioned above only passed "accidentally" because the
`pytest_runtest_makereport` hook also runs a `evalxfail.istrue()` which
triggers and evaluation, but conceptually it shouldn't do that.
Change to `pytest_runtest_call` to make the xfail checking properly
generic.
While working on improving the documentation of the
`pytest_runtest_setup` hook, I came up with this text:
> Called to perform the setup phase of the test item.
>
> The default implementation runs ``setup()`` on item and all of its
> parents (which haven't been setup yet). This includes obtaining the
> values of fixtures required by the item (which haven't been obtained
> yet).
But upon closer inspection I noticed this line at the start of
`SetupState.prepare` (which is what does the actual work for
`pytest_runtest_setup`):
self._teardown_towards(needed_collectors)
which implies that the setup phase of one item might trigger teardowns
of *previous* items. This complicates the simple explanation. It also
seems like a completely undesirable thing to do, because it breaks
isolation between tests -- e.g. a failed teardown of one item shouldn't
cause the failure of some other items just because it happens to run
after it.
So the first thing I tried was to remove that line and see if anything
breaks -- nothing did. At least pytest's own test suite runs fine. So
maybe it's just dead code?
This option checks even functions which are not annotated. It's a good
step to ensure that existing type annotation are correct.
In a Pareto fashion, the last few holdouts are always the ugliest,
beware.
The FixtureFunctionMarker attrs class already converts the params
itself.
When adding types, the previous converter composition causes some type
error, but extracting it to a standalone function fixes the issue (a
lambda is not supported by the mypy plugin, currently).
* Fix typo in capture.rst documentation
Rename ``capfsysbinary`` to ``capsysbinary`` as the former does not exist as far as i can see.
* Make Python uppercase in doc/en/capture.rst
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* Remove the sentence entirely
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
* enable tox -e docs natively on windows using a wrapper
* rename the towncrier script; run the towncrier command in a safer manner
* use subprocess.call; call exit() around main on towncrier wrapper
* change to sys.exit() instead of builtin exit()
There are state transitions start/done/suspend/resume and two additional
operations snap/writeorg.
Previously it was not well defined in what order they can be called, and
which operations are idempotent.
Formalize this and enforce using assert checks with informative error
messages if they fail (rather than random AttributeErrors).
Make it easier to read the file in progression, and avoid forward
references for upcoming type annotations.
There is one cycle, CaptureManager <-> CaptureFixture, which is hard to
untangle.
(This commit should be added to `.gitblameignore`).
`TerminalWriter`, imported recently from `py`, contains its own
incomplete wcwidth (`char_with`/`get_line_width`) implementation. The
`TerminalReporter` also needs this, but uses the external `wcwidth`
package.
This commit brings the `TerminalWriter` implementation up-to-par with
`wcwidth`, moves to implementation to a new file `_pytest._io.wcwidth`
which is used everywhere, and removes the dependency.
The differences compared to the `wcwidth` package are:
- Normalizes the string before counting.
- Uses Python's `unicodedata` instead of vendored Unicode tables. This
means the data corresponds to the Python's version Unicode version
instead of the `wcwidth`'s package version.
- Apply some optimizations.
pytest.fixture() can be used either as
@pytest.fixture
def func(): ...
or as
@pytest.fixture()
def func(): ...
or (while maybe not intended)
func = pytest.fixture(func)
so it needs to inspect internally whether it got a function in the first
positional argument or not.
Previously, there were was oddity. In the following,
func = pytest.fixture(func, autouse=True)
# OR
func = pytest.fixture(func, parms=['a', 'b'])
The result is as if `func` wasn't passed.
There isn't any reason for this special that I can understand, so remove
it.
_pytest.timing is an indirection to 'time' functions, which pytest production
code should use instead of 'time' directly.
'mock_timing' is a new fixture which then mocks those functions, allowing us
to write time-reliable tests which run instantly and are not flaky.
This was triggered by recent flaky junitxml tests on Windows related to timing
issues.
This is already done in pytest_runtest_logstart, so the fspath is
already guaranteed to have been printed (for xdist, it is disabled
anyway).
write_fspath_result is mildly expensive so it is worth avoiding calling
it twice.
The `FDCapture`/`FDCaptureBinary` classes, used by `capfd`/`capfdbinary`
fixtures and the `--capture=fd` option (set by default), redirect FDs
1/2 (stdout/stderr) to a temporary file. To do this, they need to save
the old file by duplicating the FD before redirecting it, to be restored
once finished.
Previously, if this duplicating (`os.dup()`) failed, most likely due to
that FD being invalid, the FD redirection would silently not be done. The
FD capturing also performs python-level redirection (monkeypatching
`sys.stdout`/`sys.stderr`) which would still be done, but direct writes
to the FDs would fail.
This is not great. If pytest is run with `--capture=fd`, or a test is
using `capfd`, it expects writes to the FD to work and be captured,
regardless of external circumstances.
So, instead of disabling FD capturing, keep the redirection to a
temporary file, just don't restore it after closing, because there is
nothing to restore to.
This makes a difference for e.g. pytest-xdist:
Before:
```
--dist=distmode set mode for distributing tests to exec environments. each: …
available environment. loadscope: …
grouped by file to any available environment. (default) no: …
```
After:
```
--dist=distmode set mode for distributing tests to exec environments.
each: send each test to all available environments.
load: load balance by sending any pending test to any available environment.
…
(default) no: run tests inprocess, don't distribute.
```
This might also result in unexpected changes (hard wrapping), when line
endings where used unintentionally, e.g. with:
```
help="""
some long
help text
"""
```
But the benefits from that are worth it, and it is easy to fix, as will
be done for the internal `assertmode` option.
Currently, a bad logging call, e.g.
logger.info('oops', 'first', 2)
triggers the default logging handling, which is printing an error to
stderr but otherwise continuing.
For regular programs this behavior makes sense, a bad log message
shouldn't take down the program. But during tests, it is better not to
skip over such mistakes, but propagate them to the user.
Previously, a LoggingCaptureHandler was instantiated for each test's
setup/call/teardown which turns out to be expensive.
Instead, only keep one instance and reset it between runs.
The tests came via c629f6b18 and c61ff31ffa.
The fixes from there are kind of obsoleted by 4cd08f9 (moving to importlib),
but it makes sense to keep them as integration tests in general.
The logstart/logreport/logfinish hooks don't need the stuff in
_runtest_for. The test capturing catching_logs call is irrelevant for
them, and the item-conditional sections are gone.
- Instead of making it optional, always set up a handler, but possibly
going to /dev/null. This simplifies the code by removing a lot of
conditionals. It also can replace the NullHandler() we already add.
- Change `set_log_path` to just change the stream, instead of recreating
one. Besides plugging a resource leak, it enables the next item.
- Remove the capturing_logs from _runtest_for, since it sufficiently
covered by the one in pytest_runtestloop now, which wraps all other
_runtest_for calls.
The first item alone would have had an adverse performance impact, but
the last item removes it.
Remove usage of `@contextmanager` as it is a bit slower than
hand-rolling, and also disallows re-entry which we want to use.
Removing protections around addHandler()/removeHandler(), because
logging already checks that internally.
Conceptually it doesn't check per catching_logs (and catching_logs
doesn't restore the older one either). It is just something that is
defined for each handler once.
The default message is often hard to read:
E _pytest.config.ConftestImportFailure: (local('D:\\projects\\pytest\\.tmp\\root\\foo\\conftest.py'), (<class 'RuntimeError'>, RuntimeError('some error',), <traceback object at 0x000001CCC3E39348>))
Using a shorter message is better:
E _pytest.config.ConftestImportFailure: RuntimeError: some error (from D:\projects\pytest\.tmp\root\foo\conftest.py)
And we don't really lose any information due to exception chaining.
This removes the KeyError from the traceback chain when an
conftest fails to import:
return self._conftestpath2mod[key]
E KeyError: WindowsPath('D:/projects/pytest/.tmp/root/foo/conftest.py')
During handling of the above exception, another exception occurred:
...
raise RuntimeError("some error")
E RuntimeError: some error
During handling of the above exception, another exception occurred:
...
E _pytest.config.ConftestImportFailure: (...)
By slightly changing the code, we can remove the first chain, which is often
very confusing to users and doesn't help with anything.
Fix#7223
Only filter with known failures, and explicitly keep paths of passed
arguments.
This also displays the "run-last-failure" status before collected files,
and does not update the cache with "--collect-only".
Fixes https://github.com/pytest-dev/pytest/issues/6968.
The previous commit made this possible, so utilize it.
Since legacy.py becomes pretty bare, I inlined it into __init__.py. I'm
not sure it's really "legacy" anyway!
Using a simple 50000 items benchmark with `--collect-only -k nomatch`:
Before (two commits ago):
======================== 50000 deselected in 10.31s =====================
19129345 function calls (18275596 primitive calls) in 10.634 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.001 0.001 2.270 2.270 __init__.py:149(pytest_collection_modifyitems)
1 0.036 0.036 2.270 2.270 __init__.py:104(deselect_by_keyword)
50000 0.055 0.000 2.226 0.000 legacy.py:87(matchkeyword)
After:
======================== 50000 deselected in 9.37s =========================
18029363 function calls (17175972 primitive calls) in 9.701 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 1.394 1.394 __init__.py:239(pytest_collection_modifyitems)
1 0.057 0.057 1.393 1.393 __init__.py:162(deselect_by_keyword)
The matching itself can be optimized more but that's a different story.
In current pytest, the same expression is matched against all items. But
it is re-parsed for every match.
Add support for "compiling" an expression and reusing the result. Errors
may only occur during compilation.
This is done by parsing the expression into a Python `ast.Expression`,
then `compile()`ing it into a code object. Evaluation is then done using
`eval()`.
Note: historically we used to use `eval` directly on the user input --
this is not the case here, the expression is entirely under our control
according to our grammar, we just JIT-compile it to Python as a
(completely safe) optimization.
New errors:
testing/test_setupplan.py:104:15: E741 ambiguous variable name 'l'
testing/test_setupplan.py:107:15: E741 ambiguous variable name 'l'
extra/get_issues.py:48:29: E741 ambiguous variable name 'l'
testing/test_error_diffs.py:270:32: E741 ambiguous variable name 'l'
Not so sure about it but easier to just fix.
But more importantly, is a large amount of typing-related issues there
were fixed which necessitated noqa's which can now be removed.
The `-k '-expr'` syntax is an old alias to `-k 'not expr'`. It's also
not a very convenient to have syntax that start with `-` on the CLI.
Deprecate it and suggest replacing with `not`.
---
The `-k 'expr:'` syntax discards all items until the first match and
keeps all subsequent, e.g. `-k foo` with
test_bar
test_foo
test_baz
results in `test_foo`, `test_baz`. That's a bit weird, so deprecate it
without a replacement. If someone complains we can reconsider or devise
a better alternative.
Running `pytest | head -1` and similar causes an annoying error to be
printed to stderr:
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
(or possibly even a propagating exception in older/other Python versions).
The standard UNIX behavior is to handle the EPIPE silently. To
recommended method to do this in Python is described here:
https://docs.python.org/3/library/signal.html#note-on-sigpipe
It is not appropriate to apply this recommendation to `pytest.main()`,
which is used programmatically for in-process runs. Hence, change
pytest's entrypoint to a new `pytest.console_main()` function, to be
used exclusively by pytest's CLI, and add the SIGPIPE code there.
Fixes#4375.
When setting up the warnings capture, filter strings (with the general
form `action:message:category:module:line`) are collected from the
cmdline, ini and item and applied. This happens for every test and other
cases.
To apply a string it needs to be parsed into a tuple, and it turns out
this is slow. Since we already vendor the parsing code from Python's
warnings.py, we can speed it up by caching the result. After splitting
the parsing part from the applying part, the parsing is pure and is
straightforward to cache.
An alternative is to parse ahead of time and reuse the result, however
the caching solution turns out cleaner and more general in this case.
On this benchmark:
import pytest
@pytest.mark.parametrize("x", range(5000))
def test_foo(x): pass
Before:
============================ 5000 passed in 14.11s =============================
14365646 function calls (13450775 primitive calls) in 14.536 seconds
After:
============================ 5000 passed in 13.61s =============================
13290372 function calls (12375498 primitive calls) in 14.034 seconds
- replace "tests with warnings" with just warnings: they a) might not
come from a test in the first place, and b) a single test might have
multiple warnings.
- fix usage of (unused) argument to `collapsed_location_report`
Co-authored-by: Ran Benita <ran@unusedvar.com>
Move {Passthrough,CaptureIO} to capture module, and rename Passthrough
-> Tee to match the existing terminology.
Co-authored-by: Ran Benita <ran@unusedvar.com>
Also delay calling tearDown() when --pdb is given, so users still have
access to the instance variables (which are usually cleaned up during tearDown())
when debugging.
Fix#6947
Instead of trying to handle unittest-async functions in pytest_pyfunc_call,
let the unittest framework handle them instead.
This lets us remove the hack in pytest_pyfunc_call, with the upside that
we should support any unittest-async based framework.
Also included 'asynctest' as test dependency for py37-twisted, and renamed
'twisted' to 'unittestextras' to better reflect that we install 'twisted' and
'asynctest' now.
This also fixes the problem of cleanUp functions not being properly called
for async functions.
Fix#7110Fix#6924
Previously, the expressions given to the `-m` and `-k` options were
evaluated with `eval`. This causes a few issues:
- Python keywords cannot be used.
- Constants like numbers, None, True, False are not handled correctly.
- Various syntax like numeric operators and `X if Y else Z` is supported
unintentionally.
- `eval()` is somewhat dangerous for arbitrary input.
- Can fail in many ways so requires `except Exception`.
The format we want to support is quite simple, so change to a custom
parser. This fixes the issues above, and gives us full control of the
format, so can be documented comprehensively and even be extended in the
future if we wish.
Currently this property is computed eagerly, which means
get_line_width() is computed on everything written, but that is a slow
function.
Compute it lazily, so that get_line_width() only runs when needed.
Flushing on every write is somewhat expensive.
Rely on line buffering instead (if line buffering for stdout is
disabled, there must be some reason...), and add explicit flushes when
not outputting lines.
This is how regular `print()` e.g. work so should be familiar.
The code used an O(n^2) loop. Replace list with set to make it O(n).
For backward compatibility the filesystem cache still remains a list.
On this test:
import pytest
@pytest.mark.parametrize("x", range(5000))
def test_foo(x): pass
run with `pytest --collect-only`:
Before: 0m1.251s
After: 0m0.921s
Remove this message which was shown in the warning summary of pytest's
own testsuite:
testing/test_cacheprovider.py::TestNewAPI::test_cache_writefail_cachfile_silent
testing/test_cacheprovider.py:40: PytestCacheWarning: could not create cache path /tmp/pytest-of-ran/pytest-2/test_cache_writefail_cachfile_silent0/.pytest_cache/v/test/broken
cache.set("test/broken", [])
Currently this test issues a warning which is displayed in the warning
summary (of pytest's own test suite):
testing/acceptance_test.py::TestGeneralUsage::test_early_skip
/tmp/pytest-of-ran/pytest-396/test_early_skip0/conftest.py:2: PytestDeprecationWarning: The pytest_collect_directory hook is not working.
Please use collect_ignore in conftests or pytest_collection_modifyitems.
def pytest_collect_directory():
I think the filter was meant to be `ignore` in the first place, and not
`always` which is not a valid action AFAIK.
This function is exposed and kept alive for the oejskit plugin which is
abandoned and no longer works with recent plugins, so let's prepare to
completely remove it.
Earlier pytest-bot would only print out the exception in cases of
failure but did not provide context on failing command and error from
command. This patch adds the errors from the command to the exception
message.
`Command` provides abstraction over the command to run and helps
in collecting errors from the first failing command only.
With this, we don't need to check `returncode` from each command that
we run, we can run all the commands and will have access to the error
from the first command that failed. This pattern was taken from Go.
Please refer: https://blog.golang.org/errors-are-values
The `_pytest._code._reprcompare` that was referred to previously doesn't
exist -- it was moved to other places but wasn't updated. This regressed
in f423ce9c01. Now we don't want it
anymore, so keep the status quo by explicitly removing them.
* tests: fix TypeError with test_mark_closest
It fails when trying to run it actually:
> TypeError: test_has_inherited() takes 0 positional arguments but 1 was given
* Fix testing/test_collection.py::TestCollector::test_getparent
Mostly I wanted to get rid of mentions of "MarkItem" which is something
that no longer exists, but I improved a little beyond that and annotated
some simple types.
It was surprising that `tryfirst=True` would not result in lines being
added to the beginning with `pytest_report_header`.
This is due to lines being reversed, and therefore the same applies to
`pytest_report_collectionfinish`.
TestDurations tests the `--durations=N` functionality which reports N
slowest tests, with durations <= 0.005s not shown by default.
The test relies on real time.sleep() (in addition to the code which uses
time.perf_counter()) which makes it flaky and inconsistent between
platforms.
Instead of trying to tweak it more, make it use fake time instead. The
way it is done is a little hacky but seems to work.
This hook has some functionality to provide explicit markup for the test
status. It seemed unused and wasn't tested, so I was tempted to remove
it, but I found that the pytest-rerunfailures plugin uses it, so
document it and add a test instead.
Use `testdir.syspathinsert()` with multiprocessing tests:
- test_chained_exceptions_no_reprcrash
- test_exception_handling_no_traceback
This only works currently because `_importtestmodule` changes `sys.path`
as a side-effect.
It appears to be only required on Windows though - likely due to the
multiprocessing method used there.
The dynamic scope feature added in 10bf6aac76
necessitated some wrangling of arguments in pytest.fixture(). In
particular, it deprecated positional arguments in favor of keyword-only
arguments, while keeping backward compatibility.
The way it did this avoided some code duplication but ended up being
quite hard to follow and to annotate with types.
Replace it with some straightforward code, which is not very DRY but is
simple and easy to remove when the time comes.
This changes the link anchors in "reference.html", from e.g.
`reference.html#pytest-current-test` to
`reference.html#envvar-PYTEST_CURRENT_TEST`, but I think that is OK, and
not worth adding labels for the old anchors.
twisted started to use `attr.s(eq)` argument which was added recently,
so it fails with oldattrs. One of the CI jobs ran twisted and oldattrs
together, so it started to fail.
Move the twisted code to be covered by another job, and remove it from
the job with the oldattrs.
Previously, writing to sys.stdout/stderr in text-mode (e.g.
`print('foo')`) while a `capsysbinary` fixture is active, would crash
with:
/usr/lib/python3.7/contextlib.py:119: in __exit__
next(self.gen)
E TypeError: write() argument must be str, not bytes
This is due to some confusion in the types. The relevant functions are
`snap()` and `writeorg()`. The function `snap()` returns what was
captured, and the return type should be `bytes` for the binary captures
and `str` for the regular ones. The `snap()` return value is eventually
passed to `writeorg()` to be written to the original file, so it's input
type should correspond to `snap()`. But this was incorrect for
`SysCaptureBinary`, which handled it like `str`.
To fix this, be explicit in the `snap()` and `writeorg()`
implementations, also of the other Capture types.
We can't add type annotations yet, because the current inheritance
scheme breaks Liskov Substitution and mypy would complain. To be
refactored later.
Fixes: https://github.com/pytest-dev/pytest/issues/6871
Co-authored-by: Ran Benita (some modifications & commit message)
I tried to understand what the `safe_text_dupfile()` function and
`EncodedFile` class do. Outside tests, `EncodedFile` is only used by
`safe_text_dupfile`, and `safe_text_dupfile` is only used by
`FDCaptureBinary.__init__()`. I then started to eliminate always-true
conditions based on the single call site, and in the end nothing was
left except of a couple workarounds that are still needed.
The commit which added the checks for os.dup a15afb5e48
suggests it was done for Jython. But pytest doesn't support Jython
anymore (Jython is Python 2 only).
Furthermore, it looks like the faulthandler plugin (bundled in pytest
and enabled by default) uses os.dup() unprotected and there have not
been any complaints.
So seems better to just remove these checks, and only add if someone
with a legitimate use case complains.
`KeywordMapping` returns a bool on lookup which when passed to eval
fail on certain operations such as index access and attribute access.
We catch all exceptions and raise a `UsageError`.
- it should not document the deprecated `result`; used the same as
pluggy documents itself
- add a "hookwrapper" label, that could be used by pluggy (currently it
links to the section)
- use pluggy's `hookwrappers` label for linking to its documentation
When using `(i)var` in the class docstring it would link `duration` to
`_pytest.runner.TestReport.duration`.
This moves the docstrings to the attributes properly.
This started from fixing the test, where `"xdist_strict True"` was used
as a single argument, although you typically would see `["xdist_strict",
"True"]`.
Improves the error message to mention the option that caused the error.
pytest has several instances where plugins set their own attributes on
objects they receive in hooks, like nodes and config. Since plugins are
detached from these object's definition by design, this causes a problem
for type checking because these attributes are not defined and mypy
complains.
Fix this by giving these objects a "store" which can be used by plugins
in a type-safe manner.
Currently this mechanism is private. We can consider exposing it at a
later point.
The pytest-konira plugin has not seen an update since 2011, moreover
the "project description" on PyPI points to a dubious website that
does not actually correspond to the project and instead redirects to
advertising content.
This was changed unintentionally in 45c4a8fb3 (pytest 5.3.0), but only
EnvironmentErrors might have `errno`.
Since that is not really guaranteed and it is good to have more
information this uses the string representation of the exc in the trace
message.
It should only return `True` when something is to be ignored, not
`False` otherwise typically.
This caused e.g. bad interaction with the cacheprovider (before
https://github.com/pytest-dev/pytest/pull/6448).
Passing in a tuple crashes in `_prepareconfig`:
def test_invoke_with_tuple(self):
> pytest.main(("-h",))
src/_pytest/config/__init__.py:82: in main
config = _prepareconfig(args, plugins)
src/_pytest/config/__init__.py:229: in _prepareconfig
return pluginmanager.hook.pytest_cmdline_parse(
…
src/_pytest/helpconfig.py:98: in pytest_cmdline_parse
config = outcome.get_result() # type: Config
src/_pytest/config/__init__.py:808: in pytest_cmdline_parse
self.parse(args)
src/_pytest/config/__init__.py:1017: in parse
self._preparse(args, addopts=addopts)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def _preparse(self, args: List[str], addopts: bool = True) -> None:
…
if addopts:
ini_addopts = self.getini("addopts")
if ini_addopts:
> args[:] = self._validate_args(ini_addopts, "via addopts config") + args
E TypeError: can only concatenate list (not "tuple") to list
addopts = True
args = ('-h',)
env_addopts = ''
ini_addopts = ['-rfEX', …]
src/_pytest/config/__init__.py:956: TypeError: can only concatenate list (not "tuple") to list
Might be worth handling (converting it to a list for example), but it
was documented to be a list to begin with when removing support for
strings (a7e401656).
Remove `_tmp_path_factory` and `_tmpdirhandler` from the config object.
- `_tmpdirhandler` has been deprecated since 2.8.0 (0f52856f9), when
`tmpdir_factory` has been added.
- `_tmp_path_factory` should have probably never been added there in the
first place, but maybe just used the same pattern (16e2737da).
This better reflects the inheritance / smartness with regard to raw or
encoded.
- FDCaptureBinary
- FDCapture
- SysCaptureBinary
- SysCapture
- TeeSysCapture
Mypy currently is unable to handle assigning attributes on function:
https://github.com/python/mypy/issues/2087.
pytest uses this for the outcome exceptions -- `pytest.fail.Exception`,
`pytest.exit.Exception` etc, and this is the canonical name by which they
are referred.
Initially we started working around this with type: ignores, and later
by switching e.g. `pytest.fail.Exception` with the direct exception
`Failed`. But this causes a lot of churn and is not as nice. And I also
found that some code relies on it, in skipping.py:
def pytest_configure(config):
if config.option.runxfail:
# yay a hack
import pytest
old = pytest.xfail
config._cleanup.append(lambda: setattr(pytest, "xfail", old))
def nop(*args, **kwargs):
pass
nop.Exception = xfail.Exception
setattr(pytest, "xfail", nop)
...
So it seems better to support it. Use a hack to make it work. The rest
of the commit rolls back all of the workarounds we added up to now.
`pytest.raises.Exception` also exists, but it's not used much so I kept
it as-is for now.
Hopefully in the future mypy supports this and this ugliness can be
removed.
Allows for filtering of PytestCacheWarning.
Using `_issue_warning_captured` is not necessary here, and was probably
only used because the cacheprovider misses warnings during
`pytest_sessionfinish`, which is also fixed here.
I think the usage of `_issue_warning_captured` can be removed/reduced
further, but also that this is good enough for now.
Ref: https://github.com/pytest-dev/pytest/issues/6681.
* Turn ReprTraceback into attrs class
* Use attr.asdict with serialize_repr_{crash,traceback}
* Turn ReprFileLocation into attrs class, convert py.path.local
* Use code highlighting if pygments is installed
* Use colorama constants instead of bare ascii codes
Could not find the exact equivalent of 'hl-reset' code using colorama
constants though.
* Refactor ASCII color handling into a fixture
* Revert back to using explicit color codes
* In Python 3.5 skip rest of tests that require ordered markup in colored output
Use suggestion in review and use a subplugin so hooks will only be active
if we enable faulthandler ourselves.
Fix#6575
Co-authored-by: Daniel Hahler <git@thequod.de>
ExitCode is used in several internal modules and hooks and so with type
annotations added, needs to be imported a lot.
_pytest.main, being the entry point, generally sits at the top of the
import tree.
So, it's not great to have ExitCode defined in _pytest.main, because it
will cause a lot of import cycles once type annotations are added (in
fact there is already one, which this change removes).
Move it to _pytest.config instead.
_pytest.main still imports ExitCode, so importing from there still
works, although external users should really be importing from `pytest`.
The convention is "assert result is expected". Pytest's error diffs now
reflect this. "-" means that sth. expected is missing in the result and
"+" means that there are unexpected extras in the result.
Fixes: #3333
Fix mypy errors:
src/_pytest/runner.py:36: error: "addoption" of "OptionGroup" does not return a value [func-returns-value]
src/_pytest/helpconfig.py:64: error: "addoption" of "OptionGroup" does not return a value [func-returns-value]
src/_pytest/terminal.py:67: error: "_addoption" of "OptionGroup" does not return a value [func-returns-value]
src/_pytest/terminal.py:75: error: "_addoption" of "OptionGroup" does not return a value [func-returns-value]
Fixes collection error with Python 3.5.3 (Travis):
testing/test_parseopt.py:2: in <module>
import distutils.spawn
.tox/py35-coverage/lib/python3.5/distutils/__init__.py:4: in <module>
import imp
.tox/py35-coverage/lib/python3.5/imp.py:33: in <module>
PendingDeprecationWarning, stacklevel=2)
E PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
Build log: https://travis-ci.org/blueyed/pytest/builds/648305304
This started by looking at how to get the current test item in general,
and then I noticed that it is not necessary for the capture plugin to
track it manually in the first place.
Previously this would be turned via `py.path.local("")` into the current
working directory.
This appears to be what `fspath = fn and py.path.local(fn) or None`
tries to avoid in `getfslineno`'s `TypeError` handling already, if
`Code` would raise it.
Everything was using `_pytest.compat.getfslineno` basically, which
wrapped `_pytest._code.source.getfslineno`.
This moves the extra code from there into it directly, and uses the
latter everywhere.
This helps to eventually remove the one in compat eventually, and also
causes less cyclic imports.
* Explicitly state on the PR template that we can squash commits
This way we don't need to ask every time, and users who for some reason
would not like us to squash their commits can explicitly state so.
Plugins specified with ``-p`` are now loaded after internal plugins, which
results in their hooks being called *before* the internal ones.
This makes the ``-p`` behavior consistent with ``PYTEST_PLUGINS``.
* fix/adjust test_disable_plugin_autoload
* adjust test_plugin_loading_order
`Generator` was removed in 7eb28f9eb, and this pleases mypy to correctly
complain that `FunctionMixin` has no `config` (within
`_prunetraceback`).
* typing: _prunetraceback
* minor: imports, typing
Reverts https://github.com/pytest-dev/pytest/pull/6610.
The tested `getfslineno` is `src/_pytest/_code/source.py` actually,
exported via `src/_pytest/_code/__init__.py`.
I've confused it with the one in `src/_pytest/compat.py` apparently.
It should be in `test_code` when testing `_pytest._code.getfslineno`,
not to be confused with `_pytest._code.source.getfslineno`.
Adds an extra assert (via https://github.com/pytest-dev/pytest/pull/6590).
This often might be causing for jobs to take longer than 10 minutes,
which is a timeout Codecov uses to wait for successful CI.
Also it is good in general to have CI finish faster, of course.
From the description it seemed to me as if just prefixing your methods with ``test_`` was enough, but you also need to prefix your class with ``Test``. Of course, in the reference material this is clearly stated, but I think it makes sense to mention it here as well, since you also mention the part about the methods' prefix.
While it might not help with the following, it certainly might happen as
well.
```
+ curl -S -L --connect-timeout 5 --retry 6 -s https://codecov.io/bash -o codecov-upload.sh
curl: (16) Error in the HTTP2 framing layer
```
- refactor _get_main_color/build_summary_stats_line
- factor out property _is_last_item; test_summary_stats: tr._is_last_item
- _write_progress_information_filling_space: remove color arg
- use setter for stats, handling main color
- _get_main_color: skip cache for last item
- Handle random order in test for py35.
Mypy complains about this once the function is typed:
src/_pytest/main.py:85: error: "_addoption" of "OptionGroup" does not return a value
src/_pytest/main.py:133: error: "addoption" of "OptionGroup" does not return a value
Previously, this cache was used with 3 different and mutually exclusive
key-type -> value-type combinations. Mypy can't properly type this. It's
also quite confusing.
Split to 3 different dicts instead.
Previously, _initialparts was a list whose first item was a
`py.path.local` and the rest were `str`s. This is not something that
mypy is capable of modeling. The type `List[Union[str, py.path.local]]`
is too broad and would require asserts for every access.
Instead, make each item a `Tuple[py.path.local, List[str]]`. This way
the structure is clear and the types are accurate.
To make sure any users who might have been accessing this (private)
field will not break silently, change the name to _initial_parts.
Make EncodedFile, used for captured output streams, method .write return
the number of characters written. Add test for captured stderr write.
Fixes#6557.
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
Without restoring the cwd, successive tests might fail to parse the
config (via `_pytest.config._prepareconfig()`, for when `--lsof` is
used).
And it is good practice to restore the cwd in any case anyway.
These are more "dirty" than the previous batch (that's why they were
left out). The trouble is that `compile` can return either a code object
or an AST depending on a flag, so we need to add an overload to make the
common case Union free. But it's still worthwhile.
GitHub actions already should deploy.
This is a stop gap while we figure out why coverage dropped
when removing Travis builds in #6470
(cherry picked from commit d1d7e5d41b)
Somehow in Python 3.5 on Windows this test fails with:
File "c:\hostedtoolcache\windows\python\3.5.4\x64\Lib\multiprocessing\connection.py", line 302, in _recv_bytes
overlapped=True)
OSError: [WinError 6] The handle is invalid
This only happens in this platform and Python version, decided to use
a dummy traceback as originally done in #6412.
(cherry picked from commit b9c136b809)
This allows for e.g. Jedi to infer types (it checks the name).
It was only used to support Python 3.5.0/3.5.1, where this is is not
available in the `typing` module.
Ref: https://github.com/davidhalter/jedi/issues/1472
Uses `TYPE_CHECKING = False` in `_pytest.outcomes` to avoid having to
work around circular import.
Somehow in Python 3.5 on Windows this test fails with:
File "c:\hostedtoolcache\windows\python\3.5.4\x64\Lib\multiprocessing\connection.py", line 302, in _recv_bytes
overlapped=True)
OSError: [WinError 6] The handle is invalid
This only happens in this platform and Python version, decided to use
a dummy traceback as originally done in #6412.
This is different from what pre-commit (in "linting") runs in that it
uses stubs from (test) dependencies.
It would make sense to run this on CI additionally (since there is no
"pre-commit --skip mypy", and a separate config is not worth it).
But currently it triggers a false positive though anyway
(https://github.com/erikrose/more-itertools/pull/374).
This overwrites the `codecov.yml` file in the root of the repository with
`codecov-upstream.yml` file (which contains the code-cov token)´, so PRs
and branches on the repository can upload coverage.
Suggestion from here:
https://github.com/pytest-dev/pytest/pull/6421#issuecomment-571934112
Security concerns: the token might be misused, but only to upload bogus coverage
to `codecov.io`, so the team believe this is harmless. If we decide to fallback
from this decision , we just need to revoke the token.
Related to #6369
Both references were referencing links from Python because of our intersphinx
mapping in `conf.py`:
intersphinx_mapping = {"python": ("https://docs.python.org/3", None)}
Because Python's docs explicitly define both references, Sphinx fallbacks to
them instead of generating implicit references as was expected.
Fix#6397
Tracebacks coming from remote processes crated by the multiprocess module
will contain "RemoteTracebacks" which don't have a 'reprcrash' attribute
Fix#5971
I've updated the text and incorporated the topics from #5275, so this
can now be part of the official docs, and #5275 can be closed/unpinned.
Closes#5275
This fixes some type: ignores due to typeshed update.
Newer mypy seem to ignore unannotated functions better, so add a few
minor annotations so that existing correct type:ignores make sense.
* Rearrange section about context manager to be in order
* Link to `pytest.warns` and `recwarn` since a reader going top to bottom won't have seen about those yet.
* Used only context manager form in the example; the call form is somewhat obsolete
and is mentioned in the reference docs already.
* Reuse the 'myfunction' from first example on the second one
Co-Authored-By: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-Authored-By: Hugo van Kemenade <hugovk@users.noreply.github.com>
This includes our current full matrix (windows, linux and macos), for evaluting
purposes.
We should disconsider failures when evaluating PRs.
TODO:
- deploy
- coverage
- github release notes
Even with the above missing, I still believe it would be nice to merge
this and have GitHub actions working in parallel so we can evaluate performance
and usability from now on.
The order of invocations 'python -m pytest' and 'pytest' are different in the header and the explanation. Me being lazy reading about the behaviour of 'former' looked up quickly the title and rushed to implementation to discover it actually works the other way - as stated in the documentation. So I propose to switch the order in the title to achieve consistent ordering and not confusing somebody like me again! :)
While using pytest-repeat, I noticed the previous implementation is slow
for a large number of duplicate test ids. To optimize, this commit
reduces the amount of data copied and avoids using `in` with List
(unhashable type, and therefore is very slow for many elements).
As far as the output is concerned, they are both identical so it doesn't
make sense to have both.
setup, teardown, and collect failures are already reported as "errors", "E".
Now `tox -e docs` will also include the draft changelog for the
next version (locally only).
`CHANGELOG.rst` now only points to the changelog on READTHEDOCS so
sphinx diretives can be used.
Followup to https://github.com/pytest-dev/pytest/pull/6272
Sets `PY_COLORS=0` in the environment by default, which is used by pylib.
Via https://github.com/blueyed/pytest/pull/58
(initially cherry picked from commit f153ad33d10)
This configures the default role for interpreted text (single
backticks), avoiding the need to check for / enforce double backticks.
Fixes also one instance in the existing changelog:
- Detect `pytest_` prefixed hooks using the internal plugin manager since
``pluggy`` is deprecating the ``implprefix`` argument to ``PluginManager``.
(`#3487 <https://github.com/pytest-dev/pytest/issues/3487>`_)
This feature was added in commit
007a77c2ba, but was never used in pytest
itself. A GitHub code search doesn't find any users either (only pytest
repo copies). It seems safe to clean up.
`tox -vv` is too verbose, and was only used as a hack to get the output
of durations.
As for information in logs `-v` could be used maybe still, but I've
decided to revert it for now.
This reverts commit 56cec5fa79.
Source was previously iterable because it implements `__getitem__()`,
which is apparently a thing from before `__iter__()` was introduced.
To reduce mypy's and my own confusion, implement `__iter__()` directly.
The previous test was better in that it used fakes to test all of the
real code paths. The problem with that is that it makes it impossible to
simplify the code with `isinstance` checks. So let's just simulate the
issue directly with a monkeypatch.
Add some Python 3.8 type: ignores; all are already fixed in the next
mypy release, so can be removed once we upgrade.
Also move some flake8 ignores which seem to have changed places.
Instead of giving an example of using sys and then, at the end,
advising not to use sys, just give a correct example. This is
especially helpful since mypy 0.740 has started (correctly) complaining
about sys._called_from_pytest not being present.
It currently fails with a TypeError, and was not updated since 2013 -
therefore it can be assumed that it is not important to support it.
```
____________________ ERROR collecting test_nested_marks.py _____________________
…/Vcs/pluggy/src/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:337: in traced_hookexec
return outcome.get_result()
…/Vcs/pluggy/src/pluggy/manager.py:335: in <lambda>
outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
…/Vcs/pluggy/src/pluggy/manager.py:87: in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
…/Vcs/pytest/src/_pytest/python.py:235: in pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
…/Vcs/pytest/src/_pytest/python.py:404: in _genfunctions
self.ihook.pytest_generate_tests.call_extra(methods, dict(metafunc=metafunc))
…/Vcs/pluggy/src/pluggy/hooks.py:324: in call_extra
return self(**kwargs)
…/Vcs/pluggy/src/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:337: in traced_hookexec
return outcome.get_result()
…/Vcs/pluggy/src/pluggy/manager.py:335: in <lambda>
outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
…/Vcs/pluggy/src/pluggy/manager.py:87: in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
…/Vcs/pytest/src/_pytest/python.py:130: in pytest_generate_tests
metafunc.parametrize(*marker.args, **marker.kwargs)
…/Vcs/pytest/src/_pytest/python.py:965: in parametrize
function_definition=self.definition,
…/Vcs/pytest/src/_pytest/mark/structures.py:111: in _for_parametrize
if len(param.values) != len(argnames):
E TypeError: object of type 'MarkDecorator' has no len()
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
```
This comes via hypothesis:
```
% COLUMNS=80 p testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis -vv --tb=short
============================= test session starts ==============================
platform linux -- Python 3.7.4, pytest-3.1.4.dev721+g3367bf03b.d20191112, py-1.8.1.dev11+g34f716fe, pluggy-0.13.1.dev8+ga5130ac.d20191103 -- …/Vcs/pytest/.venv/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('…/Vcs/pytest/.hypothesis/examples')
rootdir: …/Vcs/pytest, inifile: tox.ini
plugins: forked-1.1.3, hypothesis-4.44.1, cov-2.8.1, coverage-pytest-plugin-0.1, enhancements-0.0.5.dev1-gf361636-dirty, xdist-1.30.0
collected 1 item
testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis FAILED [100%]
=================================== FAILURES ===================================
______________________ TestMetafunc.test_idval_hypothesis ______________________
.venv/lib/python3.7/site-packages/hypothesis/core.py:588: in evaluate_test_data
result = self.execute(data)
.venv/lib/python3.7/site-packages/hypothesis/core.py:553: in execute
result = self.test_runner(data, run)
.venv/lib/python3.7/site-packages/hypothesis/executors.py:56: in default_new_style_executor
return function(data)
.venv/lib/python3.7/site-packages/hypothesis/core.py:536: in run
args, kwargs = data.draw(self.search_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:857: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/core.py:223: in do_draw
return self.mapped_strategy.do_draw(data)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in do_draw
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in <genexpr>
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:570: in do_draw
result = self.pack(data.draw(self.mapped_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/lazy.py:156: in do_draw
return data.draw(self.wrapped_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:570: in do_draw
result = self.pack(data.draw(self.mapped_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in do_draw
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in <genexpr>
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:508: in do_draw
return data.draw(self.element_strategies[i], label=self.branch_labels[i])
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/lazy.py:156: in do_draw
return data.draw(self.wrapped_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:570: in do_draw
result = self.pack(data.draw(self.mapped_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/lazy.py:156: in do_draw
return data.draw(self.wrapped_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:120: in do_draw
result.append(data.draw(self.element_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/numbers.py:62: in do_draw
return d.integer_range(data, self.start, self.end)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/utils.py:105: in integer_range
probe = data.draw_bits(bits)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:974: in draw_bits
self.__check_capacity(n_bytes)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:1019: in __check_capacity
self.mark_overrun()
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:1036: in mark_overrun
self.conclude_test(Status.OVERRUN)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:1027: in conclude_test
raise StopTest(self.testcounter)
E hypothesis.errors.StopTest: 0
During handling of the above exception, another exception occurred:
testing/python/metafunc.py:195: in test_idval_hypothesis
@hypothesis.settings(
.venv/lib/python3.7/site-packages/nose/__init__.py:1: in <module>
from nose.core import collector, main, run, run_exit, runmodule
.venv/lib/python3.7/site-packages/nose/core.py:12: in <module>
from nose.loader import defaultTestLoader
.venv/lib/python3.7/site-packages/nose/loader.py:21: in <module>
from nose.importer import Importer, add_path, remove_path
.venv/lib/python3.7/site-packages/nose/importer.py:12: in <module>
from imp import find_module, load_module, acquire_model1, release_model1
/usr/lib/python3.7/imp.py:33: in <module>
DeprecationWarning, stacklevel=2)
E DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
---------------------------------- Hypothesis ----------------------------------
You can add @seed(198901559535749756451579900660745168041) to this test or run pytest with --hypothesis-seed=198901559535749756451579900660745168041 to reproduce this failure.
=============================== warnings summary ===============================
testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis
…/Vcs/pytest/.venv/lib/python3.7/site-packages/unittest2/compatibility.py:143: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
class ChainMap(collections.MutableMapping):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================== short test summary info ============================
FAILED testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis - Depr...
```
This is a useful utility to abstract the caching property idiom.
It is in compat.py since eventually it will be replaced by
functools.cached_property.
Fixes#6131.
It is a little too obscure IMO, but the reason I want to drop it is that
type checking has no hope of understanding such dynamic constructs.
The warning argument wasn't used.
This is already covered by attrs.
Also, the custom implementation returns False when the types don't
match, but it's better to return `NotImplemented`. attrs does this.
This fixes/removes the previous hack of re-trying with minimum width,
which fails short when it splits strings.
This inherits from `pprint.PrettyPrinter` to override `_format` in a
minimal way to always dispatch, regardless of the given width.
Code ref: 5c0c325453/Lib/pprint.py (L170-L178)
Harden one test where it is tested.
All tests testing this:
testing/acceptance_test.py:184(TestGeneralUsage::test_not_collectable_arguments)
testing/acceptance_test.py:373(TestGeneralUsage::test_direct_addressing_notfound)
testing/acceptance_test.py:403(TestGeneralUsage::test_issue134_report_error_when_collecting_member[test_fun.py::test_a])
testing/acceptance_test.py:420(TestGeneralUsage::test_report_all_failed_collections_initargs)
testing/test_config.py:1309(test_config_blocked_default_plugins[python])
(via https://github.com/blueyed/pytest/pull/88)
Works around:
_____ ERROR collecting testing/io/test_saferepr.py _____
src/_pytest/python.py:502: in _importtestmodule
mod = self.fspath.pyimport(ensuresyspath=importmode)
.venv38/lib/python3.8/site-packages/py/_path/local.py:701: in pyimport
__import__(modname)
<frozen importlib._bootstrap>:991: in _find_and_load
???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:671: in _load_unlocked
???
src/_pytest/assertion/rewrite.py:136: in exec_module
source_stat, co = _rewrite_test(fn, self.config)
src/_pytest/assertion/rewrite.py:288: in _rewrite_test
co = compile(tree, fn, "exec", dont_inherit=True)
E File "…/Vcs/pytest/testing/io/test_saferepr.py", line 45
E None()
E ^
E SyntaxError: 'NoneType' object is not callable; perhaps you missed a comma?
This causes INTERNALERRORs with pytest-django, which uses
`pytest.fail` (derived from `BaseException`) to prevent DB access, when
pytest then tries to e.g. display the `repr()` for a Django `QuerySet`
etc.
Ref: https://github.com/pytest-dev/pytest-django/pull/776
Follow-up to 946434c61 (#5924).
Before this patch the test would look like this:
{'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
Omitting 1 identical items, use -vv to show
Right contains 1 more item:
{'new': 1}
Full diff:
{
'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
- 'sub1': {'long_a': 'substring '
+ 'sub1': {'long_a': 'substring that gets wrapped'}}},
? +++++++++++++++++ ++++
+ 'new': 1,
- 'that '
- 'gets '
- 'wrapped'}}},
}
indicate current outcome/status with color of percentage indicator
Fix type annotation, refactor _write_progress_information_filling_space
Keep code in _get_main_color as similar as possible to how it was before
Write test
Make black-compliant
Fix error in newly introduced test_collecterror
Make tests more readable by using constants and f-strings
Remove accidentally added monkeypatch
Make Python 3.5-compatible, add changelog entry
Add newline at the end of changelog file
Remove the `--fulltrace` arg from the `Option` fixture used in several
tests, but not checked for. Only use it with `test_keyboard_interrupt`.
(removes 8 tests, coverage not affected)
The function handles bytes input, however that is never used.
The function has two callers:
1)
```
if istext(left) and istext(right):
explanation = _diff_text(left, right, verbose
```
`istext` checks `isinstance(str)`.
2)
```
def _notin_text(term: str, text: str, verbose: int = 0) -> List[str]:
...
diff = _diff_text(correct_text, text, verbose
```
and `_notin_text` is called once:
```
if istext(left) and istext(right):
explanation = _notin_text(left, right, verbose
```
Without this, the second time it tries to stop in a parametrized
function it raises instead:
`ValueError: --trace can't be used with a fixture named func!`
Implementation idea, test (and changelog tweaks) thanks to blueyed
Co-Authored-By: Ronny Pfannschmidt <opensource@ronnypfannschmidt.de>
Co-Authored-By: Daniel Hahler <git@thequod.de>
Failed for me due to different indent (?) - not reproducible:
> ???
E Failed: nomatch: 'E TypeError: write() argument must be str, not bytes'
…
E and: '> def mode(self):'
E and: 'E TypeError: write() argument must be str, not bytes'
…
E remains unmatched: 'E TypeError: write() argument must be str, not bytes'
This would prevent TOXENV=py37-pexpect-… from running on e.g. FreeBSD.
And even on Windows it is pytest's job of skipping the tests then. This
was probably still from when the pexpect env was only running
pexpect-based tests.
Tidelift has launched a new marketing campaign as outlined here:
* https://forum.tidelift.com/t/task-enhancement-marketing-the-tidelift-subscription-to-your-users/321
This PR splits the previous "sponsor" information into two, Open Collective
and Tidelift (as they have very different target audiences).
Also took the opportunity to reorder some items at the end of
the contents page in a manner that I believe make more sense.
In order to make the LiteralOutputChecker lazy initialization more
amenable to type checking, I changed it to match the scheme already used
in this file to lazy-initialize PytestDoctestRunner.
Changelogs:
http://mypy-lang.blogspot.com/2019/09/mypy-730-released.htmlhttp://mypy-lang.blogspot.com/2019/10/mypy-0740-released.html
New errors:
src/_pytest/recwarn.py:77: error: Missing return statement
src/_pytest/recwarn.py:185: error: "bool" is invalid as return type for "__exit__" that always returns False
src/_pytest/recwarn.py:185: note: Use "typing_extensions.Literal[False]" as the return type or change it to "None"
src/_pytest/recwarn.py:185: note: If return type of "__exit__" implies that it may return True, the context manager may swallow exceptions
src/_pytest/recwarn.py:185: error: Return type "bool" of "__exit__" incompatible with return type "None" in supertype "catch_warnings"
src/_pytest/recwarn.py:230: error: "bool" is invalid as return type for "__exit__" that always returns False
src/_pytest/recwarn.py:230: note: Use "typing_extensions.Literal[False]" as the return type or change it to "None"
src/_pytest/recwarn.py:230: note: If return type of "__exit__" implies that it may return True, the context manager may swallow exceptions
src/_pytest/recwarn.py:230: error: Return type "bool" of "__exit__" incompatible with return type "None" in supertype "catch_warnings"
The errors are due to this new error:
https://mypy.readthedocs.io/en/latest/error_code_list.html#check-the-return-type-of-exit-exit-return
This is important when used with ``pytester``'s ``runpytest_inprocess``.
Since 07f20ccab `pytest testing/acceptance_test.py -k test_doctest_id`
would fail, since the second run would not consider the exception to be
an instance of `doctest.DocTestFailure` anymore, since the module was
re-imported, and use another failure message then in the short test
summary info (and in the report itself):
> FAILED test_doctest_id.txt::test_doctest_id.txt - doctest.DocTestFailure: <Do...
while it should be:
> FAILED test_doctest_id.txt::test_doctest_id.txt
This avoids mutating the original list to reflect on InvocationParams,
which is supposed to be an immutable snapshot of the state of pytest.main()
at the moment of invocation (see pytest-dev/pytest-xdist#478).
Massage text input for difflib when comparing pformat output of
different line lengths.
Also do not strip ndiff output on the left, which currently already
removes indenting for lines with no differences.
Before:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E Right contains 3 more items, first extra item: ' '
E Full diff:
E - ['version', 'version_info', 'sys.version', 'sys.version_info']
E + ['version',
E + 'version_info',
E + 'sys.version',
E + 'sys.version_info',
E + ' ',
E + 'sys.version',
E + 'sys.version_info']
After:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E Right contains 3 more items, first extra item: ' '
E Full diff:
E [
E 'version',
E 'version_info',
E 'sys.version',
E 'sys.version_info',
E + ' ',
E + 'sys.version',
E + 'sys.version_info',
E ]
The current idiom is to use:
assert re.match(pat, result.stdout.str())
Or
assert line in result.stdout.str()
But this does not really give good results when it fails.
Those new functions produce similar output to ther other match lines functions.
attrs 19.2 deprecated cmp in favor of the dataclass-ish eq/order duo.
This causes deprecation warnings that in turn break some of the cool new deep
object comparisons. Since we at attrs expected this to be a problem, it shipped
with helpers to write backward and forward compatible code.
This PR uses that and avoids changed to minimal versions.
Recently sometimes Azure has failed with:
++ curl -s https://codecov.io/bash
bash: /dev/fd/63: No such file or directory
This attempts to fix this by modifying report-coverage.sh slightly.
As discussed in the mailing list, pytest will join the
digital strike for the climate on Sep 20th. This will show
a closable banner on the docs until the date, and when the date
comes the banner will become a (closable) full page.
I will also pin an issue saying that the developers won't be
available on Sep 20th.
Using Path().resolve() is better than py.path.realpath because
it resolves to the correct path/drive in case-insensitive file systems (#5792):
>>> from py.path import local
>>> from pathlib import Path
>>>
>>> local('d:\\projects').realpath()
local('d:\\projects')
>>> Path('d:\\projects').resolve()
WindowsPath('D:/projects')
Fix#5819
We find that the --pastebin option to pytest sometimes fails with "HTTP
Error 400: Bad Request". We're still investigating the exact cause of
these errors, but in the meantime, a failure to upload to the pastebin
service should probably not crash pytest and cause a test failure in the
continuous-integration.
This patch catches exceptions like HTTPError that may be thrown while
trying to communicate with the pastebin service, and reports them as a
"bad response", without crashing with a backtrace or failing the entire
test suite.
`argparse.HelpFormatter` looks at `$COLUMNS` only, falling back to a
default of 80.
`py.io.get_terminal_width()` is smarter there, and could even work
better with https://github.com/pytest-dev/py/pull/219.
This ensures to use a consistent value for formatting the ini values etc.
Map `BdbQuit` exception to `outcomes.Exit`.
This is necessary since we are not wrapping `pdb.set_trace` there, and
therefore our `do_quit` is not called.
2019-07-19 02:57:25 +02:00
358 changed files with 32176 additions and 18677 deletions
Thanks for submitting a PR, your contribution is really appreciated!
Here is a quick checklist that should be present in PRs.
(please delete this text from the final description, this is just a guideline)
-->
- [ ] Target the `master` branch for bug fixes, documentation updates and trivial changes.
- [ ] Target the `features` branch for new features, improvements, and removals/deprecations.
- [ ] Include documentation when adding new features.
- [ ] Include new tests or update existing tests when applicable.
- [X] Allow maintainers to push and squash when merging my commits. Please uncheck this if you prefer to squash the commits yourself.
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
If this change fixes an issue, please:
- [ ] Add text like ``closes #XYZW`` to the PR description and/or commits (where ``XYZW`` is the issue number). See the [github docs](https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) for more information.
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/master/changelog/README.rst) for details.
- [ ] Add yourself to `AUTHORS` in alphabetical order;
Write sentences in the **past or present tense**, examples:
* *Improved verbose diff output with sequences.*
* *Terminal summary statistics now use multiple colors.*
Also make sure to end the sentence with a `.`.
- [ ] Add yourself to `AUTHORS` in alphabetical order.
or ``trivial``. You may skip creating the changelog entry if the change doesn't affect the
documented behaviour of pytest.
#. Add yourself to ``AUTHORS`` file if not there yet, in alphabetical order.
@@ -280,14 +288,13 @@ Here is a simple overview, with pytest-specific bits:
compare: your-branch-name
base-fork: pytest-dev/pytest
base: master # if it's a bugfix
base: features # if it's a feature
base: master
Writing Tests
----------------------------
~~~~~~~~~~~~~
Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/latest/reference.html#testdir>`_, as a "black-box" test.
Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/stable/reference.html#testdir>`_, as a "black-box" test.
For example, to ensure a simple test passes you can write:
@@ -324,16 +331,100 @@ one file which looks like a good fit. For example, a regression test about a bug
should go into ``test_cacheprovider.py``, given that this option is implemented in ``cacheprovider.py``.
If in doubt, go ahead and open a PR with your best guess and we can discuss this over the code.
Joining the Development Team
----------------------------
Anyone who has successfully seen through a pull request which did not
require any extra work from the development team to merge will
themselves gain commit access if they so wish (if we forget to ask please send a friendly
reminder). This does not mean your workflow to contribute changes,
reminder). This does not mean there is any change in your contribution workflow:
everyone goes through the same pull-request-and-review process and
no-one merges their own pull requests unless already approved. It does however mean you can
participate in the development process more fully since you can merge
pull requests from other contributors yourself after having reviewed
them.
Backporting bug fixes for the next patch release
------------------------------------------------
Pytest makes feature release every few weeks or months. In between, patch releases
are made to the previous feature release, containing bug fixes only. The bug fixes
usually fix regressions, but may be any change that should reach users before the
next feature release.
Suppose for example that the latest release was 1.2.3, and you want to include
a bug fix in 1.2.4 (check https://github.com/pytest-dev/pytest/releases for the
actual latest release). The procedure for this is:
#. First, make sure the bug is fixed the ``master`` branch, with a regular pull
request, as described above. An exception to this is if the bug fix is not
applicable to ``master`` anymore.
#.``git checkout origin/1.2.x -b backport-XXXX`` # use the master PR number here
#. Locate the merge commit on the PR, in the *merged* message, for example:
nicoddemus merged commit 0f8b462 into pytest-dev:master
#.``git cherry-pick -x -m1 REVISION`` # use the revision you found above (``0f8b462``).
#. Open a PR targeting ``1.2.x``:
* Prefix the message with ``[1.2.x]``.
* Delete the PR body, it usually contains a duplicate commit message.
Handling stale issues/PRs
-------------------------
Stale issues/PRs are those where pytest contributors have asked for questions/changes
and the authors didn't get around to answer/implement them yet after a somewhat long time, or
the discussion simply died because people seemed to lose interest.
There are many reasons why people don't answer questions or implement requested changes:
they might get busy, lose interest, or just forget about it,
but the fact is that this is very common in open source software.
The pytest team really appreciates every issue and pull request, but being a high-volume project
with many issues and pull requests being submitted daily, we try to reduce the number of stale
issues and PRs by regularly closing them. When an issue/pull request is closed in this manner,
it is by no means a dismissal of the topic being tackled by the issue/pull request, but it
is just a way for us to clear up the queue and make the maintainers' work more manageable. Submitters
can always reopen the issue/pull request in their own time later if it makes sense.
When to close
~~~~~~~~~~~~~
Here are a few general rules the maintainers use to decide when to close issues/PRs because
of lack of inactivity:
* Issues labeled ``question`` or ``needs information``: closed after 14 days inactive.
* Issues labeled ``proposal``: closed after six months inactive.
* Pull requests: after one month, consider pinging the author, update linked issue, or consider closing. For pull requests which are nearly finished, the team should consider finishing it up and merging it.
The above are **not hard rules**, but merely **guidelines**, and can be (and often are!) reviewed on a case-by-case basis.
Closing pull requests
~~~~~~~~~~~~~~~~~~~~~
When closing a Pull Request, it needs to be acknowledge the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
Hi <contributor>,
First of all we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
We noticed it has been awhile since you have updated this PR, however. pytest is a high activity project, with many issues/PRs being opened daily, so it is hard for us maintainers to track which PRs are ready for merging, for review, or need more attention.
So for those reasons we think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
Again we appreciate your time for working on this, and hope you might get back to this at a later time!
<bye>
Closing Issues
--------------
When a pull request is submitted to fix an issue, add text like ``closes #XYZW`` to the PR description and/or commits (where ``XYZW`` is the issue number). See the `GitHub docs <https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword>`_ for more information.
When an issue is due to user error (e.g. misunderstanding of a functionality), please politely explain to the user why the issue raised is really a non-issue and ask them to close the issue if they have no further questions. If the original requestor is unresponsive, the issue will be handled as described in the section `Handling stale issues/PRs`_ above.
The ``pytest`` framework makes it easy to write small tests, yet
scales to support complex functional testing for applications and libraries.
@@ -67,33 +71,33 @@ To execute it::
========================== 1 failed in 0.04 seconds ===========================
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started <https://docs.pytest.org/en/latest/getting-started.html#our-first-test-run>`_ for more examples.
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started <https://docs.pytest.org/en/stable/getting-started.html#our-first-test-run>`_ for more examples.
Features
--------
- Detailed info on failing `assert statements <https://docs.pytest.org/en/latest/assert.html>`_ (no need to remember ``self.assert*`` names);
- Detailed info on failing `assert statements <https://docs.pytest.org/en/stable/assert.html>`_ (no need to remember ``self.assert*`` names);
This directory contains "newsfragments" which are short files that contain a small **ReST**-formatted
text that will be added to the next ``CHANGELOG``.
The ``CHANGELOG`` will be read by users, so this description should be aimed to pytest users
The ``CHANGELOG`` will be read by **users**, so this description should be aimed to pytest users
instead of describing internal changes which are only relevant to the developers.
Make sure to use full sentences with correct case and punctuation, for example::
Make sure to use full sentences in the **past or present tense** and use punctuation, examples::
Fix issue with non-ascii messages from the ``warnings`` module.
Improved verbose diff output with sequences.
Terminal summary statistics now use multiple colors.
Each file should be named like ``<ISSUE>.<TYPE>.rst``, where
``<ISSUE>`` is an issue number, and ``<TYPE>`` is one of:
*``feature``: new user facing features, like new command-line options and new behavior.
*``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junitxml``, improved colors in terminal, etc).
*``bugfix``: fixes a reported bug.
*``bugfix``: fixes a bug.
*``doc``: documentation improvement, like rewording an entire session or adding missing docs.
*``deprecation``: feature deprecation.
*``removal``: feature removal.
*``breaking``: a change which may break existing suites, such as feature removal or behavior change.
*``vendor``: changes in packages vendored in pytest.
*``trivial``: fixing a small typo or internal change that might be noteworthy.
@@ -29,6 +31,7 @@ changelog using that instead.
If you are not sure what issue type to use, don't hesitate to ask in your PR.
``towncrier`` preserves multiple paragraphs and formatting (code blocks, lists, and so on), but for entries
other than ``features`` it is usually better to stick to a single paragraph to keep it concise. You can install
``towncrier`` and then run ``towncrier --draft``
if you want to get a preview of how your change will look in the final release notes.
other than ``features`` it is usually better to stick to a single paragraph to keep it concise.
You can also run ``tox -e docs`` to build the documentation
with the draft changelog (``doc/en/_build/html/changelog.html``) if you want to get a preview of how your change will look in the final release notes.
@@ -7,7 +7,7 @@ This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all who contributed to this release, among them:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
