This reverts commit 930a158a6a.
Regression test from Bruno Oliveira.
(cherry picked from commit fb99b5c66e)
Co-Authored-By: Bruno Oliveira <bruno@esss.co>
These are more "dirty" than the previous batch (that's why they were
left out). The trouble is that `compile` can return either a code object
or an AST depending on a flag, so we need to add an overload to make the
common case Union free. But it's still worthwhile.
Somehow in Python 3.5 on Windows this test fails with:
File "c:\hostedtoolcache\windows\python\3.5.4\x64\Lib\multiprocessing\connection.py", line 302, in _recv_bytes
overlapped=True)
OSError: [WinError 6] The handle is invalid
This only happens in this platform and Python version, decided to use
a dummy traceback as originally done in #6412.
(cherry picked from commit b9c136b809)
This allows for e.g. Jedi to infer types (it checks the name).
It was only used to support Python 3.5.0/3.5.1, where this is is not
available in the `typing` module.
Ref: https://github.com/davidhalter/jedi/issues/1472
Uses `TYPE_CHECKING = False` in `_pytest.outcomes` to avoid having to
work around circular import.
This is different from what pre-commit (in "linting") runs in that it
uses stubs from (test) dependencies.
It would make sense to run this on CI additionally (since there is no
"pre-commit --skip mypy", and a separate config is not worth it).
But currently it triggers a false positive though anyway
(https://github.com/erikrose/more-itertools/pull/374).
This overwrites the `codecov.yml` file in the root of the repository with
`codecov-upstream.yml` file (which contains the code-cov token)´, so PRs
and branches on the repository can upload coverage.
Suggestion from here:
https://github.com/pytest-dev/pytest/pull/6421#issuecomment-571934112
Security concerns: the token might be misused, but only to upload bogus coverage
to `codecov.io`, so the team believe this is harmless. If we decide to fallback
from this decision , we just need to revoke the token.
Related to #6369
Both references were referencing links from Python because of our intersphinx
mapping in `conf.py`:
intersphinx_mapping = {"python": ("https://docs.python.org/3", None)}
Because Python's docs explicitly define both references, Sphinx fallbacks to
them instead of generating implicit references as was expected.
Fix#6397
Tracebacks coming from remote processes crated by the multiprocess module
will contain "RemoteTracebacks" which don't have a 'reprcrash' attribute
Fix#5971
I've updated the text and incorporated the topics from #5275, so this
can now be part of the official docs, and #5275 can be closed/unpinned.
Closes#5275
* Rearrange section about context manager to be in order
* Link to `pytest.warns` and `recwarn` since a reader going top to bottom won't have seen about those yet.
* Used only context manager form in the example; the call form is somewhat obsolete
and is mentioned in the reference docs already.
* Reuse the 'myfunction' from first example on the second one
Co-Authored-By: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-Authored-By: Hugo van Kemenade <hugovk@users.noreply.github.com>
This includes our current full matrix (windows, linux and macos), for evaluting
purposes.
We should disconsider failures when evaluating PRs.
TODO:
- deploy
- coverage
- github release notes
Even with the above missing, I still believe it would be nice to merge
this and have GitHub actions working in parallel so we can evaluate performance
and usability from now on.
While using pytest-repeat, I noticed the previous implementation is slow
for a large number of duplicate test ids. To optimize, this commit
reduces the amount of data copied and avoids using `in` with List
(unhashable type, and therefore is very slow for many elements).
Now `tox -e docs` will also include the draft changelog for the
next version (locally only).
`CHANGELOG.rst` now only points to the changelog on READTHEDOCS so
sphinx diretives can be used.
Followup to https://github.com/pytest-dev/pytest/pull/6272
This configures the default role for interpreted text (single
backticks), avoiding the need to check for / enforce double backticks.
Fixes also one instance in the existing changelog:
- Detect `pytest_` prefixed hooks using the internal plugin manager since
``pluggy`` is deprecating the ``implprefix`` argument to ``PluginManager``.
(`#3487 <https://github.com/pytest-dev/pytest/issues/3487>`_)
`tox -vv` is too verbose, and was only used as a hack to get the output
of durations.
As for information in logs `-v` could be used maybe still, but I've
decided to revert it for now.
This reverts commit 56cec5fa79.
Source was previously iterable because it implements `__getitem__()`,
which is apparently a thing from before `__iter__()` was introduced.
To reduce mypy's and my own confusion, implement `__iter__()` directly.
The previous test was better in that it used fakes to test all of the
real code paths. The problem with that is that it makes it impossible to
simplify the code with `isinstance` checks. So let's just simulate the
issue directly with a monkeypatch.
Add some Python 3.8 type: ignores; all are already fixed in the next
mypy release, so can be removed once we upgrade.
Also move some flake8 ignores which seem to have changed places.
Instead of giving an example of using sys and then, at the end,
advising not to use sys, just give a correct example. This is
especially helpful since mypy 0.740 has started (correctly) complaining
about sys._called_from_pytest not being present.
It currently fails with a TypeError, and was not updated since 2013 -
therefore it can be assumed that it is not important to support it.
```
____________________ ERROR collecting test_nested_marks.py _____________________
…/Vcs/pluggy/src/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:337: in traced_hookexec
return outcome.get_result()
…/Vcs/pluggy/src/pluggy/manager.py:335: in <lambda>
outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
…/Vcs/pluggy/src/pluggy/manager.py:87: in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
…/Vcs/pytest/src/_pytest/python.py:235: in pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
…/Vcs/pytest/src/_pytest/python.py:404: in _genfunctions
self.ihook.pytest_generate_tests.call_extra(methods, dict(metafunc=metafunc))
…/Vcs/pluggy/src/pluggy/hooks.py:324: in call_extra
return self(**kwargs)
…/Vcs/pluggy/src/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:337: in traced_hookexec
return outcome.get_result()
…/Vcs/pluggy/src/pluggy/manager.py:335: in <lambda>
outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
…/Vcs/pluggy/src/pluggy/manager.py:87: in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
…/Vcs/pytest/src/_pytest/python.py:130: in pytest_generate_tests
metafunc.parametrize(*marker.args, **marker.kwargs)
…/Vcs/pytest/src/_pytest/python.py:965: in parametrize
function_definition=self.definition,
…/Vcs/pytest/src/_pytest/mark/structures.py:111: in _for_parametrize
if len(param.values) != len(argnames):
E TypeError: object of type 'MarkDecorator' has no len()
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
```
This comes via hypothesis:
```
% COLUMNS=80 p testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis -vv --tb=short
============================= test session starts ==============================
platform linux -- Python 3.7.4, pytest-3.1.4.dev721+g3367bf03b.d20191112, py-1.8.1.dev11+g34f716fe, pluggy-0.13.1.dev8+ga5130ac.d20191103 -- …/Vcs/pytest/.venv/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('…/Vcs/pytest/.hypothesis/examples')
rootdir: …/Vcs/pytest, inifile: tox.ini
plugins: forked-1.1.3, hypothesis-4.44.1, cov-2.8.1, coverage-pytest-plugin-0.1, enhancements-0.0.5.dev1-gf361636-dirty, xdist-1.30.0
collected 1 item
testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis FAILED [100%]
=================================== FAILURES ===================================
______________________ TestMetafunc.test_idval_hypothesis ______________________
.venv/lib/python3.7/site-packages/hypothesis/core.py:588: in evaluate_test_data
result = self.execute(data)
.venv/lib/python3.7/site-packages/hypothesis/core.py:553: in execute
result = self.test_runner(data, run)
.venv/lib/python3.7/site-packages/hypothesis/executors.py:56: in default_new_style_executor
return function(data)
.venv/lib/python3.7/site-packages/hypothesis/core.py:536: in run
args, kwargs = data.draw(self.search_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:857: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/core.py:223: in do_draw
return self.mapped_strategy.do_draw(data)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in do_draw
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in <genexpr>
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:570: in do_draw
result = self.pack(data.draw(self.mapped_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/lazy.py:156: in do_draw
return data.draw(self.wrapped_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:570: in do_draw
result = self.pack(data.draw(self.mapped_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in do_draw
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:60: in <genexpr>
return tuple(data.draw(e) for e in self.element_strategies)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:508: in do_draw
return data.draw(self.element_strategies[i], label=self.branch_labels[i])
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/lazy.py:156: in do_draw
return data.draw(self.wrapped_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/strategies.py:570: in do_draw
result = self.pack(data.draw(self.mapped_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/lazy.py:156: in do_draw
return data.draw(self.wrapped_strategy)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/collections.py:120: in do_draw
result.append(data.draw(self.element_strategy))
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:852: in draw
return strategy.do_draw(self)
.venv/lib/python3.7/site-packages/hypothesis/searchstrategy/numbers.py:62: in do_draw
return d.integer_range(data, self.start, self.end)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/utils.py:105: in integer_range
probe = data.draw_bits(bits)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:974: in draw_bits
self.__check_capacity(n_bytes)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:1019: in __check_capacity
self.mark_overrun()
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:1036: in mark_overrun
self.conclude_test(Status.OVERRUN)
.venv/lib/python3.7/site-packages/hypothesis/internal/conjecture/data.py:1027: in conclude_test
raise StopTest(self.testcounter)
E hypothesis.errors.StopTest: 0
During handling of the above exception, another exception occurred:
testing/python/metafunc.py:195: in test_idval_hypothesis
@hypothesis.settings(
.venv/lib/python3.7/site-packages/nose/__init__.py:1: in <module>
from nose.core import collector, main, run, run_exit, runmodule
.venv/lib/python3.7/site-packages/nose/core.py:12: in <module>
from nose.loader import defaultTestLoader
.venv/lib/python3.7/site-packages/nose/loader.py:21: in <module>
from nose.importer import Importer, add_path, remove_path
.venv/lib/python3.7/site-packages/nose/importer.py:12: in <module>
from imp import find_module, load_module, acquire_model1, release_model1
/usr/lib/python3.7/imp.py:33: in <module>
DeprecationWarning, stacklevel=2)
E DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
---------------------------------- Hypothesis ----------------------------------
You can add @seed(198901559535749756451579900660745168041) to this test or run pytest with --hypothesis-seed=198901559535749756451579900660745168041 to reproduce this failure.
=============================== warnings summary ===============================
testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis
…/Vcs/pytest/.venv/lib/python3.7/site-packages/unittest2/compatibility.py:143: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
class ChainMap(collections.MutableMapping):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================== short test summary info ============================
FAILED testing/python/metafunc.py::TestMetafunc::test_idval_hypothesis - Depr...
```
This is a useful utility to abstract the caching property idiom.
It is in compat.py since eventually it will be replaced by
functools.cached_property.
Fixes#6131.
It is a little too obscure IMO, but the reason I want to drop it is that
type checking has no hope of understanding such dynamic constructs.
The warning argument wasn't used.
This is already covered by attrs.
Also, the custom implementation returns False when the types don't
match, but it's better to return `NotImplemented`. attrs does this.
This fixes/removes the previous hack of re-trying with minimum width,
which fails short when it splits strings.
This inherits from `pprint.PrettyPrinter` to override `_format` in a
minimal way to always dispatch, regardless of the given width.
Code ref: 5c0c325453/Lib/pprint.py (L170-L178)
Harden one test where it is tested.
All tests testing this:
testing/acceptance_test.py:184(TestGeneralUsage::test_not_collectable_arguments)
testing/acceptance_test.py:373(TestGeneralUsage::test_direct_addressing_notfound)
testing/acceptance_test.py:403(TestGeneralUsage::test_issue134_report_error_when_collecting_member[test_fun.py::test_a])
testing/acceptance_test.py:420(TestGeneralUsage::test_report_all_failed_collections_initargs)
testing/test_config.py:1309(test_config_blocked_default_plugins[python])
(via https://github.com/blueyed/pytest/pull/88)
Works around:
_____ ERROR collecting testing/io/test_saferepr.py _____
src/_pytest/python.py:502: in _importtestmodule
mod = self.fspath.pyimport(ensuresyspath=importmode)
.venv38/lib/python3.8/site-packages/py/_path/local.py:701: in pyimport
__import__(modname)
<frozen importlib._bootstrap>:991: in _find_and_load
???
<frozen importlib._bootstrap>:975: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:671: in _load_unlocked
???
src/_pytest/assertion/rewrite.py:136: in exec_module
source_stat, co = _rewrite_test(fn, self.config)
src/_pytest/assertion/rewrite.py:288: in _rewrite_test
co = compile(tree, fn, "exec", dont_inherit=True)
E File "…/Vcs/pytest/testing/io/test_saferepr.py", line 45
E None()
E ^
E SyntaxError: 'NoneType' object is not callable; perhaps you missed a comma?
This causes INTERNALERRORs with pytest-django, which uses
`pytest.fail` (derived from `BaseException`) to prevent DB access, when
pytest then tries to e.g. display the `repr()` for a Django `QuerySet`
etc.
Ref: https://github.com/pytest-dev/pytest-django/pull/776
Follow-up to 946434c61 (#5924).
Before this patch the test would look like this:
{'env': {'sub...s wrapped'}}}} == {'env': {'sub...}}}, 'new': 1}
Omitting 1 identical items, use -vv to show
Right contains 1 more item:
{'new': 1}
Full diff:
{
'env': {'sub': {'long_a': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
- 'sub1': {'long_a': 'substring '
+ 'sub1': {'long_a': 'substring that gets wrapped'}}},
? +++++++++++++++++ ++++
+ 'new': 1,
- 'that '
- 'gets '
- 'wrapped'}}},
}
indicate current outcome/status with color of percentage indicator
Fix type annotation, refactor _write_progress_information_filling_space
Keep code in _get_main_color as similar as possible to how it was before
Write test
Make black-compliant
Fix error in newly introduced test_collecterror
Make tests more readable by using constants and f-strings
Remove accidentally added monkeypatch
Make Python 3.5-compatible, add changelog entry
Add newline at the end of changelog file
Remove the `--fulltrace` arg from the `Option` fixture used in several
tests, but not checked for. Only use it with `test_keyboard_interrupt`.
(removes 8 tests, coverage not affected)
The function handles bytes input, however that is never used.
The function has two callers:
1)
```
if istext(left) and istext(right):
explanation = _diff_text(left, right, verbose
```
`istext` checks `isinstance(str)`.
2)
```
def _notin_text(term: str, text: str, verbose: int = 0) -> List[str]:
...
diff = _diff_text(correct_text, text, verbose
```
and `_notin_text` is called once:
```
if istext(left) and istext(right):
explanation = _notin_text(left, right, verbose
```
Without this, the second time it tries to stop in a parametrized
function it raises instead:
`ValueError: --trace can't be used with a fixture named func!`
Implementation idea, test (and changelog tweaks) thanks to blueyed
Co-Authored-By: Ronny Pfannschmidt <opensource@ronnypfannschmidt.de>
Co-Authored-By: Daniel Hahler <git@thequod.de>
Failed for me due to different indent (?) - not reproducible:
> ???
E Failed: nomatch: 'E TypeError: write() argument must be str, not bytes'
…
E and: '> def mode(self):'
E and: 'E TypeError: write() argument must be str, not bytes'
…
E remains unmatched: 'E TypeError: write() argument must be str, not bytes'
This would prevent TOXENV=py37-pexpect-… from running on e.g. FreeBSD.
And even on Windows it is pytest's job of skipping the tests then. This
was probably still from when the pexpect env was only running
pexpect-based tests.
Tidelift has launched a new marketing campaign as outlined here:
* https://forum.tidelift.com/t/task-enhancement-marketing-the-tidelift-subscription-to-your-users/321
This PR splits the previous "sponsor" information into two, Open Collective
and Tidelift (as they have very different target audiences).
Also took the opportunity to reorder some items at the end of
the contents page in a manner that I believe make more sense.
In order to make the LiteralOutputChecker lazy initialization more
amenable to type checking, I changed it to match the scheme already used
in this file to lazy-initialize PytestDoctestRunner.
Changelogs:
http://mypy-lang.blogspot.com/2019/09/mypy-730-released.htmlhttp://mypy-lang.blogspot.com/2019/10/mypy-0740-released.html
New errors:
src/_pytest/recwarn.py:77: error: Missing return statement
src/_pytest/recwarn.py:185: error: "bool" is invalid as return type for "__exit__" that always returns False
src/_pytest/recwarn.py:185: note: Use "typing_extensions.Literal[False]" as the return type or change it to "None"
src/_pytest/recwarn.py:185: note: If return type of "__exit__" implies that it may return True, the context manager may swallow exceptions
src/_pytest/recwarn.py:185: error: Return type "bool" of "__exit__" incompatible with return type "None" in supertype "catch_warnings"
src/_pytest/recwarn.py:230: error: "bool" is invalid as return type for "__exit__" that always returns False
src/_pytest/recwarn.py:230: note: Use "typing_extensions.Literal[False]" as the return type or change it to "None"
src/_pytest/recwarn.py:230: note: If return type of "__exit__" implies that it may return True, the context manager may swallow exceptions
src/_pytest/recwarn.py:230: error: Return type "bool" of "__exit__" incompatible with return type "None" in supertype "catch_warnings"
The errors are due to this new error:
https://mypy.readthedocs.io/en/latest/error_code_list.html#check-the-return-type-of-exit-exit-return
This is important when used with ``pytester``'s ``runpytest_inprocess``.
Since 07f20ccab `pytest testing/acceptance_test.py -k test_doctest_id`
would fail, since the second run would not consider the exception to be
an instance of `doctest.DocTestFailure` anymore, since the module was
re-imported, and use another failure message then in the short test
summary info (and in the report itself):
> FAILED test_doctest_id.txt::test_doctest_id.txt - doctest.DocTestFailure: <Do...
while it should be:
> FAILED test_doctest_id.txt::test_doctest_id.txt
This avoids mutating the original list to reflect on InvocationParams,
which is supposed to be an immutable snapshot of the state of pytest.main()
at the moment of invocation (see pytest-dev/pytest-xdist#478).
Massage text input for difflib when comparing pformat output of
different line lengths.
Also do not strip ndiff output on the left, which currently already
removes indenting for lines with no differences.
Before:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E Right contains 3 more items, first extra item: ' '
E Full diff:
E - ['version', 'version_info', 'sys.version', 'sys.version_info']
E + ['version',
E + 'version_info',
E + 'sys.version',
E + 'sys.version_info',
E + ' ',
E + 'sys.version',
E + 'sys.version_info']
After:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...]
E Right contains 3 more items, first extra item: ' '
E Full diff:
E [
E 'version',
E 'version_info',
E 'sys.version',
E 'sys.version_info',
E + ' ',
E + 'sys.version',
E + 'sys.version_info',
E ]
The current idiom is to use:
assert re.match(pat, result.stdout.str())
Or
assert line in result.stdout.str()
But this does not really give good results when it fails.
Those new functions produce similar output to ther other match lines functions.
attrs 19.2 deprecated cmp in favor of the dataclass-ish eq/order duo.
This causes deprecation warnings that in turn break some of the cool new deep
object comparisons. Since we at attrs expected this to be a problem, it shipped
with helpers to write backward and forward compatible code.
This PR uses that and avoids changed to minimal versions.
Recently sometimes Azure has failed with:
++ curl -s https://codecov.io/bash
bash: /dev/fd/63: No such file or directory
This attempts to fix this by modifying report-coverage.sh slightly.
As discussed in the mailing list, pytest will join the
digital strike for the climate on Sep 20th. This will show
a closable banner on the docs until the date, and when the date
comes the banner will become a (closable) full page.
I will also pin an issue saying that the developers won't be
available on Sep 20th.
Using Path().resolve() is better than py.path.realpath because
it resolves to the correct path/drive in case-insensitive file systems (#5792):
>>> from py.path import local
>>> from pathlib import Path
>>>
>>> local('d:\\projects').realpath()
local('d:\\projects')
>>> Path('d:\\projects').resolve()
WindowsPath('D:/projects')
Fix#5819
We find that the --pastebin option to pytest sometimes fails with "HTTP
Error 400: Bad Request". We're still investigating the exact cause of
these errors, but in the meantime, a failure to upload to the pastebin
service should probably not crash pytest and cause a test failure in the
continuous-integration.
This patch catches exceptions like HTTPError that may be thrown while
trying to communicate with the pastebin service, and reports them as a
"bad response", without crashing with a backtrace or failing the entire
test suite.
`argparse.HelpFormatter` looks at `$COLUMNS` only, falling back to a
default of 80.
`py.io.get_terminal_width()` is smarter there, and could even work
better with https://github.com/pytest-dev/py/pull/219.
This ensures to use a consistent value for formatting the ini values etc.
The typing module on these versions have these issues:
- `typing.Pattern` cannot appear in a Union since it is not considered a
class.
- `@overload` is not supported in runtime. (On the other hand, mypy
doesn't support putting it under `if False`, so we need some runtime
hack).
Refs #5751.
The only benefit for me is to get notified about finished builds, but
that might happen to early anyway. Apart from that they are rather big
and distract from actual comments.
I was doing final tests on the script today, and forgot to change
the hardecoded "pytest-dev/pytest", which ended up publishing a
`4.99.10` release to the main repository by mistake, as my token
has access to both my fork and main repository.
I deleted the tag immeditely just a few seconds later, so hopefully
this won't cause major problems.
This change makes it safer to test this in the future, never publishing
to the main repository by mistake (as long as the tags are pushed to the
right repositories of course).
Previously it would say:
> assert '123456789012...901234567890A' == '1234567890123...901234567890B'"
This makes it look like the "3" might be different already.
This is clearer, and it is OK to have potentially one less char in the
right one:
> assert '123456789012...901234567890A' == '123456789012...901234567890B'"
This creates a separate section from 'features' for small changes which
don't usually require user intervention, such as:
* Human readable session duration
* New junitxml fields
* Improved colors in terminal
* etc.
The idea is to better match user expectations about new actual
features in the "Features" section of the changelog.
Previously, the test failed when the directory was not present,
which could have been caused for example by invoking the tests
with PYTHONDONTWRITEBYTECODE=1.
Fixes https://github.com/pytest-dev/pytest/issues/5664
Map `BdbQuit` exception to `outcomes.Exit`.
This is necessary since we are not wrapping `pdb.set_trace` there, and
therefore our `do_quit` is not called.
It seems to have been added in #1439 to fix#1178.
This was only relevant for Python 2 where it was tempting to use str (==
bytes) literals instead of unicode literals. In Python 3, it is unlikely
that anyone passes bytes to these functions.
This makes testing/ actually pick up the pytest imports -- otherwise
they are opaque and we don't actually test the types.
A single run is also a bit faster and simpler. The original reason why
we split it is no longer relevant (we fixed the problems).
This function is not called anywhere directly, and cannot be called by
the dynamic `repr_<type>()` dispatch mechanism because unicode is no
longer a type in Python 3.
This reverts commit dfe54cd82f.
The idea in the commit was to simplify the code by removing the check
and instead letting it TypeError which has the same effect.
However this type error is caught by mypy, and rather than ignoring the
error we think it's better and clearer to go back to the previous
explicit check.
A case with a fixture use both as an autouse and explititly
was raised. This case sounds too narrow to add to documentation
(and could be misleading for people learning pytest with explicitely
using an autouse fixture). At the same time there was no documentation
on the autouse vs regular fixture order, therefore this commit adds
such an information. Code sample grew and it was extracted to the file.
These doctests were expected to fail, but they were failing because of a
silly bug (I forgot to replace "{expression}" with the actual expression
to be tested), not because of the thing they were meant to be testing.
Then I had to fix one of the testcases because it was actually matching:
>>> 3.0 #doctest: +NUMBER
2.99
The doctest is saying that the actual output should match to 2 decimal
places, i.e. within 0.01 -- which it is, so it passes. I changed the
expected output to 2.98 and now it doesn't match (as we expect).
Add a very lax mypy configuration, add it to tox -e linting, and
fix/ignore the few errors that come up. The idea is to get it running
before diving in too much.
This enables:
- Progressively adding type annotations and enabling more strict
options, which will improve the codebase (IMO).
- Annotating the public API in-line, and eventually exposing it to
library users who use type checkers (with a py.typed file).
Though, none of this is done yet.
Refs https://github.com/pytest-dev/pytest/issues/3342.
Add a very lax mypy configuration, add it to tox -e linting, and
fix/ignore the few errors that come up. The idea is to get it running
before diving in too much.
This enables:
- Progressively adding type annotations and enabling more strict
options, which will improve the codebase (IMO).
- Annotating the public API in-line, and eventually exposing it to
library users who use type checkers (with a py.typed file).
Though, none of this is done yet.
Refs https://github.com/pytest-dev/pytest/issues/3342.
This is being raised by `unittest2.compatibility`:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
* Move the parts about "how to configure it" (pytest.ini vs. inline
comment) together.
* Move `--doctest-continue-on-failure` into its own sub-heading, as it
isn't related to the doctest optionflags.
When enabled, floating-point numbers only need to match as far as the
precision you have written in the expected doctest output. This avoids
false positives caused by limited floating-point precision, like this:
Expected:
0.233
Got:
0.23300000000000001
This is inspired by Sébastien Boisgérault's [numtest] but the
implementation is a bit different:
* This implementation edits the literals that are in the "got"
string (the actual output from the expression being tested), and then
proceeds to compare the strings literally. This is similar to pytest's
existing ALLOW_UNICODE and ALLOW_BYTES implementation.
* This implementation only compares floats against floats, not ints
against floats. That is, the following doctest will fail with pytest
whereas it would pass with numtest:
>>> math.py # doctest: +NUMBER
3
This behaviour should be less surprising (less false negatives) when
you enable NUMBER globally in pytest.ini.
Advantages of this implementation compared to numtest:
* Doesn't require `import numtest` at the top level of the file.
* Works with pytest (if you try to use pytest & numtest together, pytest
raises "TypeError: unbound method check_output() must be called with
NumTestOutputChecker instance as first argument (got
LiteralsOutputChecker instance instead)").
* Works with Python 3.
[numtest]: https://github.com/boisgera/numtest
These attributes can be used to access the unchanged arguments passed
to pytest.main().
The intention is to use these attributes to initialize workers in
the same manner as the master node is initialized in pytest-xdist.
Thanks for submitting a PR, your contribution is really appreciated!
Here is a quick checklist that should be present in PRs.
(please delete this text from the final description, this is just a guideline)
-->
- [ ] Target the `master` branch for bug fixes, documentation updates and trivial changes.
- [ ] Target the `features` branch for new features and removals/deprecations.
- [ ] Target the `features` branch for new features, improvements, and removals/deprecations.
- [ ] Include documentation when adding new features.
- [ ] Include new tests or update existing tests when applicable.
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/master/changelog/README.rst) for details.
- [ ] Add yourself to `AUTHORS` in alphabetical order;
Write sentences in the **past or present tense**, examples:
* *Improved verbose diff output with sequences.*
* *Terminal summary statistics now use multiple colors.*
Also make sure to end the sentence with a `.`.
- [ ] Add yourself to `AUTHORS` in alphabetical order.
This directory contains "newsfragments" which are short files that contain a small **ReST**-formatted
text that will be added to the next ``CHANGELOG``.
The ``CHANGELOG`` will be read by users, so this description should be aimed to pytest users
The ``CHANGELOG`` will be read by **users**, so this description should be aimed to pytest users
instead of describing internal changes which are only relevant to the developers.
Make sure to use full sentences with correct case and punctuation, for example::
Make sure to use full sentences in the **past or present tense** and use punctuation, examples::
Fix issue with non-ascii messages from the ``warnings`` module.
Improved verbose diff output with sequences.
Terminal summary statistics now use multiple colors.
Each file should be named like ``<ISSUE>.<TYPE>.rst``, where
``<ISSUE>`` is an issue number, and ``<TYPE>`` is one of:
*``feature``: new user facing features, like new command-line options and new behavior.
*``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junitxml``, improved colors in terminal, etc).
*``bugfix``: fixes a reported bug.
*``doc``: documentation improvement, like rewording an entire session or adding missing docs.
*``deprecation``: feature deprecation.
@@ -28,6 +31,7 @@ changelog using that instead.
If you are not sure what issue type to use, don't hesitate to ask in your PR.
``towncrier`` preserves multiple paragraphs and formatting (code blocks, lists, and so on), but for entries
other than ``features`` it is usually better to stick to a single paragraph to keep it concise. You can install
``towncrier`` and then run ``towncrier --draft``
if you want to get a preview of how your change will look in the final release notes.
other than ``features`` it is usually better to stick to a single paragraph to keep it concise.
You can also run ``tox -e docs`` to build the documentation
with the draft changelog (``doc/en/_build/changelog.html``) if you want to get a preview of how your change will look in the final release notes.
It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies.
.._`nose and unittest`: faq.html#how-does-pytest-relate-to-nose-and-unittest
The default value of ``junit_family`` option will change to ``xunit2`` in pytest 6.0, given
that this is the version supported by default in modern tools that manipulate this type of file.
In order to smooth the transition, pytest will issue a warning in case the ``--junitxml`` option
is given in the command line but ``junit_family`` is not explicitly configured in ``pytest.ini``::
PytestDeprecationWarning: The 'junit_family' default value will change to 'xunit2' in pytest 6.0.
Add 'junit_family=legacy' to your pytest.ini file to silence this warning and make your suite compatible.
In order to silence this warning, users just need to configure the ``junit_family`` option explicitly:
..code-block::ini
[pytest]
junit_family=legacy
``funcargnames`` alias for ``fixturenames``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 5.0
@@ -34,12 +55,47 @@ in places where we or plugin authors must distinguish between fixture names and
names supplied by non-fixture things such as ``pytest.mark.parametrize``.
Result log (``--result-log``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated:: 4.0
The ``--result-log`` option produces a stream of test reports which can be
analysed at runtime, but it uses a custom format which requires users to implement their own
parser.
The `pytest-reportlog <https://github.com/pytest-dev/pytest-reportlog>`__ plugin provides a ``--report-log`` option, a more standard and extensible alternative, producing
one JSON object per-line, and should cover the same use cases. Please try it out and provide feedback.
The plan is remove the ``--result-log`` option in pytest 6.0 if ``pytest-reportlog`` proves satisfactory
to all users and is deemed stable. The ``pytest-reportlog`` plugin might even be merged into the core
at some point, depending on the plans for the plugins and number of users using it.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
``pytest.config`` global
~~~~~~~~~~~~~~~~~~~~~~~~
..versionremoved:: 5.0
The ``pytest.config`` global object is deprecated. Instead use
``request.config`` (via the ``request`` fixture) or if you are a plugin author
use the ``pytest_configure(config)`` hook. Note that many hooks can also access
the ``config`` object indirectly, through ``session.config`` or ``item.config`` for example.
.._`raises message deprecated`:
``"message"`` parameter of ``pytest.raises``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
..deprecated::4.1
..versionremoved::5.0
It is a common mistake to think this parameter will match the exception message, while in fact
it only serves to provide a custom message in case the ``pytest.raises`` check fails. To prevent
@@ -70,22 +126,12 @@ If you still have concerns about this deprecation and future removal, please com
pytest.skip("test requires env in {!r}".format(envnames))
A test file using this local plugin:
@@ -413,7 +413,7 @@ the test needs:
test_someenv.py s [100%]
======================== 1 skipped in 0.12 seconds =========================
============================ 1 skipped in 0.12s ============================
and here is one that specifies exactly the environment needed:
@@ -428,7 +428,7 @@ and here is one that specifies exactly the environment needed:
test_someenv.py . [100%]
========================= 1 passed in 0.12 seconds =========================
============================ 1 passed in 0.12s =============================
The ``--markers`` option always gives you a list of available markers:
@@ -499,7 +499,7 @@ The output is as follows:
$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef>,), kwargs={})
.
1 passed in 0.12 seconds
1 passed in 0.12s
We can see that the custom marker has its argument set extended with the function ``hello_world``. This is the key difference between creating a custom marker as a callable, which invokes ``__call__`` behind the scenes, and using ``with_args``.
@@ -523,7 +523,7 @@ code you can read over all such settings. Example:
@pytest.mark.glob("class",x=2)
classTestClass(object):
classTestClass:
@pytest.mark.glob("function",x=3)
deftest_something(self):
pass
@@ -539,7 +539,7 @@ test function. From a conftest file we can read it like this:
@@ -293,7 +323,7 @@ Let's first see how it looks like at collection time:
<Function test_db_initialized[d1]>
<Function test_db_initialized[d2]>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
And then when we run the test:
@@ -312,8 +342,8 @@ And then when we run the test:
> pytest.fail("deliberately failing for demo purposes")
E Failed: deliberately failing for demo purposes
test_backends.py:6: Failed
1 failed, 1 passed in 0.12 seconds
test_backends.py:8: Failed
1 failed, 1 passed in 0.12s
The first invocation with ``db == "DB1"`` passed while the second with ``db == "DB2"`` failed. Our ``db`` fixture function has instantiated each of the DB values during the setup phase while the ``pytest_generate_tests`` generated two according calls to the ``test_db_initialized`` during the collection phase.
@@ -327,23 +357,29 @@ parameter on particular arguments. It can be done by passing list or tuple of
arguments' names to ``indirect``. In the example below there is a function ``test_indirect`` which uses
two fixtures: ``x`` and ``y``. Here we give to indirect the list, which contains the name of the
fixture ``x``. The indirect parameter will be applied to this argument only, and the value ``a``
smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@@ -638,7 +706,7 @@ So let's just do another run:
> assert b"smtp.gmail.com" in msg
E AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8\nCHUNKING'
* `Flaky Tests at Google and How We Mitigate Them <https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html>`_ by John Micco, 2016
*`Where do Google's flaky tests come from? <https://docs.google.com/document/d/1mZ0-Kc97DI_F3tf_GBW_NB_aqka-P1jVOsFfufxqUUM/edit#heading=h.ec0r4fypsleh>`_ by Jeff Listfield, 2017
*`Where do Google's flaky tests come from? <https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html>`_ by Jeff Listfield, 2017
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test::
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test:
..code-block::python
# content of test_class.py
classTestClass(object):
class TestClass:
deftest_one(self):
x="this"
assert 'h' in x
assert "h"inx
deftest_two(self):
x="hello"
asserthasattr(x, 'check')
assert hasattr(x,"check")
``pytest`` discovers all tests following its :ref:`Conventions for Python test discovery <test discovery>`, so it finds both ``test_`` prefixed functions. There is no need to subclass anything. We can simply run the module by passing its filename:
@@ -130,19 +140,21 @@ Once you develop multiple tests, you may want to group them into a class. pytest
def test_two(self):
x = "hello"
> assert hasattr(x, 'check')
> assert hasattr(x, "check")
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:8: AssertionError
1 failed, 1 passed in 0.12 seconds
1 failed, 1 passed in 0.12s
The first test passed and the second failed. You can easily see the intermediate values in the assertion to help you understand the reason for the failure.
Request a unique temporary directory for functional tests
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html#builtinfixtures>`_ to request arbitrary resources, like a unique temporary directory::
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
..code-block::python
# content of test_tmpdir.py
deftest_needsfiles(tmpdir):
@@ -168,7 +180,7 @@ List the name ``tmpdir`` in the test function signature and ``pytest`` will look
Supports passing kwarg ``extra={"auto_indent": [value]}`` to
calls to ``logging.log()`` to specify auto-indentation behavior for
a specific entry in the log. ``extra`` kwarg overrides the value specified
on the command line or in the config.
..confval:: log_cli
Enable log display during test run (also known as :ref:`"live logging" <live_logs>`).
The default is ``False``.
..code-block::ini
[pytest]
log_cli=True
..confval:: log_cli_date_format
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
