Compare commits

...

680 Commits
4.6.2 ... 5.2.2

Author SHA1 Message Date
Bruno Oliveira
b27ba97721 Preparing release version 5.2.2 2019-10-24 19:24:04 -04:00
Bruno Oliveira
2b56c7e1ce Update Tidelift docs with latest campaign (#6053)
Update Tidelift docs with latest campaign
2019-10-24 18:46:21 -03:00
Bruno Oliveira
2bee7d7c3e Update Tidelift docs with latest campaign
Tidelift has launched a new marketing campaign as outlined here:

* https://forum.tidelift.com/t/task-enhancement-marketing-the-tidelift-subscription-to-your-users/321

This PR splits the previous "sponsor" information into two, Open Collective
and Tidelift (as they have very different target audiences).

Also took the opportunity to reorder some items at the end of
the contents page in a manner that I believe make more sense.
2019-10-24 14:58:58 -03:00
Bruno Oliveira
713b9e54c3 Review rm_rf handling of FileNotFoundErrors (#6044)
Review rm_rf handling of FileNotFoundErrors
2019-10-23 19:22:26 -03:00
Bruno Oliveira
ba4b8c869c Review rm_rf handling of FileNotFoundErrors 2019-10-23 08:30:52 -03:00
Daniel Hahler
82e9013e73 Merge pull request #6004 from blueyed/fix-nf
cache: NFPlugin: keep known nodeids
2019-10-23 01:04:48 +02:00
Bruno Oliveira
14142b9113 tests: remove unnecessary test, clarify (#6013)
tests: remove unnecessary test, clarify
2019-10-20 18:36:27 -03:00
Daniel Hahler
0123b29ed7 tests: remove unnecessary test, clarify
Follow-up to https://github.com/pytest-dev/pytest/pull/6009.
2019-10-20 20:40:13 +02:00
Bruno Oliveira
16efa1bfef Merge pull request #6009 from yoavcaspi/fix_keyboardInterrupt_on_setup_show
setuponly: remove printing out/err from capman
2019-10-20 14:00:06 -03:00
Yoav Caspi
5624e366c1 add more indications to the result of the tests 2019-10-20 09:54:23 +03:00
Anthony Sottile
b88f5df4ce Merge pull request #6010 from pytest-dev/asottile-patch-2
Ensure .pytest_cache file has a newline at the end
2019-10-19 16:21:21 -07:00
Anthony Sottile
3a402811de Ensure .pytest_cache file has a newline at the end 2019-10-19 14:42:06 -07:00
Yoav Caspi
e05b33ed16 setuponly: remove printing out/err from capman 2019-10-19 22:33:19 +03:00
Florian Bruhin
2e11ea6108 Merge pull request #6006 from atugushev/update-doc-contextlib2
Update doc to use contextlib2.nullcontext
2019-10-19 14:29:16 +02:00
Albert Tugushev
18786992bb Update doc to use contextlib2.nullcontext
nullcontext has been backported in contextlib2==0.6.0
2019-10-19 15:09:37 +07:00
Daniel Hahler
3173a26388 cache: NFPlugin: keep known nodeids
Caveat: does not forget about old nodeids

Fixes https://github.com/pytest-dev/pytest/issues/5206
2019-10-19 04:53:01 +02:00
Daniel Hahler
5e7b8d813b Remove unreachable code from config._prepareconfig (#6001) 2019-10-18 23:47:09 +02:00
Philipp Loose
afac1f0021 Remove unreachable code from config._prepareconfig
The code that could trigger the execution of the removed lines was
removed with a7e4016.
2019-10-18 23:11:41 +02:00
Daniel Hahler
0783030357 Merge pull request #5992 from blueyed/tests-PDBPP_HIJACK_PDB
tests: debugging: disable pdb++ within inner tests
2019-10-18 17:50:03 +02:00
Daniel Hahler
d910175b9f Merge pull request #5993 from blueyed/filterwarnings-error-removing
tests: filterwarnings: do not crash with "(rm_rf)" warning
2019-10-18 17:38:40 +02:00
Daniel Hahler
6b2bae9392 tests: filterwarnings: do not crash with "(rm_rf)" warning
Ref: https://github.com/pytest-dev/pytest/issues/5974
2019-10-18 17:22:56 +02:00
Daniel Hahler
7ef44913a1 tests: debugging: disable pdb++ within inner tests
Ref: https://github.com/pytest-dev/pytest/pull/5306#issuecomment-495690643
2019-10-18 17:08:39 +02:00
Daniel Hahler
813ef9e88f Merge pull request #5983 from aklajnert/fix_5.2.0_announce
Remove redundant mention from 5.2.0 release notes.
2019-10-17 15:33:53 +02:00
Andrzej Klajnert
f2dd9cc63e Remove redundant mention from 5.2.0 release notes. 2019-10-17 13:17:34 +02:00
Daniel Hahler
ce8b1dfa04 ci: Travis: move py37-pexpect to another job (#5979) 2019-10-17 07:54:04 +02:00
Daniel Hahler
a73d0151a6 ci: Travis: move py37-pexpect to another job
It does not have to run all tests again by itself.
2019-10-17 02:19:46 +02:00
Bruno Oliveira
3cb1457e6d Merge pull request #5977 from blueyed/numpy
tests: keep numpy being optional
2019-10-16 20:13:29 -03:00
Daniel Hahler
90dfee5da5 tests: keep numpy being optional
Ref: https://github.com/pytest-dev/pytest/pull/5950#discussion_r335254774
2019-10-16 23:50:23 +02:00
Bruno Oliveira
77a995ffad Add missing version added/changed markers to docs (#5966)
Add missing version added/changed markers to docs
2019-10-16 10:43:45 -03:00
Bruno Oliveira
0383d43645 Add missing version added/changed markers to docs
Notice some features since 5.0 were not being properly
marked in which version they have been added/changed.
2019-10-15 19:45:58 -03:00
Bruno Oliveira
76c2a8ebbe doc: caplog: add caplog.messages (#5963)
doc: caplog: add caplog.messages
2019-10-15 19:42:15 -03:00
Daniel Hahler
71a7fd02a5 doc: caplog: add caplog.messages 2019-10-15 16:18:50 +02:00
Anthony Sottile
7bc8cb8e2b Merge pull request #5954 from blueyed/changelog-remove-dupe
changelog: #5523 was fixed in 5.0.1 already
2019-10-14 15:37:21 -07:00
Daniel Hahler
dee8d94876 changelog: #5523 was fixed in 5.0.1 already
Ref: https://github.com/pytest-dev/pytest/pull/5952#issuecomment-541801883
2019-10-14 19:20:57 +02:00
Anthony Sottile
a20880cca2 Merge pull request #5952 from nicoddemus/port-changelog
Port CHANGELOG from 4.6.6 release
2019-10-13 10:20:34 -07:00
Bruno Oliveira
ae9465215e Port CHANGELOG from 4.6.6 release 2019-10-13 11:54:02 -03:00
Bruno Oliveira
1555973487 Workaround curl bug which makes retries of fetching codecov.io/… (#5951)
Workaround curl bug which makes retries of fetching codecov.io/bash not work
2019-10-12 17:08:47 -03:00
Bruno Oliveira
3322c1e033 Casting fixture parameter to list at the beginning of parameter… (#5950)
Casting fixture parameter to list at the beginning of parameter parsing.
2019-10-12 17:02:48 -03:00
Ran Benita
7678f891f9 Workaround curl bug which makes retries of fetching codecov.io/bash not work 2019-10-12 21:52:01 +03:00
Florian Bruhin
4f2abd7ae0 Merge pull request #5948 from attomos/update-projects-doc
Documentation update to project examples
2019-10-12 17:31:24 +02:00
Victor Maryama
122cf60b27 Always creating list for consistency.
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
2019-10-12 15:46:28 +02:00
Victor Maryama
63e3d89647 Fixed linting. 2019-10-12 15:08:47 +02:00
Victor Maryama
122748a6cf Added changelog file. 2019-10-12 14:38:58 +02:00
Victor Maryama
1f639e2c22 Casting fixture parameter to list at the beginning of parameter parsing. 2019-10-12 14:33:43 +02:00
Nattaphoom Chaipreecha
83ba5eb58a Add pudb to project examples 2019-10-12 08:10:04 +07:00
Nattaphoom Chaipreecha
d07c5ba4ae Update pdb++ link (moved to GitHub) 2019-10-12 08:09:49 +07:00
Anthony Sottile
b162ab6a45 Merge pull request #5943 from nicoddemus/py2-py37-tech-docs
Add link to technical aspects issue to the py27-py34 docs
2019-10-11 11:07:58 -07:00
Bruno Oliveira
57141dc708 Add link to technical aspects issue to the py27-py34 docs 2019-10-11 08:50:38 -03:00
Bruno Oliveira
bad4ffc3a7 remove cancelled training sidebar (#5938)
remove cancelled training sidebar
2019-10-10 07:45:26 -03:00
Oliver Bestwalter
71ad5b0fbb remove cancelled training sidebar 2019-10-10 12:11:16 +02:00
Anthony Sottile
3fada8c8ee Merge pull request #5925 from asottile/fix_resource_warning_again_5088
Fix spurious ResourceWarning stderr in testsuite again
2019-10-07 16:26:38 -07:00
Anthony Sottile
271dc7f17a Merge pull request #5904 from asottile/warnings_v2
Add test to ensure _pytest is warning-clean on import
2019-10-06 19:24:07 -07:00
Anthony Sottile
19eb0590f1 Fix spurious ResourceWarning stderr in testsuite again 2019-10-06 19:11:33 -07:00
Anthony Sottile
eaa05531ed Add test to ensure _pytest is warning-clean on import 2019-10-06 14:11:08 -07:00
Bruno Oliveira
74aed6ea4c Release 5.2.1 (#5917)
Release 5.2.1
2019-10-06 12:55:15 -03:00
Bruno Oliveira
cfa9ebc91f ci: test oldest supported attrs (#5922)
ci: test oldest supported attrs
2019-10-06 12:55:03 -03:00
Daniel Hahler
b0fd8742da ci: test oldest supported attrs 2019-10-06 16:12:56 +02:00
Bruno Oliveira
12cc729f6b Preparing release version 5.2.1 2019-10-06 08:00:49 -04:00
Bruno Oliveira
1c5efffd90 Add changelog entry for #5902 2019-10-06 07:58:23 -04:00
Bruno Oliveira
8c9ea5e055 Fix warnings with attrs 19.2 and fix object assertions (#5902)
Fix warnings with attrs 19.2 and fix object assertions
2019-10-06 08:47:17 -03:00
Anthony Sottile
c58b0fb4ac Use ATTRS_EQ_FIELD for attrs 19.2 compat 2019-10-05 18:16:35 -07:00
Anthony Sottile
4011af68cd Merge pull request #5910 from pytest-dev/asottile-patch-1
Fix dynamic scoping changelog link
2019-10-03 09:39:12 -07:00
Anthony Sottile
9637b3e376 Fix dynamic scoping changelog link 2019-10-03 09:01:08 -07:00
Anthony Sottile
33c3ec66b7 Merge pull request #5898 from kevinjfoley/doc-typo-fix
Fix doc typo
2019-10-01 12:58:54 -07:00
Hynek Schlawack
a79acf279a Fix warnings with attrs 19.2 and fix object assertions
attrs 19.2 deprecated cmp in favor of the dataclass-ish eq/order duo.

This causes deprecation warnings that in turn break some of the cool new deep
object comparisons. Since we at attrs expected this to be a problem, it shipped
with helpers to write backward and forward compatible code.

This PR uses that and avoids changed to minimal versions.
2019-10-01 20:47:59 +02:00
Bruno Oliveira
9a4c0b991b Update doc: pytest section in setup.cfg (#5894)
Update doc: pytest section in setup.cfg
2019-10-01 11:45:01 -03:00
Kevin J. Foley
b490f5f979 Fix doc typo 2019-10-01 10:17:26 -04:00
tadashigaki
acfd0fd9d6 Update doc: pytest section in setup.cfg 2019-10-01 02:44:07 +09:00
Bruno Oliveira
88434f1f42 Release 5.2.0 (#5885)
Release 5.2.0
2019-09-29 10:26:06 -03:00
Florian Bruhin
4d01740be3 Merge pull request #5887 from asottile/fix_attributes_docs_pytester
Fix attribute docs in _pytest.pytester
2019-09-29 14:22:54 +02:00
Anthony Sottile
07792c7113 Fix attribute docs in _pytest.pytester 2019-09-28 18:19:53 -07:00
Bruno Oliveira
068ef90b92 Preparing release version 5.2.0 2019-09-28 21:18:37 -04:00
Bruno Oliveira
065773aa97 Use 'python3' instead of 'python3.6' on tox
This allows us to use python3.7+ to use tox
2019-09-28 21:16:20 -04:00
Anthony Sottile
b62276826c Merge pull request #5886 from nicoddemus/setup-plan-custom-items-5884
Fix --setup-only and --setup-show for custom pytest items
2019-09-28 18:08:36 -07:00
Bruno Oliveira
7bdfba3578 Fix --setup-only and --setup-show for custom pytest items
Fix #5884
2019-09-28 11:52:09 -03:00
Daniel Hahler
6bfd30d169 Merge pull request #5878 from blueyed/codecov-retry-6
ci: codecov: use 6 retries with curl
2019-09-24 10:30:23 +02:00
Daniel Hahler
7731e45615 ci: codecov: use 6 retries with curl
This should result in retries of 1+2+4+8+16+32 = 63s.

Ref: https://github.com/pytest-dev/pytest/pull/5869#issuecomment-534235437
2019-09-23 20:58:52 +02:00
Anthony Sottile
8806b1f531 Merge pull request #5875 from asottile/patch-1
Make sure to quote `sys.executable` as we're running a shell
2019-09-23 18:55:33 +02:00
Anthony Sottile
19c9e53604 Make sure to quote sys.executable as we're running a shell 2019-09-23 08:58:56 -07:00
Bruno Oliveira
c28b63135f Merge master into features (#5874)
Merge master into features
2019-09-23 12:44:20 -03:00
Bruno Oliveira
7c64d5d882 Do not call python directly but use sys.executable. Fixes #5872 (#5873)
Do not call python directly but use sys.executable. Fixes #5872
2019-09-23 12:28:00 -03:00
Bruno Oliveira
d3d9f9f668 Merge remote-tracking branch 'upstream/master' into mm 2019-09-23 12:09:01 -03:00
Bruno Oliveira
018edf2a0e Change report-coverage.sh in attempt to fix Azure (#5869)
Change report-coverage.sh in attempt to fix Azure
2019-09-23 12:01:19 -03:00
Tomáš Chvátal
04c01fb606 test_argcomplete do not call python directly #5872
Use sys.executable to detect which python we should actually be testing.
2019-09-23 16:38:15 +02:00
Daniel Hahler
ea0c7e43b6 Remove unneeded codecov options (implied with "-f") 2019-09-23 02:26:53 +02:00
Bruno Oliveira
de8fdab7a9 Change report-coverage.sh in attempt to fix Azure
Recently sometimes Azure has failed with:

++ curl -s https://codecov.io/bash
bash: /dev/fd/63: No such file or directory

This attempts to fix this by modifying report-coverage.sh slightly.
2019-09-21 16:02:48 -03:00
Daniel Hahler
c1361b48f8 Merge pull request #5868 from blueyed/mm
Merge master into features
2019-09-21 16:29:49 +02:00
Bruno Oliveira
1b4ad7774b Fix logging doc: change x.level to x.levelno (#5866)
Fix logging doc: change x.level to x.levelno
2019-09-21 11:28:40 -03:00
Daniel Hahler
409cc2946a Merge master into features 2019-09-21 16:22:48 +02:00
Bruno Oliveira
3114be9181 Revert "Show banner/full page for the Digital Climate Strike (#… (#5867)
Revert "Show banner/full page for the Digital Climate Strike (#5861)"
2019-09-21 11:09:34 -03:00
Bruno Oliveira
e4103cb02c Release version 5.1.3 (#5859)
Release version 5.1.3
2019-09-21 10:56:16 -03:00
Bruno Oliveira
217605c217 Revert "Show banner/full page for the Digital Climate Strike (#5861)"
This reverts commit c8cf748c49, reversing
changes made to 702acdba46.
2019-09-21 10:38:10 -03:00
James Cooke
2fcf21a6c7 Fix logging doc: change x.level to x.levelno 2019-09-20 18:38:47 +01:00
Bruno Oliveira
c8cf748c49 Show banner/full page for the Digital Climate Strike (#5861)
Show banner/full page for the Digital Climate Strike
2019-09-19 10:49:10 -03:00
Bruno Oliveira
249b53e623 Show banner/full page for the Ditigal Climate Strike
As discussed in the mailing list, pytest will join the
digital strike for the climate on Sep 20th. This will show
a closable banner on the docs until the date, and when the date
comes the banner will become a (closable) full page.

I will also pin an issue saying that the developers won't be
available on Sep 20th.
2019-09-19 08:26:25 -03:00
Bruno Oliveira
9669413b1f Merge pull request #5776 from aklajnert/1682-dynamic-scope
Implemented the dynamic scope feature.
2019-09-19 08:22:45 -03:00
Andrzej Klajnert
e2382e96ed Minor cleanup in tests. 2019-09-19 11:13:22 +02:00
Bruno Oliveira
1a9f4a51cb Preparing release version 5.1.3 2019-09-18 10:11:59 -03:00
Bruno Oliveira
892bdd59dc Normalize all summary durations, including quiet ones 2019-09-18 10:10:25 -03:00
Bruno Oliveira
df46afc96d Change fixture argument handling tests to unit-tests 2019-09-18 07:50:35 -03:00
Bruno Oliveira
6918d07560 Merge remote-tracking branch 'upstream/features' into aklajnert/1682-dynamic-scope 2019-09-18 07:44:18 -03:00
Daniel Hahler
c997c32004 Merge pull request #5856 from blueyed/mm
Merge master into features
2019-09-17 21:47:40 +02:00
Daniel Hahler
450409d123 Merge master into features
Conflicts:
	src/_pytest/reports.py
        (via 7259c453d, moved the type annotation; setting it to `None`
        was removed in 3c82b1cb9 already)
2019-09-17 12:46:36 +02:00
Daniel Hahler
702acdba46 Merge pull request #5811 from blueyed/fulltrace-pytest-raises
Handle --fulltrace with pytest.raises
2019-09-14 02:09:36 +02:00
Daniel Hahler
f832ac3316 Handle --fulltrace with pytest.raises
This changes `_repr_failure_py` to use `tbfilter=False` always.
2019-09-14 01:41:43 +02:00
Bruno Oliveira
9422e10322 Fix regression due to different cases on Windows (#5840)
Fix regression due to different cases on Windows
2019-09-13 18:11:12 -03:00
Bruno Oliveira
5c3b4a6f52 Add CHANGELOG entry for #5792 2019-09-12 08:05:50 -03:00
Christian Neumüller
05850d73bd Re-introduce Christian Neumüller to AUTHORS
The introduction was reverted by cd29d56
2019-09-12 08:05:50 -03:00
Bruno Oliveira
b48f51eb03 Use Path() objects to store conftest files
Using Path().resolve() is better than py.path.realpath because
it resolves to the correct path/drive in case-insensitive file systems (#5792):

>>> from py.path import local
>>> from pathlib import Path
>>>
>>> local('d:\\projects').realpath()
local('d:\\projects')
>>> Path('d:\\projects').resolve()
WindowsPath('D:/projects')

Fix #5819
2019-09-12 08:05:50 -03:00
Bruno Oliveira
cf5b544db3 Revert "Merge pull request #5792 from dynatrace-oss-contrib/bugfix/badcase"
This reverts commit 955e542210, reversing
changes made to 0215bcd84e.

Will attempt a simpler approach
2019-09-12 08:05:43 -03:00
Bruno Oliveira
73c5b7f4b1 Clarify docs by showing tox.ini considered before setup.cfg (#5839)
Clarify docs by showing tox.ini considered before setup.cfg
2019-09-11 22:14:31 -03:00
Anthony Sottile
8f2f51be6d Clarify docs by showing tox.ini considered before setup.cfg 2019-09-11 14:07:06 -07:00
Andrzej Klajnert
f2f3ced508 Fixed the fixture function signature. 2019-09-10 16:20:44 +02:00
Bruno Oliveira
23102a7d84 Update doc regarding pytest.raises (#5834)
Update doc regarding pytest.raises
2019-09-09 16:18:58 -03:00
Gene Wood
f0d538329c Update doc regarding pytest.raises
Remove reference to the `message` argument in the docs as it was deprecated in #4539
2019-09-09 12:14:09 -07:00
Bruno Oliveira
6c8bcf601c Fix pypy3.6 on windows (#5828)
Fix pypy3.6 on windows
2019-09-08 12:11:28 -03:00
Anthony Sottile
9d7b919c7d Fix pypy3.6 on windows 2019-09-07 16:49:05 -07:00
Bruno Oliveira
333e9d5c10 Merge pull request #5824 from blueyed/revert-py350
ci: Travis: do not test with 3.5.0
2019-09-06 08:05:51 -03:00
Daniel Hahler
f1b605c95e ci: Travis: do not test with 3.5.0
This causes flaky test failures (crashes).

Closes https://github.com/pytest-dev/pytest/issues/5795.
2019-09-06 12:29:17 +02:00
Bruno Oliveira
2bb8d93001 Fix for Python 4: replace unsafe PY3 with PY2 (#5820)
Fix for Python 4: replace unsafe PY3 with PY2
2019-09-05 12:39:06 -03:00
Hugo
d049b35397 Fix for Python 4: replace unsafe PY3 with PY2 2019-09-05 18:06:47 +03:00
Bruno Oliveira
8ee557f7ae Fix pythonpath anchor (#5817)
Fix pythonpath anchor
2019-09-04 22:19:57 -03:00
Gene Wood
ca3884d9bb Add Gene Wood to authors 2019-09-04 09:21:10 -07:00
Gene Wood
bc163605ab Fix anchor link from Good Practices to Pythonpath doc 2019-09-04 09:18:10 -07:00
Bruno Oliveira
1675048b35 Merge pull request #5808 from goerz/pastebin
Fix "lexer" being used when uploading to bpaste.net
2019-08-31 16:11:39 -03:00
aklajnert
10bf6aac76 Implemented the dynamic scope feature. 2019-08-31 18:12:24 +02:00
Michael Goerz
f8dd6349c1 Fix "lexer" being used when uploading to bpaste.net
Closes #5806.
2019-08-30 15:34:03 -04:00
Bruno Oliveira
8c8809e1aa Merge pull request #5805 from nicoddemus/release-5.1.2
Release 5.1.2
2019-08-30 16:05:14 -03:00
Bruno Oliveira
e56544cb58 Preparing release version 5.1.2 2019-08-30 12:43:47 -03:00
Bruno Oliveira
f9cc704b1a Replace session duration to a fix value in regendoc 2019-08-30 12:42:14 -03:00
Bruno Oliveira
bd57307a39 Merge pull request #5768 from robholt/fixture-class-instance
Fix self reference in function scoped fixtures
2019-08-30 12:31:16 -03:00
Bruno Oliveira
667c6463ab Merge pull request #5803 from nicoddemus/flaky-5795
Mark flaky test as xfail on Python 3.5.0
2019-08-30 12:30:11 -03:00
Bruno Oliveira
4e594552eb Merge pull request #5798 from aklajnert/570-indirect-fixtures
Fix the scope behavior with indirect fixtures
2019-08-30 12:29:03 -03:00
Bruno Oliveira
955dc6d18a Merge remote-tracking branch 'upstream/master' into robholt/fixture-class-instance 2019-08-30 11:21:33 -03:00
Bruno Oliveira
3ddbc7fb2a Improve CHANGELOG and add some comments
Ref: #5768
2019-08-30 11:20:19 -03:00
Bruno Oliveira
bb60736a6f Run py35 without xdist on Travis
Due to the flaky tests in 3.5.0, drop running
py35 with xdist for now in the hope we get better error
messages.

Ref: #5795
2019-08-30 10:59:50 -03:00
Bruno Oliveira
35b3b1097f Improve CHANGELOG and make test easier to understand for #570 2019-08-30 10:54:58 -03:00
Bruno Oliveira
01082fea12 Serialize/deserialize chained exceptions (#5787)
Serialize/deserialize chained exceptions
2019-08-30 07:29:48 -03:00
Bruno Oliveira
404cf0c872 Merge pull request #5764 from goerz/pastebin
Gracefully handle HTTP errors from pastebin
2019-08-30 07:29:14 -03:00
Bruno Oliveira
a511b98da9 Serialize/deserialize chained exceptions
Fix #5786
2019-08-29 20:23:44 -03:00
Andrzej Klajnert
487659d8b1 Fix the scope behavior with indirect fixtures. 2019-08-29 07:23:08 +02:00
Bruno Oliveira
955e542210 Merge pull request #5792 from dynatrace-oss-contrib/bugfix/badcase
Fix pytest with mixed up filename casing.
2019-08-28 08:44:16 -03:00
Christian Neumüller
29bb0eda27 Move _uniquepath to pathlib as unique_path.
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2019-08-28 12:29:41 +02:00
Christian Neumüller
a98270eac0 Document the bugfix. 2019-08-27 16:36:40 +02:00
Christian Neumüller
1aac64573f black formatting. 2019-08-27 16:16:45 +02:00
Michael Goerz
d47b9d04d4 Gracefully handle HTTP errors from pastebin
We find that the --pastebin option to pytest sometimes fails with "HTTP
Error 400: Bad Request". We're still investigating the exact cause of
these errors, but in the meantime, a failure to upload to the pastebin
service should probably not crash pytest and cause a test failure in the
continuous-integration.

This patch catches exceptions like HTTPError that may be thrown while
trying to communicate with the pastebin service, and reports them as a
"bad response", without crashing with a backtrace or failing the entire
test suite.
2019-08-26 23:50:46 -04:00
Anthony Sottile
5bf9f9a711 Merge pull request #5788 from nicoddemus/mm
Merge master into features
2019-08-26 18:20:07 -07:00
Bruno Oliveira
c28e428249 Merge remote-tracking branch 'upstream/master' into mm 2019-08-26 20:00:30 -03:00
Christian Neumüller
505c3340bf Fix pytest with mixed up filename casing. 2019-08-26 17:18:46 +02:00
Bruno Oliveira
7a69365486 Move TWMock class to a fixture
Using a relative import like before was not very nice
2019-08-26 11:32:57 -03:00
Bruno Oliveira
3c82b1cb97 Refactor report serialization/deserialization code
Refactoring this in order to support chained exceptions
more easily.

Related to #5786
2019-08-26 10:54:25 -03:00
Bruno Oliveira
0215bcd84e update doc (#5784)
update doc
2019-08-25 13:41:17 -03:00
linchiwei123
01b9774e3b update doc 2019-08-25 23:50:41 +08:00
Bruno Oliveira
9859d37cf6 Merge pull request #5782 from linchiwei123/patch-1
Fix TypeError
2019-08-24 15:40:49 -03:00
Bruno Oliveira
1c7aeb670a Fix linting 2019-08-24 15:01:48 -03:00
Bruno Oliveira
691c706fcc Add test for #5782 2019-08-24 14:41:06 -03:00
linchiwei123
a4adf511fc Fix TypeError
can only concatenate str (not "bytes") to str
2019-08-24 23:10:17 +08:00
Anthony Sottile
4265ab3a41 Merge pull request #5773 from asottile/release-5.1.1
Preparing release version 5.1.1
2019-08-20 14:45:46 -07:00
Anthony Sottile
b135f5af8d Preparing release version 5.1.1 2019-08-20 11:30:36 -07:00
Anthony Sottile
daff9066c0 Merge pull request #5752 from bluetech/typing-py350-fix
Fix TypeError when importing pytest on Python 3.5.0 and 3.5.1
2019-08-20 08:04:58 -07:00
Ran Benita
c2f762460f Merge pull request #5673 from bluetech/type-annotations-3
1/X Fix check_untyped_defs = True mypy errors
2019-08-20 17:20:40 +03:00
Ran Benita
43eab917a1 Fix coverage 2019-08-20 15:41:32 +03:00
Ronny Pfannschmidt
0e569faca2 Merge pull request #5769 from The-Compiler/remove-training
Remove cancelled training
2019-08-20 11:48:36 +02:00
Ran Benita
a7c235732a Pypy doesn't have sys.getrefcount(), so go back to gc 2019-08-20 11:47:22 +03:00
Ran Benita
cec2183aeb Add workaround for test_raises_cyclic_reference in Python 3.5.{0,1} 2019-08-20 11:19:25 +03:00
Florian Bruhin
c049fd85ab Remove cancelled training 2019-08-19 22:07:53 +02:00
Robert Holt
62381125e7 Fix self reference in function scoped fixtures 2019-08-19 15:57:39 -04:00
Anthony Sottile
a7ede64f42 Move @overload to compat 2019-08-18 14:54:52 -07:00
Bruno Oliveira
307652202c Merge pull request #5762 from pytest-dev/hugovk-patch-1
Docs: update tense
2019-08-18 10:20:45 -03:00
Hugo van Kemenade
a287aea00e Docs: update tense
pytest 4.6.0 was released in June 2019

https://pypi.org/project/pytest/#history
2019-08-18 13:34:59 +03:00
Anthony Sottile
6bf6265c59 Merge pull request #5760 from blueyed/F401
minor: make noqa comment more specific
2019-08-17 19:54:04 -07:00
Daniel Hahler
7c26a65865 minor: make noqa comment more specific 2019-08-17 23:48:55 +02:00
Daniel Hahler
1e3205e7cf ci: Travis: use 3.5.0
Ref: https://github.com/pytest-dev/pytest/pull/5752#issuecomment-522241225
2019-08-17 23:10:01 +02:00
Daniel Hahler
32dac18f38 Merge pull request #5755 from blueyed/fix-assert-location-with-coverage
Fix wrong location of assertion error with Coverage.py .

Reverts using-constant part from 39ba996.

Fixes https://github.com/pytest-dev/pytest/issues/5754.
2019-08-17 22:49:00 +02:00
Daniel Hahler
f05ca74d27 Merge pull request #5056 from blueyed/argparsing-width
Inject width via pylib to argparse formatter
2019-08-17 21:46:02 +02:00
Daniel Hahler
e5f4c47cd5 test_assertion: minor cleanup 2019-08-17 21:38:52 +02:00
Daniel Hahler
05bfe73cf9 Fix wrong location of assertion error with coverage.py
Reverts using-constant part from 39ba99613.

Fixes https://github.com/pytest-dev/pytest/issues/5754.
2019-08-17 21:34:13 +02:00
Daniel Hahler
2a6a1ca07d Inject width via pylib to argparse formatter
`argparse.HelpFormatter` looks at `$COLUMNS` only, falling back to a
default of 80.

`py.io.get_terminal_width()` is smarter there, and could even work
better with https://github.com/pytest-dev/py/pull/219.

This ensures to use a consistent value for formatting the ini values etc.
2019-08-17 16:51:02 +02:00
Ran Benita
9f3bfe82cf Fix TypeError when importing pytest on Python 3.5.0 and 3.5.1
The typing module on these versions have these issues:

- `typing.Pattern` cannot appear in a Union since it is not considered a
  class.

- `@overload` is not supported in runtime. (On the other hand, mypy
  doesn't support putting it under `if False`, so we need some runtime
  hack).

Refs #5751.
2019-08-17 15:41:59 +03:00
Bruno Oliveira
c3a8e609f9 Merge pull request #5750 from nicoddemus/fix-gh-publish-notes
Forward $TRAVIS_REPO_SLUG for GH publish notes
2019-08-16 15:57:01 -03:00
Ran Benita
7259c453d6 Fix some check_untyped_defs = True mypy warnings 2019-08-16 10:41:57 +03:00
Ran Benita
28761c8da1 Have AssertionRewritingHook derive from importlib.abc.MetaPathFinder
This is nice for self-documentation, and is the type required by mypy
for adding to sys.meta_path.
2019-08-16 10:41:52 +03:00
Bruno Oliveira
d9c4e646c4 Forward TRAVIS_REPO_SLUG env var for GH publish notes
Fix #5749
2019-08-15 22:50:55 -03:00
Bruno Oliveira
8ccc0177c8 Release 5.1.0 (#5748)
Release 5.1.0
2019-08-15 22:35:59 -03:00
Bruno Oliveira
eaf7ce9a99 Preparing release version 5.1.0 2019-08-15 20:00:09 -04:00
Anthony Sottile
ac3056c5a2 Merge pull request #5747 from blueyed/coveragerc-report
.coveragerc: add report section
2019-08-15 16:52:43 -07:00
Bruno Oliveira
409d61b972 Merge remote-tracking branch 'upstream/master' into release-5.1.0 2019-08-15 19:46:46 -04:00
Daniel Hahler
6ead01aacd testing/python/fixtures.py: use NotImplementedError pattern 2019-08-16 01:14:19 +02:00
Daniel Hahler
4c37dca011 .coveragerc: add report section
This will allow for "raise NotImplementedError" to indicate code not to
be covered in tests etc.
2019-08-16 01:14:19 +02:00
Daniel Hahler
44c10dbd5f Merge pull request #5746 from blueyed/codecov-comment-off
ci: add codecov.yml to turn comments off
2019-08-16 01:13:19 +02:00
Bruno Oliveira
0f11a7a73d Merge master into features (#5744)
Merge master into features
2019-08-15 19:12:37 -03:00
Daniel Hahler
d50198a3ff ci: add codecov.yml to turn comments off
The only benefit for me is to get notified about finished builds, but
that might happen to early anyway.  Apart from that they are rather big
and distract from actual comments.
2019-08-15 23:54:03 +02:00
Bruno Oliveira
ac052a98ad reword a parametrize example sentence (#5745)
reword a parametrize example sentence
2019-08-15 14:00:30 -03:00
mei-li
f4a84a8dfd reword a parametrize example sentence 2019-08-15 18:12:18 +02:00
Bruno Oliveira
1049a38cee Fix wording as suggested in review of #5741 2019-08-15 10:05:42 -03:00
Bruno Oliveira
d7f082519a Merge remote-tracking branch 'upstream/master' into mm
Conflicts:
	src/_pytest/outcomes.py
2019-08-15 10:03:52 -03:00
Bruno Oliveira
2d613a03b3 Async result warn (#5742)
Async result warn
2019-08-15 09:47:56 -03:00
Bruno Oliveira
28c6c5bb71 check that tests that are partial staticmethods are supported (#5701)
check that tests that are partial staticmethods are supported
2019-08-15 09:12:01 -03:00
Thomas Grainger
6b9d729ed3 also warn on awaitable or async iterable test results 2019-08-15 12:30:44 +01:00
Bruno Oliveira
0ba774a7c3 warn for async generator functions (#5734)
warn for async generator functions
2019-08-15 08:17:12 -03:00
Bruno Oliveira
0a62c4ac04 Merge pull request #5729 from Stranger6667/issue-5115
Do not treat warnings as errors during ``pytest_configure``.
2019-08-15 08:05:15 -03:00
Thomas Grainger
137255816e Fix collection of staticmethods defined with functools.partial
Related to #5701
2019-08-15 08:04:05 -03:00
Thomas Grainger
2f1b192fe6 Issue a warning for async gen functions
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
2019-08-15 07:54:10 -03:00
dmitry.dygalo
7183335e62 Capture warnings during `pytest_configure`
Fix #5115
2019-08-15 07:50:27 -03:00
Bruno Oliveira
0822a1e53a Merge pull request #5740 from nicoddemus/use-repo-env-var
Use TRAVIS_REPO_SLUG instead of hard-coding pytest-dev/pytest
2019-08-14 21:46:57 -03:00
Bruno Oliveira
cb94fd31c8 Use TRAVIS_REPO_SLUG instead of hard-coding pytest-dev/pytest
I was doing final tests on the script today, and forgot to change
the hardecoded "pytest-dev/pytest", which ended up publishing a
`4.99.10` release to the main repository by mistake, as my token
has access to both my fork and main repository.

I deleted the tag immeditely just a few seconds later, so hopefully
this won't cause major problems.

This change makes it safer to test this in the future, never publishing
to the main repository by mistake (as long as the tags are pushed to the
right repositories of course).
2019-08-14 20:29:16 -03:00
Bruno Oliveira
fa75d818cf pytester: spawn: skip without pexpect.spawn (#5736)
pytester: spawn: skip without pexpect.spawn
2019-08-14 18:06:06 -03:00
Daniel Hahler
1434b66c35 pytester: spawn: skip without pexpect.spawn
"pexpect" can be imported on Windows, but does not have "pexpect.spawn"
then.

Ref: https://github.com/pexpect/pexpect/blob/a803933ed53/pexpect/__init__.py#L73-L76
2019-08-13 17:54:40 +02:00
Bruno Oliveira
a24132ddc5 Publish GitHub release notes after deployment (#5723)
Publish GitHub release notes after deployment
2019-08-12 13:20:16 -03:00
Daniel Hahler
ed2425119f Merge pull request #5731 from blueyed/assert-compare-maxsize
assertrepr_compare: prefer same maxsize
2019-08-12 16:02:41 +02:00
Bruno Oliveira
a295a3ddaf test_non_ascii_paste_text: mock call to urlopen (#5728)
test_non_ascii_paste_text: mock call to urlopen
2019-08-12 10:56:38 -03:00
Bruno Oliveira
3b3ce0e799 remove %s formatting from docs (#5733)
remove %s formatting from docs
2019-08-12 10:21:31 -03:00
Bruno Oliveira
1a61265b1e Publish GitHub release notes after deployment
Fix #2933
2019-08-12 10:18:43 -03:00
Ran Benita
67ac878ccf Merge pull request #5732 from blueyed/drop-py2
assertion.rewrite: use ast.NameConstant(None) directly
2019-08-12 12:19:40 +03:00
boris
3eb4973065 remove %s formatting from docs 2019-08-12 00:09:53 -06:00
Daniel Hahler
39ba996133 assertion.rewrite: use ast.NameConstant(None) directly
`ast.parse("None")` was added/used in 3e6f1fa2d for differences between
Python 2/3, but we do not support py2 anymore.
2019-08-12 07:50:52 +02:00
Daniel Hahler
cb481a354a assertrepr_compare: prefer same maxsize
Previously it would say:

    > assert '123456789012...901234567890A' == '1234567890123...901234567890B'"

This makes it look like the "3" might be different already.

This is clearer, and it is OK to have potentially one less char in the
right one:

    > assert '123456789012...901234567890A' == '123456789012...901234567890B'"
2019-08-12 03:41:14 +02:00
Bruno Oliveira
300f78556f Show session duration in human-readable format (#5721)
Show session duration in human-readable format
2019-08-10 23:05:34 -03:00
Daniel Hahler
0db9dade65 test_non_ascii_paste_text: mock call to urlopen
Likely to fix flaky coverage due to requests failing sometimes.
Ref: f7e81dab9a...83a1f4bd66/changes
2019-08-10 23:30:49 +02:00
Daniel Hahler
a77c83a4c3 Merge pull request #5726 from jdufresne/black
Update URL: python/black → psf/black
2019-08-10 22:25:38 +02:00
Jon Dufresne
0767f080a4 Update URL: python/black → psf/black 2019-08-10 12:38:13 -07:00
Bruno Oliveira
2498aeaaa5 Merge master into features (#5717)
Merge master into features
2019-08-10 11:16:40 -03:00
Bruno Oliveira
345df99db7 Show session duration in human-readable format
Fix #5707
2019-08-10 10:45:53 -03:00
Anthony Sottile
8665f5652a Merge pull request #5722 from nicoddemus/improvements-changelog
Add new 'improvement' changelog category
2019-08-10 06:31:39 -07:00
Bruno Oliveira
beb457c75e Add new 'improvement' changelog category
This creates a separate section from 'features' for small changes which
don't usually require user intervention, such as:

* Human readable session duration
* New junitxml fields
* Improved colors in terminal
* etc.

The idea is to better match user expectations about new actual
features in the "Features" section of the changelog.
2019-08-10 09:33:51 -03:00
Bruno Oliveira
f7e81dab9a 5669: pytester: add docstrings for Testdir.copy_example (#5719)
5669: pytester: add docstrings for Testdir.copy_example
2019-08-10 09:00:42 -03:00
Ilya Stepin
ee936b27a8 pytester: fix docstrings
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
2019-08-10 09:12:04 +03:00
martbln
e0ce8b79d5 pytester: add docstrings for Testdir.copy_example 2019-08-10 00:59:14 +03:00
Bruno Oliveira
8ffa3aa65d Improve docs of pytest.importorskip (#5718)
Improve docs of pytest.importorskip
2019-08-09 18:28:44 -03:00
Tim Hoffmann
b095e0de47 Improve docs of pytest.importorskip 2019-08-09 21:35:03 +02:00
Bruno Oliveira
2f065a555f Merge remote-tracking branch 'upstream/master' into release-5.1.0 2019-08-09 12:36:19 -03:00
Bruno Oliveira
ec76f70d71 Remove unused function CallSpec2.setall (#5716)
Remove unused function CallSpec2.setall
2019-08-09 12:35:07 -03:00
Bruno Oliveira
65e6038111 Validate xunit2 files against the schema (#5605)
Validate xunit2 files against the schema
2019-08-09 12:34:21 -03:00
Daniel Hahler
9f6da8cbeb Remove unused function CallSpec2.setall
Fixes https://github.com/pytest-dev/pytest/issues/5253.
2019-08-09 16:34:48 +02:00
Bruno Oliveira
ba72b480b9 Remove xfail marker from test_runs_twice_xdist
Apperently it does not hang, let's see how it fares on CI
2019-08-09 10:55:06 -03:00
Bruno Oliveira
ba76080b59 Validate xunit2 files against the schema
Fix #5095
2019-08-09 10:55:06 -03:00
Bruno Oliveira
5ecac3c861 Add a code of conduct (#5711)
Add a code of conduct
2019-08-08 12:37:14 -03:00
Florian Bruhin
0a19439677 Add a code of conduct
See #5709 and the mails linked from there
2019-08-08 14:01:29 +02:00
Bruno Oliveira
e772bb6480 trivial doc change (#5708)
trivial doc change
2019-08-08 08:10:36 -03:00
boris
7560a7b808 trivial doc change 2019-08-07 21:49:30 -06:00
Bruno Oliveira
6c8d8a99f4 pyupgrade docs to Python 3 (#5705)
pyupgrade docs to Python 3
2019-08-07 07:27:34 -03:00
boris
78de9d4677 replace implicit code tag with colon .replace("::\n\n.. code-block", ":\n\n.. code-block") 2019-08-06 16:20:06 -07:00
boris
23a0b532db Revert "remove implicit code tags .replace("::\n\n.. code-block", "\n\n.. code-block")"
This reverts commit 9fce0bdd88.
2019-08-06 16:18:06 -07:00
boris
9fce0bdd88 remove implicit code tags .replace("::\n\n.. code-block", "\n\n.. code-block") 2019-08-06 15:46:57 -07:00
Bruno Oliveira
28fabc52bd fix grammar in test_collect_functools_partial docstr (#5703)
fix grammar in test_collect_functools_partial docstr
2019-08-06 19:24:24 -03:00
boris
5f95dce956 ran blacken-docs 2019-08-06 13:34:58 -07:00
boris
75d0b899bb ran pyupgrade-docs again 2019-08-06 13:33:21 -07:00
boris
7f90e74e02 label code blocks 2019-08-06 13:25:54 -07:00
boris
cf7761f91f ran pyupgrade-docs 2019-08-06 12:40:27 -07:00
Thomas Grainger
ef0915e1db fix grammar in test_collect_functools_partial docstr 2019-08-06 15:27:36 +01:00
Bruno Oliveira
68c486a25f Add CHANGELOG for 4.6.5 (#5696) (#5699)
Add CHANGELOG for 4.6.5 (#5696)
2019-08-05 16:25:24 -03:00
Bruno Oliveira
f5fab2bfa1 Merge master into features (#5698)
Merge master into features
2019-08-05 15:20:54 -03:00
Bruno Oliveira
f43b54aaeb Add CHANGELOG for 4.6.5 (#5696) 2019-08-05 15:09:31 -03:00
Daniel Hahler
aa06e6c8f3 Merge master into features 2019-08-05 19:12:32 +02:00
Daniel Hahler
bf3aa72a29 Merge pull request #5697 from blueyed/fix-collect-pkg-init
Fix RuntimeError when trying to collect package with "__init__.py" only
2019-08-05 18:20:28 +02:00
Daniel Hahler
198fcd8a6f Fix RuntimeError when trying to collect package with "__init__.py" only
Fixes https://github.com/pytest-dev/pytest/issues/4344.
2019-08-05 17:52:21 +02:00
Bruno Oliveira
f7e925dcc1 Add hostname and timestamp to JUnit XML testsuite tag (#5692)
Add hostname and timestamp to JUnit XML testsuite tag
2019-08-05 10:19:15 -03:00
Bruno Oliveira
cf6632a57a Rename 5471.trivial.rst to 5471.feature.rst 2019-08-05 10:19:05 -03:00
Anthony Sottile
1a66a503b6 Merge pull request #5690 from nicoddemus/custom-exit-code-docs
Mention pytest-custom_exit_code in the docs
2019-08-04 19:10:36 -07:00
Anthony Sottile
82763a293a Merge pull request #5684 from nicoddemus/errno-nomore
Use OSError subclasses instead of handling errno
2019-08-04 14:50:12 -07:00
Anthony Sottile
73e0bf92f7 Merge pull request #5694 from yashtodi94/public_access_to_exitcode_doc_change
Added info to access exitcodes directly
2019-08-04 14:48:23 -07:00
Yash Todi
0d5ed57b40 Added info to access exitcodes directly 2019-08-04 22:35:51 +05:30
Samuel Searles-Bryant
1ce45a6f67 Add hostname and timestamp to JUnit XML testsuite tag
Fix #5471
2019-08-03 15:54:39 +01:00
Bruno Oliveira
8db6642515 Mention pytest-custom_exit_code in the docs
Related to #5689
2019-08-03 10:15:56 -03:00
Daniel Hahler
29e336bd9b Merge pull request #5687 from blueyed/merge-master
Merge master into features
2019-08-03 14:12:02 +02:00
Bruno Oliveira
d3e1907899 Use OSError subclasses instead of handling errno 2019-08-02 14:17:46 -03:00
Daniel Hahler
b5b710b3ae Merge master into features
Several conflicts, mostly due to 2c402f4bd.

Conflicts:
	.pre-commit-config.yaml
	src/_pytest/outcomes.py
	src/_pytest/python_api.py
	tox.ini
2019-08-02 16:52:51 +02:00
Bruno Oliveira
0d3958e8de Handle only known functions in rm_rf (#5627)
Handle only known functions in rm_rf
2019-08-02 10:23:12 -03:00
Daniel Hahler
9064eea216 Improve rm_rf to handle only known functions
Warnings are emitted if we cannot safely remove
paths.

Fix #5626
2019-08-02 08:18:11 -03:00
Bruno Oliveira
dc6e7b9fcf Cache splitnode results to improve tests collection time (#5681)
Cache splitnode results to improve tests collection time
2019-08-01 22:32:19 -03:00
Alexey Zankevich
b43ebb7d65 Cache split nodes results to reduce long tests collection time on large test suites 2019-08-01 20:46:27 -04:00
Anthony Sottile
8a5e72c936 Merge pull request #5682 from nicoddemus/disable-shallow-copy
Disable shallow cloning because of setuptools-scm
2019-08-01 09:28:26 -07:00
Bruno Oliveira
cd924b66ca Disable shallow cloning because of setuptools-scm
setuptools-scm needs all tags to guess the version correctly
2019-08-01 12:29:39 -03:00
Anthony Sottile
1076a7e61d Merge pull request #5680 from ss18/ss18/pr0
Fix some typos
2019-08-01 08:18:48 -07:00
Bruno Oliveira
cff58457dd Fix the confusing different names of fulltrace (#5668)
Fix the confusing different names of `fulltrace`
2019-08-01 10:30:17 -03:00
Semen Zhydenko
b3f4398d64 surpressing -> suppressing 2019-08-01 15:11:38 +02:00
Semen Zhydenko
2ca47cb3f5 programatically -> programmatically 2019-08-01 15:11:26 +02:00
Semen Zhydenko
d19fe3c410 didnt -> didn't 2019-08-01 15:10:39 +02:00
Semen Zhydenko
2de145f372 wasnt -> wasn't 2019-08-01 15:10:06 +02:00
Semen Zhydenko
942fd91995 shouldnt -> shouldn't 2019-08-01 15:09:14 +02:00
Florian Bruhin
cb15e7c700 Merge pull request #5674 from helloocc/master
Doc fix: remove redundant parentheses.
2019-07-30 10:46:29 +02:00
Xixi Zhao
2959fb3198 Doc fix: remove redundant parentheses. 2019-07-30 16:18:22 +08:00
Kaiqi
aa13c625da Change the warning message 2019-07-27 21:06:29 +02:00
Kaiqi
958374addb Remove name from author 2019-07-27 17:26:52 +02:00
Kaiqi
6c2f673daf Have same name for fulltrace 2019-07-27 17:25:23 +02:00
Anthony Sottile
fd2fb36eac Merge pull request #5665 from hroncok/i5664
In test_xfail_handling, only remove __pycache__ if it exists
2019-07-25 18:35:35 -07:00
Miro Hrončok
ab39502c98 In test_xfail_handling, only remove __pycache__ if it exists
Previously, the test failed when the directory was not present,
which could have been caused for example by invoking the tests
with PYTHONDONTWRITEBYTECODE=1.

Fixes https://github.com/pytest-dev/pytest/issues/5664
2019-07-26 02:23:14 +02:00
Bruno Oliveira
50c7b5d2b5 fix setenv prepend docs (#5658)
fix setenv prepend docs
2019-07-24 20:11:05 -03:00
Thomas Grainger
3bdcd6b4f3 Update doc/en/monkeypatch.rst 2019-07-24 20:49:31 +01:00
Bruno Oliveira
d89c88478d Merge pull request #5657 from helloocc/master
Doc fix: delete useless.
2019-07-24 15:22:58 -03:00
Thomas Grainger
635916052c fix setenv prepend docs 2019-07-24 17:23:48 +01:00
Bruno Oliveira
bd34bd872a fix docs about syspath_prepend (#5656)
fix docs about syspath_prepend
2019-07-24 13:19:05 -03:00
helloocc
1d8f668e10 Add author. 2019-07-24 23:44:37 +08:00
helloocc
28343bdcbd Merge remote-tracking branch 'upstream/master' 2019-07-24 23:35:22 +08:00
Thomas Grainger
ebfe8eabf5 Update doc/en/monkeypatch.rst 2019-07-24 14:27:51 +01:00
Thomas Grainger
829cc5a242 fix docs about syspath_prepend
eg it does not modify $PATH
2019-07-24 14:27:05 +01:00
Bruno Oliveira
7fa27485df Merge pull request #5648 from nicoddemus/improve-exc-info-changelog
Make ExceptionInfo.repr change more prominent
2019-07-23 21:24:48 -03:00
Bruno Oliveira
b2839c4084 Merge pull request #5646 from nicoddemus/mark-info-docs
Update MarkDecorator docs
2019-07-23 16:42:11 -03:00
Bruno Oliveira
880e368607 Merge pull request #5653 from blueyed/fix-test
Followup: unittest: handle outcomes.Exit
2019-07-23 15:09:24 -03:00
Daniel Hahler
b9111fe677 Fix changelog 2019-07-23 19:38:33 +02:00
Daniel Hahler
401c3d1109 tests: unittest: fix/harden "test_exit_outcome"
Ref: https://github.com/pytest-dev/pytest/pull/5634#pullrequestreview-265565917
2019-07-23 19:35:51 +02:00
Bruno Oliveira
2a724a1c81 Fix fixture example and regendocs (#5651)
Fix fixture example and regendocs
2019-07-23 11:53:13 -03:00
Bruno Oliveira
52ad5a1591 Improve output when parsing an ini configuration fails (#5650)
Improve output when parsing an ini configuration fails
2019-07-23 11:17:42 -03:00
Bruno Oliveira
4abf95ba4f Merge pull request #5644 from blueyed/rm-_orig_args
Replace internal config._origargs with invocation_params.args
2019-07-23 11:03:52 -03:00
Bruno Oliveira
f163b37f6a Run regen 2019-07-23 10:00:35 -04:00
Bruno Oliveira
a82dd2f064 Fix linting 2019-07-23 10:55:22 -03:00
Bruno Oliveira
1873dc6a8a Fix fixture example in docs 2019-07-23 09:48:22 -04:00
Florian Bruhin
8c47db724c Improve output when parsing an ini configuration fails 2019-07-23 15:27:27 +02:00
Bruno Oliveira
693e9d0733 Merge pull request #5634 from blueyed/fix-unittest-exit
unittest: handle outcomes.Exit
2019-07-23 10:08:43 -03:00
Bruno Oliveira
5c09cc16f2 Add changelog entry for #5634 2019-07-23 08:55:14 -03:00
Bruno Oliveira
0824789459 Improve test for pytest.exit handling 2019-07-23 08:53:38 -03:00
Bruno Oliveira
ec4ca59bf0 Merge pull request #5631 from RonnyPfannschmidt/fix-5606
use identity checks for the mock sentinels
2019-07-23 08:48:58 -03:00
Bruno Oliveira
3c94f32e77 Make ExceptionInfo.repr change more prominent
Related to #5579
2019-07-23 08:42:03 -03:00
Bruno Oliveira
d66b6c8371 Update MarkDecorator docs
The MarkInfo class no longer exists
2019-07-23 08:07:38 -03:00
Xixi Zhao
6e687c4354 Doc fix: m1 isn't a dependency of f1.
According to the file in `example/fixtures/test_fixtures_order.py`, m1 isn't a dependency of f1.
2019-07-23 15:09:32 +08:00
William Woodall
13d8577451 README: fix typo
Closes https://github.com/pytest-dev/pytest/pull/5643.
2019-07-23 06:50:51 +02:00
Daniel Hahler
3c7438969a Replace internal config._origargs with invocation_params.args
Added in https://github.com/pytest-dev/pytest/pull/5564.
2019-07-23 06:46:56 +02:00
Xixi Zhao
66cfc66d63 Update index.rst 2019-07-22 16:01:56 +08:00
Anthony Sottile
13c4b7d212 Merge pull request #5636 from asottile/fixup_sysmodules_test
Fix ordering of sys modules snapshot
2019-07-20 12:30:18 -07:00
Ronny Pfannschmidt
8c7d9124ba switch num_mock_patch_args to do identity checks for the sentinels 2019-07-20 20:21:27 +02:00
Bruno Oliveira
240d314f36 copy test and changelog from #5607 by @niccodemus 2019-07-20 20:21:27 +02:00
Anthony Sottile
27c9d80a7e Fix ordering of sys modules snapshot 2019-07-20 11:17:30 -07:00
Bruno Oliveira
cb828ebe70 tox.ini: clean up filterwarnings (#5633)
tox.ini: clean up filterwarnings
2019-07-20 11:59:23 -03:00
Daniel Hahler
d35d09f82d unittest: handle outcomes.Exit
This is required for pytest to stop when using "quit" in pdb, and should
get honored/handled in general.
2019-07-20 07:11:49 +02:00
Daniel Hahler
f0feb6c83a fixup! tox.ini: clean up filterwarnings 2019-07-20 01:54:04 +02:00
Daniel Hahler
dcbb9c1f5a tox.ini: clean up filterwarnings
- path.local/path.readlines is not used anymore
- enhance doc for "invalid escape sequence" filter
2019-07-20 01:24:04 +02:00
Anthony Sottile
c05fcc8641 Merge pull request #5615 from bluetech/type-annotations-2
Improve mypy setup + a few minor type fixes and removals
2019-07-19 06:48:24 -07:00
Florian Bruhin
c98e7aed94 Merge pull request #5628 from pytest-dev/leipzig-training-info-1
Add info about open training in Leipzig
2019-07-19 14:57:47 +02:00
Oliver Bestwalter
c0231ae780 Add info about open training in Leipzig 2019-07-18 21:06:53 +02:00
Ran Benita
675e9507d8 Don't accept bytes message in pytest.{fail,xfail,skip}
It seems to have been added in #1439 to fix #1178.

This was only relevant for Python 2 where it was tempting to use str (==
bytes) literals instead of unicode literals. In Python 3, it is unlikely
that anyone passes bytes to these functions.
2019-07-17 22:39:10 +03:00
Ran Benita
0b532fda76 Remove unnecessary checks from SetupState
Since 4622c28ffd, _finalizers cannot
contain Nones or tuples, so these checks are not longer needed.
2019-07-17 22:39:10 +03:00
Ran Benita
7d1c697c30 Remove a no-longer-needed check if enum is available
Not needed since 4d49ba6529.
2019-07-17 22:39:10 +03:00
Ran Benita
65aee1e0c8 Allow bytes for OutcomeException(msg=...)
It's __repr__ explicitly handles it so allow it.
2019-07-17 22:39:10 +03:00
Ran Benita
8d413c1926 Allow tuple of exceptions in ExceptionInfo.errisinstance
isinstance() accepts it and some code does pass a tuple.

Fixup for commit 55a570e513.
2019-07-17 22:39:10 +03:00
Ran Benita
104f8fc836 Update mypy from 0.711 to 0.720
Release notes: http://mypy-lang.blogspot.com/2019/07/mypy-0720-released.html
2019-07-17 22:39:10 +03:00
Bruno Oliveira
157515f3c5 Fix/Remove broken links (#5613)
Fix/Remove broken links
2019-07-17 13:52:05 -03:00
Bruno Oliveira
183750fa86 CSS changes to the API reference (#5609)
CSS changes to the API reference
2019-07-17 06:24:31 -03:00
Ran Benita
fd8f92d0e7 Run mypy on src/ and testing/ together
This makes testing/ actually pick up the pytest imports -- otherwise
they are opaque and we don't actually test the types.

A single run is also a bit faster and simpler. The original reason why
we split it is no longer relevant (we fixed the problems).
2019-07-16 23:29:02 +03:00
Steffen Schroeder
1e34734f8f Fixed link to Python Classifiers 2019-07-16 21:56:04 +02:00
Steffen Schroeder
7c2e843358 Fixed link to correct anchor of Python's -w postion 2019-07-16 21:56:04 +02:00
Steffen Schroeder
79414164c2 Fixed broken links in adopt.rst 2019-07-16 21:56:03 +02:00
Steffen Schroeder
0cca7f831a Fixed broken links and non-existing anchors in release announcements 2019-07-16 21:56:03 +02:00
Steffen Schroeder
1f4ae789b8 Change link to "Where do Google's flaky tests come from?"
Although it's available, linkchecker complained about it.
It was changed to the permanent redirect.
2019-07-16 21:56:03 +02:00
Steffen Schroeder
3f46315a9d Remove link to non-existing anchor 2019-07-16 21:56:03 +02:00
Steffen Schroeder
e03b8b9e95 Update/Fix links to using projects 2019-07-16 21:56:03 +02:00
Steffen Schroeder
eb7f950e20 Fixed broken link to virtualenv 2019-07-16 21:56:03 +02:00
Steffen Schroeder
848b735a06 Fixed links to non existing GitHub users and py 2019-07-16 21:56:03 +02:00
Steffen Schroeder
7337cce332 Fixed links were content was from 'writing_plugins' to 'reference' 2019-07-16 21:56:02 +02:00
Steffen Schroeder
7a5c0a01bc Fixed links were content was moved to 'historical' 2019-07-16 21:56:02 +02:00
Ran Benita
faf222f8fb Merge pull request #5593 from bluetech/type-annotations-1
Type-annotate pytest.{exit,skip,fail,xfail,importorskip,warns,raises}
2019-07-16 22:38:20 +03:00
Ran Benita
9258fd1296 Merge pull request #5603 from bluetech/saferepr-simplify
Simplify SafeRepr a bit
2019-07-16 22:37:02 +03:00
Thomas Grainger
7440cece59 Merge pull request #5589 from graingert/fixture-yield-exit-stack
de-emphasize request.addfinalizer Fixes #5587
2019-07-16 18:33:59 +01:00
Anthony Sottile
f1332872a6 Merge pull request #5611 from mxr/patch-1
Remove language_version specification in black hooks
2019-07-16 10:15:28 -07:00
Bruno Oliveira
ac12245f5f Merge pull request #5612 from The-Compiler/new-trainings-2
Remove Europython training
2019-07-16 14:00:04 -03:00
Florian Bruhin
d9eab12ee0 Remove Europython training 2019-07-16 15:03:54 +02:00
Max R
2e756d698b Remove language_version specification in black hooks
Resolves #3840
2019-07-16 08:32:24 -04:00
Bruno Oliveira
15e235c63e Remove obsolete "importorskip('unittest.mock')" calls (#5608)
Remove obsolete "importorskip('unittest.mock')" calls
2019-07-16 04:41:58 -03:00
Bruno Oliveira
65c23017c7 Update test_getfuncargnames_patching to work with modern mock 2019-07-15 12:23:59 -03:00
Tim Hoffmann
bb7608c56f Change section titles in docs 2019-07-15 16:25:33 +02:00
Bruno Oliveira
8a3f40996a Remove obsolete "importorskip('unittest.mock')" calls 2019-07-15 11:18:05 -03:00
Tim Hoffmann
6f1d358a0c Fix some ReST indentation issues in docstrings 2019-07-15 16:16:44 +02:00
Tim Hoffmann
3ad315bcee Improve CSS layout of API reference 2019-07-15 16:15:59 +02:00
Ran Benita
129600d698 saferepr: Avoid indirect function calls
The DRY savings they provide are rather small, while they make it harder
to type-check, and IMO harder to understand.
2019-07-15 17:07:58 +03:00
Thomas Grainger
a96710dd8a demonstrate ExitStack assuming connect returns a context manager 2019-07-15 14:05:19 +01:00
Bruno Oliveira
54e08b729f More CSS fine-tuning (#5604)
More CSS fine-tuning
2019-07-15 09:42:18 -03:00
Tim Hoffmann
01606315aa More CSS fine-tuning 2019-07-15 13:40:30 +02:00
Bruno Oliveira
8f2f3bb1fa Improve CSS layout (#5602)
Improve CSS layout
2019-07-14 19:02:58 -03:00
Bruno Oliveira
499fda2349 Various cleanups in src/_pytest/python.py (#5599)
Various cleanups in src/_pytest/python.py
2019-07-14 19:00:35 -03:00
Ran Benita
c7aacc96bb saferepr: Remove unused setting of max_other
max_other is used by the superclass repr_instance, but we override it
and use maxsize instead.
2019-07-14 22:50:14 +03:00
Ran Benita
0394ebffee saferepr: Use an __init__ instead of setting attributes after construction
This will be easier to type-check, and also somewhat clearer.
2019-07-14 22:50:14 +03:00
Ran Benita
0225be53a2 saferepr: Remove dead SafeRepr.repr_unicode
This function is not called anywhere directly, and cannot be called by
the dynamic `repr_<type>()` dispatch mechanism because unicode is no
longer a type in Python 3.
2019-07-14 22:13:57 +03:00
Tim Hoffmann
da5add1294 Improve CSS layout 2019-07-14 18:58:07 +02:00
Ran Benita
11f1f79222 Allow creating ExceptionInfo from existing exc_info for better typing
This way the ExceptionInfo generic parameter can be inferred from the
passed-in exc_info. See for example the replaced cast().
2019-07-14 14:28:24 +03:00
Ran Benita
3f1fb62584 Rework ExceptionInfo to not require manual __init__ call
Mypy doesn't like calling __init__() in this way.
2019-07-14 14:28:24 +03:00
Ran Benita
14bf4cdf44 Make ExceptionInfo generic in the exception type
This way, in

    with pytest.raises(ValueError) as cm:
        ...

cm.value is a ValueError and not a BaseException.
2019-07-14 14:28:24 +03:00
Ran Benita
56dcc9e1f8 Type-annotate pytest.raises 2019-07-14 14:28:24 +03:00
Ran Benita
55a570e513 Type-annotate ExceptionInfo 2019-07-14 14:28:24 +03:00
Ran Benita
2dca68b863 Type-annotate pytest.warns 2019-07-14 14:28:24 +03:00
Ran Benita
d7ee3dac2c Type-annotate pytest.{exit,skip,fail,xfail,importorskip} 2019-07-14 14:28:24 +03:00
Ran Benita
866904ab80 Revert "Let context-managers for raises and warns handle unknown keyword arguments"
This reverts commit dfe54cd82f.

The idea in the commit was to simplify the code by removing the check
and instead letting it TypeError which has the same effect.

However this type error is caught by mypy, and rather than ignoring the
error we think it's better and clearer to go back to the previous
explicit check.
2019-07-14 14:28:21 +03:00
Ran Benita
35a57a0dfb Use flake8's extend-ignore instead of ignore
extend-ignore adds ignores in addition to flake8's existing ignores.

The default ignores currently are:
E121,E123,E126,E226,E24,E704,W503,W504
2019-07-14 11:12:47 +03:00
Florian Bruhin
6afbac29a3 Merge pull request #5600 from wojtekerbetowski/issues/3404
Add autouse fixture order information (#3404).
2019-07-13 03:22:15 -07:00
Wojtek Erbetowski
d23fbab188 Add autouse fixture order information (#3404).
A case with a fixture use both as an autouse and explititly
was raised. This case sounds too narrow to add to documentation
(and could be misleading for people learning pytest with explicitely
using an autouse fixture). At the same time there was no documentation
on the autouse vs regular fixture order, therefore this commit adds
such an information. Code sample grew and it was extracted to the file.
2019-07-13 11:43:47 +02:00
Ronny Pfannschmidt
374c4325a8 refactor finding xunit setup/teardown functions
s/_get_non_fixture_func(obj, name: str)/_get_first_non_fixture_func(obj, names: List[str])/
2019-07-13 08:13:46 +02:00
Ronny Pfannschmidt
2c071a060e refactor resolve_arg_value_types
* more explicit type checks
* expand from list+tuple to sequence
2019-07-13 08:13:46 +02:00
Bruno Oliveira
0f8b462677 Include <testsuites> root tag in generated XML (#5550)
Include <testsuites> root tag in generated XML
2019-07-12 14:35:34 -03:00
Ronny Pfannschmidt
898028cb22 remove unused _get_xunit_setup_teardown 2019-07-12 17:45:23 +02:00
Ronny Pfannschmidt
4480d3e518 generate_tests: always use call_extra
- its a noop addition in the methods is empty list
2019-07-12 17:44:17 +02:00
Bruno Oliveira
4868d0d97a Drop unittest2 support (#5565)
Drop unittest2 support
2019-07-12 11:13:51 -03:00
Bruno Oliveira
8052c7c5c6 Fix typos in a docstring (#5598)
Fix typos in a docstring
2019-07-12 07:37:50 -03:00
Ronny Pfannschmidt
13d750db20 simplify the expression which transfers mark names to keywords 2019-07-12 06:16:06 +02:00
Ronny Pfannschmidt
e98627223f remove the noop init of PyobjMixin 2019-07-12 06:14:36 +02:00
Albert Tugushev
c89e379f49 Fix typos 2019-07-12 02:35:44 +03:00
Bruno Oliveira
24a66db8d3 Remove support code for unittest 2
Also moved a pytest_runtest_makereport hook implemented in
nose.py, but nowadays makes more sense to be implemented in
unittest.py
2019-07-11 20:11:06 -03:00
Bruno Oliveira
4027254a4b Fix rmtree to remove directories with read-only files (#5588)
Fix rmtree to remove directories with read-only files
2019-07-11 18:57:03 -03:00
Bruno Oliveira
37c37963c4 Fix rmtree to remove directories with read-only files
Fix #5524
2019-07-11 18:24:53 -03:00
Bruno Oliveira
666acc9b7a doctest: Add +NUMBER option to ignore irrelevant floating-point… (#5576)
doctest: Add +NUMBER option to ignore irrelevant floating-point differences
2019-07-11 07:25:37 -03:00
David Röthlisberger
a740ef2036 docs: Document doctest +NUMBER limitation with strings
Also added an "xfail" testcase for the same.
2019-07-11 10:06:57 +01:00
David Röthlisberger
4c590e002f Fix test_doctest.test_number_non_matches
These doctests were expected to fail, but they were failing because of a
silly bug (I forgot to replace "{expression}" with the actual expression
to be tested), not because of the thing they were meant to be testing.

Then I had to fix one of the testcases because it was actually matching:

    >>> 3.0 #doctest: +NUMBER
    2.99

The doctest is saying that the actual output should match to 2 decimal
places, i.e. within 0.01 -- which it is, so it passes. I changed the
expected output to 2.98 and now it doesn't match (as we expect).
2019-07-11 09:57:44 +01:00
Anthony Sottile
26b06bdb45 Merge pull request #5595 from blueyed/typos
doc: fix typos in OPENCOLLECTIVE.rst  [ci skip]
2019-07-10 14:01:16 -07:00
Daniel Hahler
d5f1d7aebe doc: fix typos in OPENCOLLECTIVE.rst [ci skip] 2019-07-10 20:15:22 +02:00
Ran Benita
f7747f5dd6 Remove references to old-style classes in a couple error messages
These don't exist in Python 3.
2019-07-10 20:25:45 +03:00
Thomas Grainger
c224c4f1d6 de-emphasize request.addfinalizer 2019-07-10 14:52:40 +01:00
Bruno Oliveira
2c402f4bd9 [cherry-pick to master] Add rudimentary mypy type checking (#5583)
[cherry-pick to master] Add rudimentary mypy type checking
2019-07-10 07:52:32 -03:00
Bruno Oliveira
602cd5e21f Introduce Config.invocation_params (#5564)
Introduce Config.invocation_params
2019-07-10 07:46:34 -03:00
Bruno Oliveira
57e5bd0664 Improve type-checking in OutcomeException (#5580)
Improve type-checking in OutcomeException
2019-07-09 19:12:05 -03:00
Bruno Oliveira
31738155b5 Remove deprecated features (#5529)
Remove deprecated features
2019-07-09 19:04:06 -03:00
Bruno Oliveira
9db1823707 Improve type-checking in OutcomeException
Fix #5578
2019-07-09 18:26:57 -03:00
Ran Benita
89dfde9535 Add rudimentary mypy type checking
Add a very lax mypy configuration, add it to tox -e linting, and
fix/ignore the few errors that come up. The idea is to get it running
before diving in too much.

This enables:

- Progressively adding type annotations and enabling more strict
  options, which will improve the codebase (IMO).

- Annotating the public API in-line, and eventually exposing it to
  library users who use type checkers (with a py.typed file).

Though, none of this is done yet.

Refs https://github.com/pytest-dev/pytest/issues/3342.
2019-07-09 12:12:07 -07:00
Anthony Sottile
39a43dbae1 Merge pull request #5575 from bluetech/mypy-initial
Add rudimentary mypy type checking
2019-07-09 08:05:14 -07:00
Ran Benita
c1167ac552 Add rudimentary mypy type checking
Add a very lax mypy configuration, add it to tox -e linting, and
fix/ignore the few errors that come up. The idea is to get it running
before diving in too much.

This enables:

- Progressively adding type annotations and enabling more strict
  options, which will improve the codebase (IMO).

- Annotating the public API in-line, and eventually exposing it to
  library users who use type checkers (with a py.typed file).

Though, none of this is done yet.

Refs https://github.com/pytest-dev/pytest/issues/3342.
2019-07-09 10:49:17 +03:00
Bruno Oliveira
628ff4d619 Improve docs sidebar layout (#5568)
Improve docs sidebar layout
2019-07-08 19:24:48 -03:00
Bruno Oliveira
add912ff68 Update doc footer/copyright (#5569)
Update doc footer/copyright
2019-07-08 19:21:38 -03:00
Bruno Oliveira
b77c876481 Merge pull request #5563 from nicoddemus/sort-fix
Sort parametrize params to test_external_plugins_integrated
2019-07-08 19:10:45 -03:00
Bruno Oliveira
9b78a216a2 Remove unused import 2019-07-08 19:08:09 -03:00
Bruno Oliveira
7aff51ee83 Show warning about importing abcs from collections once
This is being raised by `unittest2.compatibility`:

DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
2019-07-08 17:30:15 -03:00
Bruno Oliveira
e33736c791 Merge master into features (#5559)
Merge master into features
2019-07-08 17:26:49 -03:00
David Röthlisberger
d5cc0f2a62 changelog for new NUMBERS doctest option 2019-07-08 17:16:35 +01:00
David Röthlisberger
2a23fdab9f docs: Tidy up doctest options section
* Move the parts about "how to configure it" (pytest.ini vs. inline
  comment) together.
* Move `--doctest-continue-on-failure` into its own sub-heading, as it
  isn't related to the doctest optionflags.
2019-07-08 17:07:58 +01:00
David Röthlisberger
aaa7e837cc doctest: Add +NUMBER option to ignore irrelevant floating-point differences
When enabled, floating-point numbers only need to match as far as the
precision you have written in the expected doctest output. This avoids
false positives caused by limited floating-point precision, like this:

    Expected:
        0.233
    Got:
        0.23300000000000001

This is inspired by Sébastien Boisgérault's [numtest] but the
implementation is a bit different:

* This implementation edits the literals that are in the "got"
  string (the actual output from the expression being tested), and then
  proceeds to compare the strings literally. This is similar to pytest's
  existing ALLOW_UNICODE and ALLOW_BYTES implementation.

* This implementation only compares floats against floats, not ints
  against floats. That is, the following doctest will fail with pytest
  whereas it would pass with numtest:

      >>> math.py  # doctest: +NUMBER
      3

  This behaviour should be less surprising (less false negatives) when
  you enable NUMBER globally in pytest.ini.

Advantages of this implementation compared to numtest:

* Doesn't require `import numtest` at the top level of the file.
* Works with pytest (if you try to use pytest & numtest together, pytest
  raises "TypeError: unbound method check_output() must be called with
  NumTestOutputChecker instance as first argument (got
  LiteralsOutputChecker instance instead)").
* Works with Python 3.

[numtest]: https://github.com/boisgera/numtest
2019-07-08 17:07:58 +01:00
Tim Hoffmann
71c8ca7d3d Improve sidebar layout 2019-07-07 20:32:09 +02:00
Tim Hoffmann
56a0dd7658 Update doc footer/copyright 2019-07-07 20:23:56 +02:00
Bruno Oliveira
7a82285b03 Update CHANGELOG 2019-07-06 17:29:35 -03:00
Bruno Oliveira
4cda7093f6 Add note about PYTEST_ADDOPTS 2019-07-06 17:27:54 -03:00
Bruno Oliveira
6a9bf2852a Apply review suggestions: use a simple struct for invocation params 2019-07-06 12:22:19 -03:00
Bruno Oliveira
3e669a262a Introduce Config.invocation_args and Config.invocation_plugins
These attributes can be used to access the unchanged arguments passed
to pytest.main().

The intention is to use these attributes to initialize workers in
the same manner as the master node is initialized in pytest-xdist.
2019-07-05 19:45:23 -03:00
Bruno Oliveira
f1b8431d99 Sort parametrize params to test_external_plugins_integrated
This might cause problems during collection with pytest-xdist; we
didn't see any so far mostly by luck I think.

Shame on me for letting that slip in.
2019-07-05 19:05:55 -03:00
Bruno Oliveira
a43ba78d3b Include <testsuites> root tag in generated XML
Fix #5477
2019-07-05 11:29:35 -03:00
Bruno Oliveira
2180d9ef6d Fix comment in stepwise (follow up to #5555) [skip ci] (#5560)
Fix comment in stepwise (follow up to #5555) [skip ci]
2019-07-05 10:26:37 -03:00
Bruno Oliveira
0371a3023a Fix comment in stepwise (follow up to #5555) [skip ci] 2019-07-05 09:57:10 -03:00
Bruno Oliveira
d7588b8d40 Merge remote-tracking branch 'upstream/master' into merge-master-into-features 2019-07-05 08:14:18 -03:00
Bruno Oliveira
b62b549f5f Preparing release version 5.0.1 (#5557)
Preparing release version 5.0.1
2019-07-05 08:13:37 -03:00
Bruno Oliveira
60a358fa2d Replace importlib_metadata with importlib.metadata on Python 3.… (#5539)
Replace importlib_metadata with importlib.metadata on Python 3.8+
2019-07-04 21:46:43 -03:00
Bruno Oliveira
bb29f31d22 Preparing release version 5.0.1 2019-07-04 20:01:16 -04:00
Bruno Oliveira
b1928f878d Handle xfail(strict=True) properly in --step-wise mode (#5555)
Handle xfail(strict=True) properly in --step-wise mode
2019-07-04 20:50:33 -03:00
Bruno Oliveira
8651d880a0 Handle xfail(strict=True) properly in --step-wise mode
Fix #5547
2019-07-04 20:28:37 -03:00
Bruno Oliveira
95824c588a Improve quoting in raises match failure message (#5553)
Improve quoting in raises match failure message
2019-07-04 10:25:27 -03:00
Miro Hrončok
c54cbd63c8 Replace importlib_metadata with importlib.metadata on Python 3.8+
Fixes https://github.com/pytest-dev/pytest/issues/5537
2019-07-04 15:00:10 +02:00
Bruno Oliveira
1db132290f Apply workaround for multiple short options for Python <= 3.8
Hopefully by Python 3.9 this will be fixed upstream, if not we will
need to bump the version again.

Fix #5523
2019-07-04 14:59:40 +02:00
Anthony Sottile
caa08ebd45 Improve quoting in raises match failure message 2019-07-04 05:55:26 -07:00
Bruno Oliveira
b08ae4449b Add docs about Tidelift and OpenCollective (#5527)
Add docs about Tidelift and OpenCollective
2019-07-04 07:38:17 -03:00
Anthony Sottile
776a632170 Merge pull request #5540 from bazcrown/yml2yaml
Substituted 'yml' to '.yaml' in relevant files
2019-07-03 18:47:26 -04:00
Bruno Oliveira
d74a975f82 remove documentation about setuptools integration Refs #5534 Fi… (#5546)
remove documentation about setuptools integration Refs #5534 Fixes #3814
2019-07-03 16:01:30 -03:00
Bruno Oliveira
dfe54cd82f Let context-managers for raises and warns handle unknown keyword arguments
As suggested during review
2019-07-03 13:57:28 -03:00
Thomas Grainger
2b9522e9da remove documentation about setuptools integration Refs #5534
integrating with `python setup.py test` is not good practice!
2019-07-03 16:42:53 +01:00
Bruno Oliveira
528ee3e1c5 Add upcoming trainings (#5544)
Add upcoming trainings
2019-07-03 10:45:14 -03:00
Florian Bruhin
9677099acf Add upcoming trainings 2019-07-03 14:50:32 +02:00
Michael Moore
1b0e8d73d5 Substituted 'yml' to '.yaml' in relevant files 2019-07-02 23:49:06 -07:00
Bruno Oliveira
50b846e9d3 Merge pull request #5538 from avallbona/patch-2
Added talk about pytest in PyBCN June 2019
2019-07-02 19:46:10 -03:00
Andreu Vallbona Plazas
fa9791127a Added talk about pytest in PyBCN June 2019
Added talk about pytest in PyBCN June 2019
2019-07-02 23:18:52 +02:00
Bruno Oliveira
2d9b432613 Add docs about Tidelift and OpenCollective
Give an overview of how we as an organization will use the donated money,
as well as show this information more prominently in the docs.
2019-06-30 14:13:30 -03:00
Bruno Oliveira
5d8da88e9e Merge pull request #5530 from AmirElkess/patch-1
update CONTRIBUTING.rst
2019-06-30 14:07:27 -03:00
Bruno Oliveira
0da690c8e9 Update AUTHORS (#5531)
Update AUTHORS
2019-06-30 14:07:10 -03:00
AmirElkess
e32c903fb6 Update AUTHORS 2019-06-30 18:32:07 +02:00
Bruno Oliveira
0ed7aa2db6 Make 'request' a reserved name for fixtures 2019-06-30 13:31:39 -03:00
Bruno Oliveira
85cc12e328 Move FIXTURE_FUNCTION_CALL constant to the point of error
This is no longer a deprecation so it makes sense to move it
to the place where it is needed instead of leaving it in deprecated.py
2019-06-30 13:31:39 -03:00
Bruno Oliveira
c470ade0a5 Remove 'RemovedInPytest4Warning' 2019-06-30 13:31:39 -03:00
AmirElkess
9972c78cfa update CONTRIBUTING.rst
The following edits are for the "Preparing pull requests" section only.
1. remove Python2-related commands.
2. clarifying some details for beginners. Such as that pre-commit must to be installed as admin to function properly.
3. Added "You may not create a changelog entry if the change doesn't affect the documented behaviour of Pytest." on line 270.
2019-06-30 18:29:37 +02:00
Bruno Oliveira
3a17c1b30b Remove 'pytest_logwarning' hook 2019-06-30 13:18:07 -03:00
Bruno Oliveira
7b35405033 Remove YIELD_TESTS from deprecated module as it is now an error 2019-06-30 13:18:07 -03:00
Bruno Oliveira
aa1955de72 Remove 'tmpdir_factory.ensuretemp' 2019-06-30 13:18:07 -03:00
Bruno Oliveira
7e58defc15 Remove 'pytest.config' 2019-06-30 13:18:07 -03:00
Bruno Oliveira
f2b7809d5d Move setup.cfg error message and tests to an appropriate location
Those are not really deprecated anymore and are part of the normal
code for config
2019-06-30 13:18:07 -03:00
Bruno Oliveira
647d89c444 Move code about 'pytest_plugins' error to a more appropriate place
It is no longer deprecated, but part of the normal code for 'config'
2019-06-30 13:18:07 -03:00
Bruno Oliveira
683b2632b4 Remove explicit kwargs handling from raises, warns and ParameterSet.param 2019-06-30 13:18:06 -03:00
Bruno Oliveira
13f7f27fd2 Remove 'message' parameter from pytest.raises 2019-06-30 13:18:06 -03:00
Bruno Oliveira
279733a30b Remove support for 'code as string' from pytest.raises and pytest.warns 2019-06-30 11:40:24 -03:00
Bruno Oliveira
be91c4d932 Remove Request.getfuncargvalue 2019-06-30 11:02:46 -03:00
Bruno Oliveira
497cd87fdd Apply workaround for multiple short options for Python <= 3.8 (#5526)
Apply workaround for multiple short options for Python <= 3.8
2019-06-29 12:33:00 -03:00
Bruno Oliveira
94a05e513e Run py38-xdist as part of the build instead of cron 2019-06-29 11:19:50 -03:00
Bruno Oliveira
9021194efd Apply workaround for multiple short options for Python <= 3.8
Hopefully by Python 3.9 this will be fixed upstream, if not we will
need to bump the version again.

Fix #5523
2019-06-29 11:15:28 -03:00
Bruno Oliveira
d4a76a0b99 doc(plugin_hooks): Improve documentation for writing plugin hoo… (#5517)
doc(plugin_hooks): Improve documentation for writing plugin hooks.
2019-06-29 09:39:49 -03:00
Bruno Oliveira
789a3662ce Fix linting 2019-06-29 09:11:09 -03:00
Niklas Meinzer
93a68cdfb4 doc(plugin_hooks): Improve documentation for writing plugin hooks. 2019-06-29 14:02:53 +02:00
Anthony Sottile
4f9bf028f5 Merge pull request #5522 from nicoddemus/merge-master-into-features
Merge master into features
2019-06-28 20:22:56 -07:00
Anthony Sottile
4bc0415720 Merge pull request #5521 from asottile/cherry-pick-maintenance-release
4.6.4 changelog notes
2019-06-28 19:28:44 -07:00
Anthony Sottile
844d660d5c Merge pull request #5520 from asottile/release-4.6.4
Preparing release version 4.6.4
2019-06-28 19:09:10 -07:00
Anthony Sottile
2c6453c72d Merge pull request #5509 from asottile/release-5.0.0
Preparing release version 5.0.0
2019-06-28 18:12:16 -07:00
Bruno Oliveira
e6ffa78e59 Refactoring doctests (#5518)
Refactoring doctests
2019-06-28 19:15:02 -03:00
Anthony Sottile
58bfc7736f Use shutil.which to avoid distutils+imp warning 2019-06-28 14:44:49 -07:00
Anthony Sottile
97f0a20ca9 Add notice about py35+ and move ExitCode changelog entry 2019-06-28 14:40:10 -07:00
Anthony Sottile
55d2fe076f Use importlib instead of imp in demo 2019-06-28 14:40:10 -07:00
Anthony Sottile
5e39eb91bb Correct Zac-HD's name in changelogs 2019-06-28 14:40:10 -07:00
Anthony Sottile
fd2f320485 Preparing release version 5.0.0 2019-06-28 14:39:53 -07:00
Bruno Oliveira
73d918db55 Remove astor and reproduce the original assertion expression (#5512)
Remove astor and reproduce the original assertion expression
2019-06-28 18:36:11 -03:00
Anthony Sottile
7ee244476a Remove astor and reproduce the original assertion expression 2019-06-28 13:38:52 -07:00
AmirElkess
bf39e89946 Refactoring doctests 2019-06-28 21:16:17 +02:00
Bruno Oliveira
3c9b46f781 Remove stray comment from tox.ini (#5507)
Remove stray comment from tox.ini
2019-06-27 15:12:57 -03:00
Anthony Sottile
f7bfbb557e Merge pull request #5506 from asottile/fix_no_terminal
Fix crash when discovery fails while using `-p no:terminal`
2019-06-27 10:46:43 -07:00
Anthony Sottile
45af361a67 Remove stray comment from tox.ini 2019-06-27 10:46:08 -07:00
Anthony Sottile
4e723d6750 Fix crash when discovery fails while using -p no:terminal 2019-06-27 10:24:29 -07:00
Bruno Oliveira
65fbdf2568 Don't crash with --pyargs and a filename that looks like a modu… (#5503)
Don't crash with --pyargs and a filename that looks like a module
2019-06-27 13:53:02 -03:00
Bruno Oliveira
5ef507982c Add Open Collective to FUNDING.yml (#5504)
Add Open Collective to FUNDING.yml
2019-06-27 13:49:58 -03:00
Florian Bruhin
2479a91e92 Add Open Collective to FUNDING.yml 2019-06-27 17:53:03 +02:00
Anthony Sottile
3e0e31a364 Don't crash with --pyargs and a filename that looks like a module 2019-06-27 08:32:32 -07:00
Anthony Sottile
c9923a3a5c Merge pull request #5499 from nicoddemus/cleanup-py35-rewrite
Remove py<35 compatibility code from rewrite.py
2019-06-26 21:31:54 -07:00
Bruno Oliveira
1be49e713a Remove py<35 compatibility code from rewrite.py 2019-06-26 21:44:10 -03:00
Bruno Oliveira
37fb50a3ed Features assertion pass hook (#3479)
Features assertion pass hook
2019-06-26 21:14:19 -03:00
Bruno Oliveira
790806e865 Merge pull request #5494 from Zac-HD/funcargnames-to-fixturenames
Deprecate funcargnames alias for fixturenames
2019-06-26 21:13:59 -03:00
Bruno Oliveira
2ea22218ff Cover assertions with messages when enable_assertion_pass_hook is enabled 2019-06-26 20:46:31 -03:00
Bruno Oliveira
629eb3ec6a Move formatting variables under the "has impls" if
Small optimization, move the generation of the intermediate
formatting variables inside the 'if _check_if_assertion_pass_impl():'
block.
2019-06-26 19:26:12 -03:00
Bruno Oliveira
8edf68f3c0 Add a trivial note about astor 2019-06-26 19:21:12 -03:00
Bruno Oliveira
3afee36ebb Improve docs and reference 2019-06-26 19:15:00 -03:00
Bruno Oliveira
fcbe66feba Restore proper handling of '%' in assertion messages 2019-06-26 18:51:27 -03:00
Bruno Oliveira
eb90f3d1c8 Fix default value of 'enable_assertion_pass_hook' 2019-06-26 17:54:24 -03:00
Victor Maryama
6854ff2acc Fixed import order pep8. 2019-06-26 19:05:17 +02:00
Victor Maryama
53234bf613 Added config back to AssertionWriter and fixed typo in check_if_assertion_pass_impl function call. 2019-06-26 19:00:31 +02:00
Victor Maryama
6f851e6cbb Merge remote-tracking branch 'upstream/master' into features-assertion-pass-hook-master
# Conflicts:
#	src/_pytest/assertion/rewrite.py
2019-06-26 18:12:56 +02:00
Victor Maryama
9a34d88c8d Explanation variables only defined if failed or passed with plugins implementing the hook. 2019-06-26 18:10:16 +02:00
Victor Maryama
d91a5d3cd7 Further reverting changes. 2019-06-26 18:10:16 +02:00
Victor Maryama
f755ff6af1 Black formatting. 2019-06-26 18:10:16 +02:00
Victor Maryama
d638da5821 Using ini-file option instead of cmd option. 2019-06-26 18:10:16 +02:00
Victor Maryama
0fb52416b1 Reverted changes. 2019-06-26 18:10:15 +02:00
Bruno Oliveira
6a2d844c5d Fix typo replace circuting with circuiting. (#5497)
Fix typo replace `circuting` with `circuiting`.
2019-06-26 11:09:22 -03:00
Bruno Oliveira
a48feb3261 Delete 5497.trivial.rst
Just a typo, no need for a changelog entry. :)
2019-06-26 11:09:04 -03:00
Anthony Sottile
6cf8e71dca Merge pull request #5496 from pytest-dev/stray_comment
Delete stray comment
2019-06-26 05:59:38 -07:00
Tim Gates
994c32235c Fix rst support 2019-06-26 20:46:09 +10:00
Tim Gates
d81f758285 Update changelog with trivial as per ./CONTRIBUTING.rst 2019-06-26 20:31:45 +10:00
Tim Gates
fdb6e35b1b Fix typo replace circuting with circuiting. 2019-06-26 20:23:35 +10:00
Victor Maryama
7efdd5063b Update src/_pytest/assertion/rewrite.py
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
2019-06-26 10:50:27 +02:00
Anthony Sottile
bfba33ec9e Delete stray comment 2019-06-25 20:24:13 -07:00
Anthony Sottile
3f3f3e7a29 Merge pull request #5495 from nicoddemus/blueyed/cmp-bytes
Improve comparison of byte strings (supersedes #5267)
2019-06-25 19:16:24 -07:00
Bruno Oliveira
aaf40314d1 Do not attempt to rewrite non-source files (#5490)
Do not attempt to rewrite non-source files
2019-06-25 20:43:23 -03:00
Bruno Oliveira
3f2344e8f7 Show bytes ascii representation instead of numeric value 2019-06-25 20:40:25 -03:00
Daniel Hahler
8c7eb82363 Fix/improve comparison of byte strings
Fixes https://github.com/pytest-dev/pytest/issues/5260.
2019-06-25 20:15:45 -03:00
Bruno Oliveira
1aefb24b37 Merge features into master (#5491)
Merge features into master
2019-06-25 20:02:02 -03:00
Zac Hatfield-Dodds
ed85c83154 Deprecate funcargnames alias 2019-06-26 08:53:17 +10:00
Anthony Sottile
bd647fdd8b Revert allow_abbrev=False in helper scripts 2019-06-25 14:50:07 -07:00
Victor Maryama
80ac910a24 Added msg to docstring for cleaning pyc. 2019-06-25 19:49:57 +02:00
Victor Maryama
4db5488ed8 Now dependent on command line option. 2019-06-25 19:49:05 +02:00
Victor Maryama
cfbfa53f2b Using pytester subprocess to avoid keeping references in the HookRecorder. 2019-06-25 17:46:56 +02:00
Victor Maryama
db50a975fd Reverted leak fixture test. 2019-06-25 17:23:14 +02:00
Bruno Oliveira
019455298b Merge remote-tracking branch 'upstream/features' into merge-features-into-master 2019-06-25 12:07:54 -03:00
Bruno Oliveira
64a6365227 Include pytest version in the cached pyc tags (#5484)
Include pytest version in the cached pyc tags
2019-06-25 12:07:01 -03:00
Anthony Sottile
b991810f32 Do not attempt to rewrite non-source files 2019-06-25 08:00:20 -07:00
Zac Hatfield-Dodds
4d5780facf Merge pull request #5469 from Zac-HD/disallow-abbrev
Disallow abbreviated command-line options
2019-06-25 22:38:52 +10:00
Victor Maryama
81e3f3cf95 Black formatting 2019-06-25 10:41:11 +02:00
Victor Maryama
2280f28596 Black formatting. 2019-06-25 10:36:01 +02:00
Victor Maryama
f8c9a7b86d Formatting and removed py2 support. 2019-06-25 10:35:42 +02:00
Victor Maryama
98b212cbfb Added "experimental" note. 2019-06-25 10:35:09 +02:00
Zac Hatfield-Dodds
d72fb73fa0 Do not allow abbreviated arguments 2019-06-25 13:51:33 +10:00
Bruno Oliveira
a54e2e19f5 Clarify changelog entries should be rst files (#5485)
Clarify changelog entries should be rst files
2019-06-24 22:33:40 -03:00
Bruno Oliveira
37fce6c6e5 Pickup addition positional args passed to _parse_parametrize_ar… (#5483)
Pickup addition positional args passed to _parse_parametrize_args
2019-06-24 22:05:19 -03:00
Kevin J. Foley
23aa3bb0ae Clarify changelog entries should be rst files 2019-06-24 20:55:51 -04:00
Kevin J. Foley
95714436a1 Pickup additional positional args passed to _parse_parametrize_args 2019-06-24 20:43:52 -04:00
Bruno Oliveira
f43fb13179 Include pytest version in the cached pyc tags
Fix #1671
2019-06-24 20:37:07 -03:00
Bruno Oliveira
3f5b078462 Merge pull request #5441 from nicoddemus/faulthandler-5440
Integrate pytest-faulthandler into the core
2019-06-24 20:01:15 -03:00
Bruno Oliveira
63c4802be0 minor: clarify help with reportchars (#5475)
minor: clarify help with reportchars
2019-06-24 19:59:15 -03:00
Anthony Sottile
a24933b0a6 Merge pull request #5481 from asottile/minor_py3_cleanup
Use new raise syntax in one case
2019-06-24 12:06:53 -07:00
Anthony Sottile
380ca8f880 Use new raise syntax in one case 2019-06-24 11:24:03 -07:00
Anthony Sottile
61dcb84f0d Merge pull request #5468 from asottile/switch_importlib_to_imp
Switch from deprecated imp to importlib
2019-06-24 11:22:01 -07:00
Anthony Sottile
4cd08f9b52 Switch from deprecated imp to importlib 2019-06-24 09:48:38 -07:00
Victor Maryama
52e695b329 Removed debug code. 2019-06-24 17:47:48 +02:00
Victor Maryama
9a89783fbb Assertion passed hook 2019-06-24 16:09:39 +02:00
Daniel Hahler
01a094cc43 minor: clarify help with reportchars
`-ra` / `-rA` will include "w" also.  This does not explicitly mention
it (allowing for change the behavior), but makes it a) clearer that "w"
is a recognized reportchar, and b) less confusing that `-ra
--disable-warnings` still displays them.
2019-06-24 06:07:52 +02:00
Bruno Oliveira
3ce31b6370 Change pytest-faulthandler for simplification
* The --no-faulthandler option is not necessary given that we can use
  `-p no:faulthandler`.

* The `--faulthandler-timeout` command-line option has become an ini
  option, for the reasons described in
  https://github.com/pytest-dev/pytest-faulthandler/issues/34 and
  users can still set it from the command-line.

Fix pytest-dev/pytest-faulthandler#34
2019-06-22 19:22:43 -03:00
Bruno Oliveira
3d01dd3adf Update cache.rst (#5465)
Update cache.rst
2019-06-21 21:12:30 -03:00
Bruno Oliveira
a37b902afe Integrate pytest-faulthandler into the core
* Add pytest-faulthandler files unchanged
* Adapt imports and tests
* Add code to skip registration of the external `pytest_faulthandler`
  to avoid conflicts

Fix #5440
2019-06-21 21:02:24 -03:00
curiousjazz77
12b76b6261 Update cache.rst 2019-06-21 15:48:59 -07:00
Bruno Oliveira
b713460cc7 fix typos in the resolution of #5125 (#5458)
fix typos in the resolution of #5125
2019-06-17 16:55:26 -03:00
Ronny Pfannschmidt
0627d92df2 fix typos in the resolution of #5125 2019-06-17 20:35:23 +02:00
Bruno Oliveira
b38a4e8e11 Fix stepwise crash when first collected module fails (#5446)
Fix stepwise crash when first collected module fails
2019-06-16 10:43:18 -03:00
Bruno Oliveira
689ce112e7 Small cleanups on _pytest.compat (#5451)
Small cleanups on _pytest.compat
2019-06-16 10:42:07 -03:00
Bruno Oliveira
87fc5a5455 Make pytest warnings show as from 'pytest' module instead of '_pytest.warning_types' (#5452)
Make pytest warnings show as from 'pytest' module instead of '_pytest.warning_types'
2019-06-16 10:41:40 -03:00
Bruno Oliveira
bbfc8d1501 conversion of exit codes to enum + exposure (#5420)
conversion of exit codes to enum  + exposure
2019-06-16 10:41:18 -03:00
Bruno Oliveira
ab6ed381ac Improve ExitCode docstring 2019-06-15 20:53:46 -03:00
Ronny Pfannschmidt
1cfea5f1b3 add ExitCode reference in usage 2019-06-15 21:41:55 +02:00
Daniel Hahler
cf27af734e Merge pull request #5404 from Zac-HD/helpful-mock-unwrapper
Emit warning for broken object
2019-06-15 18:54:03 +02:00
Ronny Pfannschmidt
8b3b10b14b pre-commit 2019-06-15 17:41:13 +02:00
Ronny Pfannschmidt
103d6146b0 document exitcode members 2019-06-15 17:18:21 +02:00
Ronny Pfannschmidt
065fa17124 update cangelog to fit review suggestion 2019-06-15 17:03:40 +02:00
Bruno Oliveira
43e8576ca3 Make pytest warnings show as from 'pytest' module instead of '_pytest.warning_types'
When we configure warnings as errors, users see error messages like this:

        def test():
    >       warnings.warn(pytest.PytestWarning("some warning"))
    E       _pytest.warning_types.PytestWarning: some warning

This is a problem because suggests the user should use `_pytest.warning_types.PytestWarning` to configure
their warning filters, which is not nice.

This commit changes the message to:

        def test():
    >       warnings.warn(pytest.PytestWarning("some warning"))
    E       pytest.PytestWarning: some warning
2019-06-15 11:48:34 -03:00
Bruno Oliveira
d8fa434d39 Remove Python 2-only workaround 2019-06-15 10:45:16 -03:00
Bruno Oliveira
177af032d2 Remove unused/unneeded code 2019-06-15 10:45:16 -03:00
Bruno Oliveira
355eb5adfb Small cleanups on _pytest.compat
Small improvements and cleanups
2019-06-15 10:06:37 -03:00
Ronny Pfannschmidt
2bd619ecb0 add changelog 2019-06-15 07:05:18 +02:00
Ronny Pfannschmidt
2b92fee1c3 initial conversion of exit codes to enum 2019-06-15 06:48:00 +02:00
Anthony Sottile
240828d912 Merge pull request #5448 from nicoddemus/add-hooks
Remove broken/unused PytestPluginManager.addhooks
2019-06-14 21:45:11 -07:00
Bruno Oliveira
c94e9b6145 Fix test docstring 2019-06-13 23:10:13 -03:00
Bruno Oliveira
7513d87b15 Remove broken/unused PytestPluginManager.addhooks
The function has been deprecated for ages and
the PLUGIN_MANAGER_ADDHOOKS constant doesn't even exist since 3.0.

Because the function is clearly broken, this change doesn't even
require a CHANGELOG.
2019-06-13 19:55:13 -03:00
Bruno Oliveira
bc345ac980 Remove handling of collection errors by --sw
Since then pytest itself adopted the behavior of interrupting
the test session on collection errors, so --sw no longer needs
to handle this.

The --sw behavior seems have been implemented when pytest
would continue execution even if there were collection errors.
2019-06-13 17:19:36 -03:00
Bruno Oliveira
4cc05a657d Fix --sw crash when first file in cmdline fails to collect
Fix #5444
2019-06-13 16:47:01 -03:00
Thomas Grainger
ad15efc7ea add test for stepwise attribute error Refs: #5444 2019-06-13 12:39:49 +01:00
Bruno Oliveira
e3dcf1f39d Merge pull request #5442 from patriksevallius/master
Add missing 'e' to test_mod(e).
2019-06-13 08:23:53 -03:00
patriksevallius
701d0351a6 Add missing 'e' to test_mod(e). 2019-06-13 06:01:30 +02:00
Bruno Oliveira
bad7ab721a Minor: tox: coverage: use -m with coverage-report (#5427)
Minor:  tox: coverage: use -m with coverage-report
2019-06-12 19:44:15 -03:00
Anthony Sottile
ede8b87560 Merge pull request #5438 from nicoddemus/result-log-deprecation
Postpone removal of --result-log to pytest 6.0
2019-06-12 10:45:02 -07:00
Bruno Oliveira
52780f39ce Postpone removal of --result-log to pytest 6.0
As we did not provide an alternative yet, it is better to postpone
the actual removal until we have provided a suitable and stable
alternative.

Related to #4488
2019-06-12 08:26:11 -03:00
Bruno Oliveira
d637d1b2ac Merge pull request #5435 from asottile/release-4.6.3 (#5436)
Merge pull request #5435 from asottile/release-4.6.3
2019-06-11 21:24:45 -03:00
Anthony Sottile
47447963b7 Merge pull request #5437 from asottile/reenable_pypy3
re-enable pypy3
2019-06-11 12:21:53 -07:00
Anthony Sottile
f586d627b3 re-enable pypy3 now that importlib-metadata 0.18 is released 2019-06-11 10:54:16 -07:00
Anthony Sottile
108fad1ac0 Revert "ci: Travis: remove pypy3 job for now"
This reverts commit 47022b36cb.
2019-06-11 10:53:32 -07:00
Anthony Sottile
18c2ff6625 Merge pull request #5435 from asottile/release-4.6.3
Preparing release version 4.6.3
2019-06-11 09:59:05 -07:00
Zac-HD
c5a549b5bb Emit warning for broken object 2019-06-09 22:17:40 +10:00
Daniel Hahler
75cda6de53 tox: coverage: use -m with coverage-report 2019-06-09 12:51:32 +02:00
Daniel Hahler
e868bb647d remove commented code 2019-06-09 12:51:32 +02:00
Daniel Hahler
40c5a9d9f2 Merge pull request #5409 from blueyed/twisted
ci: optimize twisted/numpy related jobs
2019-06-09 12:49:41 +02:00
Anthony Sottile
83d18096d4 Merge pull request #5421 from nicoddemus/raises-warning-message
Link deprecation docs pytest.raises 'message' warning
2019-06-08 12:44:40 -07:00
Daniel Hahler
28aa38ece6 ci: optimize twisted/pexpect related jobs
- tox: use twisted as dep only
- Azure: move twisted/numpy to main py37 job
- Travis: move twisted to main py37 build
2019-06-08 19:21:56 +02:00
Bruno Oliveira
f0cee593f2 Link deprecation docs pytest.raises 'message' warning
As commented in https://github.com/pytest-dev/pytest/issues/3974#issuecomment-499870914
2019-06-07 14:14:46 -03:00
Daniel Hahler
4f597f011d Merge pull request #5419 from blueyed/remove-pypy3
ci: Travis: remove pypy3 job for now
2019-06-07 18:05:59 +02:00
Bruno Oliveira
9f5c356784 Remove ExceptionInfo.__str__, falling back to __repr__ (#5413)
Remove ExceptionInfo.__str__, falling back to __repr__
2019-06-07 08:48:03 -03:00
Daniel Hahler
47022b36cb ci: Travis: remove pypy3 job for now
Ref: https://github.com/pytest-dev/pytest/issues/5317#issuecomment-499019928
2019-06-07 11:01:23 +02:00
Bruno Oliveira
042a10f2e2 Add slow marker in run/skip option example. (#5416)
Add `slow` marker in run/skip option example.
2019-06-06 15:00:04 -03:00
Anthony Sottile
0405697a78 Merge pull request #5415 from asottile/small_mypy_fixes
small mypy fixes
2019-06-06 10:47:58 -07:00
Ralph Giles
918268774b Add slow marker in run/skip option example.
The example implementation of a `--runslow` option results in
a `PytestUnknownMarkWarning`. Include registering the custom
mark in the example, based on the documentation in markers.rst.
2019-06-06 10:15:43 -07:00
Anthony Sottile
ccd87f9e80 small mypy fixes 2019-06-06 09:13:02 -07:00
Bruno Oliveira
65c2a81924 Remove ExceptionInfo.__str__, falling back to __repr__
Fix #5412
2019-06-06 12:20:32 -03:00
Bruno Oliveira
450d264623 ci: move coverage reporting to shared script (#5403)
ci: move coverage reporting to shared script
2019-06-06 12:04:32 -03:00
Daniel Hahler
0fd1f3038c ci: move coverage reporting to shared script 2019-06-06 16:31:39 +02:00
Bruno Oliveira
9f8b566ea9 Turn PytestDeprecationWarning into error (#5410)
Turn PytestDeprecationWarning into error
2019-06-05 22:20:14 -03:00
Bruno Oliveira
3656885d0d Fix verbosity bug in --collect-only (#5391)
Fix verbosity bug in --collect-only
2019-06-05 20:47:59 -03:00
Thomas Hisch
577b0dffe7 Fix verbosity bug in --collect-only 2019-06-05 20:10:16 -03:00
Bruno Oliveira
8f5cb461a8 Turn PytestDeprecationWarning into error
Fix #5402
2019-06-05 19:02:52 -03:00
Daniel Hahler
be84ba8a33 Merge pull request #5396 from blueyed/travis-py37-nocov
Revert "Enable coverage for 'py37' environment"
2019-06-05 12:40:46 +02:00
Daniel Hahler
d9eafbdee3 Revert "Enable coverage for 'py37' environment"
This reverts commit 6d393c5dc8.

It should not be necessary, because we have it via other jobs already.
2019-06-05 12:12:58 +02:00
Daniel Hahler
190456aeda Merge pull request #5406 from blueyed/restore-tracing
tests: restore tracing function
2019-06-05 11:50:56 +02:00
Daniel Hahler
aab5687093 tests: restore tracing function
Without this, `testing/test_pdb.py` (already without pexpect) will cause
missing test coverage afterwards (for the same process).
2019-06-05 11:26:48 +02:00
Anthony Sottile
5fdc2d7744 Merge pull request #5400 from asottile/prefixes
Clean up u' prefixes and py2 bytes conversions
2019-06-04 19:03:12 -07:00
Anthony Sottile
25cef55e4a Merge pull request #5398 from nicoddemus/kwargs-refactoring
Use keyword-only arguments in a few places
2019-06-04 18:21:44 -07:00
Anthony Sottile
be2be040f9 Clean up u' prefixes and py2 bytes conversions 2019-06-04 17:50:34 -07:00
Anthony Sottile
0f4992c223 Merge pull request #5393 from nicoddemus/unittest-self-5390
item.obj is again a bound method on TestCase function items
2019-06-04 17:48:48 -07:00
Bruno Oliveira
23cd68b667 Use keyword-only arguments in a few places 2019-06-04 21:10:59 -03:00
Bruno Oliveira
79ef04888e Merge pull request #5389 from dirk-thomas/patch-1
fix logic if importlib_metadata.PathDistribution.files is None [breaks pytest 4.6.0|1|2]
2019-06-04 20:25:21 -03:00
Bruno Oliveira
1f8fd421c4 item.obj is again a bound method on TestCase function items
Fix #5390
2019-06-04 20:22:49 -03:00
Bruno Oliveira
76d50801fd Allow pypy3 failures again (#5394)
Allow pypy3 failures again
2019-06-04 20:22:25 -03:00
Bruno Oliveira
9349c72cac Allow pypy3 failures again
Related to #5317
2019-06-04 20:17:51 -03:00
Bruno Oliveira
883db6a216 Add test for packages with broken metadata
Related to #5389
2019-06-04 17:04:15 -03:00
Dirk Thomas
898e869bcd add changelog file for #5389 2019-06-04 10:55:38 -07:00
Dirk Thomas
0a91e181af fix logic if importlib_metadata.PathDistribution.files is None 2019-06-04 10:41:58 -07:00
Bruno Oliveira
6a6b6d8195 Remove --recreate from .travis.yml (#5384)
Remove --recreate from .travis.yml
2019-06-03 22:18:17 -03:00
Anthony Sottile
b95bb29fc2 Remove --recreate from .travis.yml
Looks like this has been in the history since the beginning of time, but we should always get a blank slate anyway

Noticed this in https://github.com/crsmithdev/arrow/pull/597
2019-06-03 16:31:06 -07:00
Bruno Oliveira
2a3d643bdf Documentation: expansion of Monkeypatch to include mocked classes and dictionaries (#5315)
Documentation: expansion of Monkeypatch to include mocked classes and dictionaries
2019-06-03 19:30:29 -03:00
Bruno Oliveira
2cfd9c259f Revert "ci: Travis: add pypy3 to allowed failures temporarily" (#5378)
Revert "ci: Travis: add pypy3 to allowed failures temporarily"
2019-06-03 17:59:24 -03:00
Bruno Oliveira
927e95c65a [release notes] Merge pull request #5379 from asottile/release-4.6.2 (#5380)
[release notes] Merge pull request #5379 from asottile/release-4.6.2
2019-06-03 16:35:21 -03:00
Anthony Sottile
9657166a22 Merge pull request #5379 from asottile/release-4.6.2
Preparing release version 4.6.2
2019-06-03 12:19:42 -07:00
Bruno Oliveira
f223d26111 Add a new Exceptions section in documentation and document UsageError (#5231)
Add a new Exceptions section in documentation and document UsageError
2019-06-03 14:41:04 -03:00
Pulkit Goyal
1d6bbab2b0 Add a new Exceptions section in documentation and document UsageError 2019-06-03 14:23:56 -03:00
Daniel Hahler
606d728697 Revert "Revert "Revert "ci: Travis: add pypy3 to allowed failures temporarily""" 2019-06-03 18:54:40 +02:00
Daniel Hahler
3a8b08275d Merge pull request #5377 from blueyed/pypy3
Revert "Revert "ci: Travis: add pypy3 to allowed failures temporarily""
2019-06-03 18:33:57 +02:00
Daniel Hahler
e770db4c91 Revert "Revert "ci: Travis: add pypy3 to allowed failures temporarily""
This reverts commit a6dc283133.
2019-06-03 18:30:50 +02:00
Anthony Sottile
63099bc282 Merge pull request #5373 from asottile/revert_all_handling
Revert unrolling of `all()`
2019-06-03 09:18:47 -07:00
Bruno Oliveira
4f57d40a43 Merge master into features (#5374)
Merge master into features
2019-06-03 13:14:31 -03:00
Anthony Sottile
230f736fcd Add changelog entries for reverting all() handling 2019-06-03 08:42:03 -07:00
Anthony Sottile
1b381d5277 Revert "Unroll calls to any #5062 (#5103)"
This reverts commit 2b9ca34280, reversing
changes made to 0a57124063.
2019-06-03 08:42:03 -07:00
Anthony Sottile
2125d04501 Revert "Fix all() unroll for non-generators/non-list comprehensions (#5360)"
This reverts commit 733f43b02e, reversing
changes made to e4fe41ebb7.
2019-06-03 08:42:03 -07:00
Anthony Sottile
5976f36240 Merge pull request #5368 from asottile/pyupgrade
Automated / semi-automated python3 upgrades
2019-06-03 08:41:37 -07:00
Bruno Oliveira
ce78c9adef Fix linting 2019-06-03 12:08:50 -03:00
Bruno Oliveira
6d393c5dc8 Enable coverage for 'py37' environment 2019-06-03 12:08:02 -03:00
Anthony Sottile
8292644015 Allow multiple positions for the SyntaxError in pypy3.6 2019-06-03 12:08:02 -03:00
Anthony Sottile
96fd44e040 Update line number for warning 2019-06-03 12:08:02 -03:00
Anthony Sottile
5dcf85c17e manual: remove dependence on six 2019-06-03 12:08:02 -03:00
Anthony Sottile
ca1efd57bd git rm src/_pytest/_code/_py2traceback.py 2019-06-03 12:08:02 -03:00
Anthony Sottile
c63320cc31 codecs.open / io.open 2019-06-03 12:08:02 -03:00
Anthony Sottile
4df529e5b9 Clean up __future__ and coding: in tests 2019-06-03 12:08:02 -03:00
Anthony Sottile
a91fe1fedd pre-commit run pyupgrade --all-files 2019-06-03 12:08:02 -03:00
Anthony Sottile
3f1ec520fc pre-commit run reorder-python-imports --all-files 2019-06-03 12:08:01 -03:00
Anthony Sottile
5034399d7a pre-commit run fix-encoding-pragma --all-files 2019-06-03 12:08:01 -03:00
Anthony Sottile
99057555e1 manual changes to .pre-commit-config.yaml 2019-06-03 12:08:01 -03:00
Thomas Hisch
b8781ff868 Merge pull request #5335 from thisch/fmtregex
logging: Extend LEVELNAME_FMT_REGEX
2019-06-03 05:55:36 +02:00
Thomas Hisch
ef053193b5 logging: Extend LEVELNAME_FMT_REGEX 2019-06-03 05:53:19 +02:00
Bruno Oliveira
28ac469eaa Merge pull request #5367 from apollovy/master
Fix typo about interpreters count in doc/en/example/parametrize.html
2019-06-02 20:43:53 -03:00
Bruno Oliveira
5999782768 Use python 3 interpreters in example 2019-06-02 19:20:00 -03:00
Bruno Oliveira
6e68532bb3 Merge pull request #5362 from asottile/release-4.6.1 (#5365)
Merge pull request #5362 from asottile/release-4.6.1
2019-06-02 19:17:29 -03:00
Bruno Oliveira
602a290fcc Drop py27 and py34 support (#5318)
Drop py27 and py34 support
2019-06-02 19:14:54 -03:00
Bruno Oliveira
876df87672 Update release instructions for 4.6-maintenance (#5363)
Update release instructions for 4.6-maintenance
2019-06-02 19:14:26 -03:00
Bruno Oliveira
b7f8f53127 Revert "ci: Travis: add pypy3 to allowed failures temporarily" (#5364)
Revert "ci: Travis: add pypy3 to allowed failures temporarily"
2019-06-02 19:12:39 -03:00
Yuri Apollov
5c016d1021 Fix typo about interpreters count in doc/en/example/parametrize.html 2019-06-02 23:57:22 +03:00
Anthony Sottile
25e728c78b Update release instructions for 4.6-maintenance 2019-06-02 11:50:07 -07:00
Anthony Sottile
642ef6dc1b Merge pull request #5362 from asottile/release-4.6.1
Preparing release version 4.6.1
2019-06-02 11:47:26 -07:00
Daniel Hahler
a6dc283133 Revert "ci: Travis: add pypy3 to allowed failures temporarily"
This reverts commit 5ac498ea96.

The idea is that maybe https://github.com/pytest-dev/pytest/pull/5360
fixes the failures here also.
2019-06-02 20:19:34 +02:00
Bruno Oliveira
4d49ba6529 Drop Python 2.7 and 3.4 support
* Update setup.py requires and classifiers
* Drop Python 2.7 and 3.4 from CI
* Update docs dropping 2.7 and 3.4 support
* Fix mock imports and remove tests related to pypi's mock module
* Add py27 and 34 support docs to the sidebar
* Remove usage of six from tmpdir
* Remove six.PY* code blocks
* Remove sys.version_info related code
* Cleanup compat
* Remove obsolete safe_str
* Remove obsolete __unicode__ methods
* Remove compat.PY35 and compat.PY36: not really needed anymore
* Remove unused UNICODE_TYPES
* Remove Jython specific code
* Remove some Python 2 references from docs

Related to #5275
2019-06-02 14:39:11 -03:00
Bruno Oliveira
733f43b02e Fix all() unroll for non-generators/non-list comprehensions (#5360)
Fix all() unroll for non-generators/non-list comprehensions
2019-06-02 12:51:43 -03:00
Bruno Oliveira
acdf30a523 Fix all() unroll for non-generators/non-list comprehensions
Fix #5358
2019-06-02 12:12:14 -03:00
Anthony Sottile
e4fe41ebb7 Merge pull request #5356 from asottile/fix_parametrize_iterator
Fix `pytest.mark.parametrize` when the argvalue is an iterator
2019-06-01 15:09:43 -07:00
Anthony Sottile
cafb13c95f Fix pytest.mark.parametrize when the argvalue is an iterator 2019-06-01 15:09:17 -07:00
Bruno Oliveira
a8f4e56d8c Docs: Fix link anchor (#5355)
Docs: Fix link anchor
2019-06-01 19:09:04 -03:00
Hugo
e2ac4b782a Fix link anchor 2019-06-01 23:27:20 +03:00
Evan Kepner
2dfbed11b4 fix path expansion example 2019-05-27 23:23:18 -04:00
Evan Kepner
24c95c78e7 add updated monkeypatch examples 2019-05-27 20:55:15 -04:00
299 changed files with 9196 additions and 6234 deletions

View File

@@ -16,3 +16,11 @@ source = src/
*/lib/python*/site-packages/
*/pypy*/site-packages/
*\Lib\site-packages\
[report]
skip_covered = True
show_missing = True
exclude_lines =
\#\s*pragma: no cover
^\s*raise NotImplementedError\b
^\s*return NotImplemented\b

1
.github/FUNDING.yml vendored
View File

@@ -2,3 +2,4 @@
# * https://help.github.com/en/articles/displaying-a-sponsor-button-in-your-repository
# * https://tidelift.com/subscription/how-to-connect-tidelift-with-github
tidelift: pypi/pytest
open_collective: pytest

View File

@@ -6,7 +6,7 @@ Here is a quick checklist that should be present in PRs.
-->
- [ ] Target the `master` branch for bug fixes, documentation updates and trivial changes.
- [ ] Target the `features` branch for new features and removals/deprecations.
- [ ] Target the `features` branch for new features, improvements, and removals/deprecations.
- [ ] Include documentation when adding new features.
- [ ] Include new tests or update existing tests when applicable.

1
.gitignore vendored
View File

@@ -35,6 +35,7 @@ env/
.tox
.cache
.pytest_cache
.mypy_cache
.coverage
.coverage.*
coverage.xml

View File

@@ -1,23 +1,22 @@
exclude: doc/en/example/py2py3/test_py2.py
repos:
- repo: https://github.com/python/black
- repo: https://github.com/psf/black
rev: 19.3b0
hooks:
- id: black
args: [--safe, --quiet]
language_version: python3
- repo: https://github.com/asottile/blacken-docs
rev: v0.5.0
rev: v1.0.0
hooks:
- id: blacken-docs
additional_dependencies: [black==19.3b0]
language_version: python3
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.2.2
rev: v2.2.3
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: fix-encoding-pragma
args: [--remove]
- id: check-yaml
- id: debug-statements
exclude: _pytest/debugging.py
@@ -27,20 +26,27 @@ repos:
hooks:
- id: flake8
language_version: python3
additional_dependencies: [flake8-typing-imports==1.3.0]
- repo: https://github.com/asottile/reorder_python_imports
rev: v1.4.0
hooks:
- id: reorder-python-imports
args: ['--application-directories=.:src']
args: ['--application-directories=.:src', --py3-plus]
- repo: https://github.com/asottile/pyupgrade
rev: v1.15.0
rev: v1.18.0
hooks:
- id: pyupgrade
args: [--keep-percent-format]
args: [--py3-plus]
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.3.0
rev: v1.4.0
hooks:
- id: rst-backticks
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.720
hooks:
- id: mypy
files: ^(src/|testing/)
args: []
- repo: local
hooks:
- id: rst
@@ -53,7 +59,7 @@ repos:
name: changelog filenames
language: fail
entry: 'changelog files must be named ####.(feature|bugfix|doc|deprecation|removal|vendor|trivial).rst'
exclude: changelog/(\d+\.(feature|bugfix|doc|deprecation|removal|vendor|trivial).rst|README.rst|_template.rst)
exclude: changelog/(\d+\.(feature|improvement|bugfix|doc|deprecation|removal|vendor|trivial).rst|README.rst|_template.rst)
files: ^changelog/
- id: py-deprecated
name: py library is deprecated

View File

@@ -13,24 +13,20 @@ env:
global:
- PYTEST_ADDOPTS=-vv
# setuptools-scm needs all tags in order to obtain a proper version
git:
depth: false
install:
- python -m pip install --upgrade --pre tox
jobs:
include:
# OSX tests - first (in test stage), since they are the slower ones.
- &test-macos
os: osx
- os: osx
osx_image: xcode10.1
language: generic
# Coverage for:
# - py2 with symlink in test_cmdline_python_package_symlink.
env: TOXENV=py27-xdist PYTEST_COVERAGE=1
before_install:
- python -V
- test $(python -c 'import sys; print("%d%d" % sys.version_info[0:2])') = 27
- <<: *test-macos
env: TOXENV=py37-pexpect,py37-xdist PYTEST_COVERAGE=1
env: TOXENV=py37-xdist PYTEST_COVERAGE=1
before_install:
- which python3
- python3 -V
@@ -38,20 +34,16 @@ jobs:
- python -V
- test $(python -c 'import sys; print("%d%d" % sys.version_info[0:2])') = 37
# Full run of latest (major) supported versions, without xdist.
- env: TOXENV=py27
python: '2.7'
- env: TOXENV=py37
# Full run of latest supported version, without xdist.
# Coverage for:
# - test_sys_breakpoint_interception (via pexpect).
- env: TOXENV=py37-pexpect PYTEST_COVERAGE=1
python: '3.7'
# Coverage tracking is slow with pypy, skip it.
- env: TOXENV=pypy-xdist
python: 'pypy'
- env: TOXENV=pypy3-xdist
python: 'pypy3'
- env: TOXENV=py34-xdist
python: '3.4'
- env: TOXENV=py35-xdist
python: '3.5'
@@ -59,34 +51,19 @@ jobs:
# - pytester's LsofFdLeakChecker
# - TestArgComplete (linux only)
# - numpy
# - old attrs
# Empty PYTEST_ADDOPTS to run this non-verbose.
- env: TOXENV=py37-lsof-numpy-xdist PYTEST_COVERAGE=1 PYTEST_ADDOPTS=
# Specialized factors for py27.
- env: TOXENV=py27-nobyte-numpy-xdist
python: '2.7'
- env: TOXENV=py27-pluggymaster-xdist
python: '2.7'
- env: TOXENV=py37-lsof-oldattrs-numpy-twisted-xdist PYTEST_COVERAGE=1 PYTEST_ADDOPTS=
# Specialized factors for py37.
# Coverage for:
# - test_sys_breakpoint_interception (via pexpect).
- env: TOXENV=py37-pexpect,py37-twisted PYTEST_COVERAGE=1
- env: TOXENV=py37-pluggymaster-xdist
- env: TOXENV=py37-freeze
# Jobs only run via Travis cron jobs (currently daily).
- env: TOXENV=py38-xdist
python: '3.8-dev'
if: type = cron
- stage: baseline
# Coverage for:
# - _pytest.unittest._handle_skip (via pexpect).
env: TOXENV=py27-pexpect,py27-twisted PYTEST_COVERAGE=1
python: '2.7'
# Use py36 here for faster baseline.
- env: TOXENV=py36-xdist
env: TOXENV=py36-xdist
python: '3.6'
- env: TOXENV=linting,docs,doctesting PYTEST_COVERAGE=1
cache:
@@ -95,8 +72,17 @@ jobs:
- stage: deploy
python: '3.6'
install: pip install -U setuptools setuptools_scm
install: pip install -U setuptools setuptools_scm tox
script: skip
# token to upload github release notes: GH_RELEASE_NOTES_TOKEN
env:
- secure: "OjOeL7/0JUDkV00SsTs732e8vQjHynpbG9FKTNtZZJ+1Zn4Cib+hAlwmlBnvVukML0X60YpcfjnC4quDOIGLPsh5zeXnvJmYtAIIUNQXjWz8NhcGYrhyzuP1rqV22U68RTCdmOq3lMYU/W2acwHP7T49PwJtOiUM5kF120UAQ0Zi5EmkqkIvH8oM5mO9Dlver+/U7Htpz9rhKrHBXQNCMZI6yj2aUyukqB2PN2fjAlDbCF//+FmvYw9NjT4GeFOSkTCf4ER9yfqs7yglRfwiLtOCZ2qKQhWZNsSJDB89rxIRXWavJUjJKeY2EW2/NkomYJDpqJLIF4JeFRw/HhA47CYPeo6BJqyyNV+0CovL1frpWfi9UQw2cMbgFUkUIUk3F6DD59PHNIOX2R/HX56dQsw7WKl3QuHlCOkICXYg8F7Ta684IoKjeTX03/6QNOkURfDBwfGszY0FpbxrjCSWKom6RyZdyidnESaxv9RzjcIRZVh1rp8KMrwS1OrwRSdG0zjlsPr49hWMenN/8fKgcHTV4/r1Tj6mip0dorSRCrgUNIeRBKgmui6FS8642ab5JNKOxMteVPVR2sFuhjOQ0Jy+PmvceYY9ZMWc3+/B/KVh0dZ3hwvLGZep/vxDS2PwCA5/xw31714vT5LxidKo8yECjBynMU/wUTTS695D3NY="
addons:
apt:
packages:
# required by publish_gh_release_notes
- pandoc
after_deploy: tox -e publish_gh_release_notes
deploy:
provider: pypi
user: nicoddemus
@@ -112,9 +98,6 @@ matrix:
allow_failures:
- python: '3.8-dev'
env: TOXENV=py38-xdist
# Temporary (https://github.com/pytest-dev/pytest/pull/5334).
- env: TOXENV=pypy3-xdist
python: 'pypy3'
before_script:
- |
@@ -130,18 +113,12 @@ before_script:
export _PYTEST_TOX_EXTRA_DEP=coverage-enable-subprocess
fi
script: tox --recreate
script: tox
after_success:
- |
if [[ "$PYTEST_COVERAGE" = 1 ]]; then
set -e
# Add last TOXENV to $PATH.
PATH="$PWD/.tox/${TOXENV##*,}/bin:$PATH"
coverage combine
coverage xml
coverage report -m
bash <(curl -s https://codecov.io/bash) -Z -X gcov -X coveragepy -X search -X xcode -X gcovout -X fix -f coverage.xml -n $TOXENV-$TRAVIS_OS_NAME
env CODECOV_NAME="$TOXENV-$TRAVIS_OS_NAME" scripts/report-coverage.sh
fi
notifications:

12
AUTHORS
View File

@@ -15,6 +15,7 @@ Alexander Johnson
Alexei Kozlenok
Allan Feldman
Aly Sivji
Amir Elkess
Anatoly Bubenkoff
Anders Hovmöller
Andras Mitzki
@@ -22,6 +23,7 @@ Andras Tim
Andrea Cimatoribus
Andreas Zeidler
Andrey Paramonov
Andrzej Klajnert
Andrzej Ostrowski
Andy Freeland
Anthon van der Neut
@@ -54,6 +56,7 @@ Charnjit SiNGH (CCSJ)
Chris Lamb
Christian Boelsen
Christian Fetzer
Christian Neumüller
Christian Theunert
Christian Tismer
Christopher Gilling
@@ -70,6 +73,7 @@ Danielle Jenkins
Dave Hunt
David Díaz-Barquero
David Mohr
David Paul Röthlisberger
David Szotten
David Vierra
Daw-Ran Liou
@@ -94,6 +98,7 @@ Feng Ma
Florian Bruhin
Floris Bruynooghe
Gabriel Reis
Gene Wood
George Kussumoto
Georgy Dyuldin
Graham Horler
@@ -135,6 +140,7 @@ Kale Kundert
Katarzyna Jachim
Katerina Koukiou
Kevin Cox
Kevin J. Foley
Kodi B. Arfer
Kostis Anagnostopoulos
Kristoffer Nordström
@@ -170,6 +176,7 @@ mbyt
Michael Aquilina
Michael Birtwell
Michael Droettboom
Michael Goerz
Michael Seifert
Michal Wajszczuk
Mihai Capotă
@@ -200,11 +207,13 @@ Pulkit Goyal
Punyashloka Biswal
Quentin Pradet
Ralf Schmitt
Ralph Giles
Ran Benita
Raphael Castaneda
Raphael Pierzina
Raquel Alegre
Ravi Chandra
Robert Holt
Roberto Polli
Roland Puntaier
Romain Dorgueil
@@ -235,6 +244,7 @@ Tareq Alayan
Ted Xiao
Thomas Grainger
Thomas Hisch
Tim Hoffmann
Tim Strazny
Tom Dalton
Tom Viner
@@ -254,7 +264,9 @@ Wil Cooley
William Lee
Wim Glenn
Wouter van Ackooy
Xixi Zhao
Xuan Luong
Xuecong Liao
Yoav Caspi
Zac Hatfield-Dodds
Zoltán Máté

View File

@@ -1,6 +1,6 @@
=================
Changelog history
=================
=========
Changelog
=========
Versions follow `Semantic Versioning <https://semver.org/>`_ (``<major>.<minor>.<patch>``).
@@ -18,6 +18,616 @@ with advance notice in the **Deprecations** section of releases.
.. towncrier release notes start
pytest 5.2.2 (2019-10-24)
=========================
Bug Fixes
---------
- `#5206 <https://github.com/pytest-dev/pytest/issues/5206>`_: Fix ``--nf`` to not forget about known nodeids with partial test selection.
- `#5906 <https://github.com/pytest-dev/pytest/issues/5906>`_: Fix crash with ``KeyboardInterrupt`` during ``--setup-show``.
- `#5946 <https://github.com/pytest-dev/pytest/issues/5946>`_: Fixed issue when parametrizing fixtures with numpy arrays (and possibly other sequence-like types).
- `#6044 <https://github.com/pytest-dev/pytest/issues/6044>`_: Properly ignore ``FileNotFoundError`` exceptions when trying to remove old temporary directories,
for instance when multiple processes try to remove the same directory (common with ``pytest-xdist``
for example).
pytest 5.2.1 (2019-10-06)
=========================
Bug Fixes
---------
- `#5902 <https://github.com/pytest-dev/pytest/issues/5902>`_: Fix warnings about deprecated ``cmp`` attribute in ``attrs>=19.2``.
pytest 4.6.6 (2019-10-11)
=========================
Bug Fixes
---------
- `#5523 <https://github.com/pytest-dev/pytest/issues/5523>`_: Fixed using multiple short options together in the command-line (for example ``-vs``) in Python 3.8+.
- `#5537 <https://github.com/pytest-dev/pytest/issues/5537>`_: Replace ``importlib_metadata`` backport with ``importlib.metadata`` from the
standard library on Python 3.8+.
- `#5806 <https://github.com/pytest-dev/pytest/issues/5806>`_: Fix "lexer" being used when uploading to bpaste.net from ``--pastebin`` to "text".
- `#5902 <https://github.com/pytest-dev/pytest/issues/5902>`_: Fix warnings about deprecated ``cmp`` attribute in ``attrs>=19.2``.
Trivial/Internal Changes
------------------------
- `#5801 <https://github.com/pytest-dev/pytest/issues/5801>`_: Fixes python version checks (detected by ``flake8-2020``) in case python4 becomes a thing.
pytest 5.2.0 (2019-09-28)
=========================
Deprecations
------------
- `#1682 <https://github.com/pytest-dev/pytest/issues/1682>`_: Passing arguments to pytest.fixture() as positional arguments is deprecated - pass them
as a keyword argument instead.
Features
--------
- `#1682 <https://github.com/pytest-dev/pytest/issues/1682>`_: The ``scope`` parameter of ``@pytest.fixture`` can now be a callable that receives
the fixture name and the ``config`` object as keyword-only parameters.
See `the docs <https://docs.pytest.org/en/latest/fixture.html#dynamic-scope>`__ for more information.
- `#5764 <https://github.com/pytest-dev/pytest/issues/5764>`_: New behavior of the ``--pastebin`` option: failures to connect to the pastebin server are reported, without failing the pytest run
Bug Fixes
---------
- `#5806 <https://github.com/pytest-dev/pytest/issues/5806>`_: Fix "lexer" being used when uploading to bpaste.net from ``--pastebin`` to "text".
- `#5884 <https://github.com/pytest-dev/pytest/issues/5884>`_: Fix ``--setup-only`` and ``--setup-show`` for custom pytest items.
Trivial/Internal Changes
------------------------
- `#5056 <https://github.com/pytest-dev/pytest/issues/5056>`_: The HelpFormatter uses ``py.io.get_terminal_width`` for better width detection.
pytest 5.1.3 (2019-09-18)
=========================
Bug Fixes
---------
- `#5807 <https://github.com/pytest-dev/pytest/issues/5807>`_: Fix pypy3.6 (nightly) on windows.
- `#5811 <https://github.com/pytest-dev/pytest/issues/5811>`_: Handle ``--fulltrace`` correctly with ``pytest.raises``.
- `#5819 <https://github.com/pytest-dev/pytest/issues/5819>`_: Windows: Fix regression with conftest whose qualified name contains uppercase
characters (introduced by #5792).
pytest 5.1.2 (2019-08-30)
=========================
Bug Fixes
---------
- `#2270 <https://github.com/pytest-dev/pytest/issues/2270>`_: Fixed ``self`` reference in function-scoped fixtures defined plugin classes: previously ``self``
would be a reference to a *test* class, not the *plugin* class.
- `#570 <https://github.com/pytest-dev/pytest/issues/570>`_: Fixed long standing issue where fixture scope was not respected when indirect fixtures were used during
parametrization.
- `#5782 <https://github.com/pytest-dev/pytest/issues/5782>`_: Fix decoding error when printing an error response from ``--pastebin``.
- `#5786 <https://github.com/pytest-dev/pytest/issues/5786>`_: Chained exceptions in test and collection reports are now correctly serialized, allowing plugins like
``pytest-xdist`` to display them properly.
- `#5792 <https://github.com/pytest-dev/pytest/issues/5792>`_: Windows: Fix error that occurs in certain circumstances when loading
``conftest.py`` from a working directory that has casing other than the one stored
in the filesystem (e.g., ``c:\test`` instead of ``C:\test``).
pytest 5.1.1 (2019-08-20)
=========================
Bug Fixes
---------
- `#5751 <https://github.com/pytest-dev/pytest/issues/5751>`_: Fixed ``TypeError`` when importing pytest on Python 3.5.0 and 3.5.1.
pytest 5.1.0 (2019-08-15)
=========================
Removals
--------
- `#5180 <https://github.com/pytest-dev/pytest/issues/5180>`_: As per our policy, the following features have been deprecated in the 4.X series and are now
removed:
* ``Request.getfuncargvalue``: use ``Request.getfixturevalue`` instead.
* ``pytest.raises`` and ``pytest.warns`` no longer support strings as the second argument.
* ``message`` parameter of ``pytest.raises``.
* ``pytest.raises``, ``pytest.warns`` and ``ParameterSet.param`` now use native keyword-only
syntax. This might change the exception message from previous versions, but they still raise
``TypeError`` on unknown keyword arguments as before.
* ``pytest.config`` global variable.
* ``tmpdir_factory.ensuretemp`` method.
* ``pytest_logwarning`` hook.
* ``RemovedInPytest4Warning`` warning type.
* ``request`` is now a reserved name for fixtures.
For more information consult
`Deprecations and Removals <https://docs.pytest.org/en/latest/deprecations.html>`__ in the docs.
- `#5565 <https://github.com/pytest-dev/pytest/issues/5565>`_: Removed unused support code for `unittest2 <https://pypi.org/project/unittest2/>`__.
The ``unittest2`` backport module is no longer
necessary since Python 3.3+, and the small amount of code in pytest to support it also doesn't seem
to be used: after removed, all tests still pass unchanged.
Although our policy is to introduce a deprecation period before removing any features or support
for third party libraries, because this code is apparently not used
at all (even if ``unittest2`` is used by a test suite executed by pytest), it was decided to
remove it in this release.
If you experience a regression because of this, please
`file an issue <https://github.com/pytest-dev/pytest/issues/new>`__.
- `#5615 <https://github.com/pytest-dev/pytest/issues/5615>`_: ``pytest.fail``, ``pytest.xfail`` and ``pytest.skip`` no longer support bytes for the message argument.
This was supported for Python 2 where it was tempting to use ``"message"``
instead of ``u"message"``.
Python 3 code is unlikely to pass ``bytes`` to these functions. If you do,
please decode it to an ``str`` beforehand.
Features
--------
- `#5564 <https://github.com/pytest-dev/pytest/issues/5564>`_: New ``Config.invocation_args`` attribute containing the unchanged arguments passed to ``pytest.main()``.
- `#5576 <https://github.com/pytest-dev/pytest/issues/5576>`_: New `NUMBER <https://docs.pytest.org/en/latest/doctest.html#using-doctest-options>`__
option for doctests to ignore irrelevant differences in floating-point numbers.
Inspired by Sébastien Boisgérault's `numtest <https://github.com/boisgera/numtest>`__
extension for doctest.
Improvements
------------
- `#5471 <https://github.com/pytest-dev/pytest/issues/5471>`_: JUnit XML now includes a timestamp and hostname in the testsuite tag.
- `#5707 <https://github.com/pytest-dev/pytest/issues/5707>`_: Time taken to run the test suite now includes a human-readable representation when it takes over
60 seconds, for example::
===== 2 failed in 102.70s (0:01:42) =====
Bug Fixes
---------
- `#4344 <https://github.com/pytest-dev/pytest/issues/4344>`_: Fix RuntimeError/StopIteration when trying to collect package with "__init__.py" only.
- `#5115 <https://github.com/pytest-dev/pytest/issues/5115>`_: Warnings issued during ``pytest_configure`` are explicitly not treated as errors, even if configured as such, because it otherwise completely breaks pytest.
- `#5477 <https://github.com/pytest-dev/pytest/issues/5477>`_: The XML file produced by ``--junitxml`` now correctly contain a ``<testsuites>`` root element.
- `#5524 <https://github.com/pytest-dev/pytest/issues/5524>`_: Fix issue where ``tmp_path`` and ``tmpdir`` would not remove directories containing files marked as read-only,
which could lead to pytest crashing when executed a second time with the ``--basetemp`` option.
- `#5537 <https://github.com/pytest-dev/pytest/issues/5537>`_: Replace ``importlib_metadata`` backport with ``importlib.metadata`` from the
standard library on Python 3.8+.
- `#5578 <https://github.com/pytest-dev/pytest/issues/5578>`_: Improve type checking for some exception-raising functions (``pytest.xfail``, ``pytest.skip``, etc)
so they provide better error messages when users meant to use marks (for example ``@pytest.xfail``
instead of ``@pytest.mark.xfail``).
- `#5606 <https://github.com/pytest-dev/pytest/issues/5606>`_: Fixed internal error when test functions were patched with objects that cannot be compared
for truth values against others, like ``numpy`` arrays.
- `#5634 <https://github.com/pytest-dev/pytest/issues/5634>`_: ``pytest.exit`` is now correctly handled in ``unittest`` cases.
This makes ``unittest`` cases handle ``quit`` from pytest's pdb correctly.
- `#5650 <https://github.com/pytest-dev/pytest/issues/5650>`_: Improved output when parsing an ini configuration file fails.
- `#5701 <https://github.com/pytest-dev/pytest/issues/5701>`_: Fix collection of ``staticmethod`` objects defined with ``functools.partial``.
- `#5734 <https://github.com/pytest-dev/pytest/issues/5734>`_: Skip async generator test functions, and update the warning message to refer to ``async def`` functions.
Improved Documentation
----------------------
- `#5669 <https://github.com/pytest-dev/pytest/issues/5669>`_: Add docstring for ``Testdir.copy_example``.
Trivial/Internal Changes
------------------------
- `#5095 <https://github.com/pytest-dev/pytest/issues/5095>`_: XML files of the ``xunit2`` family are now validated against the schema by pytest's own test suite
to avoid future regressions.
- `#5516 <https://github.com/pytest-dev/pytest/issues/5516>`_: Cache node splitting function which can improve collection performance in very large test suites.
- `#5603 <https://github.com/pytest-dev/pytest/issues/5603>`_: Simplified internal ``SafeRepr`` class and removed some dead code.
- `#5664 <https://github.com/pytest-dev/pytest/issues/5664>`_: When invoking pytest's own testsuite with ``PYTHONDONTWRITEBYTECODE=1``,
the ``test_xfail_handling`` test no longer fails.
- `#5684 <https://github.com/pytest-dev/pytest/issues/5684>`_: Replace manual handling of ``OSError.errno`` in the codebase by new ``OSError`` subclasses (``PermissionError``, ``FileNotFoundError``, etc.).
pytest 5.0.1 (2019-07-04)
=========================
Bug Fixes
---------
- `#5479 <https://github.com/pytest-dev/pytest/issues/5479>`_: Improve quoting in ``raises`` match failure message.
- `#5523 <https://github.com/pytest-dev/pytest/issues/5523>`_: Fixed using multiple short options together in the command-line (for example ``-vs``) in Python 3.8+.
- `#5547 <https://github.com/pytest-dev/pytest/issues/5547>`_: ``--step-wise`` now handles ``xfail(strict=True)`` markers properly.
Improved Documentation
----------------------
- `#5517 <https://github.com/pytest-dev/pytest/issues/5517>`_: Improve "Declaring new hooks" section in chapter "Writing Plugins"
pytest 5.0.0 (2019-06-28)
=========================
Important
---------
This release is a Python3.5+ only release.
For more details, see our `Python 2.7 and 3.4 support plan <https://docs.pytest.org/en/latest/py27-py34-deprecation.html>`__.
Removals
--------
- `#1149 <https://github.com/pytest-dev/pytest/issues/1149>`_: Pytest no longer accepts prefixes of command-line arguments, for example
typing ``pytest --doctest-mod`` inplace of ``--doctest-modules``.
This was previously allowed where the ``ArgumentParser`` thought it was unambiguous,
but this could be incorrect due to delayed parsing of options for plugins.
See for example issues `#1149 <https://github.com/pytest-dev/pytest/issues/1149>`__,
`#3413 <https://github.com/pytest-dev/pytest/issues/3413>`__, and
`#4009 <https://github.com/pytest-dev/pytest/issues/4009>`__.
- `#5402 <https://github.com/pytest-dev/pytest/issues/5402>`_: **PytestDeprecationWarning are now errors by default.**
Following our plan to remove deprecated features with as little disruption as
possible, all warnings of type ``PytestDeprecationWarning`` now generate errors
instead of warning messages.
**The affected features will be effectively removed in pytest 5.1**, so please consult the
`Deprecations and Removals <https://docs.pytest.org/en/latest/deprecations.html>`__
section in the docs for directions on how to update existing code.
In the pytest ``5.0.X`` series, it is possible to change the errors back into warnings as a stop
gap measure by adding this to your ``pytest.ini`` file:
.. code-block:: ini
[pytest]
filterwarnings =
ignore::pytest.PytestDeprecationWarning
But this will stop working when pytest ``5.1`` is released.
**If you have concerns** about the removal of a specific feature, please add a
comment to `#5402 <https://github.com/pytest-dev/pytest/issues/5402>`__.
- `#5412 <https://github.com/pytest-dev/pytest/issues/5412>`_: ``ExceptionInfo`` objects (returned by ``pytest.raises``) now have the same ``str`` representation as ``repr``, which
avoids some confusion when users use ``print(e)`` to inspect the object.
This means code like:
.. code-block:: python
with pytest.raises(SomeException) as e:
...
assert "some message" in str(e)
Needs to be changed to:
.. code-block:: python
with pytest.raises(SomeException) as e:
...
assert "some message" in str(e.value)
Deprecations
------------
- `#4488 <https://github.com/pytest-dev/pytest/issues/4488>`_: The removal of the ``--result-log`` option and module has been postponed to (tentatively) pytest 6.0 as
the team has not yet got around to implement a good alternative for it.
- `#466 <https://github.com/pytest-dev/pytest/issues/466>`_: The ``funcargnames`` attribute has been an alias for ``fixturenames`` since
pytest 2.3, and is now deprecated in code too.
Features
--------
- `#3457 <https://github.com/pytest-dev/pytest/issues/3457>`_: New `pytest_assertion_pass <https://docs.pytest.org/en/latest/reference.html#_pytest.hookspec.pytest_assertion_pass>`__
hook, called with context information when an assertion *passes*.
This hook is still **experimental** so use it with caution.
- `#5440 <https://github.com/pytest-dev/pytest/issues/5440>`_: The `faulthandler <https://docs.python.org/3/library/faulthandler.html>`__ standard library
module is now enabled by default to help users diagnose crashes in C modules.
This functionality was provided by integrating the external
`pytest-faulthandler <https://github.com/pytest-dev/pytest-faulthandler>`__ plugin into the core,
so users should remove that plugin from their requirements if used.
For more information see the docs: https://docs.pytest.org/en/latest/usage.html#fault-handler
- `#5452 <https://github.com/pytest-dev/pytest/issues/5452>`_: When warnings are configured as errors, pytest warnings now appear as originating from ``pytest.`` instead of the internal ``_pytest.warning_types.`` module.
- `#5125 <https://github.com/pytest-dev/pytest/issues/5125>`_: ``Session.exitcode`` values are now coded in ``pytest.ExitCode``, an ``IntEnum``. This makes the exit code available for consumer code and are more explicit other than just documentation. User defined exit codes are still valid, but should be used with caution.
The team doesn't expect this change to break test suites or plugins in general, except in esoteric/specific scenarios.
**pytest-xdist** users should upgrade to ``1.29.0`` or later, as ``pytest-xdist`` required a compatibility fix because of this change.
Bug Fixes
---------
- `#1403 <https://github.com/pytest-dev/pytest/issues/1403>`_: Switch from ``imp`` to ``importlib``.
- `#1671 <https://github.com/pytest-dev/pytest/issues/1671>`_: The name of the ``.pyc`` files cached by the assertion writer now includes the pytest version
to avoid stale caches.
- `#2761 <https://github.com/pytest-dev/pytest/issues/2761>`_: Honor PEP 235 on case-insensitive file systems.
- `#5078 <https://github.com/pytest-dev/pytest/issues/5078>`_: Test module is no longer double-imported when using ``--pyargs``.
- `#5260 <https://github.com/pytest-dev/pytest/issues/5260>`_: Improved comparison of byte strings.
When comparing bytes, the assertion message used to show the byte numeric value when showing the differences::
def test():
> assert b'spam' == b'eggs'
E AssertionError: assert b'spam' == b'eggs'
E At index 0 diff: 115 != 101
E Use -v to get the full diff
It now shows the actual ascii representation instead, which is often more useful::
def test():
> assert b'spam' == b'eggs'
E AssertionError: assert b'spam' == b'eggs'
E At index 0 diff: b's' != b'e'
E Use -v to get the full diff
- `#5335 <https://github.com/pytest-dev/pytest/issues/5335>`_: Colorize level names when the level in the logging format is formatted using
'%(levelname).Xs' (truncated fixed width alignment), where X is an integer.
- `#5354 <https://github.com/pytest-dev/pytest/issues/5354>`_: Fix ``pytest.mark.parametrize`` when the argvalues is an iterator.
- `#5370 <https://github.com/pytest-dev/pytest/issues/5370>`_: Revert unrolling of ``all()`` to fix ``NameError`` on nested comprehensions.
- `#5371 <https://github.com/pytest-dev/pytest/issues/5371>`_: Revert unrolling of ``all()`` to fix incorrect handling of generators with ``if``.
- `#5372 <https://github.com/pytest-dev/pytest/issues/5372>`_: Revert unrolling of ``all()`` to fix incorrect assertion when using ``all()`` in an expression.
- `#5383 <https://github.com/pytest-dev/pytest/issues/5383>`_: ``-q`` has again an impact on the style of the collected items
(``--collect-only``) when ``--log-cli-level`` is used.
- `#5389 <https://github.com/pytest-dev/pytest/issues/5389>`_: Fix regressions of `#5063 <https://github.com/pytest-dev/pytest/pull/5063>`__ for ``importlib_metadata.PathDistribution`` which have their ``files`` attribute being ``None``.
- `#5390 <https://github.com/pytest-dev/pytest/issues/5390>`_: Fix regression where the ``obj`` attribute of ``TestCase`` items was no longer bound to methods.
- `#5404 <https://github.com/pytest-dev/pytest/issues/5404>`_: Emit a warning when attempting to unwrap a broken object raises an exception,
for easier debugging (`#5080 <https://github.com/pytest-dev/pytest/issues/5080>`__).
- `#5432 <https://github.com/pytest-dev/pytest/issues/5432>`_: Prevent "already imported" warnings from assertion rewriter when invoking pytest in-process multiple times.
- `#5433 <https://github.com/pytest-dev/pytest/issues/5433>`_: Fix assertion rewriting in packages (``__init__.py``).
- `#5444 <https://github.com/pytest-dev/pytest/issues/5444>`_: Fix ``--stepwise`` mode when the first file passed on the command-line fails to collect.
- `#5482 <https://github.com/pytest-dev/pytest/issues/5482>`_: Fix bug introduced in 4.6.0 causing collection errors when passing
more than 2 positional arguments to ``pytest.mark.parametrize``.
- `#5505 <https://github.com/pytest-dev/pytest/issues/5505>`_: Fix crash when discovery fails while using ``-p no:terminal``.
Improved Documentation
----------------------
- `#5315 <https://github.com/pytest-dev/pytest/issues/5315>`_: Expand docs on mocking classes and dictionaries with ``monkeypatch``.
- `#5416 <https://github.com/pytest-dev/pytest/issues/5416>`_: Fix PytestUnknownMarkWarning in run/skip example.
pytest 4.6.5 (2019-08-05)
=========================
Bug Fixes
---------
- `#4344 <https://github.com/pytest-dev/pytest/issues/4344>`_: Fix RuntimeError/StopIteration when trying to collect package with "__init__.py" only.
- `#5478 <https://github.com/pytest-dev/pytest/issues/5478>`_: Fix encode error when using unicode strings in exceptions with ``pytest.raises``.
- `#5524 <https://github.com/pytest-dev/pytest/issues/5524>`_: Fix issue where ``tmp_path`` and ``tmpdir`` would not remove directories containing files marked as read-only,
which could lead to pytest crashing when executed a second time with the ``--basetemp`` option.
- `#5547 <https://github.com/pytest-dev/pytest/issues/5547>`_: ``--step-wise`` now handles ``xfail(strict=True)`` markers properly.
- `#5650 <https://github.com/pytest-dev/pytest/issues/5650>`_: Improved output when parsing an ini configuration file fails.
pytest 4.6.4 (2019-06-28)
=========================
Bug Fixes
---------
- `#5404 <https://github.com/pytest-dev/pytest/issues/5404>`_: Emit a warning when attempting to unwrap a broken object raises an exception,
for easier debugging (`#5080 <https://github.com/pytest-dev/pytest/issues/5080>`__).
- `#5444 <https://github.com/pytest-dev/pytest/issues/5444>`_: Fix ``--stepwise`` mode when the first file passed on the command-line fails to collect.
- `#5482 <https://github.com/pytest-dev/pytest/issues/5482>`_: Fix bug introduced in 4.6.0 causing collection errors when passing
more than 2 positional arguments to ``pytest.mark.parametrize``.
- `#5505 <https://github.com/pytest-dev/pytest/issues/5505>`_: Fix crash when discovery fails while using ``-p no:terminal``.
pytest 4.6.3 (2019-06-11)
=========================
Bug Fixes
---------
- `#5383 <https://github.com/pytest-dev/pytest/issues/5383>`_: ``-q`` has again an impact on the style of the collected items
(``--collect-only``) when ``--log-cli-level`` is used.
- `#5389 <https://github.com/pytest-dev/pytest/issues/5389>`_: Fix regressions of `#5063 <https://github.com/pytest-dev/pytest/pull/5063>`__ for ``importlib_metadata.PathDistribution`` which have their ``files`` attribute being ``None``.
- `#5390 <https://github.com/pytest-dev/pytest/issues/5390>`_: Fix regression where the ``obj`` attribute of ``TestCase`` items was no longer bound to methods.
pytest 4.6.2 (2019-06-03)
=========================
Bug Fixes
---------
- `#5370 <https://github.com/pytest-dev/pytest/issues/5370>`_: Revert unrolling of ``all()`` to fix ``NameError`` on nested comprehensions.
- `#5371 <https://github.com/pytest-dev/pytest/issues/5371>`_: Revert unrolling of ``all()`` to fix incorrect handling of generators with ``if``.
- `#5372 <https://github.com/pytest-dev/pytest/issues/5372>`_: Revert unrolling of ``all()`` to fix incorrect assertion when using ``all()`` in an expression.
pytest 4.6.1 (2019-06-02)
=========================
Bug Fixes
---------
- `#5354 <https://github.com/pytest-dev/pytest/issues/5354>`_: Fix ``pytest.mark.parametrize`` when the argvalues is an iterator.
- `#5358 <https://github.com/pytest-dev/pytest/issues/5358>`_: Fix assertion rewriting of ``all()`` calls to deal with non-generators.
pytest 4.6.0 (2019-05-31)
=========================
@@ -1903,10 +2513,10 @@ Features
design. This introduces new ``Node.iter_markers(name)`` and
``Node.get_closest_marker(name)`` APIs. Users are **strongly encouraged** to
read the `reasons for the revamp in the docs
<https://docs.pytest.org/en/latest/mark.html#marker-revamp-and-iteration>`_,
<https://docs.pytest.org/en/latest/historical-notes.html#marker-revamp-and-iteration>`_,
or jump over to details about `updating existing code to use the new APIs
<https://docs.pytest.org/en/latest/mark.html#updating-code>`_. (`#3317
<https://github.com/pytest-dev/pytest/issues/3317>`_)
<https://docs.pytest.org/en/latest/historical-notes.html#updating-code>`_.
(`#3317 <https://github.com/pytest-dev/pytest/issues/3317>`_)
- Now when ``@pytest.fixture`` is applied more than once to the same function a
``ValueError`` is raised. This buggy behavior would cause surprising problems
@@ -2312,10 +2922,10 @@ Features
<https://github.com/pytest-dev/pytest/issues/3038>`_)
- New `pytest_runtest_logfinish
<https://docs.pytest.org/en/latest/writing_plugins.html#_pytest.hookspec.pytest_runtest_logfinish>`_
<https://docs.pytest.org/en/latest/reference.html#_pytest.hookspec.pytest_runtest_logfinish>`_
hook which is called when a test item has finished executing, analogous to
`pytest_runtest_logstart
<https://docs.pytest.org/en/latest/writing_plugins.html#_pytest.hookspec.pytest_runtest_start>`_.
<https://docs.pytest.org/en/latest/reference.html#_pytest.hookspec.pytest_runtest_logstart>`_.
(`#3101 <https://github.com/pytest-dev/pytest/issues/3101>`_)
- Improve performance when collecting tests using many fixtures. (`#3107
@@ -3305,7 +3915,7 @@ Bug Fixes
Thanks `@sirex`_ for the report and `@nicoddemus`_ for the PR.
* Replace ``raise StopIteration`` usages in the code by simple ``returns`` to finish generators, in accordance to `PEP-479`_ (`#2160`_).
Thanks `@tgoodlet`_ for the report and `@nicoddemus`_ for the PR.
Thanks to `@nicoddemus`_ for the PR.
* Fix internal errors when an unprintable ``AssertionError`` is raised inside a test.
Thanks `@omerhadari`_ for the PR.
@@ -3436,7 +4046,7 @@ Bug Fixes
.. _@syre: https://github.com/syre
.. _@adler-j: https://github.com/adler-j
.. _@d-b-w: https://bitbucket.org/d-b-w/
.. _@d-b-w: https://github.com/d-b-w
.. _@DuncanBetts: https://github.com/DuncanBetts
.. _@dupuy: https://bitbucket.org/dupuy/
.. _@kerrick-lyft: https://github.com/kerrick-lyft
@@ -3496,7 +4106,7 @@ Bug Fixes
.. _@adborden: https://github.com/adborden
.. _@cwitty: https://github.com/cwitty
.. _@d_b_w: https://github.com/d_b_w
.. _@d_b_w: https://github.com/d-b-w
.. _@gdyuldin: https://github.com/gdyuldin
.. _@matclab: https://github.com/matclab
.. _@MSeifert04: https://github.com/MSeifert04
@@ -3531,7 +4141,7 @@ Bug Fixes
Thanks `@axil`_ for the PR.
* Explain a bad scope value passed to ``@fixture`` declarations or
a ``MetaFunc.parametrize()`` call. Thanks `@tgoodlet`_ for the PR.
a ``MetaFunc.parametrize()`` call.
* This version includes ``pluggy-0.4.0``, which correctly handles
``VersionConflict`` errors in plugins (`#704`_).
@@ -3541,7 +4151,6 @@ Bug Fixes
.. _@philpep: https://github.com/philpep
.. _@raquel-ucl: https://github.com/raquel-ucl
.. _@axil: https://github.com/axil
.. _@tgoodlet: https://github.com/tgoodlet
.. _@vlad-dragos: https://github.com/vlad-dragos
.. _#1853: https://github.com/pytest-dev/pytest/issues/1853
@@ -3887,7 +4496,7 @@ time or change existing behaviors in order to make them less surprising/more use
* Updated docstrings with a more uniform style.
* Add stderr write for ``pytest.exit(msg)`` during startup. Previously the message was never shown.
Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@JonathonSonesen`_ and
Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@jgsonesen`_ and
`@tomviner`_ for the PR.
* No longer display the incorrect test deselection reason (`#1372`_).
@@ -3935,7 +4544,7 @@ time or change existing behaviors in order to make them less surprising/more use
Thanks to `@Stranger6667`_ for the PR.
* Fixed the total tests tally in junit xml output (`#1798`_).
Thanks to `@cryporchild`_ for the PR.
Thanks to `@cboelsen`_ for the PR.
* Fixed off-by-one error with lines from ``request.node.warn``.
Thanks to `@blueyed`_ for the PR.
@@ -4008,7 +4617,7 @@ time or change existing behaviors in order to make them less surprising/more use
.. _@BeyondEvil: https://github.com/BeyondEvil
.. _@blueyed: https://github.com/blueyed
.. _@ceridwen: https://github.com/ceridwen
.. _@cryporchild: https://github.com/cryporchild
.. _@cboelsen: https://github.com/cboelsen
.. _@csaftoiu: https://github.com/csaftoiu
.. _@d6e: https://github.com/d6e
.. _@davehunt: https://github.com/davehunt
@@ -4019,7 +4628,7 @@ time or change existing behaviors in order to make them less surprising/more use
.. _@gprasad84: https://github.com/gprasad84
.. _@graingert: https://github.com/graingert
.. _@hartym: https://github.com/hartym
.. _@JonathonSonesen: https://github.com/JonathonSonesen
.. _@jgsonesen: https://github.com/jgsonesen
.. _@kalekundert: https://github.com/kalekundert
.. _@kvas-it: https://github.com/kvas-it
.. _@marscher: https://github.com/marscher
@@ -4156,7 +4765,7 @@ time or change existing behaviors in order to make them less surprising/more use
**Changes**
* **Important**: `py.code <https://pylib.readthedocs.io/en/latest/code.html>`_ has been
* **Important**: `py.code <https://pylib.readthedocs.io/en/stable/code.html>`_ has been
merged into the ``pytest`` repository as ``pytest._code``. This decision
was made because ``py.code`` had very few uses outside ``pytest`` and the
fact that it was in a different repository made it difficult to fix bugs on

84
CODE_OF_CONDUCT.md Normal file
View File

@@ -0,0 +1,84 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at coc@pytest.org. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
The coc@pytest.org address is routed to the following people who can also be
contacted individually:
- Brianna Laugher ([@pfctdayelise](https://github.com/pfctdayelise)): brianna@laugher.id.au
- Bruno Oliveira ([@nicoddemus](https://github.com/nicoddemus)): nicoddemus@gmail.com
- Florian Bruhin ([@the-compiler](https://github.com/the-compiler)): pytest@the-compiler.org
- Ronny Pfannschmidt ([@RonnyPfannschmidt](https://github.com/RonnyPfannschmidt)): ich@ronnypfannschmidt.de
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

View File

@@ -5,8 +5,9 @@ Contribution getting started
Contributions are highly welcomed and appreciated. Every little help counts,
so do not hesitate!
.. contents:: Contribution links
.. contents::
:depth: 2
:backlinks: none
.. _submitfeedback:
@@ -166,14 +167,14 @@ Short version
#. Enable and install `pre-commit <https://pre-commit.com>`_ to ensure style-guides and code checks are followed.
#. Target ``master`` for bugfixes and doc changes.
#. Target ``features`` for new features or functionality changes.
#. Follow **PEP-8** for naming and `black <https://github.com/python/black>`_ for formatting.
#. Follow **PEP-8** for naming and `black <https://github.com/psf/black>`_ for formatting.
#. Tests are run using ``tox``::
tox -e linting,py27,py37
tox -e linting,py37
The test environments above are usually enough to cover most cases locally.
#. Write a ``changelog`` entry: ``changelog/2574.bugfix``, use issue id number
#. Write a ``changelog`` entry: ``changelog/2574.bugfix.rst``, use issue id number
and one of ``bugfix``, ``removal``, ``feature``, ``vendor``, ``doc`` or
``trivial`` for the issue type.
#. Unless your change is a trivial or a documentation fix (e.g., a typo or reword of a small section) please
@@ -217,7 +218,9 @@ Here is a simple overview, with pytest-specific bits:
If you need some help with Git, follow this quick start
guide: https://git.wiki.kernel.org/index.php/QuickStart
#. Install `pre-commit <https://pre-commit.com>`_ and its hook on the pytest repo::
#. Install `pre-commit <https://pre-commit.com>`_ and its hook on the pytest repo:
**Note: pre-commit must be installed as admin, as it will not function otherwise**::
$ pip install --user pre-commit
$ pre-commit install
@@ -237,20 +240,20 @@ Here is a simple overview, with pytest-specific bits:
#. Run all the tests
You need to have Python 2.7 and 3.7 available in your system. Now
You need to have Python 3.7 available in your system. Now
running tests is as simple as issuing this command::
$ tox -e linting,py27,py37
$ tox -e linting,py37
This command will run tests via the "tox" tool against Python 2.7 and 3.7
This command will run tests via the "tox" tool against Python 3.7
and also perform "lint" coding-style checks.
#. You can now edit your local working copy and run the tests again as necessary. Please follow PEP-8 for naming.
You can pass different options to ``tox``. For example, to run tests on Python 2.7 and pass options to pytest
You can pass different options to ``tox``. For example, to run tests on Python 3.7 and pass options to pytest
(e.g. enter pdb on failure) to pytest you can do::
$ tox -e py27 -- --pdb
$ tox -e py37 -- --pdb
Or to only run tests in a particular test module on Python 3.7::
@@ -264,9 +267,10 @@ Here is a simple overview, with pytest-specific bits:
$ git commit -a -m "<commit message>"
$ git push -u
#. Create a new changelog entry in ``changelog``. The file should be named ``<issueid>.<type>``,
#. Create a new changelog entry in ``changelog``. The file should be named ``<issueid>.<type>.rst``,
where *issueid* is the number of the issue related to the change and *type* is one of
``bugfix``, ``removal``, ``feature``, ``vendor``, ``doc`` or ``trivial``.
``bugfix``, ``removal``, ``feature``, ``vendor``, ``doc`` or ``trivial``. You may not create a
changelog entry if the change doesn't affect the documented behaviour of Pytest.
#. Add yourself to ``AUTHORS`` file if not there yet, in alphabetical order.

View File

@@ -12,6 +12,8 @@ taking a lot of time to make a new one.
#. Create a branch ``release-X.Y.Z`` with the version for the release.
* **maintenance releases**: from ``4.6-maintenance``;
* **patch releases**: from the latest ``master``;
* **minor releases**: from the latest ``features``; then merge with the latest ``master``;
@@ -24,7 +26,8 @@ taking a lot of time to make a new one.
This will generate a commit with all the changes ready for pushing.
#. Open a PR for this branch targeting ``master``.
#. Open a PR for this branch targeting ``master`` (or ``4.6-maintenance`` for
maintenance releases).
#. After all tests pass and the PR has been approved, publish to PyPI by pushing the tag::
@@ -33,7 +36,16 @@ taking a lot of time to make a new one.
Wait for the deploy to complete, then make sure it is `available on PyPI <https://pypi.org/project/pytest>`_.
#. Merge the PR into ``master``.
#. Merge the PR.
#. If this is a maintenance release, cherry-pick the CHANGELOG / announce
files to the ``master`` branch::
git fetch --all --prune
git checkout origin/master -b cherry-pick-maintenance-release
git cherry-pick --no-commit -m1 origin/4.6-maintenance
git checkout origin/master -- changelog
git commit # no arguments
#. Send an email announcement with the contents from::

44
OPENCOLLECTIVE.rst Normal file
View File

@@ -0,0 +1,44 @@
==============
OpenCollective
==============
pytest has a collective setup at `OpenCollective`_. This document describes how the core team manages
OpenCollective-related activities.
What is it
==========
Open Collective is an online funding platform for open and transparent communities.
It provides tools to raise money and share your finances in full transparency.
It is the platform of choice for individuals and companies that want to make one-time or
monthly donations directly to the project.
Funds
=====
The OpenCollective funds donated to pytest will be used to fund overall maintenance,
local sprints, merchandising (stickers to distribute in conferences for example), and future
gatherings of pytest developers (sprints).
`Core contributors`_ which are contributing on a continuous basis are free to submit invoices
to bill maintenance hours using the platform. How much each contributor should request is still an
open question, but we should use common sense and trust in the contributors, most of which know
themselves in-person. A good rule of thumb is to bill the same amount as monthly payments
contributors which participate in the `Tidelift`_ subscription. If in doubt, just ask.
Admins
======
A few people have admin access to the OpenCollective dashboard to make changes. Those people
are part of the `@pytest-dev/opencollective-admins`_ team.
`Core contributors`_ interested in helping out with OpenCollective maintenance are welcome! We don't
expect much work here other than the occasional approval of expenses from other core contributors.
Just drop a line to one of the `@pytest-dev/opencollective-admins`_ or use the mailing list.
.. _`OpenCollective`: https://opencollective.com/pytest
.. _`Tidelift`: https://tidelift.com
.. _`core contributors`: https://github.com/orgs/pytest-dev/teams/core/members
.. _`@pytest-dev/opencollective-admins`: https://github.com/orgs/pytest-dev/teams/opencollective-admins/members

View File

@@ -26,7 +26,7 @@
:target: https://dev.azure.com/pytest-dev/pytest
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/python/black
:target: https://github.com/psf/black
.. image:: https://www.codetriage.com/pytest-dev/pytest/badges/users.svg
:target: https://www.codetriage.com/pytest-dev/pytest
@@ -85,7 +85,7 @@ Features
- Can run `unittest <https://docs.pytest.org/en/latest/unittest.html>`_ (or trial),
`nose <https://docs.pytest.org/en/latest/nose.html>`_ test suites out of the box;
- Python 2.7, Python 3.4+, PyPy 2.3, Jython 2.5 (untested);
- Python 3.5+ and PyPy3;
- Rich plugin architecture, with over 315+ `external plugins <http://plugincompat.herokuapp.com>`_ and thriving community;
@@ -111,14 +111,28 @@ Consult the `Changelog <https://docs.pytest.org/en/latest/changelog.html>`__ pag
Support pytest
--------------
You can support pytest by obtaining a `Tideflift subscription`_.
`Open Collective`_ is an online funding platform for open and transparent communities.
It provide tools to raise money and share your finances in full transparency.
Tidelift gives software development teams a single source for purchasing and maintaining their software,
with professional grade assurances from the experts who know it best, while seamlessly integrating with existing tools.
It is the platform of choice for individuals and companies that want to make one-time or
monthly donations directly to the project.
See more datails in the `pytest collective`_.
.. _Open Collective: https://opencollective.com
.. _pytest collective: https://opencollective.com/pytest
.. _`Tideflift subscription`: https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=readme
pytest for enterprise
---------------------
Available as part of the Tidelift Subscription.
The maintainers of pytest and thousands of other packages are working with Tidelift to deliver commercial support and
maintenance for the open source dependencies you use to build your applications.
Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use.
`Learn more. <https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=enterprise&utm_term=repo>`_
Security
^^^^^^^^

View File

@@ -12,6 +12,9 @@ Tidelift aims to make Open Source sustainable by offering subscriptions to compa
on Open Source packages. This subscription allows it to pay maintainers of those Open Source
packages to aid sustainability of the work.
It is the perfect platform for companies that want to support Open Source packages and at the same
time obtain assurances regarding maintenance, quality and security.
Funds
=====

View File

@@ -4,9 +4,6 @@ trigger:
variables:
PYTEST_ADDOPTS: "--junitxml=build/test-results/$(tox.env).xml -vv"
python.needs_vc: False
COVERAGE_FILE: "$(Build.Repository.LocalPath)/.coverage"
COVERAGE_PROCESS_START: "$(Build.Repository.LocalPath)/.coveragerc"
PYTEST_COVERAGE: '0'
jobs:
@@ -16,44 +13,10 @@ jobs:
vmImage: "vs2017-win2016"
strategy:
matrix:
py27:
python.version: '2.7'
tox.env: 'py27'
py27-nobyte-lsof-numpy:
python.version: '2.7'
tox.env: 'py27-lsof-nobyte-numpy'
# Coverage for:
# - test_supports_breakpoint_module_global
# - test_terminal_reporter_writer_attr (without xdist)
# - "if write" branch in _pytest.assertion.rewrite
# - numpy
# - pytester's LsofFdLeakChecker (being skipped)
PYTEST_COVERAGE: '1'
py27-twisted:
python.version: '2.7'
tox.env: 'py27-twisted'
python.needs_vc: True
py27-pluggymaster-xdist:
python.version: '2.7'
tox.env: 'py27-pluggymaster-xdist'
# Coverage for:
# - except-IOError in _attempt_to_close_capture_file for py2.
# Also seen with py27-nobyte (using xdist), and py27-xdist.
# But no exception with py27-pexpect,py27-twisted,py27-numpy.
PYTEST_COVERAGE: '1'
# -- pypy2 and pypy3 are disabled for now: #5279 --
# pypy:
# python.version: 'pypy2'
# tox.env: 'pypy'
# -- pypy3 disabled for now: #5279 --
# pypy3:
# python.version: 'pypy3'
# tox.env: 'pypy3'
py34-xdist:
python.version: '3.4'
tox.env: 'py34-xdist'
# Coverage for:
# - _pytest.compat._bytes_to_ascii
PYTEST_COVERAGE: '1'
py35-xdist:
python.version: '3.5'
tox.env: 'py35-xdist'
@@ -65,7 +28,7 @@ jobs:
tox.env: 'py36-xdist'
py37:
python.version: '3.7'
tox.env: 'py37'
tox.env: 'py37-twisted-numpy'
# Coverage for:
# - _py36_windowsconsoleio_workaround (with py36+)
# - test_request_garbage (no xdist)
@@ -73,9 +36,6 @@ jobs:
py37-linting/docs/doctesting:
python.version: '3.7'
tox.env: 'linting,docs,doctesting'
py37-twisted/numpy:
python.version: '3.7'
tox.env: 'py37-twisted,py37-numpy'
py37-pluggymaster-xdist:
python.version: '3.7'
tox.env: 'py37-pluggymaster-xdist'
@@ -87,15 +47,16 @@ jobs:
versionSpec: '$(python.version)'
architecture: 'x64'
- script: choco install vcpython27
condition: eq(variables['python.needs_vc'], True)
displayName: 'Install VC for py27'
- script: python -m pip install --upgrade pip && python -m pip install tox
displayName: 'Install tox'
- script: |
call scripts/setup-coverage-vars.bat || goto :eof
- bash: |
if [[ "$PYTEST_COVERAGE" == "1" ]]; then
export _PYTEST_TOX_COVERAGE_RUN="coverage run -m"
export _PYTEST_TOX_EXTRA_DEP=coverage-enable-subprocess
export COVERAGE_FILE="$PWD/.coverage"
export COVERAGE_PROCESS_START="$PWD/.coveragerc"
fi
python -m tox -e $(tox.env)
displayName: 'Run tests'
@@ -105,9 +66,12 @@ jobs:
testRunTitle: '$(tox.env)'
condition: succeededOrFailed()
- script: call scripts\upload-coverage.bat
displayName: 'Report and upload coverage'
condition: eq(variables['PYTEST_COVERAGE'], '1')
- bash: |
if [[ "$PYTEST_COVERAGE" == 1 ]]; then
scripts/report-coverage.sh
fi
env:
CODECOV_NAME: $(tox.env)
CODECOV_TOKEN: $(CODECOV_TOKEN)
PYTEST_CODECOV_NAME: $(tox.env)
displayName: Report and upload coverage
condition: eq(variables['PYTEST_COVERAGE'], '1')

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import sys
if __name__ == "__main__":
@@ -7,7 +6,7 @@ if __name__ == "__main__":
import pstats
script = sys.argv[1:] if len(sys.argv) > 1 else ["empty.py"]
stats = cProfile.run("pytest.cmdline.main(%r)" % script, "prof")
cProfile.run("pytest.cmdline.main(%r)" % script, "prof")
p = pstats.Stats("prof")
p.strip_dirs()
p.sort_stats("cumulative")

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
# 10000 iterations, just for relative comparison
# 2.7.5 3.3.2
# FilesCompleter 75.1109 69.2116

View File

@@ -1,3 +1,2 @@
# -*- coding: utf-8 -*-
for i in range(1000):
exec("def test_func_%d(): pass" % i)

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import pytest

View File

@@ -1,6 +1,3 @@
# -*- coding: utf-8 -*-
from six.moves import range
import pytest
SKIP = True

View File

@@ -12,6 +12,7 @@ Each file should be named like ``<ISSUE>.<TYPE>.rst``, where
``<ISSUE>`` is an issue number, and ``<TYPE>`` is one of:
* ``feature``: new user facing features, like new command-line options and new behavior.
* ``improvement``: improvement of existing functionality, usually without requiring user intervention (for example, new fields being written in ``--junitxml``, improved colors in terminal, etc).
* ``bugfix``: fixes a reported bug.
* ``doc``: documentation improvement, like rewording an entire session or adding missing docs.
* ``deprecation``: feature deprecation.

7
codecov.yml Normal file
View File

@@ -0,0 +1,7 @@
coverage:
status:
project: true
patch: true
changes: true
comment: off

View File

@@ -16,7 +16,7 @@ REGENDOC_ARGS := \
--normalize "/[ \t]+\n/\n/" \
--normalize "~\$$REGENDOC_TMPDIR~/home/sweet/project~" \
--normalize "~/path/to/example~/home/sweet/project~" \
--normalize "/in \d+.\d+ seconds/in 0.12 seconds/" \
--normalize "/in \d.\d\ds/in 0.12s/" \
--normalize "@/tmp/pytest-of-.*/pytest-\d+@PYTEST_TMPDIR@" \
--normalize "@pytest-(\d+)\\.[^ ,]+@pytest-\1.x.y@" \
--normalize "@(This is pytest version )(\d+)\\.[^ ,]+@\1\2.x.y@" \

View File

@@ -4,12 +4,15 @@
<li><a href="{{ pathto('index') }}">Home</a></li>
<li><a href="{{ pathto('getting-started') }}">Install</a></li>
<li><a href="{{ pathto('contents') }}">Contents</a></li>
<li><a href="{{ pathto('reference') }}">Reference</a></li>
<li><a href="{{ pathto('reference') }}">API Reference</a></li>
<li><a href="{{ pathto('example/index') }}">Examples</a></li>
<li><a href="{{ pathto('customize') }}">Customize</a></li>
<li><a href="{{ pathto('changelog') }}">Changelog</a></li>
<li><a href="{{ pathto('contributing') }}">Contributing</a></li>
<li><a href="{{ pathto('backwards-compatibility') }}">Backwards Compatibility</a></li>
<li><a href="{{ pathto('py27-py34-deprecation') }}">Python 2.7 and 3.4 Support</a></li>
<li><a href="{{ pathto('sponsor') }}">Sponsor</a></li>
<li><a href="{{ pathto('tidelift') }}">pytest for Enterprise</a></li>
<li><a href="{{ pathto('license') }}">License</a></li>
<li><a href="{{ pathto('contact') }}">Contact Channels</a></li>
</ul>

View File

@@ -16,7 +16,7 @@
{%- block footer %}
<div class="footer">
&copy; Copyright {{ copyright }}.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> {{ sphinx_version }}.
</div>
{% if pagename == 'index' %}
</div>

View File

@@ -0,0 +1,15 @@
{#
basic/searchbox.html with heading removed.
#}
{%- if pagename != "search" and builder != "singlehtml" %}
<div id="searchbox" style="display: none" role="search">
<div class="searchformwrapper">
<form class="search" action="{{ pathto('search') }}" method="get">
<input type="text" name="q" aria-labelledby="searchlabel"
placeholder="Search"/>
<input type="submit" value="{{ _('Go') }}" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
{%- endif %}

View File

@@ -8,11 +8,12 @@
{% set page_width = '1020px' %}
{% set sidebar_width = '220px' %}
/* orange of logo is #d67c29 but we use black for links for now */
{% set link_color = '#000' %}
{% set link_hover_color = '#000' %}
/* muted version of green logo color #C9D22A */
{% set link_color = '#606413' %}
/* blue logo color */
{% set link_hover_color = '#009de0' %}
{% set base_font = 'sans-serif' %}
{% set header_font = 'serif' %}
{% set header_font = 'sans-serif' %}
@import url("basic.css");
@@ -20,7 +21,7 @@
body {
font-family: {{ base_font }};
font-size: 17px;
font-size: 16px;
background-color: white;
color: #000;
margin: 0;
@@ -78,13 +79,13 @@ div.related {
}
div.sphinxsidebar a {
color: #444;
text-decoration: none;
border-bottom: 1px dotted #999;
border-bottom: none;
}
div.sphinxsidebar a:hover {
border-bottom: 1px solid #999;
color: {{ link_hover_color }};
border-bottom: 1px solid {{ link_hover_color }};
}
div.sphinxsidebar {
@@ -106,14 +107,14 @@ div.sphinxsidebar h3,
div.sphinxsidebar h4 {
font-family: {{ header_font }};
color: #444;
font-size: 24px;
font-size: 21px;
font-weight: normal;
margin: 0 0 5px 0;
margin: 16px 0 0 0;
padding: 0;
}
div.sphinxsidebar h4 {
font-size: 20px;
font-size: 18px;
}
div.sphinxsidebar h3 a {
@@ -205,10 +206,22 @@ div.body p, div.body dd, div.body li {
line-height: 1.4em;
}
ul.simple li {
margin-bottom: 0.5em;
}
div.topic ul.simple li {
margin-bottom: 0;
}
div.topic li > p:first-child {
margin-top: 0;
margin-bottom: 0;
}
div.admonition {
background: #fafafa;
margin: 20px -30px;
padding: 10px 30px;
padding: 10px 20px;
border-top: 1px solid #ccc;
border-bottom: 1px solid #ccc;
}
@@ -217,11 +230,6 @@ div.admonition tt.xref, div.admonition a tt {
border-bottom: 1px solid #fafafa;
}
dd div.admonition {
margin-left: -60px;
padding-left: 60px;
}
div.admonition p.admonition-title {
font-family: {{ header_font }};
font-weight: normal;
@@ -231,7 +239,7 @@ div.admonition p.admonition-title {
line-height: 1;
}
div.admonition p.last {
div.admonition :last-child {
margin-bottom: 0;
}
@@ -243,7 +251,7 @@ dt:target, .highlight {
background: #FAF3E8;
}
div.note {
div.note, div.warning {
background-color: #eee;
border: 1px solid #ccc;
}
@@ -257,6 +265,11 @@ div.topic {
background-color: #eee;
}
div.topic a {
text-decoration: none;
border-bottom: none;
}
p.admonition-title {
display: inline;
}
@@ -358,21 +371,10 @@ ul, ol {
pre {
background: #eee;
padding: 7px 30px;
margin: 15px -30px;
padding: 7px 12px;
line-height: 1.3em;
}
dl pre, blockquote pre, li pre {
margin-left: -60px;
padding-left: 60px;
}
dl dl pre {
margin-left: -90px;
padding-left: 90px;
}
tt {
background-color: #ecf0f3;
color: #222;
@@ -393,6 +395,20 @@ a.reference:hover {
border-bottom: 1px solid {{ link_hover_color }};
}
li.toctree-l1 a.reference,
li.toctree-l2 a.reference,
li.toctree-l3 a.reference,
li.toctree-l4 a.reference {
border-bottom: none;
}
li.toctree-l1 a.reference:hover,
li.toctree-l2 a.reference:hover,
li.toctree-l3 a.reference:hover,
li.toctree-l4 a.reference:hover {
border-bottom: 1px solid {{ link_hover_color }};
}
a.footnote-reference {
text-decoration: none;
font-size: 0.7em;
@@ -408,6 +424,56 @@ a:hover tt {
background: #EEE;
}
#reference div.section h2 {
/* separate code elements in the reference section */
border-top: 2px solid #ccc;
padding-top: 0.5em;
}
#reference div.section h3 {
/* separate code elements in the reference section */
border-top: 1px solid #ccc;
padding-top: 0.5em;
}
dl.class, dl.function {
margin-top: 1em;
margin-bottom: 1em;
}
dl.class > dd {
border-left: 3px solid #ccc;
margin-left: 0px;
padding-left: 30px;
}
dl.field-list {
flex-direction: column;
}
dl.field-list dd {
padding-left: 4em;
border-left: 3px solid #ccc;
margin-bottom: 0.5em;
}
dl.field-list dd > ul {
list-style: none;
padding-left: 0px;
}
dl.field-list dd > ul > li li :first-child {
text-indent: 0;
}
dl.field-list dd > ul > li :first-child {
text-indent: -2em;
padding-left: 0px;
}
dl.field-list dd > p:first-child {
text-indent: -2em;
}
@media screen and (max-width: 870px) {

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
# flasky extensions. flasky pygments style based on tango style
from pygments.style import Style
from pygments.token import Comment

View File

@@ -24,11 +24,9 @@ The ideal pytest helper
- feels confident in using pytest (e.g. has explored command line options, knows how to write parametrized tests, has an idea about conftest contents)
- does not need to be an expert in every aspect!
`Pytest helpers, sign up here`_! (preferably in February, hard deadline 22 March)
Pytest helpers, sign up here! (preferably in February, hard deadline 22 March)
.. _`Pytest helpers, sign up here`: http://goo.gl/forms/nxqAhqWt1P
The ideal partner project
-----------------------------------------
@@ -40,11 +38,9 @@ The ideal partner project
- has the support of the core development team, in trying out pytest adoption
- has no tests... or 100% test coverage... or somewhere in between!
`Partner projects, sign up here`_! (by 22 March)
Partner projects, sign up here! (by 22 March)
.. _`Partner projects, sign up here`: http://goo.gl/forms/ZGyqlHiwk3
What does it mean to "adopt pytest"?
-----------------------------------------
@@ -68,11 +64,11 @@ Progressive success might look like:
It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies.
.. _`nose and unittest`: faq.html#how-does-pytest-relate-to-nose-and-unittest
.. _assert: asserts.html
.. _assert: assert.html
.. _pycmd: https://bitbucket.org/hpk42/pycmd/overview
.. _`setUp/tearDown methods`: xunit_setup.html
.. _fixtures: fixture.html
.. _markers: markers.html
.. _markers: mark.html
.. _distributed: xdist.html

View File

@@ -6,6 +6,20 @@ Release announcements
:maxdepth: 2
release-5.2.2
release-5.2.1
release-5.2.0
release-5.1.3
release-5.1.2
release-5.1.1
release-5.1.0
release-5.0.1
release-5.0.0
release-4.6.5
release-4.6.4
release-4.6.3
release-4.6.2
release-4.6.1
release-4.6.0
release-4.5.0
release-4.4.2

View File

@@ -12,7 +12,7 @@ courtesy of Benjamin Peterson. You can now safely use ``assert``
statements in test modules without having to worry about side effects
or python optimization ("-OO") options. This is achieved by rewriting
assert statements in test modules upon import, using a PEP302 hook.
See http://pytest.org/assert.html#advanced-assertion-introspection for
See https://docs.pytest.org/en/latest/assert.html for
detailed information. The work has been partly sponsored by my company,
merlinux GmbH.

View File

@@ -75,7 +75,7 @@ The py.test Development Team
**Changes**
* **Important**: `py.code <https://pylib.readthedocs.io/en/latest/code.html>`_ has been
* **Important**: `py.code <https://pylib.readthedocs.io/en/stable/code.html>`_ has been
merged into the ``pytest`` repository as ``pytest._code``. This decision
was made because ``py.code`` had very few uses outside ``pytest`` and the
fact that it was in a different repository made it difficult to fix bugs on
@@ -88,7 +88,7 @@ The py.test Development Team
**experimental**, so you definitely should not import it explicitly!
Please note that the original ``py.code`` is still available in
`pylib <https://pylib.readthedocs.io>`_.
`pylib <https://pylib.readthedocs.io/en/stable/>`_.
* ``pytest_enter_pdb`` now optionally receives the pytest config object.
Thanks `@nicoddemus`_ for the PR.

View File

@@ -66,8 +66,8 @@ The py.test Development Team
.. _#510: https://github.com/pytest-dev/pytest/issues/510
.. _#1506: https://github.com/pytest-dev/pytest/pull/1506
.. _#1496: https://github.com/pytest-dev/pytest/issue/1496
.. _#1524: https://github.com/pytest-dev/pytest/issue/1524
.. _#1496: https://github.com/pytest-dev/pytest/issues/1496
.. _#1524: https://github.com/pytest-dev/pytest/pull/1524
.. _@astraw38: https://github.com/astraw38
.. _@hackebrot: https://github.com/hackebrot

View File

@@ -20,7 +20,7 @@ Thanks to all who contributed to this release, among them:
* Jeffrey Rackauckas
* Jose Carlos Menezes
* Ronny Pfannschmidt
* Zac-HD
* Zac Hatfield-Dodds
* iwanb

View File

@@ -21,7 +21,6 @@ Thanks to all who contributed to this release, among them:
* Kyle Altendorf
* Stephan Hoyer
* Zac Hatfield-Dodds
* Zac-HD
* songbowen

View File

@@ -28,7 +28,6 @@ Thanks to all who contributed to this release, among them:
* Pulkit Goyal
* Samuel Searles-Bryant
* Zac Hatfield-Dodds
* Zac-HD
Happy testing,

View File

@@ -0,0 +1,19 @@
pytest-4.6.1
=======================================
pytest 4.6.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,18 @@
pytest-4.6.2
=======================================
pytest 4.6.2 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,21 @@
pytest-4.6.3
=======================================
pytest 4.6.3 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* Dirk Thomas
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,22 @@
pytest-4.6.4
=======================================
pytest 4.6.4 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* Thomas Grainger
* Zac Hatfield-Dodds
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,21 @@
pytest-4.6.5
=======================================
pytest 4.6.5 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* Thomas Grainger
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,46 @@
pytest-5.0.0
=======================================
The pytest team is proud to announce the 5.0.0 release!
pytest is a mature Python testing tool with more than a 2000 tests
against itself, passing on many different interpreters and platforms.
This release contains a number of bugs fixes and improvements, so users are encouraged
to take a look at the CHANGELOG:
https://docs.pytest.org/en/latest/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/latest/
As usual, you can upgrade from pypi via:
pip install -U pytest
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* Dirk Thomas
* Evan Kepner
* Florian Bruhin
* Hugo
* Kevin J. Foley
* Pulkit Goyal
* Ralph Giles
* Ronny Pfannschmidt
* Thomas Grainger
* Thomas Hisch
* Tim Gates
* Victor Maryama
* Yuri Apollov
* Zac Hatfield-Dodds
* curiousjazz77
* patriksevallius
Happy testing,
The Pytest Development Team

View File

@@ -0,0 +1,25 @@
pytest-5.0.1
=======================================
pytest 5.0.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* AmirElkess
* Andreu Vallbona Plazas
* Anthony Sottile
* Bruno Oliveira
* Florian Bruhin
* Michael Moore
* Niklas Meinzer
* Thomas Grainger
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,56 @@
pytest-5.1.0
=======================================
The pytest team is proud to announce the 5.1.0 release!
pytest is a mature Python testing tool with more than a 2000 tests
against itself, passing on many different interpreters and platforms.
This release contains a number of bugs fixes and improvements, so users are encouraged
to take a look at the CHANGELOG:
https://docs.pytest.org/en/latest/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/latest/
As usual, you can upgrade from pypi via:
pip install -U pytest
Thanks to all who contributed to this release, among them:
* Albert Tugushev
* Alexey Zankevich
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* David Röthlisberger
* Florian Bruhin
* Ilya Stepin
* Jon Dufresne
* Kaiqi
* Max R
* Miro Hrončok
* Oliver Bestwalter
* Ran Benita
* Ronny Pfannschmidt
* Samuel Searles-Bryant
* Semen Zhydenko
* Steffen Schroeder
* Thomas Grainger
* Tim Hoffmann
* William Woodall
* Wojtek Erbetowski
* Xixi Zhao
* Yash Todi
* boris
* dmitry.dygalo
* helloocc
* martbln
* mei-li
Happy testing,
The Pytest Development Team

View File

@@ -0,0 +1,24 @@
pytest-5.1.1
=======================================
pytest 5.1.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* Florian Bruhin
* Hugo van Kemenade
* Ran Benita
* Ronny Pfannschmidt
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,23 @@
pytest-5.1.2
=======================================
pytest 5.1.2 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Andrzej Klajnert
* Anthony Sottile
* Bruno Oliveira
* Christian Neumüller
* Robert Holt
* linchiwei123
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,23 @@
pytest-5.1.3
=======================================
pytest 5.1.3 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Christian Neumüller
* Daniel Hahler
* Gene Wood
* Hugo
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,35 @@
pytest-5.2.0
=======================================
The pytest team is proud to announce the 5.2.0 release!
pytest is a mature Python testing tool with more than a 2000 tests
against itself, passing on many different interpreters and platforms.
This release contains a number of bugs fixes and improvements, so users are encouraged
to take a look at the CHANGELOG:
https://docs.pytest.org/en/latest/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/latest/
As usual, you can upgrade from pypi via:
pip install -U pytest
Thanks to all who contributed to this release, among them:
* Andrzej Klajnert
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* James Cooke
* Michael Goerz
* Ran Benita
* Tomáš Chvátal
Happy testing,
The Pytest Development Team

View File

@@ -0,0 +1,23 @@
pytest-5.2.1
=======================================
pytest 5.2.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Anthony Sottile
* Bruno Oliveira
* Florian Bruhin
* Hynek Schlawack
* Kevin J. Foley
* tadashigaki
Happy testing,
The pytest Development Team

View File

@@ -0,0 +1,29 @@
pytest-5.2.2
=======================================
pytest 5.2.2 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/latest/changelog.html.
Thanks to all who contributed to this release, among them:
* Albert Tugushev
* Andrzej Klajnert
* Anthony Sottile
* Bruno Oliveira
* Daniel Hahler
* Florian Bruhin
* Nattaphoom Chaipreecha
* Oliver Bestwalter
* Philipp Loose
* Ran Benita
* Victor Maryama
* Yoav Caspi
Happy testing,
The pytest Development Team

View File

@@ -31,7 +31,7 @@ you will see the return value of the function call:
$ pytest test_assert1.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -47,7 +47,7 @@ you will see the return value of the function call:
E + where 3 = f()
test_assert1.py:6: AssertionError
========================= 1 failed in 0.12 seconds =========================
============================ 1 failed in 0.12s =============================
``pytest`` has support for showing the values of the most common subexpressions
including calls, attributes, comparisons, and binary and unary
@@ -186,7 +186,7 @@ if you run this module:
$ pytest test_assert2.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -208,7 +208,7 @@ if you run this module:
E Use -v to get the full diff
test_assert2.py:6: AssertionError
========================= 1 failed in 0.12 seconds =========================
============================ 1 failed in 0.12s =============================
Special comparisons are done for a number of cases:
@@ -238,14 +238,17 @@ file which provides an alternative explanation for ``Foo`` objects:
def pytest_assertrepr_compare(op, left, right):
if isinstance(left, Foo) and isinstance(right, Foo) and op == "==":
return ["Comparing Foo instances:", " vals: %s != %s" % (left.val, right.val)]
return [
"Comparing Foo instances:",
" vals: {} != {}".format(left.val, right.val),
]
now, given this test module:
.. code-block:: python
# content of test_foocompare.py
class Foo(object):
class Foo:
def __init__(self, val):
self.val = val
@@ -276,7 +279,7 @@ the conftest file:
E vals: 1 != 2
test_foocompare.py:12: AssertionError
1 failed in 0.12 seconds
1 failed in 0.12s
.. _assert-details:
.. _`assert introspection`:

View File

@@ -104,6 +104,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
Captured logs are available through the following properties/methods::
* caplog.messages -> list of format-interpolated log messages
* caplog.text -> string containing formatted log output
* caplog.records -> list of logging.LogRecord instances
* caplog.record_tuples -> list of (logger_name, level, message) tuples
@@ -160,9 +161,12 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
in python < 3.6 this is a pathlib2.Path
no tests ran in 0.12 seconds
no tests ran in 0.12s
You can also interactively ask for help, e.g. by typing on the Python interactive prompt something like::
You can also interactively ask for help, e.g. by typing on the Python interactive prompt something like:
.. code-block:: python
import pytest
help(pytest)

View File

@@ -33,15 +33,18 @@ Other plugins may access the `config.cache`_ object to set/get
Rerunning only failures or failures first
-----------------------------------------------
First, let's create 50 test invocation of which only 2 fail::
First, let's create 50 test invocation of which only 2 fail:
.. code-block:: python
# content of test_50.py
import pytest
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
pytest.fail("bad luck")
pytest.fail("bad luck")
If you run this for the first time you will see two failures:
@@ -57,10 +60,10 @@ If you run this for the first time you will see two failures:
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:6: Failed
test_50.py:7: Failed
_______________________________ test_num[25] _______________________________
i = 25
@@ -68,11 +71,11 @@ If you run this for the first time you will see two failures:
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:6: Failed
2 failed, 48 passed in 0.12 seconds
test_50.py:7: Failed
2 failed, 48 passed in 0.12s
If you then run it with ``--lf``:
@@ -80,7 +83,7 @@ If you then run it with ``--lf``:
$ pytest --lf
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 50 items / 48 deselected / 2 selected
@@ -96,10 +99,10 @@ If you then run it with ``--lf``:
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:6: Failed
test_50.py:7: Failed
_______________________________ test_num[25] _______________________________
i = 25
@@ -107,14 +110,14 @@ If you then run it with ``--lf``:
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:6: Failed
================= 2 failed, 48 deselected in 0.12 seconds ==================
test_50.py:7: Failed
===================== 2 failed, 48 deselected in 0.12s =====================
You have run only the two failing test from the last run, while 48 tests have
not been run ("deselected").
You have run only the two failing tests from the last run, while the 48 passing
tests have not been run ("deselected").
Now, if you run with the ``--ff`` option, all tests will be run but the first
previous failures will be executed first (as can be seen from the series
@@ -124,7 +127,7 @@ of ``FF`` and dots):
$ pytest --ff
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 50 items
@@ -140,10 +143,10 @@ of ``FF`` and dots):
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:6: Failed
test_50.py:7: Failed
_______________________________ test_num[25] _______________________________
i = 25
@@ -151,11 +154,11 @@ of ``FF`` and dots):
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:6: Failed
=================== 2 failed, 48 passed in 0.12 seconds ====================
test_50.py:7: Failed
======================= 2 failed, 48 passed in 0.12s =======================
.. _`config.cache`:
@@ -183,15 +186,19 @@ The new config.cache object
Plugins or conftest.py support code can get a cached value using the
pytest ``config`` object. Here is a basic example plugin which
implements a :ref:`fixture` which re-uses previously created state
across pytest invocations::
across pytest invocations:
.. code-block:: python
# content of test_caching.py
import pytest
import time
def expensive_computation():
print("running expensive computation...")
@pytest.fixture
def mydata(request):
val = request.config.cache.get("example/value", None)
@@ -201,6 +208,7 @@ across pytest invocations::
request.config.cache.set("example/value", val)
return val
def test_function(mydata):
assert mydata == 23
@@ -219,12 +227,12 @@ If you run this command for the first time, you can see the print statement:
> assert mydata == 23
E assert 42 == 23
test_caching.py:17: AssertionError
test_caching.py:20: AssertionError
-------------------------- Captured stdout setup ---------------------------
running expensive computation...
1 failed in 0.12 seconds
1 failed in 0.12s
If you run it a second time the value will be retrieved from
If you run it a second time, the value will be retrieved from
the cache and nothing will be printed:
.. code-block:: pytest
@@ -240,8 +248,8 @@ the cache and nothing will be printed:
> assert mydata == 23
E assert 42 == 23
test_caching.py:17: AssertionError
1 failed in 0.12 seconds
test_caching.py:20: AssertionError
1 failed in 0.12s
See the :ref:`cache-api` for more details.
@@ -256,7 +264,7 @@ You can always peek at the content of the cache using the
$ pytest --cache-show
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
@@ -269,13 +277,66 @@ You can always peek at the content of the cache using the
'test_caching.py::test_function': True,
'test_foocompare.py::test_compare': True}
cache/nodeids contains:
['test_caching.py::test_function']
['test_assert1.py::test_function',
'test_assert2.py::test_set_comparison',
'test_foocompare.py::test_compare',
'test_50.py::test_num[0]',
'test_50.py::test_num[1]',
'test_50.py::test_num[2]',
'test_50.py::test_num[3]',
'test_50.py::test_num[4]',
'test_50.py::test_num[5]',
'test_50.py::test_num[6]',
'test_50.py::test_num[7]',
'test_50.py::test_num[8]',
'test_50.py::test_num[9]',
'test_50.py::test_num[10]',
'test_50.py::test_num[11]',
'test_50.py::test_num[12]',
'test_50.py::test_num[13]',
'test_50.py::test_num[14]',
'test_50.py::test_num[15]',
'test_50.py::test_num[16]',
'test_50.py::test_num[17]',
'test_50.py::test_num[18]',
'test_50.py::test_num[19]',
'test_50.py::test_num[20]',
'test_50.py::test_num[21]',
'test_50.py::test_num[22]',
'test_50.py::test_num[23]',
'test_50.py::test_num[24]',
'test_50.py::test_num[25]',
'test_50.py::test_num[26]',
'test_50.py::test_num[27]',
'test_50.py::test_num[28]',
'test_50.py::test_num[29]',
'test_50.py::test_num[30]',
'test_50.py::test_num[31]',
'test_50.py::test_num[32]',
'test_50.py::test_num[33]',
'test_50.py::test_num[34]',
'test_50.py::test_num[35]',
'test_50.py::test_num[36]',
'test_50.py::test_num[37]',
'test_50.py::test_num[38]',
'test_50.py::test_num[39]',
'test_50.py::test_num[40]',
'test_50.py::test_num[41]',
'test_50.py::test_num[42]',
'test_50.py::test_num[43]',
'test_50.py::test_num[44]',
'test_50.py::test_num[45]',
'test_50.py::test_num[46]',
'test_50.py::test_num[47]',
'test_50.py::test_num[48]',
'test_50.py::test_num[49]',
'test_caching.py::test_function']
cache/stepwise contains:
[]
example/value contains:
42
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
``--cache-show`` takes an optional argument to specify a glob pattern for
filtering:
@@ -284,7 +345,7 @@ filtering:
$ pytest --cache-show example/*
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
@@ -292,7 +353,7 @@ filtering:
example/value contains:
42
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
Clearing Cache content
----------------------

View File

@@ -49,16 +49,21 @@ Using print statements for debugging
---------------------------------------------------
One primary benefit of the default capturing of stdout/stderr output
is that you can use print statements for debugging::
is that you can use print statements for debugging:
.. code-block:: python
# content of test_module.py
def setup_function(function):
print("setting up %s" % function)
print("setting up", function)
def test_func1():
assert True
def test_func2():
assert False
@@ -69,7 +74,7 @@ of the failing function and hide the other one:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -83,10 +88,10 @@ of the failing function and hide the other one:
> assert False
E assert False
test_module.py:9: AssertionError
test_module.py:12: AssertionError
-------------------------- Captured stdout setup ---------------------------
setting up <function test_func2 at 0xdeadbeef>
==================== 1 failed, 1 passed in 0.12 seconds ====================
======================= 1 failed, 1 passed in 0.12s ========================
Accessing captured output from a test function
---------------------------------------------------

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
#
# pytest documentation build configuration file, created by
# sphinx-quickstart on Fri Oct 8 17:54:28 2010.
@@ -16,7 +15,6 @@
#
# The full version, including alpha/beta/rc tags.
# The short X.Y version.
import datetime
import os
import sys
@@ -63,9 +61,8 @@ source_suffix = ".rst"
master_doc = "contents"
# General information about the project.
project = u"pytest"
year = datetime.datetime.utcnow().year
copyright = u"20152019 , holger krekel and pytest-dev team"
project = "pytest"
copyright = "20152019, holger krekel and pytest-dev team"
# The language for content autogenerated by Sphinx. Refer to documentation
@@ -168,18 +165,18 @@ html_favicon = "img/pytest1favi.ico"
html_sidebars = {
"index": [
"slim_searchbox.html",
"sidebarintro.html",
"globaltoc.html",
"links.html",
"sourcelink.html",
"searchbox.html",
],
"**": [
"slim_searchbox.html",
"globaltoc.html",
"relations.html",
"links.html",
"sourcelink.html",
"searchbox.html",
],
}
@@ -233,8 +230,8 @@ latex_documents = [
(
"contents",
"pytest.tex",
u"pytest Documentation",
u"holger krekel, trainer and consultant, http://merlinux.eu",
"pytest Documentation",
"holger krekel, trainer and consultant, http://merlinux.eu",
"manual",
)
]
@@ -266,16 +263,16 @@ latex_domain_indices = False
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [("usage", "pytest", u"pytest usage", [u"holger krekel at merlinux eu"], 1)]
man_pages = [("usage", "pytest", "pytest usage", ["holger krekel at merlinux eu"], 1)]
# -- Options for Epub output ---------------------------------------------------
# Bibliographic Dublin Core info.
epub_title = u"pytest"
epub_author = u"holger krekel at merlinux eu"
epub_publisher = u"holger krekel at merlinux eu"
epub_copyright = u"2013, holger krekel et alii"
epub_title = "pytest"
epub_author = "holger krekel at merlinux eu"
epub_publisher = "holger krekel at merlinux eu"
epub_copyright = "2013, holger krekel et alii"
# The language of the text. It defaults to the language option
# or en if the language is not set.

View File

@@ -1,2 +1 @@
# -*- coding: utf-8 -*-
collect_ignore = ["conf.py"]

View File

@@ -38,19 +38,24 @@ Full pytest documentation
customize
example/index
bash-completion
faq
backwards-compatibility
deprecations
py27-py34-deprecation
historical-notes
license
contributing
development_guide
sponsor
tidelift
license
contact
historical-notes
talks
projects
faq
contact
tidelift
.. only:: html

View File

@@ -107,8 +107,8 @@ check for ini-files as follows:
# first look for pytest.ini files
path/pytest.ini
path/setup.cfg # must also contain [tool:pytest] section to match
path/tox.ini # must also contain [pytest] section to match
path/setup.cfg # must also contain [tool:pytest] section to match
pytest.ini
... # all the way down to the root
@@ -134,10 +134,13 @@ progress output, you can write it into a configuration file:
.. code-block:: ini
# content of pytest.ini or tox.ini
# setup.cfg files should use [tool:pytest] section instead
[pytest]
addopts = -ra -q
# content of setup.cfg
[tool:pytest]
addopts = -ra -q
Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command
line options while the environment is in use:

View File

@@ -19,12 +19,62 @@ Below is a complete list of all pytest features which are considered deprecated.
:class:`_pytest.warning_types.PytestWarning` or subclasses, which can be filtered using
:ref:`standard warning filters <warnings>`.
``funcargnames`` alias for ``fixturenames``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 5.0
The ``FixtureRequest``, ``Metafunc``, and ``Function`` classes track the names of
their associated fixtures, with the aptly-named ``fixturenames`` attribute.
Prior to pytest 2.3, this attribute was named ``funcargnames``, and we have kept
that as an alias since. It is finally due for removal, as it is often confusing
in places where we or plugin authors must distinguish between fixture names and
names supplied by non-fixture things such as ``pytest.mark.parametrize``.
Result log (``--result-log``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 4.0
The ``--result-log`` option produces a stream of test reports which can be
analysed at runtime. It uses a custom format which requires users to implement their own
parser, but the team believes using a line-based format that can be parsed using standard
tools would provide a suitable and better alternative.
The current plan is to provide an alternative in the pytest 5.0 series and remove the ``--result-log``
option in pytest 6.0 after the new implementation proves satisfactory to all users and is deemed
stable.
The actual alternative is still being discussed in issue `#4488 <https://github.com/pytest-dev/pytest/issues/4488>`__.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
``pytest.config`` global
~~~~~~~~~~~~~~~~~~~~~~~~
.. versionremoved:: 5.0
The ``pytest.config`` global object is deprecated. Instead use
``request.config`` (via the ``request`` fixture) or if you are a plugin author
use the ``pytest_configure(config)`` hook. Note that many hooks can also access
the ``config`` object indirectly, through ``session.config`` or ``item.config`` for example.
.. _`raises message deprecated`:
``"message"`` parameter of ``pytest.raises``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 4.1
.. versionremoved:: 5.0
It is a common mistake to think this parameter will match the exception message, while in fact
it only serves to provide a custom message in case the ``pytest.raises`` check fails. To prevent
@@ -55,22 +105,12 @@ If you still have concerns about this deprecation and future removal, please com
`issue #3974 <https://github.com/pytest-dev/pytest/issues/3974>`__.
``pytest.config`` global
~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 4.1
The ``pytest.config`` global object is deprecated. Instead use
``request.config`` (via the ``request`` fixture) or if you are a plugin author
use the ``pytest_configure(config)`` hook. Note that many hooks can also access
the ``config`` object indirectly, through ``session.config`` or ``item.config`` for example.
.. _raises-warns-exec:
``raises`` / ``warns`` with a string as the second argument
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 4.1
.. versionremoved:: 5.0
Use the context manager form of these instead. When necessary, invoke ``exec``
directly.
@@ -102,26 +142,6 @@ Becomes:
Result log (``--result-log``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The ``--resultlog`` command line option has been deprecated: it is little used
and there are more modern and better alternatives, for example `pytest-tap <https://tappy.readthedocs.io/en/latest/>`_.
This feature will be effectively removed in pytest 4.0 as the team intends to include a better alternative in the core.
If you have any concerns, please don't hesitate to `open an issue <https://github.com/pytest-dev/pytest/issues>`__.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
Using ``Class`` in custom Collectors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -439,7 +459,9 @@ Internal classes accessed through ``Node``
.. versionremoved:: 4.0
Access of ``Module``, ``Function``, ``Class``, ``Instance``, ``File`` and ``Item`` through ``Node`` instances now issue
this warning::
this warning:
.. code-block:: text
usage of Function.Module is deprecated, please use pytest.Module instead

View File

@@ -29,14 +29,14 @@ then you can just invoke ``pytest`` directly:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_example.txt . [100%]
========================= 1 passed in 0.12 seconds =========================
============================ 1 passed in 0.12s =============================
By default, pytest will collect ``test*.txt`` files looking for doctest directives, but you
can pass additional globs using the ``--doctest-glob`` option (multi-allowed).
@@ -58,7 +58,7 @@ and functions, including from test modules:
$ pytest --doctest-modules
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -66,7 +66,7 @@ and functions, including from test modules:
mymodule.py . [ 50%]
test_example.txt . [100%]
========================= 2 passed in 0.12 seconds =========================
============================ 2 passed in 0.12s =============================
You can make these changes permanent in your project by
putting them into a pytest.ini file like this:
@@ -103,7 +103,7 @@ that will be used for those doctest files using the
Using 'doctest' options
-----------------------
The standard ``doctest`` module provides some `options <https://docs.python.org/3/library/doctest.html#option-flags>`__
Python's standard ``doctest`` module provides some `options <https://docs.python.org/3/library/doctest.html#option-flags>`__
to configure the strictness of doctest tests. In pytest, you can enable those flags using the
configuration file.
@@ -115,23 +115,52 @@ lengthy exception stack traces you can just write:
[pytest]
doctest_optionflags= NORMALIZE_WHITESPACE IGNORE_EXCEPTION_DETAIL
pytest also introduces new options to allow doctests to run in Python 2 and
Python 3 unchanged:
* ``ALLOW_UNICODE``: when enabled, the ``u`` prefix is stripped from unicode
strings in expected doctest output.
* ``ALLOW_BYTES``: when enabled, the ``b`` prefix is stripped from byte strings
in expected doctest output.
Alternatively, options can be enabled by an inline comment in the doc test
itself:
.. code-block:: rst
# content of example.rst
>>> get_unicode_greeting() # doctest: +ALLOW_UNICODE
'Hello'
>>> something_that_raises() # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
ValueError: ...
pytest also introduces new options:
* ``ALLOW_UNICODE``: when enabled, the ``u`` prefix is stripped from unicode
strings in expected doctest output. This allows doctests to run in Python 2
and Python 3 unchanged.
* ``ALLOW_BYTES``: similarly, the ``b`` prefix is stripped from byte strings
in expected doctest output.
* ``NUMBER``: when enabled, floating-point numbers only need to match as far as
the precision you have written in the expected doctest output. For example,
the following output would only need to match to 2 decimal places::
>>> math.pi
3.14
If you wrote ``3.1416`` then the actual output would need to match to 4
decimal places; and so on.
This avoids false positives caused by limited floating-point precision, like
this::
Expected:
0.233
Got:
0.23300000000000001
``NUMBER`` also supports lists of floating-point numbers -- in fact, it
matches floating-point numbers appearing anywhere in the output, even inside
a string! This means that it may not be appropriate to enable globally in
``doctest_optionflags`` in your configuration file.
.. versionadded:: 5.1
Continue on failure
-------------------
By default, pytest would report only the first failure for a given doctest. If
you want to continue the test even when you have failures, do:
@@ -191,15 +220,21 @@ namespace in which your doctests run. It is intended to be used within
your own fixtures to provide the tests that use them with context.
``doctest_namespace`` is a standard ``dict`` object into which you
place the objects you want to appear in the doctest namespace::
place the objects you want to appear in the doctest namespace:
.. code-block:: python
# content of conftest.py
import numpy
@pytest.fixture(autouse=True)
def add_np(doctest_namespace):
doctest_namespace['np'] = numpy
doctest_namespace["np"] = numpy
which can then be used in your doctests directly::
which can then be used in your doctests directly:
.. code-block:: python
# content of numpy.py
def arange():
@@ -219,7 +254,9 @@ Skipping tests dynamically
.. versionadded:: 4.4
You can use ``pytest.skip`` to dynamically skip doctests. For example::
You can use ``pytest.skip`` to dynamically skip doctests. For example:
.. code-block:: text
>>> import sys, pytest
>>> if sys.platform.startswith('win'):

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import _pytest._code
import pytest
from pytest import raises
@@ -21,7 +20,7 @@ def test_generative(param1, param2):
assert param1 * 2 < param2
class TestFailing(object):
class TestFailing:
def test_simple(self):
def f():
return 42
@@ -41,7 +40,7 @@ class TestFailing(object):
assert not f()
class TestSpecialisedExplanations(object):
class TestSpecialisedExplanations:
def test_eq_text(self):
assert "spam" == "eggs"
@@ -101,7 +100,7 @@ class TestSpecialisedExplanations(object):
from dataclasses import dataclass
@dataclass
class Foo(object):
class Foo:
a: int
b: str
@@ -113,7 +112,7 @@ class TestSpecialisedExplanations(object):
import attr
@attr.s
class Foo(object):
class Foo:
a = attr.ib()
b = attr.ib()
@@ -123,7 +122,7 @@ class TestSpecialisedExplanations(object):
def test_attribute():
class Foo(object):
class Foo:
b = 1
i = Foo()
@@ -131,14 +130,14 @@ def test_attribute():
def test_attribute_instance():
class Foo(object):
class Foo:
b = 1
assert Foo().b == 2
def test_attribute_failure():
class Foo(object):
class Foo:
def _get_b(self):
raise Exception("Failed to get attrib")
@@ -149,10 +148,10 @@ def test_attribute_failure():
def test_attribute_multiple():
class Foo(object):
class Foo:
b = 1
class Bar(object):
class Bar:
b = 2
assert Foo().b == Bar().b
@@ -162,7 +161,7 @@ def globf(x):
return x + 1
class TestRaises(object):
class TestRaises:
def test_raises(self):
s = "qwe"
raises(TypeError, int, s)
@@ -178,7 +177,7 @@ class TestRaises(object):
def test_reinterpret_fails_with_print_for_the_fun_of_it(self):
items = [1, 2, 3]
print("items is %r" % items)
print("items is {!r}".format(items))
a, b = items.pop()
def test_some_error(self):
@@ -191,19 +190,20 @@ class TestRaises(object):
# thanks to Matthew Scott for this test
def test_dynamic_compile_shows_nicely():
import imp
import importlib.util
import sys
src = "def foo():\n assert 1 == 0\n"
name = "abc-123"
module = imp.new_module(name)
spec = importlib.util.spec_from_loader(name, loader=None)
module = importlib.util.module_from_spec(spec)
code = _pytest._code.compile(src, name, "exec")
exec(code, module.__dict__)
sys.modules[name] = module
module.foo()
class TestMoreErrors(object):
class TestMoreErrors:
def test_complex_error(self):
def f():
return 44
@@ -253,16 +253,16 @@ class TestMoreErrors(object):
x = 0
class TestCustomAssertMsg(object):
class TestCustomAssertMsg:
def test_single_line(self):
class A(object):
class A:
a = 1
b = 2
assert A.a == b, "A.a appears not to be b"
def test_multiline(self):
class A(object):
class A:
a = 1
b = 2
@@ -271,7 +271,7 @@ class TestCustomAssertMsg(object):
), "A.a appears not to be b\nor does not appear to be b\none of those"
def test_custom_repr(self):
class JSON(object):
class JSON:
a = 1
def __repr__(self):

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import py
import pytest

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
hello = "world"

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import py
failure_demo = py.path.local(__file__).dirpath("failure_demo.py")

View File

@@ -1,9 +1,8 @@
# -*- coding: utf-8 -*-
def setup_module(module):
module.TestStateFullThing.classcount = 0
class TestStateFullThing(object):
class TestStateFullThing:
def setup_class(cls):
cls.classcount += 1

View File

@@ -18,7 +18,7 @@ example: specifying and selecting acceptance tests
return AcceptFixture(request)
class AcceptFixture(object):
class AcceptFixture:
def __init__(self, request):
if not request.config.getoption("acceptance"):
pytest.skip("specify -A to run acceptance tests")
@@ -65,7 +65,7 @@ extend the `accept example`_ by putting this in our test module:
return arg
class TestSpecialAcceptance(object):
class TestSpecialAcceptance:
def test_sometest(self, accept):
assert accept.tmpdir.join("special").check()

View File

@@ -1,2 +1 @@
# -*- coding: utf-8 -*-
collect_ignore = ["nonpython"]

View File

@@ -1,15 +1,14 @@
# -*- coding: utf-8 -*-
import pytest
@pytest.fixture("session")
@pytest.fixture(scope="session")
def setup(request):
setup = CostlySetup()
yield setup
setup.finalize()
class CostlySetup(object):
class CostlySetup:
def __init__(self):
import time

View File

@@ -1,2 +1 @@
# -*- coding: utf-8 -*-
#

View File

@@ -1,3 +1,2 @@
# -*- coding: utf-8 -*-
def test_quick(setup):
pass

View File

@@ -1,2 +1 @@
# -*- coding: utf-8 -*-
#

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
def test_something(setup):
assert setup.timecostly == 1

View File

@@ -0,0 +1,38 @@
import pytest
# fixtures documentation order example
order = []
@pytest.fixture(scope="session")
def s1():
order.append("s1")
@pytest.fixture(scope="module")
def m1():
order.append("m1")
@pytest.fixture
def f1(f3):
order.append("f1")
@pytest.fixture
def f3():
order.append("f3")
@pytest.fixture(autouse=True)
def a1():
order.append("a1")
@pytest.fixture
def f2():
order.append("f2")
def test_order(f1, m1, f2, s1):
assert order == ["s1", "m1", "a1", "f3", "f1", "f2"]

View File

@@ -33,7 +33,7 @@ You can "mark" a test function with custom metadata like this:
pass
class TestClass(object):
class TestClass:
def test_method(self):
pass
@@ -45,14 +45,14 @@ You can then restrict a test run to only run tests marked with ``webtest``:
$ pytest -v -m webtest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 3 deselected / 1 selected
test_server.py::test_send_http PASSED [100%]
================== 1 passed, 3 deselected in 0.12 seconds ==================
===================== 1 passed, 3 deselected in 0.12s ======================
Or the inverse, running all tests except the webtest ones:
@@ -60,7 +60,7 @@ Or the inverse, running all tests except the webtest ones:
$ pytest -v -m "not webtest"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 1 deselected / 3 selected
@@ -69,7 +69,7 @@ Or the inverse, running all tests except the webtest ones:
test_server.py::test_another PASSED [ 66%]
test_server.py::TestClass::test_method PASSED [100%]
================== 3 passed, 1 deselected in 0.12 seconds ==================
===================== 3 passed, 1 deselected in 0.12s ======================
Selecting tests based on their node ID
--------------------------------------
@@ -82,14 +82,14 @@ tests based on their module, class, method, or function name:
$ pytest -v test_server.py::TestClass::test_method
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED [100%]
========================= 1 passed in 0.12 seconds =========================
============================ 1 passed in 0.12s =============================
You can also select on the class:
@@ -97,14 +97,14 @@ You can also select on the class:
$ pytest -v test_server.py::TestClass
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED [100%]
========================= 1 passed in 0.12 seconds =========================
============================ 1 passed in 0.12s =============================
Or select multiple nodes:
@@ -112,7 +112,7 @@ Or select multiple nodes:
$ pytest -v test_server.py::TestClass test_server.py::test_send_http
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 2 items
@@ -120,7 +120,7 @@ Or select multiple nodes:
test_server.py::TestClass::test_method PASSED [ 50%]
test_server.py::test_send_http PASSED [100%]
========================= 2 passed in 0.12 seconds =========================
============================ 2 passed in 0.12s =============================
.. _node-id:
@@ -152,14 +152,14 @@ select tests based on their names:
$ pytest -v -k http # running with the above defined example module
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 3 deselected / 1 selected
test_server.py::test_send_http PASSED [100%]
================== 1 passed, 3 deselected in 0.12 seconds ==================
===================== 1 passed, 3 deselected in 0.12s ======================
And you can also run all tests except the ones that match the keyword:
@@ -167,7 +167,7 @@ And you can also run all tests except the ones that match the keyword:
$ pytest -k "not send_http" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 1 deselected / 3 selected
@@ -176,7 +176,7 @@ And you can also run all tests except the ones that match the keyword:
test_server.py::test_another PASSED [ 66%]
test_server.py::TestClass::test_method PASSED [100%]
================== 3 passed, 1 deselected in 0.12 seconds ==================
===================== 3 passed, 1 deselected in 0.12s ======================
Or to select "http" and "quick" tests:
@@ -184,7 +184,7 @@ Or to select "http" and "quick" tests:
$ pytest -k "http or quick" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 2 deselected / 2 selected
@@ -192,7 +192,7 @@ Or to select "http" and "quick" tests:
test_server.py::test_send_http PASSED [ 50%]
test_server.py::test_something_quick PASSED [100%]
================== 2 passed, 2 deselected in 0.12 seconds ==================
===================== 2 passed, 2 deselected in 0.12s ======================
.. note::
@@ -278,7 +278,7 @@ its test methods:
@pytest.mark.webtest
class TestClass(object):
class TestClass:
def test_startup(self):
pass
@@ -288,15 +288,14 @@ its test methods:
This is equivalent to directly applying the decorator to the
two test functions.
To remain backward-compatible with Python 2.4 you can also set a
``pytestmark`` attribute on a TestClass like this:
Due to legacy reasons, it is possible to set the ``pytestmark`` attribute on a TestClass like this:
.. code-block:: python
import pytest
class TestClass(object):
class TestClass:
pytestmark = pytest.mark.webtest
or if you need to use multiple markers you can use a list:
@@ -306,7 +305,7 @@ or if you need to use multiple markers you can use a list:
import pytest
class TestClass(object):
class TestClass:
pytestmark = [pytest.mark.webtest, pytest.mark.slowtest]
You can also set a module level marker::
@@ -337,7 +336,7 @@ apply a marker to an individual test instance:
@pytest.mark.foo
@pytest.mark.parametrize(
("n", "expected"), [(1, 2), pytest.param((1, 3), marks=pytest.mark.bar), (2, 3)]
("n", "expected"), [(1, 2), pytest.param(1, 3, marks=pytest.mark.bar), (2, 3)]
)
def test_increment(n, expected):
assert n + 1 == expected
@@ -385,7 +384,7 @@ specifies via named environments:
envnames = [mark.args[0] for mark in item.iter_markers(name="env")]
if envnames:
if item.config.getoption("-E") not in envnames:
pytest.skip("test requires env in %r" % envnames)
pytest.skip("test requires env in {!r}".format(envnames))
A test file using this local plugin:
@@ -407,14 +406,14 @@ the test needs:
$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_someenv.py s [100%]
======================== 1 skipped in 0.12 seconds =========================
============================ 1 skipped in 0.12s ============================
and here is one that specifies exactly the environment needed:
@@ -422,14 +421,14 @@ and here is one that specifies exactly the environment needed:
$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_someenv.py . [100%]
========================= 1 passed in 0.12 seconds =========================
============================ 1 passed in 0.12s =============================
The ``--markers`` option always gives you a list of available markers:
@@ -500,7 +499,7 @@ The output is as follows:
$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef>,), kwargs={})
.
1 passed in 0.12 seconds
1 passed in 0.12s
We can see that the custom marker has its argument set extended with the function ``hello_world``. This is the key difference between creating a custom marker as a callable, which invokes ``__call__`` behind the scenes, and using ``with_args``.
@@ -524,7 +523,7 @@ code you can read over all such settings. Example:
@pytest.mark.glob("class", x=2)
class TestClass(object):
class TestClass:
@pytest.mark.glob("function", x=3)
def test_something(self):
pass
@@ -540,7 +539,7 @@ test function. From a conftest file we can read it like this:
def pytest_runtest_setup(item):
for mark in item.iter_markers(name="glob"):
print("glob args=%s kwargs=%s" % (mark.args, mark.kwargs))
print("glob args={} kwargs={}".format(mark.args, mark.kwargs))
sys.stdout.flush()
Let's run this without capturing output and see what we get:
@@ -552,7 +551,7 @@ Let's run this without capturing output and see what we get:
glob args=('class',) kwargs={'x': 2}
glob args=('module',) kwargs={'x': 1}
.
1 passed in 0.12 seconds
1 passed in 0.12s
marking platform specific tests with pytest
--------------------------------------------------------------
@@ -579,7 +578,7 @@ for your particular platform, you could use the following plugin:
supported_platforms = ALL.intersection(mark.name for mark in item.iter_markers())
plat = sys.platform
if supported_platforms and plat not in supported_platforms:
pytest.skip("cannot run on platform %s" % (plat))
pytest.skip("cannot run on platform {}".format(plat))
then tests will be skipped if they were specified for a different platform.
Let's do a little test file to show how this looks like:
@@ -615,7 +614,7 @@ then you will see two tests skipped and two executed tests as expected:
$ pytest -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
@@ -624,7 +623,7 @@ then you will see two tests skipped and two executed tests as expected:
========================= short test summary info ==========================
SKIPPED [2] $REGENDOC_TMPDIR/conftest.py:13: cannot run on platform linux
=================== 2 passed, 2 skipped in 0.12 seconds ====================
======================= 2 passed, 2 skipped in 0.12s =======================
Note that if you specify a platform via the marker-command line option like this:
@@ -632,14 +631,14 @@ Note that if you specify a platform via the marker-command line option like this
$ pytest -m linux
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items / 3 deselected / 1 selected
test_plat.py . [100%]
================== 1 passed, 3 deselected in 0.12 seconds ==================
===================== 1 passed, 3 deselected in 0.12s ======================
then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests.
@@ -696,7 +695,7 @@ We can now use the ``-m option`` to select one set:
$ pytest -m interface --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items / 2 deselected / 2 selected
@@ -712,7 +711,7 @@ We can now use the ``-m option`` to select one set:
test_module.py:8: in test_interface_complex
assert 0
E assert 0
================== 2 failed, 2 deselected in 0.12 seconds ==================
===================== 2 failed, 2 deselected in 0.12s ======================
or to select both "event" and "interface" tests:
@@ -720,7 +719,7 @@ or to select both "event" and "interface" tests:
$ pytest -m "interface or event" --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items / 1 deselected / 3 selected
@@ -740,4 +739,4 @@ or to select both "event" and "interface" tests:
test_module.py:12: in test_event_simple
assert 0
E assert 0
================== 3 failed, 1 deselected in 0.12 seconds ==================
===================== 3 failed, 1 deselected in 0.12s ======================

View File

@@ -1,15 +1,14 @@
# -*- coding: utf-8 -*-
"""
module containing a parametrized tests testing cross-python
serialization via the pickle module.
"""
import distutils.spawn
import shutil
import subprocess
import textwrap
import pytest
pythonlist = ["python2.7", "python3.4", "python3.5"]
pythonlist = ["python3.5", "python3.6", "python3.7"]
@pytest.fixture(params=pythonlist)
@@ -23,9 +22,9 @@ def python2(request, python1):
return Python(request.param, python1.picklefile)
class Python(object):
class Python:
def __init__(self, version, picklefile):
self.pythonpath = distutils.spawn.find_executable(version)
self.pythonpath = shutil.which(version)
if not self.pythonpath:
pytest.skip("{!r} not found".format(version))
self.picklefile = picklefile
@@ -70,4 +69,4 @@ class Python(object):
@pytest.mark.parametrize("obj", [42, {}, {1: 3}])
def test_basic_objects(python1, python2, obj):
python1.dumps(obj)
python2.load_and_is_true("obj == %s" % obj)
python2.load_and_is_true("obj == {}".format(obj))

View File

@@ -12,14 +12,14 @@ A basic example for specifying tests in Yaml files
.. _`pytest-yamlwsgi`: http://bitbucket.org/aafshar/pytest-yamlwsgi/src/tip/pytest_yamlwsgi.py
.. _`PyYAML`: https://pypi.org/project/PyYAML/
Here is an example ``conftest.py`` (extracted from Ali Afshnars special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yml`` files and will execute the yaml-formatted content as custom tests:
Here is an example ``conftest.py`` (extracted from Ali Afshnars special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yaml`` files and will execute the yaml-formatted content as custom tests:
.. include:: nonpython/conftest.py
:literal:
You can create a simple example file:
.. include:: nonpython/test_simple.yml
.. include:: nonpython/test_simple.yaml
:literal:
and if you installed `PyYAML`_ or a compatible YAML-parser you can
@@ -27,21 +27,21 @@ now execute the test specification:
.. code-block:: pytest
nonpython $ pytest test_simple.yml
nonpython $ pytest test_simple.yaml
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collected 2 items
test_simple.yml F. [100%]
test_simple.yaml F. [100%]
================================= FAILURES =================================
______________________________ usecase: hello ______________________________
usecase execution failed
spec failed: 'some': 'other'
no further details known at this point.
==================== 1 failed, 1 passed in 0.12 seconds ====================
======================= 1 failed, 1 passed in 0.12s ========================
.. regendoc:wipe
@@ -64,20 +64,20 @@ consulted when reporting in ``verbose`` mode:
nonpython $ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collecting ... collected 2 items
test_simple.yml::hello FAILED [ 50%]
test_simple.yml::ok PASSED [100%]
test_simple.yaml::hello FAILED [ 50%]
test_simple.yaml::ok PASSED [100%]
================================= FAILURES =================================
______________________________ usecase: hello ______________________________
usecase execution failed
spec failed: 'some': 'other'
no further details known at this point.
==================== 1 failed, 1 passed in 0.12 seconds ====================
======================= 1 failed, 1 passed in 0.12s ========================
.. regendoc:wipe
@@ -88,13 +88,13 @@ interesting to just look at the collection tree:
nonpython $ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collected 2 items
<Package $REGENDOC_TMPDIR/nonpython>
<YamlFile test_simple.yml>
<YamlFile test_simple.yaml>
<YamlItem hello>
<YamlItem ok>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================

View File

@@ -1,10 +1,9 @@
# -*- coding: utf-8 -*-
# content of conftest.py
import pytest
def pytest_collect_file(parent, path):
if path.ext == ".yml" and path.basename.startswith("test"):
if path.ext == ".yaml" and path.basename.startswith("test"):
return YamlFile(path, parent)
@@ -19,7 +18,7 @@ class YamlFile(pytest.File):
class YamlItem(pytest.Item):
def __init__(self, name, parent, spec):
super(YamlItem, self).__init__(name, parent)
super().__init__(name, parent)
self.spec = spec
def runtest(self):
@@ -34,13 +33,13 @@ class YamlItem(pytest.Item):
return "\n".join(
[
"usecase execution failed",
" spec failed: %r: %r" % excinfo.value.args[1:3],
" spec failed: {1!r}: {2!r}".format(*excinfo.value.args),
" no further details known at this point.",
]
)
def reportinfo(self):
return self.fspath, 0, "usecase: %s" % self.name
return self.fspath, 0, "usecase: {}".format(self.name)
class YamlException(Exception):

View File

@@ -1,4 +1,4 @@
# test_simple.yml
# test_simple.yaml
ok:
sub1: sub1

View File

@@ -19,24 +19,30 @@ Generating parameters combinations, depending on command line
Let's say we want to execute a test with different computation
parameters and the parameter range shall be determined by a command
line argument. Let's first write a simple (do-nothing) computation test::
line argument. Let's first write a simple (do-nothing) computation test:
.. code-block:: python
# content of test_compute.py
def test_compute(param1):
assert param1 < 4
Now we add a test configuration like this::
Now we add a test configuration like this:
.. code-block:: python
# content of conftest.py
def pytest_addoption(parser):
parser.addoption("--all", action="store_true",
help="run all combinations")
parser.addoption("--all", action="store_true", help="run all combinations")
def pytest_generate_tests(metafunc):
if 'param1' in metafunc.fixturenames:
if metafunc.config.getoption('all'):
if "param1" in metafunc.fixturenames:
if metafunc.config.getoption("all"):
end = 5
else:
end = 2
@@ -48,7 +54,7 @@ This means that we only run 2 tests if we do not pass ``--all``:
$ pytest -q test_compute.py
.. [100%]
2 passed in 0.12 seconds
2 passed in 0.12s
We run only two computations, so we see two dots.
let's run the full monty:
@@ -66,8 +72,8 @@ let's run the full monty:
> assert param1 < 4
E assert 4 < 4
test_compute.py:3: AssertionError
1 failed, 4 passed in 0.12 seconds
test_compute.py:4: AssertionError
1 failed, 4 passed in 0.12s
As expected when running the full range of ``param1`` values
we'll get an error on the last one.
@@ -83,7 +89,9 @@ Running pytest with ``--collect-only`` will show the generated IDs.
Numbers, strings, booleans and None will have their usual string representation
used in the test ID. For other objects, pytest will make a string based on
the argument name::
the argument name:
.. code-block:: python
# content of test_time.py
@@ -112,7 +120,7 @@ the argument name::
def idfn(val):
if isinstance(val, (datetime,)):
# note this wouldn't show any hours/minutes/seconds
return val.strftime('%Y%m%d')
return val.strftime("%Y%m%d")
@pytest.mark.parametrize("a,b,expected", testdata, ids=idfn)
@@ -120,12 +128,18 @@ the argument name::
diff = a - b
assert diff == expected
@pytest.mark.parametrize("a,b,expected", [
pytest.param(datetime(2001, 12, 12), datetime(2001, 12, 11),
timedelta(1), id='forward'),
pytest.param(datetime(2001, 12, 11), datetime(2001, 12, 12),
timedelta(-1), id='backward'),
])
@pytest.mark.parametrize(
"a,b,expected",
[
pytest.param(
datetime(2001, 12, 12), datetime(2001, 12, 11), timedelta(1), id="forward"
),
pytest.param(
datetime(2001, 12, 11), datetime(2001, 12, 12), timedelta(-1), id="backward"
),
],
)
def test_timedistance_v3(a, b, expected):
diff = a - b
assert diff == expected
@@ -144,7 +158,7 @@ objects, they are still using the default pytest representation:
$ pytest test_time.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 8 items
@@ -158,7 +172,7 @@ objects, they are still using the default pytest representation:
<Function test_timedistance_v3[forward]>
<Function test_timedistance_v3[backward]>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
In ``test_timedistance_v3``, we used ``pytest.param`` to specify the test IDs
together with the actual data, instead of listing them separately.
@@ -171,10 +185,13 @@ A quick port of "testscenarios"
Here is a quick port to run tests configured with `test scenarios`_,
an add-on from Robert Collins for the standard unittest framework. We
only have to work a bit to construct the correct arguments for pytest's
:py:func:`Metafunc.parametrize`::
:py:func:`Metafunc.parametrize`:
.. code-block:: python
# content of test_scenarios.py
def pytest_generate_tests(metafunc):
idlist = []
argvalues = []
@@ -182,13 +199,15 @@ only have to work a bit to construct the correct arguments for pytest's
idlist.append(scenario[0])
items = scenario[1].items()
argnames = [x[0] for x in items]
argvalues.append(([x[1] for x in items]))
argvalues.append([x[1] for x in items])
metafunc.parametrize(argnames, argvalues, ids=idlist, scope="class")
scenario1 = ('basic', {'attribute': 'value'})
scenario2 = ('advanced', {'attribute': 'value2'})
class TestSampleWithScenarios(object):
scenario1 = ("basic", {"attribute": "value"})
scenario2 = ("advanced", {"attribute": "value2"})
class TestSampleWithScenarios:
scenarios = [scenario1, scenario2]
def test_demo1(self, attribute):
@@ -203,14 +222,14 @@ this is a fully self-contained example which you can run with:
$ pytest test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
test_scenarios.py .... [100%]
========================= 4 passed in 0.12 seconds =========================
============================ 4 passed in 0.12s =============================
If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function:
@@ -218,7 +237,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
$ pytest --collect-only test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
@@ -229,7 +248,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
<Function test_demo1[advanced]>
<Function test_demo2[advanced]>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
Note that we told ``metafunc.parametrize()`` that your scenario values
should be considered class-scoped. With pytest-2.3 this leads to a
@@ -243,12 +262,16 @@ Deferring the setup of parametrized resources
The parametrization of test functions happens at collection
time. It is a good idea to setup expensive resources like DB
connections or subprocess only when the actual test is run.
Here is a simple example how you can achieve that, first
the actual test requiring a ``db`` object::
Here is a simple example how you can achieve that. This test
requires a ``db`` object fixture:
.. code-block:: python
# content of test_backends.py
import pytest
def test_db_initialized(db):
# a dummy test
if db.__class__.__name__ == "DB2":
@@ -256,20 +279,27 @@ the actual test requiring a ``db`` object::
We can now add a test configuration that generates two invocations of
the ``test_db_initialized`` function and also implements a factory that
creates a database object for the actual test invocations::
creates a database object for the actual test invocations:
.. code-block:: python
# content of conftest.py
import pytest
def pytest_generate_tests(metafunc):
if 'db' in metafunc.fixturenames:
metafunc.parametrize("db", ['d1', 'd2'], indirect=True)
class DB1(object):
def pytest_generate_tests(metafunc):
if "db" in metafunc.fixturenames:
metafunc.parametrize("db", ["d1", "d2"], indirect=True)
class DB1:
"one database object"
class DB2(object):
class DB2:
"alternative database object"
@pytest.fixture
def db(request):
if request.param == "d1":
@@ -285,7 +315,7 @@ Let's first see how it looks like at collection time:
$ pytest test_backends.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -293,7 +323,7 @@ Let's first see how it looks like at collection time:
<Function test_db_initialized[d1]>
<Function test_db_initialized[d2]>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
And then when we run the test:
@@ -312,8 +342,8 @@ And then when we run the test:
> pytest.fail("deliberately failing for demo purposes")
E Failed: deliberately failing for demo purposes
test_backends.py:6: Failed
1 failed, 1 passed in 0.12 seconds
test_backends.py:8: Failed
1 failed, 1 passed in 0.12s
The first invocation with ``db == "DB1"`` passed while the second with ``db == "DB2"`` failed. Our ``db`` fixture function has instantiated each of the DB values during the setup phase while the ``pytest_generate_tests`` generated two according calls to the ``test_db_initialized`` during the collection phase.
@@ -327,23 +357,29 @@ parameter on particular arguments. It can be done by passing list or tuple of
arguments' names to ``indirect``. In the example below there is a function ``test_indirect`` which uses
two fixtures: ``x`` and ``y``. Here we give to indirect the list, which contains the name of the
fixture ``x``. The indirect parameter will be applied to this argument only, and the value ``a``
will be passed to respective fixture function::
will be passed to respective fixture function:
.. code-block:: python
# content of test_indirect_list.py
import pytest
@pytest.fixture(scope='function')
@pytest.fixture(scope="function")
def x(request):
return request.param * 3
@pytest.fixture(scope='function')
@pytest.fixture(scope="function")
def y(request):
return request.param * 2
@pytest.mark.parametrize('x, y', [('a', 'b')], indirect=['x'])
def test_indirect(x,y):
assert x == 'aaa'
assert y == 'b'
@pytest.mark.parametrize("x, y", [("a", "b")], indirect=["x"])
def test_indirect(x, y):
assert x == "aaa"
assert y == "b"
The result of this test will be successful:
@@ -351,14 +387,14 @@ The result of this test will be successful:
$ pytest test_indirect_list.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
<Module test_indirect_list.py>
<Function test_indirect[a-b]>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
.. regendoc:wipe
@@ -370,23 +406,28 @@ Parametrizing test methods through per-class configuration
Here is an example ``pytest_generate_tests`` function implementing a
parametrization scheme similar to Michael Foord's `unittest
parametrizer`_ but in a lot less code::
parametrizer`_ but in a lot less code:
.. code-block:: python
# content of ./test_parametrize.py
import pytest
def pytest_generate_tests(metafunc):
# called once per each test function
funcarglist = metafunc.cls.params[metafunc.function.__name__]
argnames = sorted(funcarglist[0])
metafunc.parametrize(argnames, [[funcargs[name] for name in argnames]
for funcargs in funcarglist])
metafunc.parametrize(
argnames, [[funcargs[name] for name in argnames] for funcargs in funcarglist]
)
class TestClass(object):
class TestClass:
# a map specifying multiple argument sets for a test method
params = {
'test_equals': [dict(a=1, b=2), dict(a=3, b=3), ],
'test_zerodivision': [dict(a=1, b=0), ],
"test_equals": [dict(a=1, b=2), dict(a=3, b=3)],
"test_zerodivision": [dict(a=1, b=0)],
}
def test_equals(self, a, b):
@@ -412,8 +453,8 @@ argument sets to use for each test function. Let's run it:
> assert a == b
E assert 1 == 2
test_parametrize.py:18: AssertionError
1 failed, 2 passed in 0.12 seconds
test_parametrize.py:21: AssertionError
1 failed, 2 passed in 0.12s
Indirect parametrization with multiple fixtures
--------------------------------------------------------------
@@ -429,15 +470,16 @@ is to be run with different sets of arguments for its three arguments:
.. literalinclude:: multipython.py
Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize):
Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (3 interpreters times 3 interpreters times 3 objects to serialize/deserialize):
.. code-block:: pytest
. $ pytest -rs -q multipython.py
...sss...sssssssss...sss... [100%]
ssssssssssssssssssssssss... [100%]
========================= short test summary info ==========================
SKIPPED [15] $REGENDOC_TMPDIR/CWD/multipython.py:31: 'python3.4' not found
12 passed, 15 skipped in 0.12 seconds
SKIPPED [12] $REGENDOC_TMPDIR/CWD/multipython.py:30: 'python3.5' not found
SKIPPED [12] $REGENDOC_TMPDIR/CWD/multipython.py:30: 'python3.6' not found
3 passed, 24 skipped in 0.12s
Indirect parametrization of optional implementations/imports
--------------------------------------------------------------------
@@ -446,36 +488,47 @@ If you want to compare the outcomes of several implementations of a given
API, you can write test functions that receive the already imported implementations
and get skipped in case the implementation is not importable/available. Let's
say we have a "base" implementation and the other (possibly optimized ones)
need to provide similar results::
need to provide similar results:
.. code-block:: python
# content of conftest.py
import pytest
@pytest.fixture(scope="session")
def basemod(request):
return pytest.importorskip("base")
@pytest.fixture(scope="session", params=["opt1", "opt2"])
def optmod(request):
return pytest.importorskip(request.param)
And then a base implementation of a simple function::
And then a base implementation of a simple function:
.. code-block:: python
# content of base.py
def func1():
return 1
And an optimized version::
And an optimized version:
.. code-block:: python
# content of opt1.py
def func1():
return 1.0001
And finally a little test module::
And finally a little test module:
.. code-block:: python
# content of test_module.py
def test_func1(basemod, optmod):
assert round(basemod.func1(), 3) == round(optmod.func1(), 3)
@@ -486,7 +539,7 @@ If you run this with reporting for skips enabled:
$ pytest -rs test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -494,8 +547,8 @@ If you run this with reporting for skips enabled:
test_module.py .s [100%]
========================= short test summary info ==========================
SKIPPED [1] $REGENDOC_TMPDIR/conftest.py:11: could not import 'opt2': No module named 'opt2'
=================== 1 passed, 1 skipped in 0.12 seconds ====================
SKIPPED [1] $REGENDOC_TMPDIR/conftest.py:13: could not import 'opt2': No module named 'opt2'
======================= 1 passed, 1 skipped in 0.12s =======================
You'll see that we don't have an ``opt2`` module and thus the second test run
of our ``test_func1`` was skipped. A few notes:
@@ -548,16 +601,16 @@ Then run ``pytest`` with verbose mode and with only the ``basic`` marker:
$ pytest -v -m basic
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 17 items / 14 deselected / 3 selected
collecting ... collected 18 items / 15 deselected / 3 selected
test_pytest_param_example.py::test_eval[1+7-8] PASSED [ 33%]
test_pytest_param_example.py::test_eval[basic_2+4] PASSED [ 66%]
test_pytest_param_example.py::test_eval[basic_6*9] XFAIL [100%]
============ 2 passed, 14 deselected, 1 xfailed in 0.12 seconds ============
=============== 2 passed, 15 deselected, 1 xfailed in 0.12s ================
As the result:
@@ -578,22 +631,28 @@ Use :func:`pytest.raises` with the
in which some tests raise exceptions and others do not.
It is helpful to define a no-op context manager ``does_not_raise`` to serve
as a complement to ``raises``. For example::
as a complement to ``raises``. For example:
.. code-block:: python
from contextlib import contextmanager
import pytest
@contextmanager
def does_not_raise():
yield
@pytest.mark.parametrize('example_input,expectation', [
(3, does_not_raise()),
(2, does_not_raise()),
(1, does_not_raise()),
(0, pytest.raises(ZeroDivisionError)),
])
@pytest.mark.parametrize(
"example_input,expectation",
[
(3, does_not_raise()),
(2, does_not_raise()),
(1, does_not_raise()),
(0, pytest.raises(ZeroDivisionError)),
],
)
def test_division(example_input, expectation):
"""Test how much I know division."""
with expectation:
@@ -603,14 +662,20 @@ In the example above, the first three test cases should run unexceptionally,
while the fourth should raise ``ZeroDivisionError``.
If you're only supporting Python 3.7+, you can simply use ``nullcontext``
to define ``does_not_raise``::
to define ``does_not_raise``:
.. code-block:: python
from contextlib import nullcontext as does_not_raise
Or, if you're supporting Python 3.3+ you can use::
Or, if you're supporting Python 3.3+ you can use:
.. code-block:: python
from contextlib import ExitStack as does_not_raise
Or, if desired, you can ``pip install contextlib2`` and use::
Or, if desired, you can ``pip install contextlib2`` and use:
from contextlib2 import ExitStack as does_not_raise
.. code-block:: python
from contextlib2 import nullcontext as does_not_raise

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import sys
import pytest

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
def test_exception_syntax():
try:
0 / 0

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
# run this with $ pytest --collect-only test_collectonly.py
#
@@ -7,7 +6,7 @@ def test_function():
pass
class TestClass(object):
class TestClass:
def test_method(self):
pass

View File

@@ -31,7 +31,7 @@ you will see that ``pytest`` only collects test-modules, which do not match the
.. code-block:: pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 5 items
@@ -131,12 +131,15 @@ Here is an example:
This would make ``pytest`` look for tests in files that match the ``check_*
.py`` glob-pattern, ``Check`` prefixes in classes, and functions and methods
that match ``*_check``. For example, if we have::
that match ``*_check``. For example, if we have:
.. code-block:: python
# content of check_myapp.py
class CheckMyApp(object):
class CheckMyApp:
def simple_check(self):
pass
def complex_check(self):
pass
@@ -146,7 +149,7 @@ The test collection would look like this:
$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 2 items
@@ -155,7 +158,7 @@ The test collection would look like this:
<Function simple_check>
<Function complex_check>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
You can check for multiple glob patterns by adding a space between the patterns:
@@ -208,7 +211,7 @@ You can always peek at the collection tree without running tests like this:
. $ pytest --collect-only pythoncollection.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 3 items
@@ -218,7 +221,7 @@ You can always peek at the collection tree without running tests like this:
<Function test_method>
<Function test_anothermethod>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
.. _customizing-test-collection:
@@ -238,7 +241,9 @@ You can easily instruct ``pytest`` to discover tests from every Python file:
However, many projects will have a ``setup.py`` which they don't want to be
imported. Moreover, there may files only importable by a specific python
version. For such cases you can dynamically define files to be ignored by
listing them in a ``conftest.py`` file::
listing them in a ``conftest.py`` file:
.. code-block:: python
# content of conftest.py
import sys
@@ -247,7 +252,9 @@ listing them in a ``conftest.py`` file::
if sys.version_info[0] > 2:
collect_ignore.append("pkg/module_py2.py")
and then if you have a module file like this::
and then if you have a module file like this:
.. code-block:: python
# content of pkg/module_py2.py
def test_only_on_python2():
@@ -256,10 +263,12 @@ and then if you have a module file like this::
except Exception, e:
pass
and a ``setup.py`` dummy file like this::
and a ``setup.py`` dummy file like this:
.. code-block:: python
# content of setup.py
0/0 # will raise exception if imported
0 / 0 # will raise exception if imported
If you run with a Python 2 interpreter then you will find the one test and will
leave out the ``setup.py`` file:
@@ -283,19 +292,21 @@ file will be left out:
$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 0 items
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
It's also possible to ignore files based on Unix shell-style wildcards by adding
patterns to ``collect_ignore_glob``.
The following example ``conftest.py`` ignores the file ``setup.py`` and in
addition all files that end with ``*_py2.py`` when executed with a Python 3
interpreter::
interpreter:
.. code-block:: python
# content of conftest.py
import sys

View File

@@ -9,7 +9,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
assertion $ pytest failure_demo.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/assertion
collected 44 items
@@ -26,7 +26,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert param1 * 2 < param2
E assert (3 * 2) < 6
failure_demo.py:21: AssertionError
failure_demo.py:20: AssertionError
_________________________ TestFailing.test_simple __________________________
self = <failure_demo.TestFailing object at 0xdeadbeef>
@@ -43,7 +43,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + where 42 = <function TestFailing.test_simple.<locals>.f at 0xdeadbeef>()
E + and 43 = <function TestFailing.test_simple.<locals>.g at 0xdeadbeef>()
failure_demo.py:32: AssertionError
failure_demo.py:31: AssertionError
____________________ TestFailing.test_simple_multiline _____________________
self = <failure_demo.TestFailing object at 0xdeadbeef>
@@ -51,7 +51,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
def test_simple_multiline(self):
> otherfunc_multi(42, 6 * 9)
failure_demo.py:35:
failure_demo.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = 42, b = 54
@@ -60,7 +60,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert a == b
E assert 42 == 54
failure_demo.py:16: AssertionError
failure_demo.py:15: AssertionError
___________________________ TestFailing.test_not ___________________________
self = <failure_demo.TestFailing object at 0xdeadbeef>
@@ -73,7 +73,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E assert not 42
E + where 42 = <function TestFailing.test_not.<locals>.f at 0xdeadbeef>()
failure_demo.py:41: AssertionError
failure_demo.py:40: AssertionError
_________________ TestSpecialisedExplanations.test_eq_text _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -84,7 +84,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E - spam
E + eggs
failure_demo.py:46: AssertionError
failure_demo.py:45: AssertionError
_____________ TestSpecialisedExplanations.test_eq_similar_text _____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -97,7 +97,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + foo 2 bar
E ? ^
failure_demo.py:49: AssertionError
failure_demo.py:48: AssertionError
____________ TestSpecialisedExplanations.test_eq_multiline_text ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -110,7 +110,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + eggs
E bar
failure_demo.py:52: AssertionError
failure_demo.py:51: AssertionError
______________ TestSpecialisedExplanations.test_eq_long_text _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -119,7 +119,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
a = "1" * 100 + "a" + "2" * 100
b = "1" * 100 + "b" + "2" * 100
> assert a == b
E AssertionError: assert '111111111111...2222222222222' == '1111111111111...2222222222222'
E AssertionError: assert '111111111111...2222222222222' == '111111111111...2222222222222'
E Skipping 90 identical leading characters in diff, use -v to show
E Skipping 91 identical trailing characters in diff, use -v to show
E - 1111111111a222222222
@@ -127,7 +127,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + 1111111111b222222222
E ? ^
failure_demo.py:57: AssertionError
failure_demo.py:56: AssertionError
_________ TestSpecialisedExplanations.test_eq_long_text_multiline __________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -136,7 +136,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
a = "1\n" * 100 + "a" + "2\n" * 100
b = "1\n" * 100 + "b" + "2\n" * 100
> assert a == b
E AssertionError: assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n1...n2\n2\n2\n2\n'
E AssertionError: assert '1\n1\n1\n1\n...n2\n2\n2\n2\n' == '1\n1\n1\n1\n...n2\n2\n2\n2\n'
E Skipping 190 identical leading characters in diff, use -v to show
E Skipping 191 identical trailing characters in diff, use -v to show
E 1
@@ -147,7 +147,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E
E ...Full output truncated (7 lines hidden), use '-vv' to show
failure_demo.py:62: AssertionError
failure_demo.py:61: AssertionError
_________________ TestSpecialisedExplanations.test_eq_list _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -158,7 +158,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E At index 2 diff: 2 != 3
E Use -v to get the full diff
failure_demo.py:65: AssertionError
failure_demo.py:64: AssertionError
______________ TestSpecialisedExplanations.test_eq_list_long _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -171,7 +171,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E At index 100 diff: 1 != 2
E Use -v to get the full diff
failure_demo.py:70: AssertionError
failure_demo.py:69: AssertionError
_________________ TestSpecialisedExplanations.test_eq_dict _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -189,7 +189,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:73: AssertionError
failure_demo.py:72: AssertionError
_________________ TestSpecialisedExplanations.test_eq_set __________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -207,7 +207,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:76: AssertionError
failure_demo.py:75: AssertionError
_____________ TestSpecialisedExplanations.test_eq_longer_list ______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -218,7 +218,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E Right contains one more item: 3
E Use -v to get the full diff
failure_demo.py:79: AssertionError
failure_demo.py:78: AssertionError
_________________ TestSpecialisedExplanations.test_in_list _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -227,7 +227,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert 1 in [0, 2, 3, 4, 5]
E assert 1 in [0, 2, 3, 4, 5]
failure_demo.py:82: AssertionError
failure_demo.py:81: AssertionError
__________ TestSpecialisedExplanations.test_not_in_text_multiline __________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -235,7 +235,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
def test_not_in_text_multiline(self):
text = "some multiline\ntext\nwhich\nincludes foo\nand a\ntail"
> assert "foo" not in text
E AssertionError: assert 'foo' not in 'some multiline\ntext\nw...ncludes foo\nand a\ntail'
E AssertionError: assert 'foo' not in 'some multil...nand a\ntail'
E 'foo' is contained here:
E some multiline
E text
@@ -246,7 +246,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:86: AssertionError
failure_demo.py:85: AssertionError
___________ TestSpecialisedExplanations.test_not_in_text_single ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -259,7 +259,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E single foo line
E ? +++
failure_demo.py:90: AssertionError
failure_demo.py:89: AssertionError
_________ TestSpecialisedExplanations.test_not_in_text_single_long _________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -267,12 +267,12 @@ Here is a nice run of several failures and how ``pytest`` presents things:
def test_not_in_text_single_long(self):
text = "head " * 50 + "foo " + "tail " * 20
> assert "foo" not in text
E AssertionError: assert 'foo' not in 'head head head head hea...ail tail tail tail tail '
E AssertionError: assert 'foo' not in 'head head h...l tail tail '
E 'foo' is contained here:
E head head foo tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail
E ? +++
failure_demo.py:94: AssertionError
failure_demo.py:93: AssertionError
______ TestSpecialisedExplanations.test_not_in_text_single_long_term _______
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -280,12 +280,12 @@ Here is a nice run of several failures and how ``pytest`` presents things:
def test_not_in_text_single_long_term(self):
text = "head " * 50 + "f" * 70 + "tail " * 20
> assert "f" * 70 not in text
E AssertionError: assert 'fffffffffff...ffffffffffff' not in 'head head he...l tail tail '
E AssertionError: assert 'fffffffffff...ffffffffffff' not in 'head head h...l tail tail '
E 'ffffffffffffffffff...fffffffffffffffffff' is contained here:
E head head fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffftail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail tail
E ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
failure_demo.py:98: AssertionError
failure_demo.py:97: AssertionError
______________ TestSpecialisedExplanations.test_eq_dataclass _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -294,19 +294,19 @@ Here is a nice run of several failures and how ``pytest`` presents things:
from dataclasses import dataclass
@dataclass
class Foo(object):
class Foo:
a: int
b: str
left = Foo(1, "b")
right = Foo(1, "c")
> assert left == right
E AssertionError: assert TestSpecialis...oo(a=1, b='b') == TestSpecialise...oo(a=1, b='c')
E AssertionError: assert TestSpecialis...oo(a=1, b='b') == TestSpecialis...oo(a=1, b='c')
E Omitting 1 identical items, use -vv to show
E Differing attributes:
E b: 'b' != 'c'
failure_demo.py:110: AssertionError
failure_demo.py:109: AssertionError
________________ TestSpecialisedExplanations.test_eq_attrs _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -315,7 +315,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
import attr
@attr.s
class Foo(object):
class Foo:
a = attr.ib()
b = attr.ib()
@@ -327,11 +327,11 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E Differing attributes:
E b: 'b' != 'c'
failure_demo.py:122: AssertionError
failure_demo.py:121: AssertionError
______________________________ test_attribute ______________________________
def test_attribute():
class Foo(object):
class Foo:
b = 1
i = Foo()
@@ -339,11 +339,11 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E assert 1 == 2
E + where 1 = <failure_demo.test_attribute.<locals>.Foo object at 0xdeadbeef>.b
failure_demo.py:130: AssertionError
failure_demo.py:129: AssertionError
_________________________ test_attribute_instance __________________________
def test_attribute_instance():
class Foo(object):
class Foo:
b = 1
> assert Foo().b == 2
@@ -351,11 +351,11 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + where 1 = <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef>.b
E + where <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef> = <class 'failure_demo.test_attribute_instance.<locals>.Foo'>()
failure_demo.py:137: AssertionError
failure_demo.py:136: AssertionError
__________________________ test_attribute_failure __________________________
def test_attribute_failure():
class Foo(object):
class Foo:
def _get_b(self):
raise Exception("Failed to get attrib")
@@ -364,7 +364,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
i = Foo()
> assert i.b == 2
failure_demo.py:148:
failure_demo.py:147:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <failure_demo.test_attribute_failure.<locals>.Foo object at 0xdeadbeef>
@@ -373,14 +373,14 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> raise Exception("Failed to get attrib")
E Exception: Failed to get attrib
failure_demo.py:143: Exception
failure_demo.py:142: Exception
_________________________ test_attribute_multiple __________________________
def test_attribute_multiple():
class Foo(object):
class Foo:
b = 1
class Bar(object):
class Bar:
b = 2
> assert Foo().b == Bar().b
@@ -390,7 +390,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + and 2 = <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef>.b
E + where <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef> = <class 'failure_demo.test_attribute_multiple.<locals>.Bar'>()
failure_demo.py:158: AssertionError
failure_demo.py:157: AssertionError
__________________________ TestRaises.test_raises __________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -400,7 +400,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> raises(TypeError, int, s)
E ValueError: invalid literal for int() with base 10: 'qwe'
failure_demo.py:168: ValueError
failure_demo.py:167: ValueError
______________________ TestRaises.test_raises_doesnt _______________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -409,7 +409,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> raises(IOError, int, "3")
E Failed: DID NOT RAISE <class 'OSError'>
failure_demo.py:171: Failed
failure_demo.py:170: Failed
__________________________ TestRaises.test_raise ___________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -418,7 +418,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> raise ValueError("demo error")
E ValueError: demo error
failure_demo.py:174: ValueError
failure_demo.py:173: ValueError
________________________ TestRaises.test_tupleerror ________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -427,18 +427,18 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> a, b = [1] # NOQA
E ValueError: not enough values to unpack (expected 2, got 1)
failure_demo.py:177: ValueError
failure_demo.py:176: ValueError
______ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ______
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_reinterpret_fails_with_print_for_the_fun_of_it(self):
items = [1, 2, 3]
print("items is %r" % items)
print("items is {!r}".format(items))
> a, b = items.pop()
E TypeError: 'int' object is not iterable
E TypeError: cannot unpack non-iterable int object
failure_demo.py:182: TypeError
failure_demo.py:181: TypeError
--------------------------- Captured stdout call ---------------------------
items is [1, 2, 3]
________________________ TestRaises.test_some_error ________________________
@@ -449,16 +449,17 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> if namenotexi: # NOQA
E NameError: name 'namenotexi' is not defined
failure_demo.py:185: NameError
failure_demo.py:184: NameError
____________________ test_dynamic_compile_shows_nicely _____________________
def test_dynamic_compile_shows_nicely():
import imp
import importlib.util
import sys
src = "def foo():\n assert 1 == 0\n"
name = "abc-123"
module = imp.new_module(name)
spec = importlib.util.spec_from_loader(name, loader=None)
module = importlib.util.module_from_spec(spec)
code = _pytest._code.compile(src, name, "exec")
exec(code, module.__dict__)
sys.modules[name] = module
@@ -487,7 +488,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:214:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
failure_demo.py:12: in somefunc
failure_demo.py:11: in somefunc
otherfunc(x, y)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
@@ -497,7 +498,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert a == b
E assert 44 == 43
failure_demo.py:8: AssertionError
failure_demo.py:7: AssertionError
___________________ TestMoreErrors.test_z1_unpack_error ____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -515,7 +516,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
def test_z2_type_error(self):
items = 3
> a, b = items
E TypeError: 'int' object is not iterable
E TypeError: cannot unpack non-iterable int object
failure_demo.py:222: TypeError
______________________ TestMoreErrors.test_startswith ______________________
@@ -598,7 +599,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
def test_single_line(self):
class A(object):
class A:
a = 1
b = 2
@@ -613,7 +614,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
def test_multiline(self):
class A(object):
class A:
a = 1
b = 2
@@ -632,7 +633,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
def test_custom_repr(self):
class JSON(object):
class JSON:
a = 1
def __repr__(self):
@@ -649,4 +650,4 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a
failure_demo.py:282: AssertionError
======================== 44 failed in 0.12 seconds =========================
============================ 44 failed in 0.12s ============================

View File

@@ -65,7 +65,7 @@ Let's run this without supplying our new option:
test_sample.py:6: AssertionError
--------------------------- Captured stdout call ---------------------------
first
1 failed in 0.12 seconds
1 failed in 0.12s
And now with supplying a command line option:
@@ -89,7 +89,7 @@ And now with supplying a command line option:
test_sample.py:6: AssertionError
--------------------------- Captured stdout call ---------------------------
second
1 failed in 0.12 seconds
1 failed in 0.12s
You can see that the command line option arrived in our test. This
completes the basic pattern. However, one often rather wants to process
@@ -127,12 +127,12 @@ directory with the above conftest.py:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 0 items
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
.. _`excontrolskip`:
@@ -157,6 +157,10 @@ line option to control skipping of ``pytest.mark.slow`` marked tests:
)
def pytest_configure(config):
config.addinivalue_line("markers", "slow: mark test as slow to run")
def pytest_collection_modifyitems(config, items):
if config.getoption("--runslow"):
# --runslow given in cli: do not skip slow tests
@@ -188,7 +192,7 @@ and when running it will see a skipped "slow" test:
$ pytest -rs # "-rs" means report details on the little 's'
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -197,7 +201,7 @@ and when running it will see a skipped "slow" test:
========================= short test summary info ==========================
SKIPPED [1] test_module.py:8: need --runslow option to run
=================== 1 passed, 1 skipped in 0.12 seconds ====================
======================= 1 passed, 1 skipped in 0.12s =======================
Or run it including the ``slow`` marked test:
@@ -205,14 +209,14 @@ Or run it including the ``slow`` marked test:
$ pytest --runslow
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
test_module.py .. [100%]
========================= 2 passed in 0.12 seconds =========================
============================ 2 passed in 0.12s =============================
Writing well integrated assertion helpers
--------------------------------------------------
@@ -234,7 +238,7 @@ Example:
def checkconfig(x):
__tracebackhide__ = True
if not hasattr(x, "config"):
pytest.fail("not configured: %s" % (x,))
pytest.fail("not configured: {}".format(x))
def test_something():
@@ -257,7 +261,7 @@ Let's run our little function:
E Failed: not configured: 42
test_checkconfig.py:11: Failed
1 failed in 0.12 seconds
1 failed in 0.12s
If you only want to hide certain exceptions, you can set ``__tracebackhide__``
to a callable which gets the ``ExceptionInfo`` object. You can for example use
@@ -276,7 +280,7 @@ this to make sure unexpected exception types aren't hidden:
def checkconfig(x):
__tracebackhide__ = operator.methodcaller("errisinstance", ConfigException)
if not hasattr(x, "config"):
raise ConfigException("not configured: %s" % (x,))
raise ConfigException("not configured: {}".format(x))
def test_something():
@@ -348,13 +352,13 @@ which will add the string to the test header accordingly:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
project deps: mylib-1.1
rootdir: $REGENDOC_TMPDIR
collected 0 items
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
.. regendoc:wipe
@@ -377,14 +381,14 @@ which will add info only when run with "--v":
$ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
info1: did you know that ...
did you?
rootdir: $REGENDOC_TMPDIR
collecting ... collected 0 items
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
and nothing when run plainly:
@@ -392,12 +396,12 @@ and nothing when run plainly:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 0 items
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
profiling test duration
--------------------------
@@ -432,7 +436,7 @@ Now we can profile which test functions execute the slowest:
$ pytest --durations=3
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
@@ -443,7 +447,7 @@ Now we can profile which test functions execute the slowest:
0.30s call test_some_are_slow.py::test_funcslow2
0.20s call test_some_are_slow.py::test_funcslow1
0.10s call test_some_are_slow.py::test_funcfast
========================= 3 passed in 0.12 seconds =========================
============================ 3 passed in 0.12s =============================
incremental testing - test steps
---------------------------------------------------
@@ -474,7 +478,7 @@ an ``incremental`` marker which is to be used on classes:
if "incremental" in item.keywords:
previousfailed = getattr(item.parent, "_previousfailed", None)
if previousfailed is not None:
pytest.xfail("previous test failed (%s)" % previousfailed.name)
pytest.xfail("previous test failed ({})".format(previousfailed.name))
These two hook implementations work together to abort incremental-marked
tests in a class. Here is a test module example:
@@ -487,7 +491,7 @@ tests in a class. Here is a test module example:
@pytest.mark.incremental
class TestUserHandling(object):
class TestUserHandling:
def test_login(self):
pass
@@ -507,7 +511,7 @@ If we run this:
$ pytest -rx
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
@@ -527,7 +531,7 @@ If we run this:
========================= short test summary info ==========================
XFAIL test_step.py::TestUserHandling::test_deletion
reason: previous test failed (test_modification)
============== 1 failed, 2 passed, 1 xfailed in 0.12 seconds ===============
================== 1 failed, 2 passed, 1 xfailed in 0.12s ==================
We'll see that ``test_deletion`` was not executed because ``test_modification``
failed. It is reported as an "expected failure".
@@ -552,7 +556,7 @@ Here is an example for making a ``db`` fixture available in a directory:
import pytest
class DB(object):
class DB:
pass
@@ -591,7 +595,7 @@ We can run this:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 7 items
@@ -640,7 +644,7 @@ We can run this:
E assert 0
a/test_db2.py:2: AssertionError
========== 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ==========
============= 3 failed, 2 passed, 1 xfailed, 1 error in 0.12s ==============
The two test modules in the ``a`` directory see the same ``db`` fixture instance
while the one test in the sister-directory ``b`` doesn't see it. We could of course
@@ -680,7 +684,7 @@ case we just write some information out to a ``failures`` file:
with open("failures", mode) as f:
# let's also access a fixture for the fun of it
if "tmpdir" in item.fixturenames:
extra = " (%s)" % item.funcargs["tmpdir"]
extra = " ({})".format(item.funcargs["tmpdir"])
else:
extra = ""
@@ -705,7 +709,7 @@ and run them:
$ pytest test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -729,7 +733,7 @@ and run them:
E assert 0
test_module.py:6: AssertionError
========================= 2 failed in 0.12 seconds =========================
============================ 2 failed in 0.12s =============================
you will have a "failures" file which contains the failing test ids:
@@ -809,7 +813,7 @@ and run it:
$ pytest -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
@@ -844,7 +848,7 @@ and run it:
E assert 0
test_module.py:19: AssertionError
==================== 2 failed, 1 error in 0.12 seconds =====================
======================== 2 failed, 1 error in 0.12s ========================
You'll see that the fixture finalizers could use the precise reporting
information.

View File

@@ -5,30 +5,36 @@ A session-scoped fixture effectively has access to all
collected test items. Here is an example of a fixture
function which walks all collected tests and looks
if their test class defines a ``callme`` method and
calls it::
calls it:
.. code-block:: python
# content of conftest.py
import pytest
@pytest.fixture(scope="session", autouse=True)
def callattr_ahead_of_alltests(request):
print("callattr_ahead_of_alltests called")
seen = set([None])
seen = {None}
session = request.node
for item in session.items:
cls = item.getparent(pytest.Class)
if cls not in seen:
if hasattr(cls.obj, "callme"):
cls.obj.callme()
cls.obj.callme()
seen.add(cls)
test classes may now define a ``callme`` method which
will be called ahead of running any tests::
will be called ahead of running any tests:
.. code-block:: python
# content of test_module.py
class TestHello(object):
class TestHello:
@classmethod
def callme(cls):
print("callme called!")
@@ -39,16 +45,20 @@ will be called ahead of running any tests::
def test_method2(self):
print("test_method1 called")
class TestOther(object):
class TestOther:
@classmethod
def callme(cls):
print("callme other called")
def test_other(self):
print("test other")
# works with unittest as well ...
import unittest
class SomeTest(unittest.TestCase):
@classmethod
def callme(self):
@@ -71,4 +81,4 @@ If you run this without output capturing:
.test other
.test_unit1 method called
.
4 passed in 0.12 seconds
4 passed in 0.12s

View File

@@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
import pytest
xfail = pytest.mark.xfail

View File

@@ -15,7 +15,9 @@ Running an existing test suite with pytest
Say you want to contribute to an existing repository somewhere.
After pulling the code into your development space using some
flavor of version control and (optionally) setting up a virtualenv
you will want to run::
you will want to run:
.. code-block:: bash
cd <repository>
pip install -e . # Environment dependent alternatives include

View File

@@ -49,20 +49,25 @@ argument. For each argument name, a fixture function with that name provides
the fixture object. Fixture functions are registered by marking them with
:py:func:`@pytest.fixture <_pytest.python.fixture>`. Let's look at a simple
self-contained test module containing a fixture and a test function
using it::
using it:
.. code-block:: python
# content of ./test_smtpsimple.py
import pytest
@pytest.fixture
def smtp_connection():
import smtplib
return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
def test_ehlo(smtp_connection):
response, msg = smtp_connection.ehlo()
assert response == 250
assert 0 # for demo purposes
assert 0 # for demo purposes
Here, the ``test_ehlo`` needs the ``smtp_connection`` fixture value. pytest
will discover and call the :py:func:`@pytest.fixture <_pytest.python.fixture>`
@@ -72,7 +77,7 @@ marked ``smtp_connection`` fixture function. Running the test looks like this:
$ pytest test_smtpsimple.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -87,11 +92,11 @@ marked ``smtp_connection`` fixture function. Running the test looks like this:
def test_ehlo(smtp_connection):
response, msg = smtp_connection.ehlo()
assert response == 250
> assert 0 # for demo purposes
> assert 0 # for demo purposes
E assert 0
test_smtpsimple.py:11: AssertionError
========================= 1 failed in 0.12 seconds =========================
test_smtpsimple.py:14: AssertionError
============================ 1 failed in 0.12s =============================
In the failure traceback we see that the test function was called with a
``smtp_connection`` argument, the ``smtplib.SMTP()`` instance created by the fixture
@@ -180,12 +185,15 @@ Possible values for ``scope`` are: ``function``, ``class``, ``module``, ``packag
The next example puts the fixture function into a separate ``conftest.py`` file
so that tests from multiple test modules in the directory can
access the fixture function::
access the fixture function:
.. code-block:: python
# content of conftest.py
import pytest
import smtplib
@pytest.fixture(scope="module")
def smtp_connection():
return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
@@ -193,16 +201,20 @@ access the fixture function::
The name of the fixture again is ``smtp_connection`` and you can access its
result by listing the name ``smtp_connection`` as an input parameter in any
test or fixture function (in or below the directory where ``conftest.py`` is
located)::
located):
.. code-block:: python
# content of test_module.py
def test_ehlo(smtp_connection):
response, msg = smtp_connection.ehlo()
assert response == 250
assert b"smtp.gmail.com" in msg
assert 0 # for demo purposes
def test_noop(smtp_connection):
response, msg = smtp_connection.noop()
assert response == 250
@@ -215,7 +227,7 @@ inspect what is going on and can now run the tests:
$ pytest test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
@@ -234,7 +246,7 @@ inspect what is going on and can now run the tests:
> assert 0 # for demo purposes
E assert 0
test_module.py:6: AssertionError
test_module.py:7: AssertionError
________________________________ test_noop _________________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@@ -245,8 +257,8 @@ inspect what is going on and can now run the tests:
> assert 0 # for demo purposes
E assert 0
test_module.py:11: AssertionError
========================= 2 failed in 0.12 seconds =========================
test_module.py:13: AssertionError
============================ 2 failed in 0.12s =============================
You see the two ``assert 0`` failing and more importantly you can also see
that the same (module-scoped) ``smtp_connection`` object was passed into the
@@ -289,51 +301,59 @@ are finalized when the last test of a *package* finishes.
Use this new feature sparingly and please make sure to report any issues you find.
Higher-scoped fixtures are instantiated first
---------------------------------------------
.. _dynamic scope:
Dynamic scope
^^^^^^^^^^^^^
.. versionadded:: 5.2
In some cases, you might want to change the scope of the fixture without changing the code.
To do that, pass a callable to ``scope``. The callable must return a string with a valid scope
and will be executed only once - during the fixture definition. It will be called with two
keyword arguments - ``fixture_name`` as a string and ``config`` with a configuration object.
This can be especially useful when dealing with fixtures that need time for setup, like spawning
a docker container. You can use the command-line argument to control the scope of the spawned
containers for different environments. See the example below.
.. code-block:: python
def determine_scope(fixture_name, config):
if config.getoption("--keep-containers"):
return "session"
return "function"
@pytest.fixture(scope=determine_scope)
def docker_container():
yield spawn_container()
Order: Higher-scoped fixtures are instantiated first
----------------------------------------------------
Within a function request for features, fixture of higher-scopes (such as ``session``) are instantiated first than
lower-scoped fixtures (such as ``function`` or ``class``). The relative order of fixtures of same scope follows
the declared order in the test function and honours dependencies between fixtures.
the declared order in the test function and honours dependencies between fixtures. Autouse fixtures will be
instantiated before explicitly used fixtures.
Consider the code below:
.. code-block:: python
.. literalinclude:: example/fixtures/test_fixtures_order.py
@pytest.fixture(scope="session")
def s1():
pass
@pytest.fixture(scope="module")
def m1():
pass
@pytest.fixture
def f1(tmpdir):
pass
@pytest.fixture
def f2():
pass
def test_foo(f1, m1, f2, s1):
...
The fixtures requested by ``test_foo`` will be instantiated in the following order:
The fixtures requested by ``test_order`` will be instantiated in the following order:
1. ``s1``: is the highest-scoped fixture (``session``).
2. ``m1``: is the second highest-scoped fixture (``module``).
3. ``tmpdir``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point
because it is a dependency of ``f1``.
4. ``f1``: is the first ``function``-scoped fixture in ``test_foo`` parameter list.
5. ``f2``: is the last ``function``-scoped fixture in ``test_foo`` parameter list.
3. ``a1``: is a ``function``-scoped ``autouse`` fixture: it will be instantiated before other fixtures
within the same scope.
4. ``f3``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point
5. ``f1``: is the first ``function``-scoped fixture in ``test_order`` parameter list.
6. ``f2``: is the last ``function``-scoped fixture in ``test_order`` parameter list.
.. _`finalization`:
@@ -371,7 +391,7 @@ Let's execute it:
$ pytest -s -q --tb=no
FFteardown smtp
2 failed in 0.12 seconds
2 failed in 0.12s
We see that the ``smtp_connection`` instance is finalized after the two
tests finished execution. Note that if we decorated our fixture
@@ -400,6 +420,34 @@ The ``smtp_connection`` connection will be closed after the test finished
execution because the ``smtp_connection`` object automatically closes when
the ``with`` statement ends.
Using the contextlib.ExitStack context manager finalizers will always be called
regardless if the fixture *setup* code raises an exception. This is handy to properly
close all resources created by a fixture even if one of them fails to be created/acquired:
.. code-block:: python
# content of test_yield3.py
import contextlib
import pytest
@contextlib.contextmanager
def connect(port):
... # create connection
yield
... # close connection
@pytest.fixture
def equipments():
with contextlib.ExitStack() as stack:
yield [stack.enter_context(connect(port)) for port in ("C1", "C3", "C28")]
In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still
be properly closed.
Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the
*teardown* code (after the ``yield``) will not be called.
@@ -428,27 +476,39 @@ Here's the ``smtp_connection`` fixture changed to use ``addfinalizer`` for clean
return smtp_connection # provide the fixture value
Here's the ``equipments`` fixture changed to use ``addfinalizer`` for cleanup:
.. code-block:: python
# content of test_yield3.py
import contextlib
import functools
import pytest
@contextlib.contextmanager
def connect(port):
... # create connection
yield
... # close connection
@pytest.fixture
def equipments(request):
r = []
for port in ("C1", "C3", "C28"):
cm = connect(port)
equip = cm.__enter__()
request.addfinalizer(functools.partial(cm.__exit__, None, None, None))
r.append(equip)
return r
Both ``yield`` and ``addfinalizer`` methods work similarly by calling their code after the test
ends, but ``addfinalizer`` has two key differences over ``yield``:
1. It is possible to register multiple finalizer functions.
2. Finalizers will always be called regardless if the fixture *setup* code raises an exception.
This is handy to properly close all resources created by a fixture even if one of them
fails to be created/acquired::
@pytest.fixture
def equipments(request):
r = []
for port in ('C1', 'C3', 'C28'):
equip = connect(port)
request.addfinalizer(equip.disconnect)
r.append(equip)
return r
In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still
be properly closed. Of course, if an exception happens before the finalize function is
registered then it will not be executed.
ends. Of course, if an exception happens before the finalize function is registered then it
will not be executed.
.. _`request-context`:
@@ -459,18 +519,21 @@ Fixtures can introspect the requesting test context
Fixture functions can accept the :py:class:`request <FixtureRequest>` object
to introspect the "requesting" test function, class or module context.
Further extending the previous ``smtp_connection`` fixture example, let's
read an optional server URL from the test module which uses our fixture::
read an optional server URL from the test module which uses our fixture:
.. code-block:: python
# content of conftest.py
import pytest
import smtplib
@pytest.fixture(scope="module")
def smtp_connection(request):
server = getattr(request.module, "smtpserver", "smtp.gmail.com")
smtp_connection = smtplib.SMTP(server, 587, timeout=5)
yield smtp_connection
print("finalizing %s (%s)" % (smtp_connection, server))
print("finalizing {} ({})".format(smtp_connection, server))
smtp_connection.close()
We use the ``request.module`` attribute to optionally obtain an
@@ -482,15 +545,18 @@ again, nothing much has changed:
$ pytest -s -q --tb=no
FFfinalizing <smtplib.SMTP object at 0xdeadbeef> (smtp.gmail.com)
2 failed in 0.12 seconds
2 failed in 0.12s
Let's quickly create another test module that actually sets the
server URL in its module namespace::
server URL in its module namespace:
.. code-block:: python
# content of test_anothersmtp.py
smtpserver = "mail.python.org" # will be read by smtp fixture
def test_showhelo(smtp_connection):
assert 0, smtp_connection.helo()
@@ -502,7 +568,7 @@ Running it:
F [100%]
================================= FAILURES =================================
______________________________ test_showhelo _______________________________
test_anothersmtp.py:5: in test_showhelo
test_anothersmtp.py:6: in test_showhelo
assert 0, smtp_connection.helo()
E AssertionError: (250, b'mail.python.org')
E assert 0
@@ -522,16 +588,14 @@ of a fixture is needed multiple times in a single test. Instead of returning
data directly, the fixture instead returns a function which generates the data.
This function can then be called multiple times in the test.
Factories can have have parameters as needed::
Factories can have parameters as needed:
.. code-block:: python
@pytest.fixture
def make_customer_record():
def _make_customer_record(name):
return {
"name": name,
"orders": []
}
return {"name": name, "orders": []}
return _make_customer_record
@@ -541,7 +605,9 @@ Factories can have have parameters as needed::
customer_2 = make_customer_record("Mike")
customer_3 = make_customer_record("Meredith")
If the data created by the factory requires managing, the fixture can take care of that::
If the data created by the factory requires managing, the fixture can take care of that:
.. code-block:: python
@pytest.fixture
def make_customer_record():
@@ -580,18 +646,20 @@ configured in multiple ways.
Extending the previous example, we can flag the fixture to create two
``smtp_connection`` fixture instances which will cause all tests using the fixture
to run twice. The fixture function gets access to each parameter
through the special :py:class:`request <FixtureRequest>` object::
through the special :py:class:`request <FixtureRequest>` object:
.. code-block:: python
# content of conftest.py
import pytest
import smtplib
@pytest.fixture(scope="module",
params=["smtp.gmail.com", "mail.python.org"])
@pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"])
def smtp_connection(request):
smtp_connection = smtplib.SMTP(request.param, 587, timeout=5)
yield smtp_connection
print("finalizing %s" % smtp_connection)
print("finalizing {}".format(smtp_connection))
smtp_connection.close()
The main change is the declaration of ``params`` with
@@ -616,7 +684,7 @@ So let's just do another run:
> assert 0 # for demo purposes
E assert 0
test_module.py:6: AssertionError
test_module.py:7: AssertionError
________________________ test_noop[smtp.gmail.com] _________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@@ -627,7 +695,7 @@ So let's just do another run:
> assert 0 # for demo purposes
E assert 0
test_module.py:11: AssertionError
test_module.py:13: AssertionError
________________________ test_ehlo[mail.python.org] ________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@@ -638,7 +706,7 @@ So let's just do another run:
> assert b"smtp.gmail.com" in msg
E AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8\nCHUNKING'
test_module.py:5: AssertionError
test_module.py:6: AssertionError
-------------------------- Captured stdout setup ---------------------------
finalizing <smtplib.SMTP object at 0xdeadbeef>
________________________ test_noop[mail.python.org] ________________________
@@ -651,10 +719,10 @@ So let's just do another run:
> assert 0 # for demo purposes
E assert 0
test_module.py:11: AssertionError
test_module.py:13: AssertionError
------------------------- Captured stdout teardown -------------------------
finalizing <smtplib.SMTP object at 0xdeadbeef>
4 failed in 0.12 seconds
4 failed in 0.12s
We see that our two test functions each ran twice, against the different
``smtp_connection`` instances. Note also, that with the ``mail.python.org``
@@ -672,28 +740,35 @@ Numbers, strings, booleans and None will have their usual string
representation used in the test ID. For other objects, pytest will
make a string based on the argument name. It is possible to customise
the string used in a test ID for a certain fixture value by using the
``ids`` keyword argument::
``ids`` keyword argument:
.. code-block:: python
# content of test_ids.py
import pytest
@pytest.fixture(params=[0, 1], ids=["spam", "ham"])
def a(request):
return request.param
def test_a(a):
pass
def idfn(fixture_value):
if fixture_value == 0:
return "eggs"
else:
return None
@pytest.fixture(params=[0, 1], ids=idfn)
def b(request):
return request.param
def test_b(b):
pass
@@ -708,7 +783,7 @@ Running the above tests results in the following test IDs being used:
$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 10 items
@@ -726,7 +801,7 @@ Running the above tests results in the following test IDs being used:
<Function test_ehlo[mail.python.org]>
<Function test_noop[mail.python.org]>
======================= no tests ran in 0.12 seconds =======================
========================== no tests ran in 0.12s ===========================
.. _`fixture-parametrize-marks`:
@@ -736,14 +811,19 @@ Using marks with parametrized fixtures
:func:`pytest.param` can be used to apply marks in values sets of parametrized fixtures in the same way
that they can be used with :ref:`@pytest.mark.parametrize <@pytest.mark.parametrize>`.
Example::
Example:
.. code-block:: python
# content of test_fixture_marks.py
import pytest
@pytest.fixture(params=[0, 1, pytest.param(2, marks=pytest.mark.skip)])
def data_set(request):
return request.param
def test_data(data_set):
pass
@@ -753,7 +833,7 @@ Running this test will *skip* the invocation of ``data_set`` with value ``2``:
$ pytest test_fixture_marks.py -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 3 items
@@ -762,7 +842,7 @@ Running this test will *skip* the invocation of ``data_set`` with value ``2``:
test_fixture_marks.py::test_data[1] PASSED [ 66%]
test_fixture_marks.py::test_data[2] SKIPPED [100%]
=================== 2 passed, 1 skipped in 0.12 seconds ====================
======================= 2 passed, 1 skipped in 0.12s =======================
.. _`interdependent fixtures`:
@@ -774,20 +854,25 @@ can use other fixtures themselves. This contributes to a modular design
of your fixtures and allows re-use of framework-specific fixtures across
many projects. As a simple example, we can extend the previous example
and instantiate an object ``app`` where we stick the already defined
``smtp_connection`` resource into it::
``smtp_connection`` resource into it:
.. code-block:: python
# content of test_appsetup.py
import pytest
class App(object):
class App:
def __init__(self, smtp_connection):
self.smtp_connection = smtp_connection
@pytest.fixture(scope="module")
def app(smtp_connection):
return App(smtp_connection)
def test_smtp_connection_exists(app):
assert app.smtp_connection
@@ -798,7 +883,7 @@ Here we declare an ``app`` fixture which receives the previously defined
$ pytest -v test_appsetup.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 2 items
@@ -806,7 +891,7 @@ Here we declare an ``app`` fixture which receives the previously defined
test_appsetup.py::test_smtp_connection_exists[smtp.gmail.com] PASSED [ 50%]
test_appsetup.py::test_smtp_connection_exists[mail.python.org] PASSED [100%]
========================= 2 passed in 0.12 seconds =========================
============================ 2 passed in 0.12s =============================
Due to the parametrization of ``smtp_connection``, the test will run twice with two
different ``App`` instances and respective smtp servers. There is no
@@ -836,31 +921,40 @@ this eases testing of applications which create and use global state.
The following example uses two parametrized fixtures, one of which is
scoped on a per-module basis, and all the functions perform ``print`` calls
to show the setup/teardown flow::
to show the setup/teardown flow:
.. code-block:: python
# content of test_module.py
import pytest
@pytest.fixture(scope="module", params=["mod1", "mod2"])
def modarg(request):
param = request.param
print(" SETUP modarg %s" % param)
print(" SETUP modarg", param)
yield param
print(" TEARDOWN modarg %s" % param)
print(" TEARDOWN modarg", param)
@pytest.fixture(scope="function", params=[1,2])
@pytest.fixture(scope="function", params=[1, 2])
def otherarg(request):
param = request.param
print(" SETUP otherarg %s" % param)
print(" SETUP otherarg", param)
yield param
print(" TEARDOWN otherarg %s" % param)
print(" TEARDOWN otherarg", param)
def test_0(otherarg):
print(" RUN test0 with otherarg %s" % otherarg)
print(" RUN test0 with otherarg", otherarg)
def test_1(modarg):
print(" RUN test1 with modarg %s" % modarg)
print(" RUN test1 with modarg", modarg)
def test_2(otherarg, modarg):
print(" RUN test2 with otherarg %s and modarg %s" % (otherarg, modarg))
print(" RUN test2 with otherarg {} and modarg {}".format(otherarg, modarg))
Let's run the tests in verbose mode and with looking at the print-output:
@@ -869,7 +963,7 @@ Let's run the tests in verbose mode and with looking at the print-output:
$ pytest -v -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 8 items
@@ -907,7 +1001,7 @@ Let's run the tests in verbose mode and with looking at the print-output:
TEARDOWN modarg mod2
========================= 8 passed in 0.12 seconds =========================
============================ 8 passed in 0.12s =============================
You can see that the parametrized module-scoped ``modarg`` resource caused an
ordering of test execution that lead to the fewest possible "active" resources.
@@ -935,7 +1029,9 @@ current working directory but otherwise do not care for the concrete
directory. Here is how you can use the standard `tempfile
<http://docs.python.org/library/tempfile.html>`_ and pytest fixtures to
achieve it. We separate the creation of the fixture into a conftest.py
file::
file:
.. code-block:: python
# content of conftest.py
@@ -943,19 +1039,23 @@ file::
import tempfile
import os
@pytest.fixture()
def cleandir():
newpath = tempfile.mkdtemp()
os.chdir(newpath)
and declare its use in a test module via a ``usefixtures`` marker::
and declare its use in a test module via a ``usefixtures`` marker:
.. code-block:: python
# content of test_setenv.py
import os
import pytest
@pytest.mark.usefixtures("cleandir")
class TestDirectoryInit(object):
class TestDirectoryInit:
def test_cwd_starts_empty(self):
assert os.listdir(os.getcwd()) == []
with open("myfile", "w") as f:
@@ -973,7 +1073,7 @@ to verify our fixture is activated and the tests pass:
$ pytest -q
.. [100%]
2 passed in 0.12 seconds
2 passed in 0.12s
You can specify multiple fixtures like this:
@@ -1032,25 +1132,32 @@ without declaring a function argument explicitly or a `usefixtures`_ decorator.
As a practical example, suppose we have a database fixture which has a
begin/rollback/commit architecture and we want to automatically surround
each test method by a transaction and a rollback. Here is a dummy
self-contained implementation of this idea::
self-contained implementation of this idea:
.. code-block:: python
# content of test_db_transact.py
import pytest
class DB(object):
class DB:
def __init__(self):
self.intransaction = []
def begin(self, name):
self.intransaction.append(name)
def rollback(self):
self.intransaction.pop()
@pytest.fixture(scope="module")
def db():
return DB()
class TestClass(object):
class TestClass:
@pytest.fixture(autouse=True)
def transact(self, request, db):
db.begin(request.function.__name__)
@@ -1074,7 +1181,7 @@ If we run it, we get two passing tests:
$ pytest -q
.. [100%]
2 passed in 0.12 seconds
2 passed in 0.12s
Here is how autouse fixtures work in other scopes:
@@ -1098,7 +1205,9 @@ Here is how autouse fixtures work in other scopes:
Note that the above ``transact`` fixture may very well be a fixture that
you want to make available in your project without having it generally
active. The canonical way to do that is to put the transact definition
into a conftest.py file **without** using ``autouse``::
into a conftest.py file **without** using ``autouse``:
.. code-block:: python
# content of conftest.py
@pytest.fixture
@@ -1107,10 +1216,12 @@ into a conftest.py file **without** using ``autouse``::
yield
db.rollback()
and then e.g. have a TestClass using it by declaring the need::
and then e.g. have a TestClass using it by declaring the need:
.. code-block:: python
@pytest.mark.usefixtures("transact")
class TestClass(object):
class TestClass:
def test_method1(self):
...

View File

@@ -122,4 +122,4 @@ Resources
* Google:
* `Flaky Tests at Google and How We Mitigate Them <https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html>`_ by John Micco, 2016
* `Where do Google's flaky tests come from? <https://docs.google.com/document/d/1mZ0-Kc97DI_F3tf_GBW_NB_aqka-P1jVOsFfufxqUUM/edit#heading=h.ec0r4fypsleh>`_ by Jeff Listfield, 2017
* `Where do Google's flaky tests come from? <https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html>`_ by Jeff Listfield, 2017

View File

@@ -21,19 +21,23 @@ funcarg for a test function is required. If a factory wants to
re-use a resource across different scopes, it often used
the ``request.cached_setup()`` helper to manage caching of
resources. Here is a basic example how we could implement
a per-session Database object::
a per-session Database object:
.. code-block:: python
# content of conftest.py
class Database(object):
class Database:
def __init__(self):
print("database instance created")
def destroy(self):
print("database instance destroyed")
def pytest_funcarg__db(request):
return request.cached_setup(setup=DataBase,
teardown=lambda db: db.destroy,
scope="session")
return request.cached_setup(
setup=DataBase, teardown=lambda db: db.destroy, scope="session"
)
There are several limitations and difficulties with this approach:
@@ -68,7 +72,9 @@ Direct scoping of fixture/funcarg factories
Instead of calling cached_setup() with a cache scope, you can use the
:ref:`@pytest.fixture <pytest.fixture>` decorator and directly state
the scope::
the scope:
.. code-block:: python
@pytest.fixture(scope="session")
def db(request):
@@ -90,11 +96,13 @@ Previously, funcarg factories could not directly cause parametrization.
You needed to specify a ``@parametrize`` decorator on your test function
or implement a ``pytest_generate_tests`` hook to perform
parametrization, i.e. calling a test multiple times with different value
sets. pytest-2.3 introduces a decorator for use on the factory itself::
sets. pytest-2.3 introduces a decorator for use on the factory itself:
.. code-block:: python
@pytest.fixture(params=["mysql", "pg"])
def db(request):
... # use request.param
... # use request.param
Here the factory will be invoked twice (with the respective "mysql"
and "pg" values set as ``request.param`` attributes) and all of
@@ -107,7 +115,9 @@ allow to re-use already written factories because effectively
parametrized via
:py:func:`~_pytest.python.Metafunc.parametrize(indirect=True)` calls.
Of course it's perfectly fine to combine parametrization and scoping::
Of course it's perfectly fine to combine parametrization and scoping:
.. code-block:: python
@pytest.fixture(scope="session", params=["mysql", "pg"])
def db(request):
@@ -128,7 +138,9 @@ No ``pytest_funcarg__`` prefix when using @fixture decorator
When using the ``@fixture`` decorator the name of the function
denotes the name under which the resource can be accessed as a function
argument::
argument:
.. code-block:: python
@pytest.fixture()
def db(request):
@@ -137,7 +149,9 @@ argument::
The name under which the funcarg resource can be requested is ``db``.
You can still use the "old" non-decorator way of specifying funcarg factories
aka::
aka:
.. code-block:: python
def pytest_funcarg__db(request):
...

View File

@@ -1,9 +1,9 @@
Installation and Getting Started
===================================
**Pythons**: Python 2.7, 3.4, 3.5, 3.6, 3.7, Jython, PyPy-2.3
**Pythons**: Python 3.5, 3.6, 3.7, PyPy3
**Platforms**: Unix/Posix and Windows
**Platforms**: Linux and Windows
**PyPI package name**: `pytest <https://pypi.org/project/pytest/>`_
@@ -28,19 +28,22 @@ Install ``pytest``
.. code-block:: bash
$ pytest --version
This is pytest version 4.x.y, imported from $PYTHON_PREFIX/lib/python3.6/site-packages/pytest.py
This is pytest version 5.x.y, imported from $PYTHON_PREFIX/lib/python3.7/site-packages/pytest.py
.. _`simpletest`:
Create your first test
----------------------------------------------------------
Create a simple test function with just four lines of code::
Create a simple test function with just four lines of code:
.. code-block:: python
# content of test_sample.py
def func(x):
return x + 1
def test_answer():
assert func(3) == 5
@@ -50,7 +53,7 @@ Thats it. You can now execute the test function:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -65,8 +68,8 @@ Thats it. You can now execute the test function:
E assert 4 == 5
E + where 4 = func(3)
test_sample.py:5: AssertionError
========================= 1 failed in 0.12 seconds =========================
test_sample.py:6: AssertionError
============================ 1 failed in 0.12s =============================
This test returns a failure report because ``func(3)`` does not return ``5``.
@@ -83,13 +86,18 @@ Run multiple tests
Assert that a certain exception is raised
--------------------------------------------------------------
Use the :ref:`raises <assertraises>` helper to assert that some code raises an exception::
Use the :ref:`raises <assertraises>` helper to assert that some code raises an exception:
.. code-block:: python
# content of test_sysexit.py
import pytest
def f():
raise SystemExit(1)
def test_mytest():
with pytest.raises(SystemExit):
f()
@@ -100,22 +108,24 @@ Execute the test function with “quiet” reporting mode:
$ pytest -q test_sysexit.py
. [100%]
1 passed in 0.12 seconds
1 passed in 0.12s
Group multiple tests in a class
--------------------------------------------------------------
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test::
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test:
.. code-block:: python
# content of test_class.py
class TestClass(object):
class TestClass:
def test_one(self):
x = "this"
assert 'h' in x
assert "h" in x
def test_two(self):
x = "hello"
assert hasattr(x, 'check')
assert hasattr(x, "check")
``pytest`` discovers all tests following its :ref:`Conventions for Python test discovery <test discovery>`, so it finds both ``test_`` prefixed functions. There is no need to subclass anything. We can simply run the module by passing its filename:
@@ -130,19 +140,21 @@ Once you develop multiple tests, you may want to group them into a class. pytest
def test_two(self):
x = "hello"
> assert hasattr(x, 'check')
> assert hasattr(x, "check")
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:8: AssertionError
1 failed, 1 passed in 0.12 seconds
1 failed, 1 passed in 0.12s
The first test passed and the second failed. You can easily see the intermediate values in the assertion to help you understand the reason for the failure.
Request a unique temporary directory for functional tests
--------------------------------------------------------------
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html#builtinfixtures>`_ to request arbitrary resources, like a unique temporary directory::
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
.. code-block:: python
# content of test_tmpdir.py
def test_needsfiles(tmpdir):
@@ -168,7 +180,7 @@ List the name ``tmpdir`` in the test function signature and ``pytest`` will look
test_tmpdir.py:3: AssertionError
--------------------------- Captured stdout call ---------------------------
PYTEST_TMPDIR/test_needsfiles0
1 failed in 0.12 seconds
1 failed in 0.12s
More info on tmpdir handling is available at :ref:`Temporary directories and files <tmpdir handling>`.

View File

@@ -7,19 +7,22 @@ Good Integration Practices
Install package with pip
-------------------------------------------------
For development, we recommend you use venv_ for virtual environments
(or virtualenv_ for Python 2.7) and
For development, we recommend you use venv_ for virtual environments and
pip_ for installing your application and any dependencies,
as well as the ``pytest`` package itself.
This ensures your code and dependencies are isolated from your system Python installation.
Next, place a ``setup.py`` file in the root of your package with the following minimum content::
Next, place a ``setup.py`` file in the root of your package with the following minimum content:
.. code-block:: python
from setuptools import setup, find_packages
setup(name="PACKAGENAME", packages=find_packages())
Where ``PACKAGENAME`` is the name of your package. You can then install your package in "editable" mode by running from the same directory::
Where ``PACKAGENAME`` is the name of your package. You can then install your package in "editable" mode by running from the same directory:
.. code-block:: bash
pip install -e .
@@ -61,7 +64,9 @@ Tests outside application code
Putting tests into an extra directory outside your actual application code
might be useful if you have many functional tests or for other reasons want
to keep tests separate from actual application code (often a good idea)::
to keep tests separate from actual application code (often a good idea):
.. code-block:: text
setup.py
mypkg/
@@ -83,7 +88,7 @@ This has the following benefits:
.. note::
See :ref:`pythonpath` for more information about the difference between calling ``pytest`` and
See :ref:`pytest vs python -m pytest` for more information about the difference between calling ``pytest`` and
``python -m pytest``.
Note that using this scheme your test files must have **unique names**, because
@@ -93,7 +98,9 @@ be imported as ``test_app`` and ``test_view`` top-level modules by adding ``test
``sys.path``.
If you need to have test modules with the same name, you might add ``__init__.py`` files to your
``tests`` folder and subfolders, changing them to packages::
``tests`` folder and subfolders, changing them to packages:
.. code-block:: text
setup.py
mypkg/
@@ -115,7 +122,9 @@ This is problematic if you are using a tool like `tox`_ to test your package in
because you want to test the *installed* version of your package, not the local code from the repository.
In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
sub-directory of your root::
sub-directory of your root:
.. code-block:: text
setup.py
src/
@@ -141,7 +150,9 @@ Tests as part of application code
Inlining test directories into your application package
is useful if you have direct relation between tests and application modules and
want to distribute them along with your application::
want to distribute them along with your application:
.. code-block:: text
setup.py
mypkg/
@@ -154,7 +165,9 @@ want to distribute them along with your application::
test_view.py
...
In this scheme, it is easy to run your tests using the ``--pyargs`` option::
In this scheme, it is easy to run your tests using the ``--pyargs`` option:
.. code-block:: bash
pytest --pyargs mypkg
@@ -220,101 +233,4 @@ against your source code checkout, helping to detect packaging
glitches.
Integrating with setuptools / ``python setup.py test`` / ``pytest-runner``
--------------------------------------------------------------------------
You can integrate test runs into your setuptools based project
with the `pytest-runner <https://pypi.org/project/pytest-runner/>`_ plugin.
Add this to ``setup.py`` file:
.. code-block:: python
from setuptools import setup
setup(
# ...,
setup_requires=["pytest-runner", ...],
tests_require=["pytest", ...],
# ...,
)
And create an alias into ``setup.cfg`` file:
.. code-block:: ini
[aliases]
test=pytest
If you now type::
python setup.py test
this will execute your tests using ``pytest-runner``. As this is a
standalone version of ``pytest`` no prior installation whatsoever is
required for calling the test command. You can also pass additional
arguments to pytest such as your test directory or other
options using ``--addopts``.
You can also specify other pytest-ini options in your ``setup.cfg`` file
by putting them into a ``[tool:pytest]`` section:
.. code-block:: ini
[tool:pytest]
addopts = --verbose
python_files = testing/*/*.py
Manual Integration
^^^^^^^^^^^^^^^^^^
If for some reason you don't want/can't use ``pytest-runner``, you can write
your own setuptools Test command for invoking pytest.
.. code-block:: python
import sys
from setuptools.command.test import test as TestCommand
class PyTest(TestCommand):
user_options = [("pytest-args=", "a", "Arguments to pass to pytest")]
def initialize_options(self):
TestCommand.initialize_options(self)
self.pytest_args = ""
def run_tests(self):
import shlex
# import here, cause outside the eggs aren't loaded
import pytest
errno = pytest.main(shlex.split(self.pytest_args))
sys.exit(errno)
setup(
# ...,
tests_require=["pytest"],
cmdclass={"pytest": PyTest},
)
Now if you run::
python setup.py test
this will download ``pytest`` if needed and then run your tests
as you would expect it to. You can pass a single string of arguments
using the ``--pytest-args`` or ``-a`` command-line option. For example::
python setup.py test -a "--durations=5"
is equivalent to running ``pytest --durations=5``.
.. include:: links.inc

View File

@@ -28,7 +28,7 @@ To execute it:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
@@ -44,7 +44,7 @@ To execute it:
E + where 4 = inc(3)
test_sample.py:6: AssertionError
========================= 1 failed in 0.12 seconds =========================
============================ 1 failed in 0.12s =============================
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used.
See :ref:`Getting Started <getstarted>` for more examples.
@@ -61,7 +61,7 @@ Features
- Can run :ref:`unittest <unittest>` (including trial) and :ref:`nose <noseintegration>` test suites out of the box;
- Python 2.7, Python 3.4+, PyPy 2.3, Jython 2.5 (untested);
- Python 3.5+ and PyPy 3;
- Rich plugin architecture, with over 315+ `external plugins <http://plugincompat.herokuapp.com>`_ and thriving community;
@@ -83,6 +83,39 @@ Changelog
Consult the :ref:`Changelog <changelog>` page for fixes and enhancements of each version.
Support pytest
--------------
`Open Collective`_ is an online funding platform for open and transparent communities.
It provide tools to raise money and share your finances in full transparency.
It is the platform of choice for individuals and companies that want to make one-time or
monthly donations directly to the project.
See more datails in the `pytest collective`_.
.. _Open Collective: https://opencollective.com
.. _pytest collective: https://opencollective.com/pytest
pytest for enterprise
---------------------
Available as part of the Tidelift Subscription.
The maintainers of pytest and thousands of other packages are working with Tidelift to deliver commercial support and
maintenance for the open source dependencies you use to build your applications.
Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use.
`Learn more. <https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=enterprise&utm_term=repo>`_
Security
^^^^^^^^
pytest has never been associated with a security vunerability, but in any case, to report a
security vulnerability please use the `Tidelift security contact <https://tidelift.com/security>`_.
Tidelift will coordinate the fix and disclosure.
License
-------

View File

@@ -9,7 +9,7 @@ Distributed under the terms of the `MIT`_ license, pytest is free and open sourc
The MIT License (MIT)
Copyright (c) 2004-2017 Holger Krekel and others
Copyright (c) 2004-2019 Holger Krekel and others
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in

View File

@@ -14,7 +14,7 @@
.. _`distribute docs`:
.. _`distribute`: https://pypi.org/project/distribute/
.. _`pip`: https://pypi.org/project/pip/
.. _`venv`: https://docs.python.org/3/library/venv.html/
.. _`venv`: https://docs.python.org/3/library/venv.html
.. _`virtualenv`: https://pypi.org/project/virtualenv/
.. _hudson: http://hudson-ci.org/
.. _jenkins: http://jenkins-ci.org/

View File

@@ -70,7 +70,9 @@ caplog fixture
^^^^^^^^^^^^^^
Inside tests it is possible to change the log level for the captured log
messages. This is supported by the ``caplog`` fixture::
messages. This is supported by the ``caplog`` fixture:
.. code-block:: python
def test_foo(caplog):
caplog.set_level(logging.INFO)
@@ -78,59 +80,69 @@ messages. This is supported by the ``caplog`` fixture::
By default the level is set on the root logger,
however as a convenience it is also possible to set the log level of any
logger::
logger:
.. code-block:: python
def test_foo(caplog):
caplog.set_level(logging.CRITICAL, logger='root.baz')
caplog.set_level(logging.CRITICAL, logger="root.baz")
pass
The log levels set are restored automatically at the end of the test.
It is also possible to use a context manager to temporarily change the log
level inside a ``with`` block::
level inside a ``with`` block:
.. code-block:: python
def test_bar(caplog):
with caplog.at_level(logging.INFO):
pass
Again, by default the level of the root logger is affected but the level of any
logger can be changed instead with::
logger can be changed instead with:
.. code-block:: python
def test_bar(caplog):
with caplog.at_level(logging.CRITICAL, logger='root.baz'):
with caplog.at_level(logging.CRITICAL, logger="root.baz"):
pass
Lastly all the logs sent to the logger during the test run are made available on
the fixture in the form of both the ``logging.LogRecord`` instances and the final log text.
This is useful for when you want to assert on the contents of a message::
This is useful for when you want to assert on the contents of a message:
.. code-block:: python
def test_baz(caplog):
func_under_test()
for record in caplog.records:
assert record.levelname != 'CRITICAL'
assert 'wally' not in caplog.text
assert record.levelname != "CRITICAL"
assert "wally" not in caplog.text
For all the available attributes of the log records see the
``logging.LogRecord`` class.
You can also resort to ``record_tuples`` if all you want to do is to ensure,
that certain messages have been logged under a given logger name with a given
severity and message::
severity and message:
.. code-block:: python
def test_foo(caplog):
logging.getLogger().info('boo %s', 'arg')
logging.getLogger().info("boo %s", "arg")
assert caplog.record_tuples == [
('root', logging.INFO, 'boo arg'),
]
assert caplog.record_tuples == [("root", logging.INFO, "boo arg")]
You can call ``caplog.clear()`` to reset the captured log records in a test::
You can call ``caplog.clear()`` to reset the captured log records in a test:
.. code-block:: python
def test_something_with_clearing_records(caplog):
some_method_that_creates_log_records()
caplog.clear()
your_test_method()
assert ['Foo'] == [rec.message for rec in caplog.records]
assert ["Foo"] == [rec.message for rec in caplog.records]
The ``caplog.records`` attribute contains records from the current stage only, so
@@ -149,7 +161,7 @@ the records for the ``setup`` and ``call`` stages during teardown like so:
yield window
for when in ("setup", "call"):
messages = [
x.message for x in caplog.get_records(when) if x.level == logging.WARNING
x.message for x in caplog.get_records(when) if x.levelno == logging.WARNING
]
if messages:
pytest.fail(

View File

@@ -40,7 +40,7 @@ You can register custom marks in your ``pytest.ini`` file like this:
Note that everything after the ``:`` is an optional description.
Alternatively, you can register new markers programatically in a
Alternatively, you can register new markers programmatically in a
:ref:`pytest_configure <initialization-hooks>` hook:
.. code-block:: python

View File

@@ -8,46 +8,218 @@ Sometimes tests need to invoke functionality which depends
on global settings or which invokes code which cannot be easily
tested such as network access. The ``monkeypatch`` fixture
helps you to safely set/delete an attribute, dictionary item or
environment variable or to modify ``sys.path`` for importing.
environment variable, or to modify ``sys.path`` for importing.
The ``monkeypatch`` fixture provides these helper methods for safely patching and mocking
functionality in tests:
.. code-block:: python
monkeypatch.setattr(obj, name, value, raising=True)
monkeypatch.delattr(obj, name, raising=True)
monkeypatch.setitem(mapping, name, value)
monkeypatch.delitem(obj, name, raising=True)
monkeypatch.setenv(name, value, prepend=False)
monkeypatch.delenv(name, raising=True)
monkeypatch.syspath_prepend(path)
monkeypatch.chdir(path)
All modifications will be undone after the requesting
test function or fixture has finished. The ``raising``
parameter determines if a ``KeyError`` or ``AttributeError``
will be raised if the target of the set/deletion operation does not exist.
Consider the following scenarios:
1. Modifying the behavior of a function or the property of a class for a test e.g.
there is an API call or database connection you will not make for a test but you know
what the expected output should be. Use :py:meth:`monkeypatch.setattr` to patch the
function or property with your desired testing behavior. This can include your own functions.
Use :py:meth:`monkeypatch.delattr` to remove the function or property for the test.
2. Modifying the values of dictionaries e.g. you have a global configuration that
you want to modify for certain test cases. Use :py:meth:`monkeypatch.setitem` to patch the
dictionary for the test. :py:meth:`monkeypatch.delitem` can be used to remove items.
3. Modifying environment variables for a test e.g. to test program behavior if an
environment variable is missing, or to set multiple values to a known variable.
:py:meth:`monkeypatch.setenv` and :py:meth:`monkeypatch.delenv` can be used for
these patches.
4. Use ``monkeypatch.setenv("PATH", value, prepend=os.pathsep)`` to modify ``$PATH``, and
:py:meth:`monkeypatch.chdir` to change the context of the current working directory
during a test.
5. Use :py:meth:`monkeypatch.syspath_prepend` to modify ``sys.path`` which will also
call :py:meth:`pkg_resources.fixup_namespace_packages` and :py:meth:`importlib.invalidate_caches`.
See the `monkeypatch blog post`_ for some introduction material
and a discussion of its motivation.
.. _`monkeypatch blog post`: http://tetamap.wordpress.com/2009/03/03/monkeypatching-in-unit-tests-done-right/
Simple example: monkeypatching functions
----------------------------------------
If you want to pretend that ``os.expanduser`` returns a certain
directory, you can use the :py:meth:`monkeypatch.setattr` method to
patch this function before calling into a function which uses it::
Consider a scenario where you are working with user directories. In the context of
testing, you do not want your test to depend on the running user. ``monkeypatch``
can be used to patch functions dependent on the user to always return a
specific value.
# content of test_module.py
import os.path
def getssh(): # pseudo application code
return os.path.join(os.path.expanduser("~admin"), '.ssh')
In this example, :py:meth:`monkeypatch.setattr` is used to patch ``Path.home``
so that the known testing path ``Path("/abc")`` is always used when the test is run.
This removes any dependency on the running user for testing purposes.
:py:meth:`monkeypatch.setattr` must be called before the function which will use
the patched function is called.
After the test function finishes the ``Path.home`` modification will be undone.
def test_mytest(monkeypatch):
def mockreturn(path):
return '/abc'
monkeypatch.setattr(os.path, 'expanduser', mockreturn)
.. code-block:: python
# contents of test_module.py with source code and the test
from pathlib import Path
def getssh():
"""Simple function to return expanded homedir ssh path."""
return Path.home() / ".ssh"
def test_getssh(monkeypatch):
# mocked return function to replace Path.home
# always return '/abc'
def mockreturn():
return Path("/abc")
# Application of the monkeypatch to replace Path.home
# with the behavior of mockreturn defined above.
monkeypatch.setattr(Path, "home", mockreturn)
# Calling getssh() will use mockreturn in place of Path.home
# for this test with the monkeypatch.
x = getssh()
assert x == '/abc/.ssh'
assert x == Path("/abc/.ssh")
Monkeypatching returned objects: building mock classes
------------------------------------------------------
:py:meth:`monkeypatch.setattr` can be used in conjunction with classes to mock returned
objects from functions instead of values.
Imagine a simple function to take an API url and return the json response.
.. code-block:: python
# contents of app.py, a simple API retrieval example
import requests
def get_json(url):
"""Takes a URL, and returns the JSON."""
r = requests.get(url)
return r.json()
We need to mock ``r``, the returned response object for testing purposes.
The mock of ``r`` needs a ``.json()`` method which returns a dictionary.
This can be done in our test file by defining a class to represent ``r``.
.. code-block:: python
# contents of test_app.py, a simple test for our API retrieval
# import requests for the purposes of monkeypatching
import requests
# our app.py that includes the get_json() function
# this is the previous code block example
import app
# custom class to be the mock return value
# will override the requests.Response returned from requests.get
class MockResponse:
# mock json() method always returns a specific testing dictionary
@staticmethod
def json():
return {"mock_key": "mock_response"}
def test_get_json(monkeypatch):
# Any arguments may be passed and mock_get() will always return our
# mocked object, which only has the .json() method.
def mock_get(*args, **kwargs):
return MockResponse()
# apply the monkeypatch for requests.get to mock_get
monkeypatch.setattr(requests, "get", mock_get)
# app.get_json, which contains requests.get, uses the monkeypatch
result = app.get_json("https://fakeurl")
assert result["mock_key"] == "mock_response"
``monkeypatch`` applies the mock for ``requests.get`` with our ``mock_get`` function.
The ``mock_get`` function returns an instance of the ``MockResponse`` class, which
has a ``json()`` method defined to return a known testing dictionary and does not
require any outside API connection.
You can build the ``MockResponse`` class with the appropriate degree of complexity for
the scenario you are testing. For instance, it could include an ``ok`` property that
always returns ``True``, or return different values from the ``json()`` mocked method
based on input strings.
This mock can be shared across tests using a ``fixture``:
.. code-block:: python
# contents of test_app.py, a simple test for our API retrieval
import pytest
import requests
# app.py that includes the get_json() function
import app
# custom class to be the mock return value of requests.get()
class MockResponse:
@staticmethod
def json():
return {"mock_key": "mock_response"}
# monkeypatched requests.get moved to a fixture
@pytest.fixture
def mock_response(monkeypatch):
"""Requests.get() mocked to return {'mock_key':'mock_response'}."""
def mock_get(*args, **kwargs):
return MockResponse()
monkeypatch.setattr(requests, "get", mock_get)
# notice our test uses the custom fixture instead of monkeypatch directly
def test_get_json(mock_response):
result = app.get_json("https://fakeurl")
assert result["mock_key"] == "mock_response"
Furthermore, if the mock was designed to be applied to all tests, the ``fixture`` could
be moved to a ``conftest.py`` file and use the with ``autouse=True`` option.
Here our test function monkeypatches ``os.path.expanduser`` and
then calls into a function that calls it. After the test function
finishes the ``os.path.expanduser`` modification will be undone.
Global patch example: preventing "requests" from remote operations
------------------------------------------------------------------
If you want to prevent the "requests" library from performing http
requests in all your tests, you can do::
requests in all your tests, you can do:
# content of conftest.py
.. code-block:: python
# contents of conftest.py
import pytest
@pytest.fixture(autouse=True)
def no_requests(monkeypatch):
"""Remove requests.sessions.Session.request for all tests."""
monkeypatch.delattr("requests.sessions.Session.request")
This autouse fixture will be executed for each test function and it
@@ -85,7 +257,7 @@ Monkeypatching environment variables
------------------------------------
If you are working with environment variables you often need to safely change the values
or delete them from the system for testing purposes. ``Monkeypatch`` provides a mechanism
or delete them from the system for testing purposes. ``monkeypatch`` provides a mechanism
to do this using the ``setenv`` and ``delenv`` method. Our example code to test:
.. code-block:: python
@@ -100,7 +272,7 @@ to do this using the ``setenv`` and ``delenv`` method. Our example code to test:
username = os.getenv("USER")
if username is None:
raise EnvironmentError("USER environment is not set.")
raise OSError("USER environment is not set.")
return username.lower()
@@ -124,13 +296,14 @@ both paths can be safely tested without impacting the running environment:
"""Remove the USER env var and assert EnvironmentError is raised."""
monkeypatch.delenv("USER", raising=False)
with pytest.raises(EnvironmentError):
with pytest.raises(OSError):
_ = get_os_user_lower()
This behavior can be moved into ``fixture`` structures and shared across tests:
.. code-block:: python
# contents of our test file e.g. test_code.py
import pytest
@@ -144,16 +317,122 @@ This behavior can be moved into ``fixture`` structures and shared across tests:
monkeypatch.delenv("USER", raising=False)
# Notice the tests reference the fixtures for mocks
# notice the tests reference the fixtures for mocks
def test_upper_to_lower(mock_env_user):
assert get_os_user_lower() == "testinguser"
def test_raise_exception(mock_env_missing):
with pytest.raises(EnvironmentError):
with pytest.raises(OSError):
_ = get_os_user_lower()
Monkeypatching dictionaries
---------------------------
:py:meth:`monkeypatch.setitem` can be used to safely set the values of dictionaries
to specific values during tests. Take this simplified connection string example:
.. code-block:: python
# contents of app.py to generate a simple connection string
DEFAULT_CONFIG = {"user": "user1", "database": "db1"}
def create_connection_string(config=None):
"""Creates a connection string from input or defaults."""
config = config or DEFAULT_CONFIG
return f"User Id={config['user']}; Location={config['database']};"
For testing purposes we can patch the ``DEFAULT_CONFIG`` dictionary to specific values.
.. code-block:: python
# contents of test_app.py
# app.py with the connection string function (prior code block)
import app
def test_connection(monkeypatch):
# Patch the values of DEFAULT_CONFIG to specific
# testing values only for this test.
monkeypatch.setitem(app.DEFAULT_CONFIG, "user", "test_user")
monkeypatch.setitem(app.DEFAULT_CONFIG, "database", "test_db")
# expected result based on the mocks
expected = "User Id=test_user; Location=test_db;"
# the test uses the monkeypatched dictionary settings
result = app.create_connection_string()
assert result == expected
You can use the :py:meth:`monkeypatch.delitem` to remove values.
.. code-block:: python
# contents of test_app.py
import pytest
# app.py with the connection string function
import app
def test_missing_user(monkeypatch):
# patch the DEFAULT_CONFIG t be missing the 'user' key
monkeypatch.delitem(app.DEFAULT_CONFIG, "user", raising=False)
# Key error expected because a config is not passed, and the
# default is now missing the 'user' entry.
with pytest.raises(KeyError):
_ = app.create_connection_string()
The modularity of fixtures gives you the flexibility to define
separate fixtures for each potential mock and reference them in the needed tests.
.. code-block:: python
# contents of test_app.py
import pytest
# app.py with the connection string function
import app
# all of the mocks are moved into separated fixtures
@pytest.fixture
def mock_test_user(monkeypatch):
"""Set the DEFAULT_CONFIG user to test_user."""
monkeypatch.setitem(app.DEFAULT_CONFIG, "user", "test_user")
@pytest.fixture
def mock_test_database(monkeypatch):
"""Set the DEFAULT_CONFIG database to test_db."""
monkeypatch.setitem(app.DEFAULT_CONFIG, "database", "test_db")
@pytest.fixture
def mock_missing_default_user(monkeypatch):
"""Remove the user key from DEFAULT_CONFIG"""
monkeypatch.delitem(app.DEFAULT_CONFIG, "user", raising=False)
# tests reference only the fixture mocks that are needed
def test_connection(mock_test_user, mock_test_database):
expected = "User Id=test_user; Location=test_db;"
result = app.create_connection_string()
assert result == expected
def test_missing_user(mock_missing_default_user):
with pytest.raises(KeyError):
_ = app.create_connection_string()
.. currentmodule:: _pytest.monkeypatch

View File

@@ -46,7 +46,7 @@ Unsupported idioms / known issues
<https://github.com/pytest-dev/pytest/issues/377/>`_.
- nose imports test modules with the same import path (e.g.
``tests.test_mod``) but different file system paths
``tests.test_mode``) but different file system paths
(e.g. ``tests/test_mode.py`` and ``other/tests/test_mode.py``)
by extending sys.path/import semantics. pytest does not do that
but there is discussion in `#268 <https://github.com/pytest-dev/pytest/issues/268>`_ for adding some support. Note that

Some files were not shown because too many files have changed in this diff Show More