floraachy/pytest2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

#1642 Resolve conflicts

Browse Source
This commit is contained in:
feuillemorte
2018-02-01 00:18:28 +03:00
parent 741b571f3b f2fb841b29
commit 9f1772e679
79 changed files with 1476 additions and 600 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
doc/en/announce/index.rst
View File

@@ -6,6 +6,7 @@ Release announcements
:maxdepth: 2
release-3.4.0
release-3.3.2
release-3.3.1
release-3.3.0

52
doc/en/announce/release-3.4.0.rst Normal file
View File

@@ -0,0 +1,52 @@
pytest-3.4.0
=======================================
The pytest team is proud to announce the 3.4.0 release!
pytest is a mature Python testing tool with more than a 1600 tests
against itself, passing on many different interpreters and platforms.
This release contains a number of bugs fixes and improvements, so users are encouraged
to take a look at the CHANGELOG:
http://doc.pytest.org/en/latest/changelog.html
For complete documentation, please visit:
http://docs.pytest.org
As usual, you can upgrade from pypi via:
pip install -U pytest
Thanks to all who contributed to this release, among them:
* Aaron
* Alan Velasco
* Anders Hovmöller
* Andrew Toolan
* Anthony Sottile
* Aron Coyle
* Brian Maissy
* Bruno Oliveira
* Cyrus Maden
* Florian Bruhin
* Henk-Jaap Wagenaar
* Ian Lesperance
* Jon Dufresne
* Jurko Gospodnetić
* Kate
* Kimberly
* Per A. Brodtkorb
* Pierre-Alexandre Fonta
* Raphael Castaneda
* Ronny Pfannschmidt
* ST John
* Segev Finer
* Thomas Hisch
* Tzu-ping Chung
* feuillemorte
Happy testing,
The Pytest Development Team

89
doc/en/backwards-compatibility.rst
View File

@@ -15,91 +15,6 @@ We will only remove deprecated functionality in major releases (e.g. if we depre
Deprecation Roadmap
-------------------
This page lists deprecated features and when we plan to remove them. It is important to list the feature, the version where it got deprecated and the version we plan to remove it.
We track deprecation and removal of features using milestones and the `deprecation <https://github.com/pytest-dev/pytest/issues?q=label%3A%22type%3A+deprecation%22>`_ and `removal <https://github.com/pytest-dev/pytest/labels/type%3A%20removal>`_ labels on GitHub.
Following our deprecation policy, we should aim to keep features for *at least* two minor versions after it was considered deprecated.
Future Releases
~~~~~~~~~~~~~~~
3.4
^^^
**Old style classes**
Issue: `#2147 <https://github.com/pytest-dev/pytest/issues/2147>`_.
Deprecated in ``3.2``.
4.0
^^^
**Yield tests**
Deprecated in ``3.0``.
**pytest-namespace hook**
deprecated in ``3.2``.
**Marks in parameter sets**
Deprecated in ``3.2``.
**--result-log**
Deprecated in ``3.0``.
See `#830 <https://github.com/pytest-dev/pytest/issues/830>`_ for more information. Suggested alternative: `pytest-tap <https://pypi.python.org/pypi/pytest-tap>`_.
**metafunc.addcall**
Issue: `#2876 <https://github.com/pytest-dev/pytest/issues/2876>`_.
Deprecated in ``3.3``.
**pytest_plugins in non-toplevel conftests**
There is a deep conceptual confusion as ``conftest.py`` files themselves are activated/deactivated based on path, but the plugins they depend on aren't.
Issue: `#2639 <https://github.com/pytest-dev/pytest/issues/2639>`_.
Not yet officially deprecated.
**passing a single string to pytest.main()**
Pass a list of strings to ``pytest.main()`` instead.
Deprecated in ``3.1``.
**[pytest] section in setup.cfg**
Use ``[tool:pytest]`` instead for compatibility with other tools.
Deprecated in ``3.0``.
Past Releases
~~~~~~~~~~~~~
3.0
^^^
* The following deprecated commandline options were removed:
* ``--genscript``: no longer supported;
* ``--no-assert``: use ``--assert=plain`` instead;
* ``--nomagic``: use ``--assert=plain`` instead;
* ``--report``: use ``-r`` instead;
* Removed all ``py.test-X*`` entry points. The versioned, suffixed entry points
were never documented and a leftover from a pre-virtualenv era. These entry
points also created broken entry points in wheels, so removing them also
removes a source of confusion for users.
3.3
^^^
* Dropped support for EOL Python 2.6 and 3.3.
Following our deprecation policy, after starting issuing deprecation warnings we keep features for *at least* two minor versions before considering removal.

4
doc/en/builtin.rst
View File

@@ -116,6 +116,10 @@ You can ask for available builtin or project-custom
Add extra xml properties to the tag for the calling test.
The fixture is callable with ``(name, value)``, with value being automatically
xml-encoded.
record_xml_attribute
Add extra xml attributes to the tag for the calling test.
The fixture is callable with ``(name, value)``, with value being automatically
xml-encoded
caplog
Access and control log capturing.

2
doc/en/cache.rst
View File

@@ -225,7 +225,7 @@ You can always peek at the content of the cache using the
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
cachedir: $REGENDOC_TMPDIR/.cache
cachedir: $REGENDOC_TMPDIR/.pytest_cache
------------------------------- cache values -------------------------------
cache/lastfailed contains:
{'test_caching.py::test_function': True}

42
doc/en/customize.rst
View File

@@ -152,11 +152,25 @@ above will show verbose output because ``-v`` overwrites ``-q``.
Builtin configuration file options
----------------------------------------------
Here is a list of builtin configuration options that may be written in a ``pytest.ini``, ``tox.ini`` or ``setup.cfg``
file, usually located at the root of your repository. All options must be under a ``[pytest]`` section
(``[tool:pytest]`` for ``setup.cfg`` files).
Configuration file options may be overwritten in the command-line by using ``-o/--override``, which can also be
passed multiple times. The expected format is ``name=value``. For example::
pytest -o console_output_style=classic -o cache_dir=/tmp/mycache
.. confval:: minversion
Specifies a minimal pytest version required for running tests.
minversion = 2.1 # will fail if we run with pytest-2.0
.. code-block:: ini
# content of pytest.ini
[pytest]
minversion = 3.0 # will fail if we run with pytest-2.8
.. confval:: addopts
@@ -165,6 +179,7 @@ Builtin configuration file options
.. code-block:: ini
# content of pytest.ini
[pytest]
addopts = --maxfail=2 -rf # exit after 2 failures, report fail info
@@ -332,6 +347,31 @@ Builtin configuration file options
[pytest]
console_output_style = classic
.. confval:: empty_parameter_set_mark
.. versionadded:: 3.4
Allows to pick the action for empty parametersets in parameterization
* ``skip`` skips tests with a empty parameterset (default)
* ``xfail`` marks tests with a empty parameterset as xfail(run=False)
.. code-block:: ini
# content of pytest.ini
[pytest]
empty_parameter_set_mark = xfail
.. note::
The default value of this option is planned to change to ``xfail`` in future releases
as this is considered less error prone, see `#3155`_ for more details.
.. _`#3155`: https://github.com/pytest-dev/pytest/issues/3155
.. confval:: rootdir
Sets a :ref:`rootdir <rootdir>` directory. Directory may be relative or absolute path.

6
doc/en/example/assertion/failure_demo.py
View File

@@ -157,12 +157,14 @@ class TestRaises(object):
# thanks to Matthew Scott for this test
def test_dynamic_compile_shows_nicely():
import imp
import sys
src = 'def foo():\n assert 1 == 0\n'
name = 'abc-123'
module = py.std.imp.new_module(name)
module = imp.new_module(name)
code = _pytest._code.compile(src, name, 'exec')
py.builtin.exec_(code, module.__dict__)
py.std.sys.modules[name] = module
sys.modules[name] = module
module.foo()

20
doc/en/example/markers.rst
View File

@@ -32,7 +32,7 @@ You can then restrict a test run to only run tests marked with ``webtest``::
$ pytest -v -m webtest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
@@ -46,7 +46,7 @@ Or the inverse, running all tests except the webtest ones::
$ pytest -v -m "not webtest"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
@@ -67,7 +67,7 @@ tests based on their module, class, method, or function name::
$ pytest -v test_server.py::TestClass::test_method
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 1 item
@@ -80,7 +80,7 @@ You can also select on the class::
$ pytest -v test_server.py::TestClass
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 1 item
@@ -93,7 +93,7 @@ Or select multiple nodes::
$ pytest -v test_server.py::TestClass test_server.py::test_send_http
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 2 items
@@ -131,7 +131,7 @@ select tests based on their names::
$ pytest -v -k http # running with the above defined example module
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
@@ -145,7 +145,7 @@ And you can also run all tests except the ones that match the keyword::
$ pytest -k "not send_http" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
@@ -161,7 +161,7 @@ Or to select "http" and "quick" tests::
$ pytest -k "http or quick" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
@@ -432,7 +432,7 @@ The output is as follows::
$ pytest -q -s
Marker info name=my_marker args=(<function hello_world at 0xdeadbeef>,) kwars={}
. [100%]
.
1 passed in 0.12 seconds
We can see that the custom marker has its argument set extended with the function ``hello_world``. This is the key difference between creating a custom marker as a callable, which invokes ``__call__`` behind the scenes, and using ``with_args``.
@@ -477,7 +477,7 @@ Let's run this without capturing output and see what we get::
glob args=('function',) kwargs={'x': 3}
glob args=('class',) kwargs={'x': 2}
glob args=('module',) kwargs={'x': 1}
. [100%]
.
1 passed in 0.12 seconds
marking platform specific tests with pytest

2
doc/en/example/nonpython.rst
View File

@@ -60,7 +60,7 @@ consulted when reporting in ``verbose`` mode::
nonpython $ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
collecting ... collected 2 items

34
doc/en/example/reportingdemo.rst
View File

@@ -411,22 +411,24 @@ get on the terminal - we are working on that)::
____________________ test_dynamic_compile_shows_nicely _____________________
def test_dynamic_compile_shows_nicely():
import imp
import sys
src = 'def foo():\n assert 1 == 0\n'
name = 'abc-123'
module = py.std.imp.new_module(name)
module = imp.new_module(name)
code = _pytest._code.compile(src, name, 'exec')
py.builtin.exec_(code, module.__dict__)
py.std.sys.modules[name] = module
sys.modules[name] = module
> module.foo()
failure_demo.py:166:
failure_demo.py:168:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def foo():
> assert 1 == 0
E AssertionError
<2-codegen 'abc-123' $REGENDOC_TMPDIR/assertion/failure_demo.py:163>:2: AssertionError
<2-codegen 'abc-123' $REGENDOC_TMPDIR/assertion/failure_demo.py:165>:2: AssertionError
____________________ TestMoreErrors.test_complex_error _____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -438,7 +440,7 @@ get on the terminal - we are working on that)::
return 43
> somefunc(f(), g())
failure_demo.py:176:
failure_demo.py:178:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
failure_demo.py:9: in somefunc
otherfunc(x,y)
@@ -460,7 +462,7 @@ get on the terminal - we are working on that)::
> a,b = l
E ValueError: not enough values to unpack (expected 2, got 0)
failure_demo.py:180: ValueError
failure_demo.py:182: ValueError
____________________ TestMoreErrors.test_z2_type_error _____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -470,7 +472,7 @@ get on the terminal - we are working on that)::
> a,b = l
E TypeError: 'int' object is not iterable
failure_demo.py:184: TypeError
failure_demo.py:186: TypeError
______________________ TestMoreErrors.test_startswith ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -483,7 +485,7 @@ get on the terminal - we are working on that)::
E + where False = <built-in method startswith of str object at 0xdeadbeef>('456')
E + where <built-in method startswith of str object at 0xdeadbeef> = '123'.startswith
failure_demo.py:189: AssertionError
failure_demo.py:191: AssertionError
__________________ TestMoreErrors.test_startswith_nested ___________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -500,7 +502,7 @@ get on the terminal - we are working on that)::
E + where '123' = <function TestMoreErrors.test_startswith_nested.<locals>.f at 0xdeadbeef>()
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef>()
failure_demo.py:196: AssertionError
failure_demo.py:198: AssertionError
_____________________ TestMoreErrors.test_global_func ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -511,7 +513,7 @@ get on the terminal - we are working on that)::
E + where False = isinstance(43, float)
E + where 43 = globf(42)
failure_demo.py:199: AssertionError
failure_demo.py:201: AssertionError
_______________________ TestMoreErrors.test_instance _______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -522,7 +524,7 @@ get on the terminal - we are working on that)::
E assert 42 != 42
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef>.x
failure_demo.py:203: AssertionError
failure_demo.py:205: AssertionError
_______________________ TestMoreErrors.test_compare ________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -532,7 +534,7 @@ get on the terminal - we are working on that)::
E assert 11 < 5
E + where 11 = globf(10)
failure_demo.py:206: AssertionError
failure_demo.py:208: AssertionError
_____________________ TestMoreErrors.test_try_finally ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -543,7 +545,7 @@ get on the terminal - we are working on that)::
> assert x == 0
E assert 1 == 0
failure_demo.py:211: AssertionError
failure_demo.py:213: AssertionError
___________________ TestCustomAssertMsg.test_single_line ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@@ -557,7 +559,7 @@ get on the terminal - we are working on that)::
E assert 1 == 2
E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_single_line.<locals>.A'>.a
failure_demo.py:222: AssertionError
failure_demo.py:224: AssertionError
____________________ TestCustomAssertMsg.test_multiline ____________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@@ -574,7 +576,7 @@ get on the terminal - we are working on that)::
E assert 1 == 2
E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_multiline.<locals>.A'>.a
failure_demo.py:228: AssertionError
failure_demo.py:230: AssertionError
___________________ TestCustomAssertMsg.test_custom_repr ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@@ -594,7 +596,7 @@ get on the terminal - we are working on that)::
E assert 1 == 2
E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a
failure_demo.py:238: AssertionError
failure_demo.py:240: AssertionError
============================= warnings summary =============================
None
Metafunc.addcall is deprecated and scheduled to be removed in pytest 4.0.

20
doc/en/example/simple.rst
View File

@@ -332,7 +332,7 @@ which will add info only when run with "--v"::
$ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
info1: did you know that ...
did you?
rootdir: $REGENDOC_TMPDIR, inifile:
@@ -385,9 +385,9 @@ Now we can profile which test functions execute the slowest::
test_some_are_slow.py ... [100%]
========================= slowest 3 test durations =========================
0.31s call test_some_are_slow.py::test_funcslow2
0.20s call test_some_are_slow.py::test_funcslow1
0.17s call test_some_are_slow.py::test_funcfast
0.58s call test_some_are_slow.py::test_funcslow2
0.41s call test_some_are_slow.py::test_funcslow1
0.10s call test_some_are_slow.py::test_funcfast
========================= 3 passed in 0.12 seconds =========================
incremental testing - test steps
@@ -537,7 +537,7 @@ We can run this::
file $REGENDOC_TMPDIR/b/test_error.py, line 1
def test_root(db): # no db here, will error out
E fixture 'db' not found
> available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_xml_property, recwarn, tmpdir, tmpdir_factory
> available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_xml_attribute, record_xml_property, recwarn, tmpdir, tmpdir_factory
> use 'pytest --fixtures [testpath]' for help on them.
$REGENDOC_TMPDIR/b/test_error.py:1
@@ -731,7 +731,7 @@ and run it::
test_module.py Esetting up a test failed! test_module.py::test_setup_fails
Fexecuting test failed test_module.py::test_call_fails
F [100%]
F
================================== ERRORS ==================================
____________________ ERROR at setup of test_setup_fails ____________________
@@ -826,15 +826,20 @@ Instead of freezing the pytest runner as a separate executable, you can make
your frozen program work as the pytest runner by some clever
argument handling during program startup. This allows you to
have a single executable, which is usually more convenient.
Please note that the mechanism for plugin discovery used by pytest
(setupttools entry points) doesn't work with frozen executables so pytest
can't find any third party plugins automatically. To include third party plugins
like ``pytest-timeout`` they must be imported explicitly and passed on to pytest.main.
.. code-block:: python
# contents of app_main.py
import sys
import pytest_timeout # Third party plugin
if len(sys.argv) > 1 and sys.argv[1] == '--pytest':
import pytest
sys.exit(pytest.main(sys.argv[2:]))
sys.exit(pytest.main(sys.argv[2:], plugins=[pytest_timeout]))
else:
# normal application execution: at this point argv can be parsed
# by your argument-parsing library of choice as usual
@@ -845,3 +850,4 @@ This allows you to execute tests using the frozen
application with standard ``pytest`` command-line options::
./app_main --pytest --verbose --tb=long --junitxml=results.xml test-suite/

2
doc/en/example/special.rst
View File

@@ -68,5 +68,5 @@ If you run this without output capturing::
.test_method1 called
.test other
.test_unit1 method called
. [100%]
.
4 passed in 0.12 seconds

24
doc/en/fixture.rst
View File

@@ -286,7 +286,7 @@ tests.
Let's execute it::
$ pytest -s -q --tb=no
FF [100%]teardown smtp
FFteardown smtp
2 failed in 0.12 seconds
@@ -391,7 +391,7 @@ We use the ``request.module`` attribute to optionally obtain an
again, nothing much has changed::
$ pytest -s -q --tb=no
FF [100%]finalizing <smtplib.SMTP object at 0xdeadbeef> (smtp.gmail.com)
FFfinalizing <smtplib.SMTP object at 0xdeadbeef> (smtp.gmail.com)
2 failed in 0.12 seconds
@@ -612,7 +612,7 @@ Here we declare an ``app`` fixture which receives the previously defined
$ pytest -v test_appsetup.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 2 items
@@ -681,40 +681,40 @@ Let's run the tests in verbose mode and with looking at the print-output::
$ pytest -v -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
cachedir: .pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 8 items
test_module.py::test_0[1] SETUP otherarg 1
RUN test0 with otherarg 1
PASSED [ 12%] TEARDOWN otherarg 1
PASSED TEARDOWN otherarg 1
test_module.py::test_0[2] SETUP otherarg 2
RUN test0 with otherarg 2
PASSED [ 25%] TEARDOWN otherarg 2
PASSED TEARDOWN otherarg 2
test_module.py::test_1[mod1] SETUP modarg mod1
RUN test1 with modarg mod1
PASSED [ 37%]
PASSED
test_module.py::test_2[1-mod1] SETUP otherarg 1
RUN test2 with otherarg 1 and modarg mod1
PASSED [ 50%] TEARDOWN otherarg 1
PASSED TEARDOWN otherarg 1
test_module.py::test_2[2-mod1] SETUP otherarg 2
RUN test2 with otherarg 2 and modarg mod1
PASSED [ 62%] TEARDOWN otherarg 2
PASSED TEARDOWN otherarg 2
test_module.py::test_1[mod2] TEARDOWN modarg mod1
SETUP modarg mod2
RUN test1 with modarg mod2
PASSED [ 75%]
PASSED
test_module.py::test_2[1-mod2] SETUP otherarg 1
RUN test2 with otherarg 1 and modarg mod2
PASSED [ 87%] TEARDOWN otherarg 1
PASSED TEARDOWN otherarg 1
test_module.py::test_2[2-mod2] SETUP otherarg 2
RUN test2 with otherarg 2 and modarg mod2
PASSED [100%] TEARDOWN otherarg 2
PASSED TEARDOWN otherarg 2
TEARDOWN modarg mod2

86
doc/en/getting-started.rst
View File

@@ -7,32 +7,34 @@ Installation and Getting Started
**PyPI package name**: `pytest <http://pypi.python.org/pypi/pytest>`_
**dependencies**: `py <http://pypi.python.org/pypi/py>`_,
**Dependencies**: `py <http://pypi.python.org/pypi/py>`_,
`colorama (Windows) <http://pypi.python.org/pypi/colorama>`_,
**documentation as PDF**: `download latest <https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf>`_
**Documentation as PDF**: `download latest <https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf>`_
``pytest`` is a framework that makes building simple and scalable tests easy. Tests are expressive and readable—no boilerplate code required. Get started in minutes with a small unit test or complex functional test for your application or library.
.. _`getstarted`:
.. _installation:
.. _`installation`:
Installation
Install ``pytest``
----------------------------------------
Installation::
1. Run the following command in your command line::
pip install -U pytest
To check your installation has installed the correct version::
2. Check that you installed the correct version::
$ pytest --version
This is pytest version 3.x.y, imported from $PYTHON_PREFIX/lib/python3.5/site-packages/pytest.py
.. _`simpletest`:
Our first test run
Create your first test
----------------------------------------------------------
Let's create a first test file with a simple test function::
Create a simple test function with just four lines of code::
# content of test_sample.py
def func(x):
@@ -41,7 +43,7 @@ Let's create a first test file with a simple test function::
def test_answer():
assert func(3) == 5
That's it. You can execute the test function now::
That’s it. You can now execute the test function::
$ pytest
=========================== test session starts ============================
@@ -62,30 +64,22 @@ That's it. You can execute the test function now::
test_sample.py:5: AssertionError
========================= 1 failed in 0.12 seconds =========================
We got a failure report because our little ``func(3)`` call did not return ``5``.
This test returns a failure report because ``func(3)`` does not return ``5``.
.. note::
You can simply use the ``assert`` statement for asserting test
expectations. pytest's :ref:`assert introspection` will intelligently
report intermediate values of the assert expression freeing
you from the need to learn the many names of `JUnit legacy methods`_.
You can use the ``assert`` statement to verify test expectations. pytest’s `Advanced assertion introspection <http://docs.python.org/reference/simple_stmts.html#the-assert-statement>`_ will intelligently report intermediate values of the assert expression so you can avoid the many names `of JUnit legacy methods <http://docs.python.org/library/unittest.html#test-cases>`_.
.. _`JUnit legacy methods`: http://docs.python.org/library/unittest.html#test-cases
.. _`assert statement`: http://docs.python.org/reference/simple_stmts.html#the-assert-statement
Running multiple tests
Run multiple tests
----------------------------------------------------------
``pytest`` will run all files in the current directory and its subdirectories of the form test_*.py or \*_test.py. More generally, it follows :ref:`standard test discovery rules <test discovery>`.
``pytest`` will run all files of the form test_*.py or \*_test.py in the current directory and its subdirectories. More generally, it follows :ref:`standard test discovery rules <test discovery>`.
Asserting that a certain exception is raised
Assert that a certain exception is raised
--------------------------------------------------------------
If you want to assert that some code raises an exception you can
use the ``raises`` helper::
Use the ``raises`` helper to assert that some code raises an exception::
# content of test_sysexit.py
import pytest
@@ -96,18 +90,16 @@ use the ``raises`` helper::
with pytest.raises(SystemExit):
f()
Running it with, this time in "quiet" reporting mode::
Execute the test function with “quiet” reporting mode::
$ pytest -q test_sysexit.py
. [100%]
1 passed in 0.12 seconds
Grouping multiple tests in a class
Group multiple tests in a class
--------------------------------------------------------------
Once you start to have more than a few tests it often makes sense
to group tests logically, in classes and modules. Let's write a class
containing two tests::
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test::
# content of test_class.py
class TestClass(object):
@@ -119,9 +111,7 @@ containing two tests::
x = "hello"
assert hasattr(x, 'check')
The two tests are found because of the standard :ref:`test discovery`.
There is no need to subclass anything. We can simply
run the module by passing its filename::
``pytest`` discovers all tests following its :ref:`Conventions for Python test discovery <test discovery>`, so it finds both ``test_`` prefixed functions. There is no need to subclass anything. We can simply run the module by passing its filename::
$ pytest -q test_class.py
.F [100%]
@@ -139,26 +129,19 @@ run the module by passing its filename::
test_class.py:8: AssertionError
1 failed, 1 passed in 0.12 seconds
The first test passed, the second failed. Again we can easily see
the intermediate values used in the assertion, helping us to
understand the reason for the failure.
The first test passed and the second failed. You can easily see the intermediate values in the assertion to help you understand the reason for the failure.
Going functional: requesting a unique temporary directory
Request a unique temporary directory for functional tests
--------------------------------------------------------------
For functional tests one often needs to create some files
and pass them to application objects. pytest provides
:ref:`builtinfixtures` which allow to request arbitrary
resources, for example a unique temporary directory::
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html#builtinfixtures>`_ to request arbitrary resources, like a unique temporary directory::
# content of test_tmpdir.py
def test_needsfiles(tmpdir):
print (tmpdir)
assert 0
We list the name ``tmpdir`` in the test function signature and
``pytest`` will lookup and call a fixture factory to create the resource
before performing the test function call. Let's just run it::
List the name ``tmpdir`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory::
$ pytest -q test_tmpdir.py
F [100%]
@@ -177,22 +160,21 @@ before performing the test function call. Let's just run it::
PYTEST_TMPDIR/test_needsfiles0
1 failed in 0.12 seconds
Before the test runs, a unique-per-test-invocation temporary directory
was created. More info at :ref:`tmpdir handling`.
More info on tmpdir handling is available at `Temporary directories and files <tmpdir handling>`_.
You can find out what kind of builtin :ref:`fixtures` exist by typing::
Find out what kind of builtin ```pytest`` fixtures <fixtures>`_ exist with the command::
pytest --fixtures # shows builtin and custom fixtures
Where to go next
Continue reading
-------------------------------------
Here are a few suggestions where to go next:
Check out additional pytest resources to help you customize tests for your unique workflow:
* :ref:`cmdline` for command line invocation examples
* :ref:`good practices <goodpractices>` for virtualenv, test layout
* :ref:`existingtestsuite` for working with pre-existing tests
* :ref:`fixtures` for providing a functional baseline to your tests
* :ref:`plugins` managing and writing plugins
* ":ref:`cmdline`" for command line invocation examples
* ":ref:`goodpractices`" for virtualenv and test layouts
* ":ref:`existingtestsuite`" for working with pre-existing tests
* ":ref:`fixtures`" for providing a functional baseline to your tests
* ":ref:`plugins`" for managing and writing plugins
.. include:: links.inc

131
doc/en/logging.rst
View File

@@ -3,24 +3,11 @@
Logging
-------
.. versionadded 3.3.0
.. versionadded:: 3.3
.. versionchanged:: 3.4
.. note::
This feature is a drop-in replacement for the `pytest-catchlog
<https://pypi.org/project/pytest-catchlog/>`_ plugin and they will conflict
with each other. The backward compatibility API with ``pytest-capturelog``
has been dropped when this feature was introduced, so if for that reason you
still need ``pytest-catchlog`` you can disable the internal feature by
adding to your ``pytest.ini``:
.. code-block:: ini
[pytest]
addopts=-p no:logging
Log messages are captured by default and for each failed test will be shown in
the same manner as captured stdout and stderr.
pytest captures log messages of level ``WARNING`` or above automatically and displays them in their own section
for each failed test in the same manner as captured stdout and stderr.
Running without options::
@@ -29,7 +16,7 @@ Running without options::
Shows failed tests like so::
----------------------- Captured stdlog call ----------------------
test_reporting.py 26 INFO text going to logger
test_reporting.py 26 WARNING text going to logger
----------------------- Captured stdout call ----------------------
text going to stdout
----------------------- Captured stderr call ----------------------
@@ -37,11 +24,10 @@ Shows failed tests like so::
==================== 2 failed in 0.02 seconds =====================
By default each captured log message shows the module, line number, log level
and message. Showing the exact module and line number is useful for testing and
debugging. If desired the log format and date format can be specified to
anything that the logging module supports.
and message.
Running pytest specifying formatting options::
If desired the log and date format can be specified to
anything that the logging module supports by passing specific formatting options::
pytest --log-format="%(asctime)s %(levelname)s %(message)s" \
--log-date-format="%Y-%m-%d %H:%M:%S"
@@ -49,14 +35,14 @@ Running pytest specifying formatting options::
Shows failed tests like so::
----------------------- Captured stdlog call ----------------------
2010-04-10 14:48:44 INFO text going to logger
2010-04-10 14:48:44 WARNING text going to logger
----------------------- Captured stdout call ----------------------
text going to stdout
----------------------- Captured stderr call ----------------------
text going to stderr
==================== 2 failed in 0.02 seconds =====================
These options can also be customized through a configuration file:
These options can also be customized through ``pytest.ini`` file:
.. code-block:: ini
@@ -69,7 +55,7 @@ with::
pytest --no-print-logs
Or in you ``pytest.ini``:
Or in the ``pytest.ini`` file:
.. code-block:: ini
@@ -85,6 +71,10 @@ Shows failed tests in the normal manner as no logs were captured::
text going to stderr
==================== 2 failed in 0.02 seconds =====================
caplog fixture
^^^^^^^^^^^^^^
Inside tests it is possible to change the log level for the captured log
messages. This is supported by the ``caplog`` fixture::
@@ -92,7 +82,7 @@ messages. This is supported by the ``caplog`` fixture::
caplog.set_level(logging.INFO)
pass
By default the level is set on the handler used to catch the log messages,
By default the level is set on the root logger,
however as a convenience it is also possible to set the log level of any
logger::
@@ -100,14 +90,16 @@ logger::
caplog.set_level(logging.CRITICAL, logger='root.baz')
pass
The log levels set are restored automatically at the end of the test.
It is also possible to use a context manager to temporarily change the log
level::
level inside a ``with`` block::
def test_bar(caplog):
with caplog.at_level(logging.INFO):
pass
Again, by default the level of the handler is affected but the level of any
Again, by default the level of the root logger is affected but the level of any
logger can be changed instead with::
def test_bar(caplog):
@@ -115,7 +107,7 @@ logger can be changed instead with::
pass
Lastly all the logs sent to the logger during the test run are made available on
the fixture in the form of both the LogRecord instances and the final log text.
the fixture in the form of both the ``logging.LogRecord`` instances and the final log text.
This is useful for when you want to assert on the contents of a message::
def test_baz(caplog):
@@ -146,12 +138,41 @@ You can call ``caplog.clear()`` to reset the captured log records in a test::
your_test_method()
assert ['Foo'] == [rec.message for rec in caplog.records]
The ``caplop.records`` attribute contains records from the current stage only, so
inside the ``setup`` phase it contains only setup logs, same with the ``call`` and
``teardown`` phases.
To access logs from other stages, use the ``caplog.get_records(when)`` method. As an example,
if you want to make sure that tests which use a certain fixture never log any warnings, you can inspect
the records for the ``setup`` and ``call`` stages during teardown like so:
.. code-block:: python
@pytest.fixture
def window(caplog):
window = create_window()
yield window
for when in ('setup', 'call'):
messages = [x.message for x in caplog.get_records(when) if x.level == logging.WARNING]
if messages:
pytest.fail('warning messages encountered during testing: {}'.format(messages))
caplog fixture API
~~~~~~~~~~~~~~~~~~
.. autoclass:: _pytest.logging.LogCaptureFixture
:members:
.. _live_logs:
Live Logs
^^^^^^^^^
By default, pytest will output any logging records with a level higher or
equal to WARNING. In order to actually see these logs in the console you have to
disable pytest output capture by passing ``-s``.
By setting the :confval:`log_cli` configuration option to ``true``, pytest will output
logging records as they are emitted directly into the console.
You can specify the logging level for which log records with equal or higher
level are printed to the console by passing ``--log-cli-level``. This setting
@@ -190,3 +211,49 @@ option names are:
* ``log_file_level``
* ``log_file_format``
* ``log_file_date_format``
.. _log_release_notes:
Release notes
^^^^^^^^^^^^^
This feature was introduced as a drop-in replacement for the `pytest-catchlog
<https://pypi.org/project/pytest-catchlog/>`_ plugin and they conflict
with each other. The backward compatibility API with ``pytest-capturelog``
has been dropped when this feature was introduced, so if for that reason you
still need ``pytest-catchlog`` you can disable the internal feature by
adding to your ``pytest.ini``:
.. code-block:: ini
[pytest]
addopts=-p no:logging
.. _log_changes_3_4:
Incompatible changes in pytest 3.4
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This feature was introduced in ``3.3`` and some **incompatible changes** have been
made in ``3.4`` after community feedback:
* Log levels are no longer changed unless explicitly requested by the :confval:`log_level` configuration
or ``--log-level`` command-line options. This allows users to configure logger objects themselves.
* :ref:`Live Logs <live_logs>` is now disabled by default and can be enabled setting the
:confval:`log_cli` configuration option to ``true``. When enabled, the verbosity is increased so logging for each
test is visible.
* :ref:`Live Logs <live_logs>` are now sent to ``sys.stdout`` and no longer require the ``-s`` command-line option
to work.
If you want to partially restore the logging behavior of version ``3.3``, you can add this options to your ``ini``
file:
.. code-block:: ini
[pytest]
log_cli=true
log_level=NOTSET
More details about the discussion that lead to this changes can be read in
issue `#3013 <https://github.com/pytest-dev/pytest/issues/3013>`_.

2
doc/en/parametrize.rst
View File

@@ -123,7 +123,7 @@ To get all combinations of multiple parametrized arguments you can stack
def test_foo(x, y):
pass
This will run the test with the arguments set to ``x=0/y=2``,``x=1/y=2``,
This will run the test with the arguments set to ``x=0/y=2``, ``x=1/y=2``,
``x=0/y=3``, and ``x=1/y=3`` exhausting parameters in the order of the decorators.
.. _`pytest_generate_tests`:

70
doc/en/usage.rst
View File

@@ -256,6 +256,66 @@ This will add an extra property ``example_key="1"`` to the generated
Also please note that using this feature will break any schema verification.
This might be a problem when used with some CI servers.
record_xml_attribute
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. versionadded:: 3.4
To add an additional xml attribute to a testcase element, you can use
``record_xml_attribute`` fixture. This can also be used to override existing values:
.. code-block:: python
def test_function(record_xml_attribute):
record_xml_attribute("assertions", "REQ-1234")
record_xml_attribute("classname", "custom_classname")
print('hello world')
assert True
Unlike ``record_xml_property``, this will not add a new child element.
Instead, this will add an attribute ``assertions="REQ-1234"`` inside the generated
``testcase`` tag and override the default ``classname`` with ``"classname=custom_classname"``:
.. code-block:: xml
<testcase classname="custom_classname" file="test_function.py" line="0" name="test_function" time="0.003" assertions="REQ-1234">
<system-out>
hello world
</system-out>
</testcase>
.. warning::
``record_xml_attribute`` is an experimental feature, and its interface might be replaced
by something more powerful and general in future versions. The
functionality per-se will be kept, however.
Using this over ``record_xml_property`` can help when using ci tools to parse the xml report.
However, some parsers are quite strict about the elements and attributes that are allowed.
Many tools use an xsd schema (like the example below) to validate incoming xml.
Make sure you are using attribute names that are allowed by your parser.
Below is the Scheme used by Jenkins to validate the XML report:
.. code-block:: xml
<xs:element name="testcase">
<xs:complexType>
<xs:sequence>
<xs:element ref="skipped" minOccurs="0" maxOccurs="1"/>
<xs:element ref="error" minOccurs="0" maxOccurs="unbounded"/>
<xs:element ref="failure" minOccurs="0" maxOccurs="unbounded"/>
<xs:element ref="system-out" minOccurs="0" maxOccurs="unbounded"/>
<xs:element ref="system-err" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute name="name" type="xs:string" use="required"/>
<xs:attribute name="assertions" type="xs:string" use="optional"/>
<xs:attribute name="time" type="xs:string" use="optional"/>
<xs:attribute name="classname" type="xs:string" use="optional"/>
<xs:attribute name="status" type="xs:string" use="optional"/>
</xs:complexType>
</xs:element>
LogXML: add_global_property
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -389,4 +449,14 @@ hook was invoked::
*** test run reporting finishing
.. note::
Calling ``pytest.main()`` will result in importing your tests and any modules
that they import. Due to the caching mechanism of python's import system,
making subsequent calls to ``pytest.main()`` from the same process will not
reflect changes to those files between the calls. For this reason, making
multiple calls to ``pytest.main()`` from the same process (in order to re-run
tests, for example) is not recommended.
.. include:: links.inc

6
doc/en/warnings.rst
View File

@@ -112,6 +112,12 @@ decorator or to all tests in a module by setting the ``pytestmark`` variable:
pytestmark = pytest.mark.filterwarnings('error')
.. note::
Except for these features, pytest does not change the python warning filter; it only captures
and displays the warnings which are issued with respect to the currently configured filter,
including changes to the filter made by test functions or by the system under test.
.. note::
``DeprecationWarning`` and ``PendingDeprecationWarning`` are hidden by the standard library

13
doc/en/writing_plugins.rst
View File

@@ -583,11 +583,22 @@ pytest hook reference
Initialization, command line and configuration hooks
----------------------------------------------------
Bootstrapping hooks
~~~~~~~~~~~~~~~~~~~
Bootstrapping hooks called for plugins registered early enough (internal and setuptools plugins).
.. autofunction:: pytest_load_initial_conftests
.. autofunction:: pytest_cmdline_preparse
.. autofunction:: pytest_cmdline_parse
.. autofunction:: pytest_addoption
.. autofunction:: pytest_cmdline_main
Initialization hooks
~~~~~~~~~~~~~~~~~~~~
Initialization hooks called for plugins and ``conftest.py`` files.
.. autofunction:: pytest_addoption
.. autofunction:: pytest_configure
.. autofunction:: pytest_unconfigure