floraachy/pytest2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Remove gendoc normalization for '=' and '_' headers

Browse Source 
Using the default for non-atty terminals (80) so the new
progress indicator aligns correctly.
This commit is contained in:
Bruno Oliveira
2017-11-23 15:33:41 +00:00
parent b533c2600a
commit e0d236c031
21 changed files with 404 additions and 379 deletions
Download Patch File Download Diff File Expand all files Collapse all files

132
doc/en/example/markers.rst
View File

@@ -30,32 +30,32 @@ You can "mark" a test function with custom metadata like this::
You can then restrict a test run to only run tests marked with ``webtest``::
$ pytest -v -m webtest
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
test_server.py::test_send_http PASSED
test_server.py::test_send_http PASSED [100%]
======= 3 tests deselected ========
======= 1 passed, 3 deselected in 0.12 seconds ========
============================ 3 tests deselected ============================
================== 1 passed, 3 deselected in 0.12 seconds ==================
Or the inverse, running all tests except the webtest ones::
$ pytest -v -m "not webtest"
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
test_server.py::test_something_quick PASSED
test_server.py::test_another PASSED
test_server.py::TestClass::test_method PASSED
test_server.py::test_something_quick PASSED [ 33%]
test_server.py::test_another PASSED [ 66%]
test_server.py::TestClass::test_method PASSED [100%]
======= 1 tests deselected ========
======= 3 passed, 1 deselected in 0.12 seconds ========
============================ 1 tests deselected ============================
================== 3 passed, 1 deselected in 0.12 seconds ==================
Selecting tests based on their node ID
--------------------------------------
@@ -65,42 +65,42 @@ arguments to select only specified tests. This makes it easy to select
tests based on their module, class, method, or function name::
$ pytest -v test_server.py::TestClass::test_method
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED
test_server.py::TestClass::test_method PASSED [100%]
======= 1 passed in 0.12 seconds ========
========================= 1 passed in 0.12 seconds =========================
You can also select on the class::
$ pytest -v test_server.py::TestClass
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED
test_server.py::TestClass::test_method PASSED [100%]
======= 1 passed in 0.12 seconds ========
========================= 1 passed in 0.12 seconds =========================
Or select multiple nodes::
$ pytest -v test_server.py::TestClass test_server.py::test_send_http
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 2 items
test_server.py::TestClass::test_method PASSED
test_server.py::test_send_http PASSED
test_server.py::TestClass::test_method PASSED [ 50%]
test_server.py::test_send_http PASSED [100%]
======= 2 passed in 0.12 seconds ========
========================= 2 passed in 0.12 seconds =========================
.. _node-id:
@@ -129,47 +129,47 @@ exact match on markers that ``-m`` provides. This makes it easy to
select tests based on their names::
$ pytest -v -k http # running with the above defined example module
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
test_server.py::test_send_http PASSED
test_server.py::test_send_http PASSED [100%]
======= 3 tests deselected ========
======= 1 passed, 3 deselected in 0.12 seconds ========
============================ 3 tests deselected ============================
================== 1 passed, 3 deselected in 0.12 seconds ==================
And you can also run all tests except the ones that match the keyword::
$ pytest -k "not send_http" -v
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
test_server.py::test_something_quick PASSED
test_server.py::test_another PASSED
test_server.py::TestClass::test_method PASSED
test_server.py::test_something_quick PASSED [ 33%]
test_server.py::test_another PASSED [ 66%]
test_server.py::TestClass::test_method PASSED [100%]
======= 1 tests deselected ========
======= 3 passed, 1 deselected in 0.12 seconds ========
============================ 1 tests deselected ============================
================== 3 passed, 1 deselected in 0.12 seconds ==================
Or to select "http" and "quick" tests::
$ pytest -k "http or quick" -v
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 4 items
test_server.py::test_send_http PASSED
test_server.py::test_something_quick PASSED
test_server.py::test_send_http PASSED [ 50%]
test_server.py::test_something_quick PASSED [100%]
======= 2 tests deselected ========
======= 2 passed, 2 deselected in 0.12 seconds ========
============================ 2 tests deselected ============================
================== 2 passed, 2 deselected in 0.12 seconds ==================
.. note::
@@ -354,26 +354,26 @@ and an example invocations specifying a different environment than what
the test needs::
$ pytest -E stage2
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 1 item
test_someenv.py s
test_someenv.py s [100%]
======= 1 skipped in 0.12 seconds ========
======================== 1 skipped in 0.12 seconds =========================
and here is one that specifies exactly the environment needed::
$ pytest -E stage1
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 1 item
test_someenv.py .
test_someenv.py . [100%]
======= 1 passed in 0.12 seconds ========
========================= 1 passed in 0.12 seconds =========================
The ``--markers`` option always gives you a list of available markers::
@@ -432,7 +432,7 @@ The output is as follows::
$ pytest -q -s
Marker info name=my_marker args=(<function hello_world at 0xdeadbeef>,) kwars={}
.
. [100%]
1 passed in 0.12 seconds
We can see that the custom marker has its argument set extended with the function ``hello_world``. This is the key difference between creating a custom marker as a callable, which invokes ``__call__`` behind the scenes, and using ``with_args``.
@@ -477,7 +477,7 @@ Let's run this without capturing output and see what we get::
glob args=('function',) kwargs={'x': 3}
glob args=('class',) kwargs={'x': 2}
glob args=('module',) kwargs={'x': 1}
.
. [100%]
1 passed in 0.12 seconds
marking platform specific tests with pytest
@@ -530,29 +530,29 @@ Let's do a little test file to show how this looks like::
then you will see two tests skipped and two executed tests as expected::
$ pytest -rs # this option reports skip reasons
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
test_plat.py s.s.
======= short test summary info ========
test_plat.py s.s. [100%]
========================= short test summary info ==========================
SKIP [2] $REGENDOC_TMPDIR/conftest.py:13: cannot run on platform linux
======= 2 passed, 2 skipped in 0.12 seconds ========
=================== 2 passed, 2 skipped in 0.12 seconds ====================
Note that if you specify a platform via the marker-command line option like this::
$ pytest -m linux
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
test_plat.py .
test_plat.py . [100%]
======= 3 tests deselected ========
======= 1 passed, 3 deselected in 0.12 seconds ========
============================ 3 tests deselected ============================
================== 1 passed, 3 deselected in 0.12 seconds ==================
then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests.
@@ -596,47 +596,47 @@ We want to dynamically define two markers and can do it in a
We can now use the ``-m option`` to select one set::
$ pytest -m interface --tb=short
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
test_module.py FF
test_module.py FF [100%]
======= FAILURES ========
_______ test_interface_simple ________
================================= FAILURES =================================
__________________________ test_interface_simple ___________________________
test_module.py:3: in test_interface_simple
assert 0
E assert 0
_______ test_interface_complex ________
__________________________ test_interface_complex __________________________
test_module.py:6: in test_interface_complex
assert 0
E assert 0
======= 2 tests deselected ========
======= 2 failed, 2 deselected in 0.12 seconds ========
============================ 2 tests deselected ============================
================== 2 failed, 2 deselected in 0.12 seconds ==================
or to select both "event" and "interface" tests::
$ pytest -m "interface or event" --tb=short
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
test_module.py FFF
test_module.py FFF [100%]
======= FAILURES ========
_______ test_interface_simple ________
================================= FAILURES =================================
__________________________ test_interface_simple ___________________________
test_module.py:3: in test_interface_simple
assert 0
E assert 0
_______ test_interface_complex ________
__________________________ test_interface_complex __________________________
test_module.py:6: in test_interface_complex
assert 0
E assert 0
_______ test_event_simple ________
____________________________ test_event_simple _____________________________
test_module.py:9: in test_event_simple
assert 0
E assert 0
======= 1 tests deselected ========
======= 3 failed, 1 deselected in 0.12 seconds ========
============================ 1 tests deselected ============================
================== 3 failed, 1 deselected in 0.12 seconds ==================

26
doc/en/example/nonpython.rst
View File

@@ -26,19 +26,19 @@ and if you installed `PyYAML`_ or a compatible YAML-parser you can
now execute the test specification::
nonpython $ pytest test_simple.yml
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
collected 2 items
test_simple.yml F.
test_simple.yml F. [100%]
======= FAILURES ========
_______ usecase: hello ________
================================= FAILURES =================================
______________________________ usecase: hello ______________________________
usecase execution failed
spec failed: 'some': 'other'
no further details known at this point.
======= 1 failed, 1 passed in 0.12 seconds ========
==================== 1 failed, 1 passed in 0.12 seconds ====================
.. regendoc:wipe
@@ -58,21 +58,21 @@ your own domain specific testing language this way.
consulted when reporting in ``verbose`` mode::
nonpython $ pytest -v
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
collecting ... collected 2 items
test_simple.yml::hello FAILED
test_simple.yml::ok PASSED
test_simple.yml::hello FAILED [ 50%]
test_simple.yml::ok PASSED [100%]
======= FAILURES ========
_______ usecase: hello ________
================================= FAILURES =================================
______________________________ usecase: hello ______________________________
usecase execution failed
spec failed: 'some': 'other'
no further details known at this point.
======= 1 failed, 1 passed in 0.12 seconds ========
==================== 1 failed, 1 passed in 0.12 seconds ====================
.. regendoc:wipe
@@ -80,7 +80,7 @@ While developing your custom test collection and execution it's also
interesting to just look at the collection tree::
nonpython $ pytest --collect-only
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
collected 2 items
@@ -88,4 +88,4 @@ interesting to just look at the collection tree::
<YamlItem 'hello'>
<YamlItem 'ok'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================

59
doc/en/example/parametrize.rst
View File

@@ -45,16 +45,16 @@ Now we add a test configuration like this::
This means that we only run 2 tests if we do not pass ``--all``::
$ pytest -q test_compute.py
..
.. [100%]
2 passed in 0.12 seconds
We run only two computations, so we see two dots.
let's run the full monty::
$ pytest -q --all
....F
======= FAILURES ========
_______ test_compute[4] ________
....F [100%]
================================= FAILURES =================================
_____________________________ test_compute[4] ______________________________
param1 = 4
@@ -138,7 +138,7 @@ objects, they are still using the default pytest representation::
$ pytest test_time.py --collect-only
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 8 items
@@ -152,7 +152,7 @@ objects, they are still using the default pytest representation::
<Function 'test_timedistance_v3[forward]'>
<Function 'test_timedistance_v3[backward]'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
In ``test_timedistance_v3``, we used ``pytest.param`` to specify the test IDs
together with the actual data, instead of listing them separately.
@@ -194,20 +194,20 @@ only have to work a bit to construct the correct arguments for pytest's
this is a fully self-contained example which you can run with::
$ pytest test_scenarios.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
test_scenarios.py ....
test_scenarios.py .... [100%]
======= 4 passed in 0.12 seconds ========
========================= 4 passed in 0.12 seconds =========================
If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function::
$ pytest --collect-only test_scenarios.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
@@ -219,7 +219,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
<Function 'test_demo1[advanced]'>
<Function 'test_demo2[advanced]'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
Note that we told ``metafunc.parametrize()`` that your scenario values
should be considered class-scoped. With pytest-2.3 this leads to a
@@ -272,7 +272,7 @@ creates a database object for the actual test invocations::
Let's first see how it looks like at collection time::
$ pytest test_backends.py --collect-only
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 2 items
@@ -280,14 +280,14 @@ Let's first see how it looks like at collection time::
<Function 'test_db_initialized[d1]'>
<Function 'test_db_initialized[d2]'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
And then when we run the test::
$ pytest -q test_backends.py
.F
======= FAILURES ========
_______ test_db_initialized[d2] ________
.F [100%]
================================= FAILURES =================================
_________________________ test_db_initialized[d2] __________________________
db = <conftest.DB2 object at 0xdeadbeef>
@@ -333,14 +333,14 @@ will be passed to respective fixture function::
The result of this test will be successful::
$ pytest test_indirect_list.py --collect-only
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 1 item
<Module 'test_indirect_list.py'>
<Function 'test_indirect[a-b]'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
.. regendoc:wipe
@@ -381,9 +381,9 @@ Our test generator looks up a class-level definition which specifies which
argument sets to use for each test function. Let's run it::
$ pytest -q
F..
======= FAILURES ========
_______ TestClass.test_equals[1-2] ________
F.. [100%]
================================= FAILURES =================================
________________________ TestClass.test_equals[1-2] ________________________
self = <test_parametrize.TestClass object at 0xdeadbeef>, a = 1, b = 2
@@ -411,10 +411,11 @@ is to be run with different sets of arguments for its three arguments:
Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize)::
. $ pytest -rs -q multipython.py
sssssssssssssss.........sss.........sss.........
======= short test summary info ========
SKIP [21] $REGENDOC_TMPDIR/CWD/multipython.py:24: 'python2.6' not found
27 passed, 21 skipped in 0.12 seconds
ssssssssssssssssssssssss... [100%]
========================= short test summary info ==========================
SKIP [12] $REGENDOC_TMPDIR/CWD/multipython.py:24: 'python2.7' not found
SKIP [12] $REGENDOC_TMPDIR/CWD/multipython.py:24: 'python3.4' not found
3 passed, 24 skipped in 0.12 seconds
Indirect parametrization of optional implementations/imports
--------------------------------------------------------------------
@@ -460,16 +461,16 @@ And finally a little test module::
If you run this with reporting for skips enabled::
$ pytest -rs test_module.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 2 items
test_module.py .s
======= short test summary info ========
test_module.py .s [100%]
========================= short test summary info ==========================
SKIP [1] $REGENDOC_TMPDIR/conftest.py:11: could not import 'opt2'
======= 1 passed, 1 skipped in 0.12 seconds ========
=================== 1 passed, 1 skipped in 0.12 seconds ====================
You'll see that we don't have a ``opt2`` module and thus the second test run
of our ``test_func1`` was skipped. A few notes:

12
doc/en/example/pythoncollection.rst
View File

@@ -116,7 +116,7 @@ that match ``*_check``. For example, if we have::
then the test collection looks like this::
$ pytest --collect-only
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 2 items
@@ -126,7 +126,7 @@ then the test collection looks like this::
<Function 'simple_check'>
<Function 'complex_check'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
.. note::
@@ -162,7 +162,7 @@ Finding out what is collected
You can always peek at the collection tree without running tests like this::
. $ pytest --collect-only pythoncollection.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 3 items
@@ -173,7 +173,7 @@ You can always peek at the collection tree without running tests like this::
<Function 'test_method'>
<Function 'test_anothermethod'>
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
.. _customizing-test-collection:
@@ -231,9 +231,9 @@ If you run with a Python 3 interpreter both the one test and the ``setup.py``
file will be left out::
$ pytest --collect-only
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 0 items
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================

94
doc/en/example/reportingdemo.rst
View File

@@ -10,15 +10,15 @@ not showing the nice colors here in the HTML that you
get on the terminal - we are working on that)::
assertion $ pytest failure_demo.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR/assertion, inifile:
collected 42 items
failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [100%]
======= FAILURES ========
_______ test_generative[0] ________
================================= FAILURES =================================
____________________________ test_generative[0] ____________________________
param1 = 3, param2 = 6
@@ -27,7 +27,7 @@ get on the terminal - we are working on that)::
E assert (3 * 2) < 6
failure_demo.py:16: AssertionError
_______ TestFailing.test_simple ________
_________________________ TestFailing.test_simple __________________________
self = <failure_demo.TestFailing object at 0xdeadbeef>
@@ -43,7 +43,7 @@ get on the terminal - we are working on that)::
E + and 43 = <function TestFailing.test_simple.<locals>.g at 0xdeadbeef>()
failure_demo.py:29: AssertionError
_______ TestFailing.test_simple_multiline ________
____________________ TestFailing.test_simple_multiline _____________________
self = <failure_demo.TestFailing object at 0xdeadbeef>
@@ -63,7 +63,7 @@ get on the terminal - we are working on that)::
E assert 42 == 54
failure_demo.py:12: AssertionError
_______ TestFailing.test_not ________
___________________________ TestFailing.test_not ___________________________
self = <failure_demo.TestFailing object at 0xdeadbeef>
@@ -75,7 +75,7 @@ get on the terminal - we are working on that)::
E + where 42 = <function TestFailing.test_not.<locals>.f at 0xdeadbeef>()
failure_demo.py:39: AssertionError
_______ TestSpecialisedExplanations.test_eq_text ________
_________________ TestSpecialisedExplanations.test_eq_text _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -86,7 +86,7 @@ get on the terminal - we are working on that)::
E + eggs
failure_demo.py:43: AssertionError
_______ TestSpecialisedExplanations.test_eq_similar_text ________
_____________ TestSpecialisedExplanations.test_eq_similar_text _____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -99,7 +99,7 @@ get on the terminal - we are working on that)::
E ? ^
failure_demo.py:46: AssertionError
_______ TestSpecialisedExplanations.test_eq_multiline_text ________
____________ TestSpecialisedExplanations.test_eq_multiline_text ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -112,7 +112,7 @@ get on the terminal - we are working on that)::
E bar
failure_demo.py:49: AssertionError
_______ TestSpecialisedExplanations.test_eq_long_text ________
______________ TestSpecialisedExplanations.test_eq_long_text _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -129,7 +129,7 @@ get on the terminal - we are working on that)::
E ? ^
failure_demo.py:54: AssertionError
_______ TestSpecialisedExplanations.test_eq_long_text_multiline ________
_________ TestSpecialisedExplanations.test_eq_long_text_multiline __________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -149,7 +149,7 @@ get on the terminal - we are working on that)::
E ...Full output truncated (7 lines hidden), use '-vv' to show
failure_demo.py:59: AssertionError
_______ TestSpecialisedExplanations.test_eq_list ________
_________________ TestSpecialisedExplanations.test_eq_list _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -160,7 +160,7 @@ get on the terminal - we are working on that)::
E Use -v to get the full diff
failure_demo.py:62: AssertionError
_______ TestSpecialisedExplanations.test_eq_list_long ________
______________ TestSpecialisedExplanations.test_eq_list_long _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -173,7 +173,7 @@ get on the terminal - we are working on that)::
E Use -v to get the full diff
failure_demo.py:67: AssertionError
_______ TestSpecialisedExplanations.test_eq_dict ________
_________________ TestSpecialisedExplanations.test_eq_dict _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -191,7 +191,7 @@ get on the terminal - we are working on that)::
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:70: AssertionError
_______ TestSpecialisedExplanations.test_eq_set ________
_________________ TestSpecialisedExplanations.test_eq_set __________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -209,7 +209,7 @@ get on the terminal - we are working on that)::
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:73: AssertionError
_______ TestSpecialisedExplanations.test_eq_longer_list ________
_____________ TestSpecialisedExplanations.test_eq_longer_list ______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -220,7 +220,7 @@ get on the terminal - we are working on that)::
E Use -v to get the full diff
failure_demo.py:76: AssertionError
_______ TestSpecialisedExplanations.test_in_list ________
_________________ TestSpecialisedExplanations.test_in_list _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -229,7 +229,7 @@ get on the terminal - we are working on that)::
E assert 1 in [0, 2, 3, 4, 5]
failure_demo.py:79: AssertionError
_______ TestSpecialisedExplanations.test_not_in_text_multiline ________
__________ TestSpecialisedExplanations.test_not_in_text_multiline __________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -248,7 +248,7 @@ get on the terminal - we are working on that)::
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:83: AssertionError
_______ TestSpecialisedExplanations.test_not_in_text_single ________
___________ TestSpecialisedExplanations.test_not_in_text_single ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -261,7 +261,7 @@ get on the terminal - we are working on that)::
E ? +++
failure_demo.py:87: AssertionError
_______ TestSpecialisedExplanations.test_not_in_text_single_long ________
_________ TestSpecialisedExplanations.test_not_in_text_single_long _________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@@ -287,7 +287,7 @@ get on the terminal - we are working on that)::
E ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
failure_demo.py:95: AssertionError
_______ test_attribute ________
______________________________ test_attribute ______________________________
def test_attribute():
class Foo(object):
@@ -298,7 +298,7 @@ get on the terminal - we are working on that)::
E + where 1 = <failure_demo.test_attribute.<locals>.Foo object at 0xdeadbeef>.b
failure_demo.py:102: AssertionError
_______ test_attribute_instance ________
_________________________ test_attribute_instance __________________________
def test_attribute_instance():
class Foo(object):
@@ -309,7 +309,7 @@ get on the terminal - we are working on that)::
E + where <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef> = <class 'failure_demo.test_attribute_instance.<locals>.Foo'>()
failure_demo.py:108: AssertionError
_______ test_attribute_failure ________
__________________________ test_attribute_failure __________________________
def test_attribute_failure():
class Foo(object):
@@ -329,7 +329,7 @@ get on the terminal - we are working on that)::
E Exception: Failed to get attrib
failure_demo.py:114: Exception
_______ test_attribute_multiple ________
_________________________ test_attribute_multiple __________________________
def test_attribute_multiple():
class Foo(object):
@@ -344,7 +344,7 @@ get on the terminal - we are working on that)::
E + where <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef> = <class 'failure_demo.test_attribute_multiple.<locals>.Bar'>()
failure_demo.py:125: AssertionError
_______ TestRaises.test_raises ________
__________________________ TestRaises.test_raises __________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -359,7 +359,7 @@ get on the terminal - we are working on that)::
E ValueError: invalid literal for int() with base 10: 'qwe'
<0-codegen $PYTHON_PREFIX/lib/python3.5/site-packages/_pytest/python_api.py:580>:1: ValueError
_______ TestRaises.test_raises_doesnt ________
______________________ TestRaises.test_raises_doesnt _______________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -368,7 +368,7 @@ get on the terminal - we are working on that)::
E Failed: DID NOT RAISE <class 'OSError'>
failure_demo.py:137: Failed
_______ TestRaises.test_raise ________
__________________________ TestRaises.test_raise ___________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -377,7 +377,7 @@ get on the terminal - we are working on that)::
E ValueError: demo error
failure_demo.py:140: ValueError
_______ TestRaises.test_tupleerror ________
________________________ TestRaises.test_tupleerror ________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -399,7 +399,7 @@ get on the terminal - we are working on that)::
failure_demo.py:148: TypeError
--------------------------- Captured stdout call ---------------------------
l is [1, 2, 3]
_______ TestRaises.test_some_error ________
________________________ TestRaises.test_some_error ________________________
self = <failure_demo.TestRaises object at 0xdeadbeef>
@@ -408,7 +408,7 @@ get on the terminal - we are working on that)::
E NameError: name 'namenotexi' is not defined
failure_demo.py:151: NameError
_______ test_dynamic_compile_shows_nicely ________
____________________ test_dynamic_compile_shows_nicely _____________________
def test_dynamic_compile_shows_nicely():
src = 'def foo():\n assert 1 == 0\n'
@@ -427,7 +427,7 @@ get on the terminal - we are working on that)::
E AssertionError
<2-codegen 'abc-123' $REGENDOC_TMPDIR/assertion/failure_demo.py:163>:2: AssertionError
_______ TestMoreErrors.test_complex_error ________
____________________ TestMoreErrors.test_complex_error _____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -451,7 +451,7 @@ get on the terminal - we are working on that)::
E assert 44 == 43
failure_demo.py:6: AssertionError
_______ TestMoreErrors.test_z1_unpack_error ________
___________________ TestMoreErrors.test_z1_unpack_error ____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -461,7 +461,7 @@ get on the terminal - we are working on that)::
E ValueError: not enough values to unpack (expected 2, got 0)
failure_demo.py:180: ValueError
_______ TestMoreErrors.test_z2_type_error ________
____________________ TestMoreErrors.test_z2_type_error _____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -471,7 +471,7 @@ get on the terminal - we are working on that)::
E TypeError: 'int' object is not iterable
failure_demo.py:184: TypeError
_______ TestMoreErrors.test_startswith ________
______________________ TestMoreErrors.test_startswith ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -484,7 +484,7 @@ get on the terminal - we are working on that)::
E + where <built-in method startswith of str object at 0xdeadbeef> = '123'.startswith
failure_demo.py:189: AssertionError
_______ TestMoreErrors.test_startswith_nested ________
__________________ TestMoreErrors.test_startswith_nested ___________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -501,7 +501,7 @@ get on the terminal - we are working on that)::
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef>()
failure_demo.py:196: AssertionError
_______ TestMoreErrors.test_global_func ________
_____________________ TestMoreErrors.test_global_func ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -512,7 +512,7 @@ get on the terminal - we are working on that)::
E + where 43 = globf(42)
failure_demo.py:199: AssertionError
_______ TestMoreErrors.test_instance ________
_______________________ TestMoreErrors.test_instance _______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -523,7 +523,7 @@ get on the terminal - we are working on that)::
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef>.x
failure_demo.py:203: AssertionError
_______ TestMoreErrors.test_compare ________
_______________________ TestMoreErrors.test_compare ________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -533,7 +533,7 @@ get on the terminal - we are working on that)::
E + where 11 = globf(10)
failure_demo.py:206: AssertionError
_______ TestMoreErrors.test_try_finally ________
_____________________ TestMoreErrors.test_try_finally ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@@ -544,7 +544,7 @@ get on the terminal - we are working on that)::
E assert 1 == 0
failure_demo.py:211: AssertionError
_______ TestCustomAssertMsg.test_single_line ________
___________________ TestCustomAssertMsg.test_single_line ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@@ -558,7 +558,7 @@ get on the terminal - we are working on that)::
E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_single_line.<locals>.A'>.a
failure_demo.py:222: AssertionError
_______ TestCustomAssertMsg.test_multiline ________
____________________ TestCustomAssertMsg.test_multiline ____________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@@ -575,7 +575,7 @@ get on the terminal - we are working on that)::
E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_multiline.<locals>.A'>.a
failure_demo.py:228: AssertionError
_______ TestCustomAssertMsg.test_custom_repr ________
___________________ TestCustomAssertMsg.test_custom_repr ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@@ -595,4 +595,10 @@ get on the terminal - we are working on that)::
E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a
failure_demo.py:238: AssertionError
======= 42 failed in 0.12 seconds ========
============================= warnings summary =============================
None
Metafunc.addcall is deprecated and scheduled to be removed in pytest 4.0.
Please use Metafunc.parametrize instead.
-- Docs: http://doc.pytest.org/en/latest/warnings.html
================== 42 failed, 1 warnings in 0.12 seconds ===================

122
doc/en/example/simple.rst
View File

@@ -41,9 +41,9 @@ provide the ``cmdopt`` through a :ref:`fixture function <fixture function>`:
Let's run this without supplying our new option::
$ pytest -q test_sample.py
F
======= FAILURES ========
_______ test_answer ________
F [100%]
================================= FAILURES =================================
_______________________________ test_answer ________________________________
cmdopt = 'type1'
@@ -63,9 +63,9 @@ Let's run this without supplying our new option::
And now with supplying a command line option::
$ pytest -q --cmdopt=type2
F
======= FAILURES ========
_______ test_answer ________
F [100%]
================================= FAILURES =================================
_______________________________ test_answer ________________________________
cmdopt = 'type2'
@@ -112,12 +112,12 @@ of subprocesses close to your CPU. Running in an empty
directory with the above conftest.py::
$ pytest
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 0 items
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
.. _`excontrolskip`:
@@ -166,28 +166,28 @@ We can now write a test module like this:
and when running it will see a skipped "slow" test::
$ pytest -rs # "-rs" means report details on the little 's'
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 2 items
test_module.py .s
======= short test summary info ========
test_module.py .s [100%]
========================= short test summary info ==========================
SKIP [1] test_module.py:8: need --runslow option to run
======= 1 passed, 1 skipped in 0.12 seconds ========
=================== 1 passed, 1 skipped in 0.12 seconds ====================
Or run it including the ``slow`` marked test::
$ pytest --runslow
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 2 items
test_module.py ..
test_module.py .. [100%]
======= 2 passed in 0.12 seconds ========
========================= 2 passed in 0.12 seconds =========================
Writing well integrated assertion helpers
--------------------------------------------------
@@ -218,9 +218,9 @@ unless the ``--full-trace`` command line option is specified.
Let's run our little function::
$ pytest -q test_checkconfig.py
F
======= FAILURES ========
_______ test_something ________
F [100%]
================================= FAILURES =================================
______________________________ test_something ______________________________
def test_something():
> checkconfig(42)
@@ -305,13 +305,13 @@ It's easy to present extra information in a ``pytest`` run:
which will add the string to the test header accordingly::
$ pytest
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
project deps: mylib-1.1
rootdir: $REGENDOC_TMPDIR, inifile:
collected 0 items
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
.. regendoc:wipe
@@ -330,7 +330,7 @@ display more information if applicable:
which will add info only when run with "--v"::
$ pytest -v
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.5
cachedir: .cache
info1: did you know that ...
@@ -338,17 +338,17 @@ which will add info only when run with "--v"::
rootdir: $REGENDOC_TMPDIR, inifile:
collecting ... collected 0 items
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
and nothing when run plainly::
$ pytest
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 0 items
======= no tests ran in 0.12 seconds ========
======================= no tests ran in 0.12 seconds =======================
profiling test duration
--------------------------
@@ -377,18 +377,18 @@ out which tests are the slowest. Let's make an artificial test suite:
Now we can profile which test functions execute the slowest::
$ pytest --durations=3
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 3 items
test_some_are_slow.py ...
test_some_are_slow.py ... [100%]
======= slowest 3 test durations ========
========================= slowest 3 test durations =========================
0.30s call test_some_are_slow.py::test_funcslow2
0.20s call test_some_are_slow.py::test_funcslow1
0.10s call test_some_are_slow.py::test_funcfast
======= 3 passed in 0.12 seconds ========
========================= 3 passed in 0.12 seconds =========================
incremental testing - test steps
---------------------------------------------------
@@ -443,18 +443,18 @@ tests in a class. Here is a test module example:
If we run this::
$ pytest -rx
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 4 items
test_step.py .Fx.
======= short test summary info ========
test_step.py .Fx. [100%]
========================= short test summary info ==========================
XFAIL test_step.py::TestUserHandling::()::test_deletion
reason: previous test failed (test_modification)
======= FAILURES ========
_______ TestUserHandling.test_modification ________
================================= FAILURES =================================
____________________ TestUserHandling.test_modification ____________________
self = <test_step.TestUserHandling object at 0xdeadbeef>
@@ -463,7 +463,7 @@ If we run this::
E assert 0
test_step.py:9: AssertionError
======= 1 failed, 2 passed, 1 xfailed in 0.12 seconds ========
============== 1 failed, 2 passed, 1 xfailed in 0.12 seconds ===============
We'll see that ``test_deletion`` was not executed because ``test_modification``
failed. It is reported as an "expected failure".
@@ -522,27 +522,27 @@ the ``db`` fixture:
We can run this::
$ pytest
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 7 items
test_step.py .Fx.
a/test_db.py F
a/test_db2.py F
b/test_error.py E
test_step.py .Fx. [ 57%]
a/test_db.py F [ 71%]
a/test_db2.py F [ 85%]
b/test_error.py E [100%]
======= ERRORS ========
_______ ERROR at setup of test_root ________
================================== ERRORS ==================================
_______________________ ERROR at setup of test_root ________________________
file $REGENDOC_TMPDIR/b/test_error.py, line 1
def test_root(db): # no db here, will error out
E fixture 'db' not found
> available fixtures: cache, capfd, capsys, doctest_namespace, monkeypatch, pytestconfig, record_xml_property, recwarn, tmpdir, tmpdir_factory
> available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_xml_property, recwarn, tmpdir, tmpdir_factory
> use 'pytest --fixtures [testpath]' for help on them.
$REGENDOC_TMPDIR/b/test_error.py:1
======= FAILURES ========
_______ TestUserHandling.test_modification ________
================================= FAILURES =================================
____________________ TestUserHandling.test_modification ____________________
self = <test_step.TestUserHandling object at 0xdeadbeef>
@@ -551,7 +551,7 @@ We can run this::
E assert 0
test_step.py:9: AssertionError
_______ test_a1 ________
_________________________________ test_a1 __________________________________
db = <conftest.DB object at 0xdeadbeef>
@@ -561,7 +561,7 @@ We can run this::
E assert 0
a/test_db.py:2: AssertionError
_______ test_a2 ________
_________________________________ test_a2 __________________________________
db = <conftest.DB object at 0xdeadbeef>
@@ -571,7 +571,7 @@ We can run this::
E assert 0
a/test_db2.py:2: AssertionError
======= 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ========
========== 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ==========
The two test modules in the ``a`` directory see the same ``db`` fixture instance
while the one test in the sister-directory ``b`` doesn't see it. We could of course
@@ -630,15 +630,15 @@ if you then have failing tests:
and run them::
$ pytest test_module.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 2 items
test_module.py FF
test_module.py FF [100%]
======= FAILURES ========
_______ test_fail1 ________
================================= FAILURES =================================
________________________________ test_fail1 ________________________________
tmpdir = local('PYTEST_TMPDIR/test_fail10')
@@ -647,14 +647,14 @@ and run them::
E assert 0
test_module.py:2: AssertionError
_______ test_fail2 ________
________________________________ test_fail2 ________________________________
def test_fail2():
> assert 0
E assert 0
test_module.py:4: AssertionError
======= 2 failed in 0.12 seconds ========
========================= 2 failed in 0.12 seconds =========================
you will have a "failures" file which contains the failing test ids::
@@ -724,17 +724,17 @@ if you then have failing tests:
and run it::
$ pytest -s test_module.py
======= test session starts ========
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y
rootdir: $REGENDOC_TMPDIR, inifile:
collected 3 items
test_module.py Esetting up a test failed! test_module.py::test_setup_fails
Fexecuting test failed test_module.py::test_call_fails
F
F [100%]
======= ERRORS ========
_______ ERROR at setup of test_setup_fails ________
================================== ERRORS ==================================
____________________ ERROR at setup of test_setup_fails ____________________
@pytest.fixture
def other():
@@ -742,8 +742,8 @@ and run it::
E assert 0
test_module.py:6: AssertionError
======= FAILURES ========
_______ test_call_fails ________
================================= FAILURES =================================
_____________________________ test_call_fails ______________________________
something = None
@@ -752,14 +752,14 @@ and run it::
E assert 0
test_module.py:12: AssertionError
_______ test_fail2 ________
________________________________ test_fail2 ________________________________
def test_fail2():
> assert 0
E assert 0
test_module.py:15: AssertionError
======= 2 failed, 1 error in 0.12 seconds ========
==================== 2 failed, 1 error in 0.12 seconds =====================
You'll see that the fixture finalizers could use the precise reporting
information.

2
doc/en/example/special.rst
View File

@@ -68,5 +68,5 @@ If you run this without output capturing::
.test_method1 called
.test other
.test_unit1 method called
.
. [100%]
4 passed in 0.12 seconds