From 6fd57ec786d27dfc59c22b0438dac65049538069 Mon Sep 17 00:00:00 2001 From: holger krekel Date: Wed, 6 Jun 2012 16:34:13 +0200 Subject: [PATCH] extend marker section with a platform example --- doc/en/example/markers.txt | 104 ++++++++++++++++++++++++++++++++----- 1 file changed, 91 insertions(+), 13 deletions(-) diff --git a/doc/en/example/markers.txt b/doc/en/example/markers.txt index 1f1e11c31..1d87f9617 100644 --- a/doc/en/example/markers.txt +++ b/doc/en/example/markers.txt @@ -26,7 +26,7 @@ You can then restrict a test run to only run tests marked with ``webtest``:: $ py.test -v -m webtest =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 -- /home/hpk/venv/0/bin/python + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 -- /home/hpk/venv/1/bin/python collecting ... collected 2 items test_server.py:3: test_send_http PASSED @@ -38,13 +38,13 @@ Or the inverse, running all tests except the webtest ones:: $ py.test -v -m "not webtest" =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 -- /home/hpk/venv/0/bin/python + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 -- /home/hpk/venv/1/bin/python collecting ... collected 2 items test_server.py:6: test_something_quick PASSED ================= 1 tests deselected by "-m 'not webtest'" ================= - ================== 1 passed, 1 deselected in 0.00 seconds ================== + ================== 1 passed, 1 deselected in 0.01 seconds ================== Registering markers ------------------------------------- @@ -143,7 +143,7 @@ the given argument:: $ py.test -k send_http # running with the above defined examples =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 collecting ... collected 4 items test_server.py . @@ -155,7 +155,7 @@ And you can also run all tests except the ones that match the keyword:: $ py.test -k-send_http =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 collecting ... collected 4 items test_mark_classlevel.py .. @@ -168,7 +168,7 @@ Or to only select the class:: $ py.test -kTestClass =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 collecting ... collected 4 items test_mark_classlevel.py .. @@ -223,23 +223,23 @@ the test needs:: $ py.test -E stage2 =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 collecting ... collected 1 items test_someenv.py s - ======================== 1 skipped in 0.00 seconds ========================= + ======================== 1 skipped in 0.01 seconds ========================= and here is one that specifies exactly the environment needed:: $ py.test -E stage1 =========================== test session starts ============================ - platform linux2 -- Python 2.7.1 -- pytest-2.2.4 + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 collecting ... collected 1 items test_someenv.py . - ========================= 1 passed in 0.00 seconds ========================= + ========================= 1 passed in 0.01 seconds ========================= The ``--markers`` option always gives you a list of available markers:: @@ -265,6 +265,8 @@ Reading markers which were set from multiple places If you are heavily using markers in your test suite you may encounter the case where a marker is applied several times to a test function. From plugin code you can read over all such settings. Example:: +.. regendoc:wipe + # content of test_mark_three_times.py import pytest pytestmark = pytest.mark.glob("module", x=1) @@ -279,19 +281,95 @@ Here we have the marker "glob" applied three times to the same test function. From a conftest file we can read it like this:: # content of conftest.py + import sys def pytest_runtest_setup(item): g = getattr(item.obj, 'glob', None) if g is not None: for info in g: print ("glob args=%s kwargs=%s" %(info.args, info.kwargs)) + sys.stdout.flush() Let's run this without capturing output and see what we get:: $ py.test -q -s - collecting ... collected 2 items - .. - 2 passed in 0.01 seconds + collecting ... collected 1 items glob args=('function',) kwargs={'x': 3} glob args=('class',) kwargs={'x': 2} glob args=('module',) kwargs={'x': 1} + . + 1 passed in 0.01 seconds + +marking platform specific tests with pytest +-------------------------------------------------------------- + +Consider you have a test suite which marks tests for particular platforms, +namely ``pytest.mark.osx``, ``pytest.mark.win32`` etc. and you +also have tests that run on all platforms and have no specific +marker. If you now want to have a way to only run the tests +for your particular platform, you could use the following plugin:: + +.. regendoc:wipe + + # content of conftest.py + # + import sys + import pytest + + ALL = set("osx linux2 win32".split()) + + def pytest_runtest_setup(item): + if isinstance(item, item.Function): + plat = sys.platform + if not hasattr(item.obj, plat): + if ALL.intersection(set(item.obj.__dict__)): + pytest.skip("cannot run on platform %s" %(plat)) + +then tests will be skipped if they were specified for a different platform. +Let's do a little test file to show how this looks like:: + + # content of test_plat.py + + import pytest + + @pytest.mark.osx + def test_if_apple_is_evil(): + pass + + @pytest.mark.linux2 + def test_if_linux_works(): + pass + + @pytest.mark.win32 + def test_if_win32_crashes(): + pass + + def test_runs_everywhere(): + pass + +then you will see two test skipped and two executed tests as expected:: + + $ py.test -rs # this option reports skip reasons + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 + collecting ... collected 4 items + + test_plat.py s.s. + ========================= short test summary info ========================== + SKIP [2] /home/hpk/tmp/doc-exec-222/conftest.py:12: cannot run on platform linux2 + + =================== 2 passed, 2 skipped in 0.01 seconds ==================== + +Note that if you specify a platform via the marker-command line option like this:: + + $ py.test -m linux2 + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- pytest-2.2.5.dev1 + collecting ... collected 4 items + + test_plat.py . + + =================== 3 tests deselected by "-m 'linux2'" ==================== + ================== 1 passed, 3 deselected in 0.01 seconds ================== + +then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests.