generalize skipping
- rename pytest_xfail to pytest_skip - dynamic "skipif" and "xfail" decorators - move most skipping code to the plugin also coming with this commit: - extend mark keyword to accept positional args + docs - fix a few documentation related issues - leave version as "trunk" for now --HG-- branch : trunk
This commit is contained in:
@@ -1,6 +1,11 @@
|
||||
Changes between 1.0.2 and '1.1.0b1'
|
||||
=====================================
|
||||
|
||||
* generalized skipping: a new way to mark python functions with skipif or xfail
|
||||
at function, class and modules level based on platform or sys-module attributes.
|
||||
|
||||
* extend py.test.mark decorator to allow for positional args
|
||||
|
||||
* introduce and test "py.cleanup -d" to remove empty directories
|
||||
|
||||
* fix issue #59 - robustify unittest test collection
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import py
|
||||
from py.__.rest.resthtml import convert_rest_html, strip_html_header
|
||||
from _py.rest.resthtml import convert_rest_html, strip_html_header
|
||||
|
||||
html = py.xml.html
|
||||
|
||||
|
||||
@@ -125,22 +125,11 @@ a PDB `Python debugger`_ when a test fails.
|
||||
advanced skipping of tests
|
||||
-------------------------------
|
||||
|
||||
If you want to skip tests you can use ``py.test.skip`` within
|
||||
test or setup functions. Example::
|
||||
py.test has builtin support for skipping tests or expecting
|
||||
failures on tests on certain platforms. Apart from the
|
||||
minimal py.test style also unittest- and nose-style tests
|
||||
can make use of this feature.
|
||||
|
||||
def test_hello():
|
||||
if sys.platform != "win32":
|
||||
py.test.skip("only win32 supported")
|
||||
|
||||
You can also use a helper to skip on a failing import::
|
||||
|
||||
docutils = py.test.importorskip("docutils")
|
||||
|
||||
or to skip if a library does not have the right version::
|
||||
|
||||
docutils = py.test.importorskip("docutils", minversion="0.3")
|
||||
|
||||
The version will be read from the specified module's ``__version__`` attribute.
|
||||
|
||||
.. _`funcargs mechanism`: funcargs.html
|
||||
.. _`unittest.py`: http://docs.python.org/library/unittest.html
|
||||
|
||||
@@ -276,6 +276,7 @@ methods in a convenient way.
|
||||
.. _`conftest plugin`: customize.html#conftestplugin
|
||||
|
||||
.. _`funcarg factory`:
|
||||
.. _factory:
|
||||
|
||||
funcarg factories: setting up test function arguments
|
||||
==============================================================
|
||||
|
||||
@@ -139,6 +139,15 @@ hook specification sourcecode
|
||||
# distributed testing
|
||||
# -------------------------------------------------------------------------
|
||||
|
||||
def pytest_gwmanage_newgateway(gateway, platinfo):
|
||||
""" called on new raw gateway creation. """
|
||||
|
||||
def pytest_gwmanage_rsyncstart(source, gateways):
|
||||
""" called before rsyncing a directory to remote gateways takes place. """
|
||||
|
||||
def pytest_gwmanage_rsyncfinish(source, gateways):
|
||||
""" called after rsyncing a directory to remote gateways takes place. """
|
||||
|
||||
def pytest_testnodeready(node):
|
||||
""" Test Node is ready to operate. """
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
plugins for Python test functions
|
||||
=================================
|
||||
|
||||
xfail_ mark python test functions as expected-to-fail and report them separately.
|
||||
skipping_ mark python test functions, classes or modules for conditional
|
||||
|
||||
figleaf_ write and report coverage data with 'figleaf'.
|
||||
|
||||
|
||||
@@ -14,22 +14,29 @@ By default, all filename parts and class/function names of a test
|
||||
function are put into the set of keywords for a given test. You can
|
||||
specify additional kewords like this::
|
||||
|
||||
@py.test.mark.webtest
|
||||
@py.test.mark.webtest
|
||||
def test_send_http():
|
||||
...
|
||||
|
||||
This will set an attribute 'webtest' on the given test function
|
||||
and by default all such attributes signal keywords. You can
|
||||
also set values in this attribute which you could read from
|
||||
a hook in order to do something special with respect to
|
||||
the test function::
|
||||
This will set an attribute 'webtest' to True on the given test function.
|
||||
You can read the value 'webtest' from the functions __dict__ later.
|
||||
|
||||
@py.test.mark.timeout(seconds=5)
|
||||
You can also set values for an attribute which are put on an empty
|
||||
dummy object::
|
||||
|
||||
@py.test.mark.webtest(firefox=30)
|
||||
def test_receive():
|
||||
...
|
||||
|
||||
This will set the "timeout" attribute with a Marker object
|
||||
that has a 'seconds' attribute.
|
||||
after which ``test_receive.webtest.firefox == 30`` holds true.
|
||||
|
||||
In addition to keyword arguments you can also use positional arguments::
|
||||
|
||||
@py.test.mark.webtest("triangular")
|
||||
def test_receive():
|
||||
...
|
||||
|
||||
after which ``test_receive.webtest._1 == 'triangular`` hold true.
|
||||
|
||||
Start improving this plugin in 30 seconds
|
||||
=========================================
|
||||
|
||||
@@ -5,22 +5,23 @@
|
||||
.. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_monkeypatch.py
|
||||
.. _`pytest_keyword.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_keyword.py
|
||||
.. _`pastebin`: pastebin.html
|
||||
.. _`skipping`: skipping.html
|
||||
.. _`plugins`: index.html
|
||||
.. _`pytest_capture.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_capture.py
|
||||
.. _`pytest_doctest.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_doctest.py
|
||||
.. _`capture`: capture.html
|
||||
.. _`pytest_nose.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_nose.py
|
||||
.. _`pytest_restdoc.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_restdoc.py
|
||||
.. _`xfail`: xfail.html
|
||||
.. _`restdoc`: restdoc.html
|
||||
.. _`pytest_pastebin.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_pastebin.py
|
||||
.. _`pytest_figleaf.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_figleaf.py
|
||||
.. _`pytest_hooklog.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_hooklog.py
|
||||
.. _`pytest_skipping.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_skipping.py
|
||||
.. _`checkout the py.test development version`: ../../download.html#checkout
|
||||
.. _`pytest_helpconfig.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_helpconfig.py
|
||||
.. _`oejskit`: oejskit.html
|
||||
.. _`doctest`: doctest.html
|
||||
.. _`get in contact`: ../../contact.html
|
||||
.. _`pytest_xfail.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_xfail.py
|
||||
.. _`pytest_capture.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_capture.py
|
||||
.. _`figleaf`: figleaf.html
|
||||
.. _`customize`: ../customize.html
|
||||
.. _`hooklog`: hooklog.html
|
||||
@@ -30,7 +31,6 @@
|
||||
.. _`monkeypatch`: monkeypatch.html
|
||||
.. _`resultlog`: resultlog.html
|
||||
.. _`keyword`: keyword.html
|
||||
.. _`restdoc`: restdoc.html
|
||||
.. _`django`: django.html
|
||||
.. _`pytest_unittest.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_unittest.py
|
||||
.. _`nose`: nose.html
|
||||
|
||||
115
doc/test/plugin/skipping.txt
Normal file
115
doc/test/plugin/skipping.txt
Normal file
@@ -0,0 +1,115 @@
|
||||
|
||||
pytest_skipping plugin
|
||||
======================
|
||||
|
||||
mark python test functions, classes or modules for conditional
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
skipping (skipif) or as expected-to-fail (xfail). Both declarations
|
||||
lead to special reporting and both can be systematically associated
|
||||
with functions, whole classes or modules. The difference between
|
||||
the two is that 'xfail' will still execute test functions
|
||||
but it will revert the outcome. A passing test is now
|
||||
a failure and failing test is expected. All skip conditions
|
||||
are reported at the end of test run through the terminal
|
||||
reporter.
|
||||
|
||||
.. _skipif:
|
||||
|
||||
skip a test function conditionally
|
||||
-------------------------------------------
|
||||
|
||||
Here is an example for skipping a test function on Python3::
|
||||
|
||||
@py.test.mark.skipif("sys.version_info >= (3,0)")
|
||||
def test_function():
|
||||
...
|
||||
|
||||
Conditions are specified as python expressions
|
||||
and can access the ``sys`` module. They can also
|
||||
access the config object and thus depend on command
|
||||
line or conftest options::
|
||||
|
||||
@py.test.mark.skipif("config.getvalue('db') is None")
|
||||
def test_function(...):
|
||||
...
|
||||
|
||||
conditionally mark a function as "expected to fail"
|
||||
-------------------------------------------------------
|
||||
|
||||
You can use the ``xfail`` keyword to mark your test functions as
|
||||
'expected to fail'::
|
||||
|
||||
@py.test.mark.xfail
|
||||
def test_hello():
|
||||
...
|
||||
|
||||
This test will be executed but no traceback will be reported
|
||||
when it fails. Instead terminal reporting will list it in the
|
||||
"expected to fail" or "unexpectedly passing" sections.
|
||||
As with skipif_ you may selectively expect a failure
|
||||
depending on platform::
|
||||
|
||||
@py.test.mark.xfail("sys.version_info >= (3,0)")
|
||||
def test_function():
|
||||
...
|
||||
|
||||
skip/xfail a whole test class or module
|
||||
-------------------------------------------
|
||||
|
||||
Instead of marking single functions you can skip
|
||||
a whole class of tests when runnign on a specific
|
||||
platform::
|
||||
|
||||
class TestSomething:
|
||||
skipif = "sys.platform == 'win32'"
|
||||
|
||||
Or you can mark all test functions as expected
|
||||
to fail for a specific test configuration::
|
||||
|
||||
xfail = "config.getvalue('db') == 'mysql'"
|
||||
|
||||
|
||||
skip if a dependency cannot be imported
|
||||
---------------------------------------------
|
||||
|
||||
You can use a helper to skip on a failing import::
|
||||
|
||||
docutils = py.test.importorskip("docutils")
|
||||
|
||||
You can use this helper at module level or within
|
||||
a test or setup function.
|
||||
|
||||
You can aslo skip if a library does not have the right version::
|
||||
|
||||
docutils = py.test.importorskip("docutils", minversion="0.3")
|
||||
|
||||
The version will be read from the specified module's ``__version__`` attribute.
|
||||
|
||||
|
||||
dynamically skip from within a test or setup
|
||||
-------------------------------------------------
|
||||
|
||||
If you want to skip the execution of a test you can call
|
||||
``py.test.skip()`` within a test, a setup or from a
|
||||
`funcarg factory`_ function. Example::
|
||||
|
||||
def test_function():
|
||||
if not valid_config():
|
||||
py.test.skip("unsuppored configuration")
|
||||
|
||||
.. _`funcarg factory`: ../funcargs.html#factory
|
||||
|
||||
Start improving this plugin in 30 seconds
|
||||
=========================================
|
||||
|
||||
|
||||
1. Download `pytest_skipping.py`_ plugin source code
|
||||
2. put it somewhere as ``pytest_skipping.py`` into your import path
|
||||
3. a subsequent ``py.test`` run will use your local version
|
||||
|
||||
Checkout customize_, other plugins_ or `get in contact`_.
|
||||
|
||||
.. include:: links.txt
|
||||
@@ -1,34 +0,0 @@
|
||||
|
||||
pytest_xfail plugin
|
||||
===================
|
||||
|
||||
mark python test functions as expected-to-fail and report them separately.
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
usage
|
||||
------------
|
||||
|
||||
Use the generic mark decorator to mark your test functions as
|
||||
'expected to fail'::
|
||||
|
||||
@py.test.mark.xfail
|
||||
def test_hello():
|
||||
...
|
||||
|
||||
This test will be executed but no traceback will be reported
|
||||
when it fails. Instead terminal reporting will list it in the
|
||||
"expected to fail" section or "unexpectedly passing" section.
|
||||
|
||||
Start improving this plugin in 30 seconds
|
||||
=========================================
|
||||
|
||||
|
||||
1. Download `pytest_xfail.py`_ plugin source code
|
||||
2. put it somewhere as ``pytest_xfail.py`` into your import path
|
||||
3. a subsequent ``py.test`` run will use your local version
|
||||
|
||||
Checkout customize_, other plugins_ or `get in contact`_.
|
||||
|
||||
.. include:: links.txt
|
||||
Reference in New Issue
Block a user