generalize skipping

- rename pytest_xfail to pytest_skip
- dynamic "skipif" and "xfail" decorators
- move most skipping code to the plugin

also coming with this commit:
- extend mark keyword to accept positional args + docs
- fix a few documentation related issues
- leave version as "trunk" for now

--HG--
branch : trunk
This commit is contained in:
holger krekel
2009-10-15 16:18:57 +02:00
parent 5e21e39125
commit 3ca770b420
28 changed files with 532 additions and 241 deletions

View File

@@ -1,6 +1,11 @@
Changes between 1.0.2 and '1.1.0b1'
=====================================
* generalized skipping: a new way to mark python functions with skipif or xfail
at function, class and modules level based on platform or sys-module attributes.
* extend py.test.mark decorator to allow for positional args
* introduce and test "py.cleanup -d" to remove empty directories
* fix issue #59 - robustify unittest test collection

View File

@@ -1,5 +1,5 @@
import py
from py.__.rest.resthtml import convert_rest_html, strip_html_header
from _py.rest.resthtml import convert_rest_html, strip_html_header
html = py.xml.html

View File

@@ -125,22 +125,11 @@ a PDB `Python debugger`_ when a test fails.
advanced skipping of tests
-------------------------------
If you want to skip tests you can use ``py.test.skip`` within
test or setup functions. Example::
py.test has builtin support for skipping tests or expecting
failures on tests on certain platforms. Apart from the
minimal py.test style also unittest- and nose-style tests
can make use of this feature.
def test_hello():
if sys.platform != "win32":
py.test.skip("only win32 supported")
You can also use a helper to skip on a failing import::
docutils = py.test.importorskip("docutils")
or to skip if a library does not have the right version::
docutils = py.test.importorskip("docutils", minversion="0.3")
The version will be read from the specified module's ``__version__`` attribute.
.. _`funcargs mechanism`: funcargs.html
.. _`unittest.py`: http://docs.python.org/library/unittest.html

View File

@@ -276,6 +276,7 @@ methods in a convenient way.
.. _`conftest plugin`: customize.html#conftestplugin
.. _`funcarg factory`:
.. _factory:
funcarg factories: setting up test function arguments
==============================================================

View File

@@ -139,6 +139,15 @@ hook specification sourcecode
# distributed testing
# -------------------------------------------------------------------------
def pytest_gwmanage_newgateway(gateway, platinfo):
""" called on new raw gateway creation. """
def pytest_gwmanage_rsyncstart(source, gateways):
""" called before rsyncing a directory to remote gateways takes place. """
def pytest_gwmanage_rsyncfinish(source, gateways):
""" called after rsyncing a directory to remote gateways takes place. """
def pytest_testnodeready(node):
""" Test Node is ready to operate. """

View File

@@ -2,7 +2,7 @@
plugins for Python test functions
=================================
xfail_ mark python test functions as expected-to-fail and report them separately.
skipping_ mark python test functions, classes or modules for conditional
figleaf_ write and report coverage data with 'figleaf'.

View File

@@ -14,22 +14,29 @@ By default, all filename parts and class/function names of a test
function are put into the set of keywords for a given test. You can
specify additional kewords like this::
@py.test.mark.webtest
@py.test.mark.webtest
def test_send_http():
...
This will set an attribute 'webtest' on the given test function
and by default all such attributes signal keywords. You can
also set values in this attribute which you could read from
a hook in order to do something special with respect to
the test function::
This will set an attribute 'webtest' to True on the given test function.
You can read the value 'webtest' from the functions __dict__ later.
@py.test.mark.timeout(seconds=5)
You can also set values for an attribute which are put on an empty
dummy object::
@py.test.mark.webtest(firefox=30)
def test_receive():
...
This will set the "timeout" attribute with a Marker object
that has a 'seconds' attribute.
after which ``test_receive.webtest.firefox == 30`` holds true.
In addition to keyword arguments you can also use positional arguments::
@py.test.mark.webtest("triangular")
def test_receive():
...
after which ``test_receive.webtest._1 == 'triangular`` hold true.
Start improving this plugin in 30 seconds
=========================================

View File

@@ -5,22 +5,23 @@
.. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_monkeypatch.py
.. _`pytest_keyword.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_keyword.py
.. _`pastebin`: pastebin.html
.. _`skipping`: skipping.html
.. _`plugins`: index.html
.. _`pytest_capture.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_capture.py
.. _`pytest_doctest.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_doctest.py
.. _`capture`: capture.html
.. _`pytest_nose.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_nose.py
.. _`pytest_restdoc.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_restdoc.py
.. _`xfail`: xfail.html
.. _`restdoc`: restdoc.html
.. _`pytest_pastebin.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_pastebin.py
.. _`pytest_figleaf.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_figleaf.py
.. _`pytest_hooklog.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_hooklog.py
.. _`pytest_skipping.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_skipping.py
.. _`checkout the py.test development version`: ../../download.html#checkout
.. _`pytest_helpconfig.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_helpconfig.py
.. _`oejskit`: oejskit.html
.. _`doctest`: doctest.html
.. _`get in contact`: ../../contact.html
.. _`pytest_xfail.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_xfail.py
.. _`pytest_capture.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_capture.py
.. _`figleaf`: figleaf.html
.. _`customize`: ../customize.html
.. _`hooklog`: hooklog.html
@@ -30,7 +31,6 @@
.. _`monkeypatch`: monkeypatch.html
.. _`resultlog`: resultlog.html
.. _`keyword`: keyword.html
.. _`restdoc`: restdoc.html
.. _`django`: django.html
.. _`pytest_unittest.py`: http://bitbucket.org/hpk42/py-trunk/raw/trunk/py/test/plugin/pytest_unittest.py
.. _`nose`: nose.html

View File

@@ -0,0 +1,115 @@
pytest_skipping plugin
======================
mark python test functions, classes or modules for conditional
.. contents::
:local:
skipping (skipif) or as expected-to-fail (xfail). Both declarations
lead to special reporting and both can be systematically associated
with functions, whole classes or modules. The difference between
the two is that 'xfail' will still execute test functions
but it will revert the outcome. A passing test is now
a failure and failing test is expected. All skip conditions
are reported at the end of test run through the terminal
reporter.
.. _skipif:
skip a test function conditionally
-------------------------------------------
Here is an example for skipping a test function on Python3::
@py.test.mark.skipif("sys.version_info >= (3,0)")
def test_function():
...
Conditions are specified as python expressions
and can access the ``sys`` module. They can also
access the config object and thus depend on command
line or conftest options::
@py.test.mark.skipif("config.getvalue('db') is None")
def test_function(...):
...
conditionally mark a function as "expected to fail"
-------------------------------------------------------
You can use the ``xfail`` keyword to mark your test functions as
'expected to fail'::
@py.test.mark.xfail
def test_hello():
...
This test will be executed but no traceback will be reported
when it fails. Instead terminal reporting will list it in the
"expected to fail" or "unexpectedly passing" sections.
As with skipif_ you may selectively expect a failure
depending on platform::
@py.test.mark.xfail("sys.version_info >= (3,0)")
def test_function():
...
skip/xfail a whole test class or module
-------------------------------------------
Instead of marking single functions you can skip
a whole class of tests when runnign on a specific
platform::
class TestSomething:
skipif = "sys.platform == 'win32'"
Or you can mark all test functions as expected
to fail for a specific test configuration::
xfail = "config.getvalue('db') == 'mysql'"
skip if a dependency cannot be imported
---------------------------------------------
You can use a helper to skip on a failing import::
docutils = py.test.importorskip("docutils")
You can use this helper at module level or within
a test or setup function.
You can aslo skip if a library does not have the right version::
docutils = py.test.importorskip("docutils", minversion="0.3")
The version will be read from the specified module's ``__version__`` attribute.
dynamically skip from within a test or setup
-------------------------------------------------
If you want to skip the execution of a test you can call
``py.test.skip()`` within a test, a setup or from a
`funcarg factory`_ function. Example::
def test_function():
if not valid_config():
py.test.skip("unsuppored configuration")
.. _`funcarg factory`: ../funcargs.html#factory
Start improving this plugin in 30 seconds
=========================================
1. Download `pytest_skipping.py`_ plugin source code
2. put it somewhere as ``pytest_skipping.py`` into your import path
3. a subsequent ``py.test`` run will use your local version
Checkout customize_, other plugins_ or `get in contact`_.
.. include:: links.txt

View File

@@ -1,34 +0,0 @@
pytest_xfail plugin
===================
mark python test functions as expected-to-fail and report them separately.
.. contents::
:local:
usage
------------
Use the generic mark decorator to mark your test functions as
'expected to fail'::
@py.test.mark.xfail
def test_hello():
...
This test will be executed but no traceback will be reported
when it fails. Instead terminal reporting will list it in the
"expected to fail" section or "unexpectedly passing" section.
Start improving this plugin in 30 seconds
=========================================
1. Download `pytest_xfail.py`_ plugin source code
2. put it somewhere as ``pytest_xfail.py`` into your import path
3. a subsequent ``py.test`` run will use your local version
Checkout customize_, other plugins_ or `get in contact`_.
.. include:: links.txt