floraachy
/
pytest2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[svn r63222] refactor test-features doc, make references to others 

--HG--
branch : trunk
This commit is contained in:
hpk 2009-03-23 11:01:15 +01:00
parent af39c9850e
commit c4fe18608d
3 changed files with 174 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
py/doc/download.txt
View File

@ -1,5 +1,3 @@
"easy_install py"
===================================================

2
py/doc/test-dist.txt
View File

@ -74,6 +74,8 @@ new socket host with something like this::
py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
.. _`atonce`:
Running tests on many platforms at once
-------------------------------------------------------------

305
py/doc/test-features.txt
View File

@ -1,137 +1,59 @@
automatic collection of tests on all levels
-------------------------------------------
.. contents:: Basic features
:depth: 1
The automated test collection process walks the current
directory (or the directory given as a command line argument)
and all its subdirectories and collects python modules with a
leading ``test_`` or trailing ``_test`` filename. From each
test module every function with a leading ``test_`` or class with
a leading ``Test`` name is collected. The collecting process can
be customized at directory, module or class level. (see
`collection process`_ for some implementation details).
py.test: cross-project general testing tool
==================================================
py.test is a standalone-tool that collects and runs tests for
your Python application and modules. py.test works across
linux, windows and osx and on Python 2.3 - Python 2.6.
It aims to support *unit-tests* and *functional tests* written
in Python and is used in projects that run more than 10000
tests regularly.
py.test presents a clean and powerful command line interface
and strives to generally make testing a fun effort.
automatically collects and executes tests
===============================================
py.test discovers tests automatically by inspect specified
directories or files. By default, it collects all python
modules a leading ``test_`` or trailing ``_test`` filename.
From each test module every function with a leading ``test_``
or class with a leading ``Test`` name is collected.
.. _`generative tests`:
.. _`collection process`: impl-test.html#collection-process
assert with the ``assert`` statement
------------------------------------
load-balance tests to multiple CPUs
===================================
``py.test`` allows to use the standard python
``assert statement`` for verifying expectations
and values in Python tests. For example, you can
write the following in your tests::
For large test suites you can distribute your
tests to multiple CPUs by issuing for example::
assert hasattr(x, 'attribute')
py.test -n 3
to state that your object has a certain ``attribute``. In case this
assertion fails you will see the value of ``x``. Intermediate
values are computed by executing the assert expression a second time.
If you execute code with side effects, e.g. read from a file like this::
Read more on `distributed testing`_.
assert f.read() != '...'
.. _`distributed testing`: test-dist.html
then you may get a warning from pytest if that assertions
first failed and then succeeded.
Distribute tests across machines
===================================
asserting expected exceptions
----------------------------------------------
py.test supports the sending of tests to
remote ssh-accounts or socket servers.
It can `ad-hoc run your test on multiple
platforms one a single test run`. Ad-hoc
means that there are **no installation
requirements whatsoever** on the remote side.
In order to write assertions about exceptions, you use
one of two forms::
py.test.raises(Exception, func, *args, **kwargs)
py.test.raises(Exception, "func(*args, **kwargs)")
both of which execute the given function with args and kwargs and
asserts that the given ``Exception`` is raised. The reporter will
provide you with helpful output in case of failures such as *no
exception* or *wrong exception*.
dynamically skipping tests
----------------------------------------
If you want to skip tests you can use ``py.test.skip`` within
test or setup functions. Example::
py.test.skip("message")
You can also use a helper to skip on a failing import::
docutils = py.test.importorskip("docutils")
or to skip if a library does not have the right version::
docutils = py.test.importorskip("docutils", minversion="0.3")
The version will be read from the module's ``__version__`` attribute.
generative tests: yielding more tests
-------------------------------------
*Generative tests* are test methods that are *generator functions* which
``yield`` callables and their arguments. This is most useful for running a
test function multiple times against different parameters. Example::
def test_generative():
for x in (42,17,49):
yield check, x
def check(arg):
assert arg % 7 == 0 # second generated tests fails!
Note that ``test_generative()`` will cause three tests
to get run, notably ``check(42)``, ``check(17)`` and ``check(49)``
of which the middle one will obviously fail.
To make it easier to distinguish the generated tests it is possible to specify an explicit name for them, like for example::
def test_generative():
for x in (42,17,49):
yield "case %d" % x, check, x
.. _`selection by keyword`:
selecting/unselecting tests by keyword
---------------------------------------------
Pytest's keyword mechanism provides a powerful way to
group and selectively run tests in your test code base.
You can selectively run tests by specifiying a keyword
on the command line. Examples:
py.test -k test_simple
py.test -k "-test_simple"
will run all tests matching (or not matching) the
"test_simple" keyword. Note that you need to quote
the keyword if "-" is recognized as an indicator
for a commandline option. Lastly, you may use
py.test. -k "test_simple:"
which will run all tests after the expression has *matched once*, i.e.
all tests that are seen after a test that matches the "test_simple"
keyword.
By default, all filename parts and
class/function names of a test function are put into the set
of keywords for a given test. You may specify additional
kewords like this::
@py.test.mark(webtest=True)
def test_send_http():
...
testing with multiple python versions / executables
---------------------------------------------------
With ``--tx EXECUTABLE`` you can specify a python
executable (e.g. ``python2.2``) with which the tests
will be executed.
.. _`ad-hoc run your test on multiple platforms one a single test run`: test-dist.html#atonce
extensive debugging support
===================================
testing starts immediately
--------------------------
@ -174,6 +96,40 @@ this prevents more advanced usages: running tests
distributedly or selectively, or in "looponfailing" mode,
will cause them to run in random order.
assert with the ``assert`` statement
----------------------------------------
``py.test`` allows to use the standard python
``assert statement`` for verifying expectations
and values in Python tests. For example, you can
write the following in your tests::
assert hasattr(x, 'attribute')
to state that your object has a certain ``attribute``. In case this
assertion fails you will see the value of ``x``. Intermediate
values are computed by executing the assert expression a second time.
If you execute code with side effects, e.g. read from a file like this::
assert f.read() != '...'
then you may get a warning from pytest if that assertions
first failed and then succeeded.
asserting expected exceptions
----------------------------------------
In order to write assertions about exceptions, you use
one of two forms::
py.test.raises(Exception, func, *args, **kwargs)
py.test.raises(Exception, "func(*args, **kwargs)")
both of which execute the given function with args and kwargs and
asserts that the given ``Exception`` is raised. The reporter will
provide you with helpful output in case of failures such as *no
exception* or *wrong exception*.
useful tracebacks, recursion detection
--------------------------------------
@ -182,7 +138,7 @@ failure. Try::
py.test py/doc/example/pytest/failure_demo.py
to see a variety of 17 tracebacks, each tailored to a different
to see a variety of tracebacks, each representing a different
failure situation.
``py.test`` uses the same order for presenting tracebacks as Python
@ -206,6 +162,68 @@ them be found by the test runner. Besides being easier, it also allows
you to write test classes that subclass from application level
classes.
testing for deprecated APIs
------------------------------
In your tests you can use ``py.test.deprecated_call(func, *args, **kwargs)``
to test that a particular function call triggers a DeprecationWarning.
This is useful for testing phasing out of old APIs in your projects.
advanced test selection / skipping
=========================================================
dynamically skipping tests
-------------------------------
If you want to skip tests you can use ``py.test.skip`` within
test or setup functions. Example::
py.test.skip("message")
You can also use a helper to skip on a failing import::
docutils = py.test.importorskip("docutils")
or to skip if a library does not have the right version::
docutils = py.test.importorskip("docutils", minversion="0.3")
The version will be read from the module's ``__version__`` attribute.
.. _`selection by keyword`:
selecting/unselecting tests by keyword
---------------------------------------------
Pytest's keyword mechanism provides a powerful way to
group and selectively run tests in your test code base.
You can selectively run tests by specifiying a keyword
on the command line. Examples::
py.test -k test_simple
py.test -k "-test_simple"
will run all tests matching (or not matching) the
"test_simple" keyword. Note that you need to quote
the keyword if "-" is recognized as an indicator
for a commandline option. Lastly, you may use::
py.test. -k "test_simple:"
which will run all tests after the expression has *matched once*, i.e.
all tests that are seen after a test that matches the "test_simple"
keyword.
By default, all filename parts and
class/function names of a test function are put into the set
of keywords for a given test. You may specify additional
kewords like this::
@py.test.mark(webtest=True)
def test_send_http():
...
disabling a test class
----------------------
@ -219,22 +237,43 @@ For example, in order to avoid running some tests on Win32::
def test_xxx(self):
...
testing for deprecated APIs
------------------------------
generative tests: yielding parametrized tests
====================================================
In your tests you can use ``py.test.deprecated_call(func, *args, **kwargs)``
to test that a particular function call triggers a DeprecationWarning.
This is useful for testing phasing out of old APIs in your projects.
*Generative tests* are test methods that are *generator functions* which
``yield`` callables and their arguments. This is most useful for running a
test function multiple times against different parameters. Example::
doctest support
-------------------
def test_generative():
for x in (42,17,49):
yield check, x
def check(arg):
assert arg % 7 == 0 # second generated tests fails!
If you want to integrate doctests, ``py.test`` now by default
picks up files matching the ``test_*.txt`` or ``*_test.txt``
patterns and processes them as text files containing doctests.
This is an experimental feature and likely to change
its implementation.
Note that ``test_generative()`` will cause three tests
to get run, notably ``check(42)``, ``check(17)`` and ``check(49)``
of which the middle one will obviously fail.
To make it easier to distinguish the generated tests it is possible to specify an explicit name for them, like for example::
def test_generative():
for x in (42,17,49):
yield "case %d" % x, check, x
extensible plugin system
=========================================
py.test itself consists of many plugins
and you can easily write new `py.test plugins`_
for these purposes:
* reporting extensions
* customizing collection and run of tests
* running non-python tests
* managing test state setup
.. _`py.test plugins`: test-plugins.html
.. _`reStructured Text`: http://docutils.sourceforge.net
.. _`Python debugger`: http://docs.python.org/lib/module-pdb.html