floraachy/pytest2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[svn r64029] * move test docs into their own subdir

Browse Source 
* don't use sphinx for now, put the support code to attic
* remove some old docs

--HG--
branch : trunk
This commit is contained in:
hpk
2009-04-13 15:58:26 +02:00
parent 0324f84a78
commit 236f84d6d4
26 changed files with 47 additions and 1609 deletions
Download Patch File Download Diff File Expand all files Collapse all files

293
doc/test/features.txt Normal file
View File

@@ -0,0 +1,293 @@
==================================================
py.test Features
==================================================
py.test is an extensible tool for running all kinds
of tests one one or more machines. It supports a variety
of testing methods for your Python application and modules,
including unit, functional, integration and doc-testing.
It is used in projects that run more than 10000 tests
daily as well as single-python-module projects.
py.test presents a clean and powerful command line interface
and strives to generally make testing a fun effort.
py.test 1.0 works across linux, windows and osx
and on Python 2.3 - Python 2.6.
More detailed feature list:
.. contents::
:depth: 1
automatically collects and executes tests
===============================================
py.test discovers tests automatically by inspect specified
directories or files. By default, it collects all python
modules a leading ``test_`` or trailing ``_test`` filename.
From each test module every function with a leading ``test_``
or class with a leading ``Test`` name is collected.
.. _`generative tests`:
.. _`collection process`: ext.html#collection-process
Rapidly write integration, functional, unit tests
===================================================
py.test provides
load-balance tests to multiple CPUs
===================================
For large test suites you can distribute your
tests to multiple CPUs by issuing for example::
py.test -n 3
Read more on `distributed testing`_.
.. _`distributed testing`: dist.html
Distribute tests across machines
===================================
py.test supports the sending of tests to
remote ssh-accounts or socket servers.
It can `ad-hoc run your test on multiple
platforms one a single test run`. Ad-hoc
means that there are **no installation
requirements whatsoever** on the remote side.
.. _`ad-hoc run your test on multiple platforms one a single test run`: dist.html#atonce
extensive debugging support
===================================
testing starts immediately
--------------------------
Testing starts as soon as the first ``test item``
is collected. The collection process is iterative
and does not need to complete before your first
test items are executed.
support for modules containing tests
--------------------------------------
As ``py.test`` operates as a separate cmdline
tool you can easily have a command line utility and
some tests in the same file.
debug with the ``print`` statement
----------------------------------
By default, ``py.test`` catches text written to stdout/stderr during
the execution of each individual test. This output will only be
displayed however if the test fails; you will not see it
otherwise. This allows you to put debugging print statements in your
code without being overwhelmed by all the output that might be
generated by tests that do not fail.
Each failing test that produced output during the running of the test
will have its output displayed in the ``recorded stdout`` section.
The catching of stdout/stderr output can be disabled using the
``--nocapture`` option to the ``py.test`` tool. Any output will
in this case be displayed as soon as it is generated.
test execution order
--------------------------------
Tests usually run in the order in which they appear in the files.
However, tests should not rely on running one after another, as
this prevents more advanced usages: running tests
distributedly or selectively, or in "looponfailing" mode,
will cause them to run in random order.
assert with the ``assert`` statement
----------------------------------------
``py.test`` allows to use the standard python
``assert statement`` for verifying expectations
and values in Python tests. For example, you can
write the following in your tests::
assert hasattr(x, 'attribute')
to state that your object has a certain ``attribute``. In case this
assertion fails you will see the value of ``x``. Intermediate
values are computed by executing the assert expression a second time.
If you execute code with side effects, e.g. read from a file like this::
assert f.read() != '...'
then you may get a warning from pytest if that assertions
first failed and then succeeded.
asserting expected exceptions
----------------------------------------
In order to write assertions about exceptions, you use
one of two forms::
py.test.raises(Exception, func, *args, **kwargs)
py.test.raises(Exception, "func(*args, **kwargs)")
both of which execute the given function with args and kwargs and
asserts that the given ``Exception`` is raised. The reporter will
provide you with helpful output in case of failures such as *no
exception* or *wrong exception*.
useful tracebacks, recursion detection
--------------------------------------
A lot of care is taken to present nice tracebacks in case of test
failure. Try::
py.test py/doc/example/pytest/failure_demo.py
to see a variety of tracebacks, each representing a different
failure situation.
``py.test`` uses the same order for presenting tracebacks as Python
itself: the oldest function call is shown first, and the most recent call is
shown last. A ``py.test`` reported traceback starts with your
failing test function. If the maximum recursion depth has been
exceeded during the running of a test, for instance because of
infinite recursion, ``py.test`` will indicate where in the
code the recursion was taking place. You can inhibit
traceback "cutting" magic by supplying ``--fulltrace``.
There is also the possibility of using ``--tb=short`` to get regular CPython
tracebacks. Or you can use ``--tb=no`` to not show any tracebacks at all.
no inheritance requirement
--------------------------
Test classes are recognized by their leading ``Test`` name. Unlike
``unitest.py``, you don't need to inherit from some base class to make
them be found by the test runner. Besides being easier, it also allows
you to write test classes that subclass from application level
classes.
testing for deprecated APIs
------------------------------
In your tests you can use ``py.test.deprecated_call(func, *args, **kwargs)``
to test that a particular function call triggers a DeprecationWarning.
This is useful for testing phasing out of old APIs in your projects.
advanced test selection / skipping
=========================================================
dynamically skipping tests
-------------------------------
If you want to skip tests you can use ``py.test.skip`` within
test or setup functions. Example::
py.test.skip("message")
You can also use a helper to skip on a failing import::
docutils = py.test.importorskip("docutils")
or to skip if a library does not have the right version::
docutils = py.test.importorskip("docutils", minversion="0.3")
The version will be read from the module's ``__version__`` attribute.
.. _`selection by keyword`:
selecting/unselecting tests by keyword
---------------------------------------------
Pytest's keyword mechanism provides a powerful way to
group and selectively run tests in your test code base.
You can selectively run tests by specifiying a keyword
on the command line. Examples::
py.test -k test_simple
py.test -k "-test_simple"
will run all tests matching (or not matching) the
"test_simple" keyword. Note that you need to quote
the keyword if "-" is recognized as an indicator
for a commandline option. Lastly, you may use::
py.test. -k "test_simple:"
which will run all tests after the expression has *matched once*, i.e.
all tests that are seen after a test that matches the "test_simple"
keyword.
By default, all filename parts and
class/function names of a test function are put into the set
of keywords for a given test. You may specify additional
kewords like this::
@py.test.mark(webtest=True)
def test_send_http():
...
disabling a test class
----------------------
If you want to disable a complete test class you
can set the class-level attribute ``disabled``.
For example, in order to avoid running some tests on Win32::
class TestPosixOnly:
disabled = sys.platform == 'win32'
def test_xxx(self):
...
generative tests: yielding parametrized tests
====================================================
*Generative tests* are test methods that are *generator functions* which
``yield`` callables and their arguments. This is most useful for running a
test function multiple times against different parameters. Example::
def test_generative():
for x in (42,17,49):
yield check, x
def check(arg):
assert arg % 7 == 0 # second generated tests fails!
Note that ``test_generative()`` will cause three tests
to get run, notably ``check(42)``, ``check(17)`` and ``check(49)``
of which the middle one will obviously fail.
To make it easier to distinguish the generated tests it is possible to specify an explicit name for them, like for example::
def test_generative():
for x in (42,17,49):
yield "case %d" % x, check, x
extensible plugin system
=========================================
py.test itself consists of many plugins
and you can easily write new `py.test plugins`_
for these purposes:
* reporting extensions
* customizing collection and run of tests
* running non-python tests
* managing test state setup
.. _`py.test plugins`: plugins.html
.. _`reStructured Text`: http://docutils.sourceforge.net
.. _`Python debugger`: http://docs.python.org/lib/module-pdb.html
.. _nose: http://somethingaboutorange.com/mrl/projects/nose/