remove dist-testing and looponfail code from core. there remain some (pytest_runner particularly) tests that test both plain and dist modes which cannot be easily dis-entangled. food for thought.
--HG-- branch : trunk
This commit is contained in:
holger krekel
18
doc/test/dist.html
Normal file
18
doc/test/dist.html
Normal file
@@ -0,0 +1,18 @@
|
||||
<html>
|
||||
<head>
|
||||
<meta http-equiv="refresh" content=" 1 ; URL=plugin/xdist.html" />
|
||||
</head>
|
||||
|
||||
<body>
|
||||
<script type="text/javascript">
|
||||
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
|
||||
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
|
||||
</script>
|
||||
<script type="text/javascript">
|
||||
try {
|
||||
var pageTracker = _gat._getTracker("UA-7597274-3");
|
||||
pageTracker._trackPageview();
|
||||
} catch(err) {}</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -13,8 +13,6 @@ funcargs_: powerful parametrized test function setup
|
||||
|
||||
`plugins`_: list of available plugins with usage examples and feature details.
|
||||
|
||||
`distributed testing`_: ad-hoc run tests on multiple CPUs and platforms
|
||||
|
||||
customize_: configuration, customization, extensions
|
||||
|
||||
changelog_: history of changes covering last releases
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_capture plugin
|
||||
=====================
|
||||
|
||||
configurable per-test stdout/stderr capturing mechanisms.
|
||||
=========================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_doctest plugin
|
||||
=====================
|
||||
|
||||
collect and execute doctests from modules and test files.
|
||||
=========================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,17 +1,41 @@
|
||||
pytest_figleaf plugin
|
||||
=====================
|
||||
|
||||
add options to drive and report python test coverage using the 'figleaf' package.
|
||||
report test coverage using the 'figleaf' package.
|
||||
=================================================
|
||||
|
||||
Install the `pytest-figleaf`_ plugin to use figleaf coverage testing::
|
||||
|
||||
easy_install pytest-figleaf
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
or::
|
||||
|
||||
pip install pytest-figleaf
|
||||
Usage
|
||||
---------------
|
||||
|
||||
This will make py.test have figleaf related options.
|
||||
after pip or easy_install mediated installation of ``pytest-figleaf`` you can type::
|
||||
|
||||
.. _`pytest-figleaf`: http://bitbucket.org/hpk42/pytest-figleaf/
|
||||
py.test --figleaf [...]
|
||||
|
||||
to enable figleaf coverage in your test run. A default ".figleaf" data file
|
||||
and "html" directory will be created. You can use ``--fig-data``
|
||||
and ``fig-html`` to modify the paths.
|
||||
|
||||
command line options
|
||||
--------------------
|
||||
|
||||
|
||||
``--figleaf``
|
||||
trace python coverage with figleaf and write HTML for files below the current working dir
|
||||
``--fig-data=dir``
|
||||
set tracing file, default: ".figleaf".
|
||||
``--fig-html=dir``
|
||||
set html reporting dir, default "html".
|
||||
|
||||
Start improving this plugin in 30 seconds
|
||||
=========================================
|
||||
|
||||
|
||||
1. Download `pytest_figleaf.py`_ plugin source code
|
||||
2. put it somewhere as ``pytest_figleaf.py`` into your import path
|
||||
3. a subsequent ``py.test`` run will use your local version
|
||||
|
||||
Checkout customize_, other plugins_ or `get in contact`_.
|
||||
|
||||
.. include:: links.txt
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_genscript plugin
|
||||
=======================
|
||||
|
||||
generate standalone test script to be distributed along with an application.
|
||||
============================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_helpconfig plugin
|
||||
========================
|
||||
|
||||
provide version info, conftest/environment config names.
|
||||
========================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_hooklog plugin
|
||||
=====================
|
||||
|
||||
log invocations of extension hooks to a file.
|
||||
=============================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -8,7 +8,7 @@ mark_ generic mechanism for marking python functions.
|
||||
|
||||
pdb_ interactive debugging with the Python Debugger.
|
||||
|
||||
figleaf_ (external) for testing with Titus' figleaf coverage module
|
||||
figleaf_ report test coverage using the 'figleaf' package.
|
||||
|
||||
coverage_ (external) for testing with Ned's coverage module
|
||||
|
||||
@@ -21,13 +21,15 @@ recwarn_ helpers for asserting deprecation and other warnings.
|
||||
tmpdir_ provide temporary directories to test functions.
|
||||
|
||||
|
||||
testing domains
|
||||
===============
|
||||
other testing domains, misc
|
||||
===========================
|
||||
|
||||
oejskit_ (external) run javascript tests in real life browsers
|
||||
|
||||
django_ (external) for testing django applications
|
||||
|
||||
xdist_ loop on failing tests, distribute test runs to CPUs and hosts.
|
||||
|
||||
genscript_ generate standalone test script to be distributed along with an application.
|
||||
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
.. _`pytest_logxml.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_logxml.py
|
||||
.. _`helpconfig`: helpconfig.html
|
||||
.. _`terminal`: terminal.html
|
||||
.. _`pytest_recwarn.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_recwarn.py
|
||||
.. _`unittest`: unittest.html
|
||||
.. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_monkeypatch.py
|
||||
@@ -15,11 +15,14 @@
|
||||
.. _`pytest_nose.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_nose.py
|
||||
.. _`pytest_restdoc.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_restdoc.py
|
||||
.. _`restdoc`: restdoc.html
|
||||
.. _`xdist`: xdist.html
|
||||
.. _`pytest_pastebin.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_pastebin.py
|
||||
.. _`pytest_tmpdir.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_tmpdir.py
|
||||
.. _`terminal`: terminal.html
|
||||
.. _`pytest_figleaf.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_figleaf.py
|
||||
.. _`pytest_hooklog.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_hooklog.py
|
||||
.. _`logxml`: logxml.html
|
||||
.. _`helpconfig`: helpconfig.html
|
||||
.. _`plugin.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/plugin.py
|
||||
.. _`pytest_skipping.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_skipping.py
|
||||
.. _`checkout the py.test development version`: ../../install.html#checkout
|
||||
.. _`pytest_helpconfig.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.2.0a1/py/plugin/pytest_helpconfig.py
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_logxml plugin
|
||||
====================
|
||||
|
||||
logging of test results in JUnit-XML format, for use with Hudson
|
||||
================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_mark plugin
|
||||
==================
|
||||
|
||||
generic mechanism for marking python functions.
|
||||
===============================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_monkeypatch plugin
|
||||
=========================
|
||||
|
||||
safely patch object attributes, dicts and environment variables.
|
||||
================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_nose plugin
|
||||
==================
|
||||
|
||||
nose-compatibility plugin: allow to run nose test suites natively.
|
||||
==================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_pastebin plugin
|
||||
======================
|
||||
|
||||
submit failure or test session information to a pastebin service.
|
||||
=================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_pdb plugin
|
||||
=================
|
||||
|
||||
interactive debugging with the Python Debugger.
|
||||
===============================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_recwarn plugin
|
||||
=====================
|
||||
|
||||
helpers for asserting deprecation and other warnings.
|
||||
=====================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_restdoc plugin
|
||||
=====================
|
||||
|
||||
perform ReST syntax, local and remote reference tests on .rst/.txt files.
|
||||
=========================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_resultlog plugin
|
||||
=======================
|
||||
|
||||
non-xml machine-readable logging of test results.
|
||||
=================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_skipping plugin
|
||||
======================
|
||||
|
||||
advanced skipping for python test functions, classes or modules.
|
||||
================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_terminal plugin
|
||||
======================
|
||||
|
||||
Implements terminal reporting of the full testing process.
|
||||
==========================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_tmpdir plugin
|
||||
====================
|
||||
|
||||
provide temporary directories to test functions.
|
||||
================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
|
||||
pytest_unittest plugin
|
||||
======================
|
||||
|
||||
automatically discover and run traditional "unittest.py" style tests.
|
||||
=====================================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
181
doc/test/plugin/xdist.txt
Normal file
181
doc/test/plugin/xdist.txt
Normal file
@@ -0,0 +1,181 @@
|
||||
|
||||
loop on failing tests, distribute test runs to CPUs and hosts.
|
||||
==============================================================
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
|
||||
The `pytest-xdist`_ plugin extends py.test with some unique
|
||||
test execution modes:
|
||||
|
||||
* Looponfail: run your tests in a subprocess. After it finishes py.test
|
||||
waits until a file in your project changes and then re-runs only the
|
||||
failing tests. This is repeated until all tests pass after which again
|
||||
a full run is performed.
|
||||
|
||||
* Load-balancing: if you have multiple CPUs or hosts you can use
|
||||
those for a combined test run. This allows to speed up
|
||||
development or to use special resources of remote machines.
|
||||
|
||||
* Multi-Platform coverage: you can specify different Python interpreters
|
||||
or different platforms and run tests in parallel on all of them.
|
||||
|
||||
Before running tests remotely, ``py.test`` efficiently synchronizes your
|
||||
program source code to the remote place. All test results
|
||||
are reported back and displayed to your local test session.
|
||||
You may specify different Python versions and interpreters.
|
||||
|
||||
|
||||
Usage examples
|
||||
---------------------
|
||||
|
||||
Speed up test runs by sending tests to multiple CPUs
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
To send tests to multiple CPUs, type::
|
||||
|
||||
py.test -n NUM
|
||||
|
||||
Especially for longer running tests or tests requiring
|
||||
a lot of IO this can lead to considerable speed ups.
|
||||
|
||||
|
||||
Running tests in a Python subprocess
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
To instantiate a python2.4 sub process and send tests to it, you may type::
|
||||
|
||||
py.test -d --tx popen//python=python2.4
|
||||
|
||||
This will start a subprocess which is run with the "python2.4"
|
||||
Python interpreter, found in your system binary lookup path.
|
||||
|
||||
If you prefix the --tx option value like this::
|
||||
|
||||
--tx 3*popen//python=python2.4
|
||||
|
||||
then three subprocesses would be created and tests
|
||||
will be load-balanced across these three processes.
|
||||
|
||||
|
||||
Sending tests to remote SSH accounts
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
Suppose you have a package ``mypkg`` which contains some
|
||||
tests that you can successfully run locally. And you
|
||||
have a ssh-reachable machine ``myhost``. Then
|
||||
you can ad-hoc distribute your tests by typing::
|
||||
|
||||
py.test -d --tx ssh=myhostpopen --rsyncdir mypkg mypkg
|
||||
|
||||
This will synchronize your ``mypkg`` package directory
|
||||
to an remote ssh account and then locally collect tests
|
||||
and send them to remote places for execution.
|
||||
|
||||
You can specify multiple ``--rsyncdir`` directories
|
||||
to be sent to the remote side.
|
||||
|
||||
**NOTE:** For py.test to collect and send tests correctly
|
||||
you not only need to make sure all code and tests
|
||||
directories are rsynced, but that any test (sub) directory
|
||||
also has an ``__init__.py`` file because internally
|
||||
py.test references tests as a fully qualified python
|
||||
module path. **You will otherwise get strange errors**
|
||||
during setup of the remote side.
|
||||
|
||||
Sending tests to remote Socket Servers
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
Download the single-module `socketserver.py`_ Python program
|
||||
and run it like this::
|
||||
|
||||
python socketserver.py
|
||||
|
||||
It will tell you that it starts listening on the default
|
||||
port. You can now on your home machine specify this
|
||||
new socket host with something like this::
|
||||
|
||||
py.test -d --tx socket=192.168.1.102:8888 --rsyncdir mypkg mypkg
|
||||
|
||||
|
||||
.. _`atonce`:
|
||||
|
||||
Running tests on many platforms at once
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
The basic command to run tests on multiple platforms is::
|
||||
|
||||
py.test --dist=each --tx=spec1 --tx=spec2
|
||||
|
||||
If you specify a windows host, an OSX host and a Linux
|
||||
environment this command will send each tests to all
|
||||
platforms - and report back failures from all platforms
|
||||
at once. The specifications strings use the `xspec syntax`_.
|
||||
|
||||
.. _`xspec syntax`: http://codespeak.net/execnet/trunk/basics.html#xspec
|
||||
|
||||
.. _`socketserver.py`: http://codespeak.net/svn/py/dist/py/execnet/script/socketserver.py
|
||||
|
||||
.. _`execnet`: http://codespeak.net/execnet
|
||||
|
||||
Specifying test exec environments in a conftest.py
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
Instead of specifying command line options, you can
|
||||
put options values in a ``conftest.py`` file like this::
|
||||
|
||||
pytest_option_tx = ['ssh=myhost//python=python2.5', 'popen//python=python2.5']
|
||||
pytest_option_dist = True
|
||||
|
||||
Any commandline ``--tx`` specifictions will add to the list of available execution
|
||||
environments.
|
||||
|
||||
Specifying "rsync" dirs in a conftest.py
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
In your ``mypkg/conftest.py`` you may specify directories to synchronise
|
||||
or to exclude::
|
||||
|
||||
rsyncdirs = ['.', '../plugins']
|
||||
rsyncignore = ['_cache']
|
||||
|
||||
These directory specifications are relative to the directory
|
||||
where the ``conftest.py`` is found.
|
||||
|
||||
command line options
|
||||
--------------------
|
||||
|
||||
|
||||
``-f, --looponfail``
|
||||
run tests in subprocess, wait for modified files and re-run failing test set until all pass.
|
||||
``-n numprocesses``
|
||||
shortcut for '--dist=load --tx=NUM*popen'
|
||||
``--boxed``
|
||||
box each test run in a separate process (unix)
|
||||
``--dist=distmode``
|
||||
set mode for distributing tests to exec environments.
|
||||
|
||||
each: send each test to each available environment.
|
||||
|
||||
load: send each test to available environment.
|
||||
|
||||
(default) no: run tests inprocess, don't distribute.
|
||||
``--tx=xspec``
|
||||
add a test execution environment. some examples: --tx popen//python=python2.5 --tx socket=192.168.1.102:8888 --tx ssh=user@codespeak.net//chdir=testcache
|
||||
``-d``
|
||||
load-balance tests. shortcut for '--dist=load'
|
||||
``--rsyncdir=dir1``
|
||||
add directory for rsyncing to remote tx nodes.
|
||||
|
||||
Start improving this plugin in 30 seconds
|
||||
=========================================
|
||||
|
||||
|
||||
1. Download `plugin.py`_ plugin source code
|
||||
2. put it somewhere as ``plugin.py`` into your import path
|
||||
3. a subsequent ``py.test`` run will use your local version
|
||||
|
||||
Checkout customize_, other plugins_ or `get in contact`_.
|
||||
|
||||
.. include:: links.txt
|
||||
Reference in New Issue
Block a user