diff --git a/_pytest/python.py b/_pytest/python.py index bdef136f2..4155a8366 100644 --- a/_pytest/python.py +++ b/_pytest/python.py @@ -907,12 +907,15 @@ class Function(FunctionMixin, pytest.Item): class FuncargRequest: - """ A request for function arguments from a test function. + """ (old-style) A request for function arguments from a test function. Note that there is an optional ``param`` attribute in case there was an invocation to metafunc.addcall(param=...). If no such call was done in a ``pytest_generate_tests`` - hook, the attribute will not be present. + hook, the attribute will not be present. Note that + as of pytest-2.3 you probably rather want to use the + testcontext object and mark your factory with a ``@pytest.factory`` + marker. """ def __init__(self, pyfuncitem): diff --git a/doc/en/apiref.txt b/doc/en/apiref.txt index 1add0221f..1f41c688a 100644 --- a/doc/en/apiref.txt +++ b/doc/en/apiref.txt @@ -11,6 +11,8 @@ py.test reference documentation customize.txt assert.txt funcargs.txt + funcarg_compare.txt + setup.txt xunit_setup.txt capture.txt monkeypatch.txt diff --git a/doc/en/conf.py b/doc/en/conf.py index fac7b07e4..0bd3cd6c8 100644 --- a/doc/en/conf.py +++ b/doc/en/conf.py @@ -17,7 +17,7 @@ # # The full version, including alpha/beta/rc tags. # The short X.Y version. -version = release = "2.3.0.dev8" +version = release = "2.3.0.dev9" import sys, os diff --git a/doc/en/example/index.txt b/doc/en/example/index.txt index 3a31c5611..e0b628316 100644 --- a/doc/en/example/index.txt +++ b/doc/en/example/index.txt @@ -21,4 +21,3 @@ need more examples or have questions. Also take a look at the :ref:`comprehensiv markers.txt pythoncollection.txt nonpython.txt - newexamples.txt diff --git a/doc/en/funcarg_compare.txt b/doc/en/funcarg_compare.txt index 3f9284002..3d12706e4 100644 --- a/doc/en/funcarg_compare.txt +++ b/doc/en/funcarg_compare.txt @@ -1,55 +1,23 @@ -V5: changes to new resource/setup facilities +.. _`funcargcompare`: + +============================================================= +pytest-2.3: reasoning for the new funcarg and setup functions ============================================================= **Target audience**: Reading this document requires basic knowledge of -python testing, xUnit setup methods and the basic pytest funcarg mechanism, -see http://pytest.org/latest/funcargs.html - - -**Changes**: This V5 draft is based on incorporating and thinking about -feedback on previous versions provided by Floris Bruynooghe, Carl Meyer, -Ronny Pfannschmidt and Samuele Pedroni. I have also now implemented it -which triggered a number of refinements as well. The main changes are: - -* Collapse funcarg factory decorators into a single "@resource" one. - You can specify scopes and params with it. When using the decorator - the "pytest_funcarg__" prefix is not allowed and the old-style - ``request`` object cannot be received. - -* funcarg resource factories can now use funcargs themselves - -* Drop setup/directory scope from this draft - -* introduce a new @setup decorator similar to the @funcarg one - except that setup-markers cannot define parametriation themselves. - Instead they can easily depend on a parametrized funcarg (which - must not be visible at test function signatures). - -* drop consideration of setup_X support for funcargs because - it is less flexible and probably causes more implementation - troubles than the current @setup approach which can share - a lot of logic with the @funcarg one. - -* tests are grouped by parametrized funcargs and according to scope - (sounds like a small thing but is a big deal) - -* make the new-style funcargs/setup use a "testcontext" object - which offers test context info and addfinalizer() methods but no - getfuncargvalue()/cached_setup()/applymarker anymore. Reason - being that getfuncargvalue()/cached_setup breaks other features - such as sorting by resource-scope and parametrization - +python testing, xUnit setup methods and the (previous) basic pytest +funcarg mechanism, see http://pytest.org/2.2.4/funcargs.html .. currentmodule:: _pytest Shortcomings of the previous pytest_funcarg__ mechanism ---------------------------------------------------------- +=========================================================== -The previous funcarg mechanism calls a factory each time a -funcarg for a test function is testcontexted. If a factory wants -t re-use a resource across different scopes, it often used -the ``testcontext.cached_setup()`` helper to manage caching of +The pre pytest-2.3 funcarg mechanism calls a factory each time a +funcarg for a test function is required. If a factory wants to +re-use a resource across different scopes, it often used +the ``request.cached_setup()`` helper to manage caching of resources. Here is a basic example how we could implement a per-session Database object:: @@ -95,10 +63,9 @@ new facilities. Direct scoping of funcarg factories -------------------------------------------------------- -Instead of calling cached_setup(), you can decorate your factory -to state its scope:: +Instead of calling cached_setup(), you can use the :ref:`@pytest.factory <@pytest.factory>` decorator and directly state the scope:: - @pytest.mark.resource(scope="session") + @pytest.factory(scope="session") def db(testcontext): # factory will only be invoked once per session - db = DataBase() @@ -108,25 +75,21 @@ to state its scope:: This factory implementation does not need to call ``cached_setup()`` anymore because it will only be invoked once per session. Moreover, the ``testcontext.addfinalizer()`` registers a finalizer according to the specified -resource scope on which the factory function is operating. With this new -scoping, the still existing ``cached_setup()`` should be much less used -but will remain for compatibility reasons and for the case where you -still want to have your factory get called on a per-item basis. +resource scope on which the factory function is operating. Direct parametrization of funcarg resource factories ---------------------------------------------------------- -.. note:: Implemented - Previously, funcarg factories could not directly cause parametrization. -You needed to specify a ``@parametrize`` or implement a ``pytest_generate_tests`` hook to perform parametrization, i.e. calling a test multiple times -with different value sets. pytest-2.X introduces a decorator for use -on the factory itself:: +You needed to specify a ``@parametrize`` decorator on your test function +or implement a ``pytest_generate_tests`` hook to perform +parametrization, i.e. calling a test multiple times with different value +sets. pytest-2.3 introduces a decorator for use on the factory itself:: - @pytest.mark.resource(params=["mysql", "pg"]) + @pytest.factory(params=["mysql", "pg"]) def pytest_funcarg__db(testcontext): - ... + ... # use testcontext.param Here the factory will be invoked twice (with the respective "mysql" and "pg" values set as ``testcontext.param`` attributes) and and all of @@ -141,7 +104,7 @@ functions/classes were parametrized via Of course it's perfectly fine to combine parametrization and scoping:: - @pytest.mark.resource(scope="session", params=["mysql", "pg"]) + @pytest.factory(scope="session", params=["mysql", "pg"]) def pytest_funcarg__db(testcontext): if testcontext.param == "mysql": db = MySQL() @@ -155,26 +118,23 @@ receiving the values created by the two respective invocations to the factory function. -No ``pytest_funcarg__`` prefix when using @resource decorator +No ``pytest_funcarg__`` prefix when using @factory decorator ------------------------------------------------------------------- - -.. note:: Implemented - When using the ``@funcarg`` decorator the name of the function -does not need to (and in fact cannot) use the ``pytest_funcarg__`` -naming:: +denotes the name under which the resource can be accessed as a function +argument:: - @pytest.mark.resource + @pytest.factory() def db(testcontext): ... The name under which the funcarg resource can be requested is ``db``. -You can also use the "old" non-decorator way of specifying funcarg factories +You can still use the "old" non-decorator way of specifying funcarg factories aka:: - def pytest_funcarg__db(testcontext): + def pytest_funcarg__db(request): ... It is recommended to use the resource decorator, however. @@ -183,8 +143,6 @@ It is recommended to use the resource decorator, however. solving per-session setup / the new @setup marker -------------------------------------------------------------- -.. note:: Implemented, at least working for basic situations. - pytest for a long time offered a pytest_configure and a pytest_sessionstart hook which are often used to setup global resources. This suffers from several problems: @@ -201,7 +159,7 @@ several problems: fact that this hook is actually used for reporting, in particular the test-header with platform/custom information. -Moreover, it is today not easy to define a scoped setup from plugins or +Moreover, it was not easy to define a scoped setup from plugins or conftest files other than to implement a ``pytest_runtest_setup()`` hook and caring for scoping/caching yourself. And it's virtually impossible to do this with parametrization as ``pytest_runtest_setup()`` is called @@ -209,20 +167,17 @@ during test execution and parametrization happens at collection time. It follows that pytest_configure/session/runtest_setup are often not appropriate for implementing common fixture needs. Therefore, -pytest-2.X introduces a new :ref:`@pytest.setup` marker which takes -an optional "scope" parameter. +pytest-2.3 introduces a new :ref:`@pytest.setup ` marker +for setup functions and it accepts an optional "scope" parameter. -See :ref:`new_setup` for examples. +See :ref:`setup` for more explanation and examples. funcarg and setup discovery now happens at collection time --------------------------------------------------------------------- -.. note:: - Partially implemented - collectonly shows no extra information however. - -pytest-2.X takes care to discover funcarg factories and @setup methods +pytest-2.3 takes care to discover funcarg factories and @setup methods at collection time. This is more efficient especially for large test suites. -Moreover, a call to "py.test --collectonly" should be able to show -a lot of setup-information and thus presents a nice method to get an +Moreover, a call to "py.test --collectonly" should be able to in the future +show a lot of setup-information and thus presents a nice method to get an overview of resource management in your project. diff --git a/doc/en/funcargs.txt b/doc/en/funcargs.txt index 5999cf6d1..4cfb95c80 100644 --- a/doc/en/funcargs.txt +++ b/doc/en/funcargs.txt @@ -1,75 +1,110 @@ -============================================================== -Injecting objects into test functions (funcargs) -============================================================== - -.. currentmodule:: _pytest.python - +.. _resources: .. _`funcargs`: .. _`funcarg mechanism`: -Dependency injection through function arguments -================================================= +======================================================= +funcargs: resource injection and parametrization +======================================================= .. note:: - This section describes the pytest mechanisms prior - to the pytest-2.3 release. If you haven't used these - features yet, it makes more sense to stop here and read - :ref:`resources` instead. - -py.test lets you inject objects into test invocations and precisely -control their life cycle in relation to the overall test execution. Moreover, -you can run a test function multiple times injecting different objects. - -The basic mechanism for injecting objects is also called the -*funcarg mechanism* because objects are ultimately injected -by calling a test function with it as an argument. Unlike the -classical xUnit approach *funcargs* relate more to `Dependency Injection`_ -because they help to de-couple test code from objects required for -them to execute. At test writing time you do not need to care for the -details of how your required resources are constructed or if they -live through a function, class, module or session scope. + pytest-2.3 introduces major refinements to the original funcarg + mechanism introduced to pytest-2.0. While the old way + remains fully supported, it is recommended to use the refined + mechanisms. See also the `compatibility notes`_ and the detailed + :ref:`reasoning for the new funcarg and setup functions `. .. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection -To create a value with which to call a test function a factory function -is called which gets full access to the test function context and can -register finalizers or invoke lifecycle-caching helpers. The factory -can be implemented in same test class or test module, in a -per-directory ``conftest.py`` file or in an external plugin. This -allows total de-coupling of test and setup code. +Introduction +==================== -A test function may be invoked multiple times in which case we -speak of :ref:`parametrized testing `. This can be -very useful if you want to test e.g. against different database backends -or with multiple numerical arguments sets and want to reuse the same set -of test functions. +py.test supports the injection of objects into test and setup functions +and flexibly control their life cycle in relation to the overall test +execution. Moreover, you can run a test function multiple times +injecting different objects. -py.test comes with some :ref:`builtinresources` and there are some refined usages in the examples section. +The basic mechanism for injecting objects is called the *funcarg +mechanism* because objects are **injected** when a test or setup +function states it as an **argument**. The injected argument is +created by a call to a registered **funcarg factory**. This approach is +an example of `Dependency Injection`_ and helps to de-couple test code +from the setup of required objects: at test writing time you do not need +to care for the details of where and how your required resources are +constructed, if they are shared on a per-class, module or session basis, +or if your test function is invoked multiple times with differently +configured resource instances. -.. _funcarg: +When a test function is invoked multiple times with different arguments we +speak of **parametrized testing**. This is useful if you want to test +e.g. against different database backends or want to write a parametrized +test function, checking that certain inputs lead to certain outputs. +You can parametrize funcarg factories, parametrize test function +arguments or even implement your own parametrization scheme through a +plugin hook. -Basic injection example --------------------------------- +pytest additionally offers powerful xunit-style :ref:`setup functions ` for the cases where you need to create implicit test state +that is not passed explicitely to test functions. -Let's look at a simple self-contained test module:: +Concretely, there are three main means of funcarg management: + +* a `@pytest.factory`_ marker to define resource factories, + their scoping and parametrization. Factories can themselves + receive resources through their function arguments, easing + the setup of `interdependent resources`_. Factories can use + the special `testcontext`_ object to access details from where + the factory or setup function is called and for registering finalizers. + +* a `@pytest.mark.parametrize`_ marker for executing test functions + multiple times with different argument sets, + +* a `pytest_generate_tests`_ plugin hook marker for implementing + your parametrization for a test function which may depend on + command line options, class/module attributes etc. + +Apart from making it easy to manage your own test resources +pytest also comes with some :ref:`builtinresources` which +you can use without defining them yourself. Third-party plugins +offer yet more domain-specific funcarg resources (for example the +`pytest-django plugin `_) so +that after plugin installation you can simply use them in +your test and setup functions. This all contributes to high +re-useability of test resource management and goes far beyond what can +be done with the classical xUnit style approach which encodes resource +setup statically into the test source code, leading to duplicate and +hard-to change fixtures. + +.. _`@pytest.factory`: + +``@pytest.factory``: Creating parametrized, scoped resources +===================================================================== + +Basic funcarg injection example +----------------------------------------------------------- + +Let's look at a simple self-contained test module using a factory +and a funcarg:: # content of ./test_simplefactory.py - def pytest_funcarg__myfuncarg(request): + import pytest + + @pytest.factory() + def myfuncarg(): return 42 def test_function(myfuncarg): assert myfuncarg == 17 -This test function needs an injected object named ``myfuncarg``. -py.test will automatically discover and call the ``pytest_funcarg__myfuncarg`` -factory. Running the test looks like this:: +Here, the ``test_function`` needs an object named ``myfuncarg`` and thus +py.test will discover and call the ``@pytest.factory`` marked ``myfuncarg`` +factory function. Running the tests looks like this:: $ py.test test_simplefactory.py =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2 - plugins: xdist, bugzilla, cache, oejskit, pep8, cov + platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 + plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov collecting ... collected 1 items test_simplefactory.py F @@ -83,7 +118,7 @@ factory. Running the test looks like this:: > assert myfuncarg == 17 E assert 42 == 17 - test_simplefactory.py:5: AssertionError + test_simplefactory.py:8: AssertionError ========================= 1 failed in 0.02 seconds ========================= This shows that the test function was called with a ``myfuncarg`` @@ -93,14 +128,12 @@ how py.test comes to call the test function this way: 1. py.test :ref:`finds ` the ``test_function`` because of the ``test_`` prefix. The test function needs a function argument named ``myfuncarg``. A matching factory function is discovered by - looking for the name ``pytest_funcarg__myfuncarg``. + looking for a factory function named ``myfuncarg``. -2. ``pytest_funcarg__myfuncarg(request)`` is called and - returns the value for ``myfuncarg``. +2. ``myfuncarg()`` is called to create a value ``42``. -3. the test function can now be called: ``test_function(42)``. - This results in the above exception because of the assertion - mismatch. +3. ``test_function(42)`` is now called and results in the above + reported exception because of the assertion mismatch. Note that if you misspell a function argument or want to use one that isn't available, you'll see an error @@ -114,130 +147,517 @@ with a list of available function arguments. to see available function arguments. -The request object passed to factories ------------------------------------------ -Each funcarg factory receives a :py:class:`~_pytest.python.FuncargRequest` object which -provides methods to manage caching and finalization in the context of the -test invocation as well as several attributes of the the underlying test item. In fact, as of version pytest-2.3, the request API is implemented on all Item -objects and therefore the request object has general :py:class:`Node attributes and methods <_pytest.main.Node>` attributes. This is a backward compatible -change so no changes are neccessary for pre-2.3 funcarg factories. +Location independency of funcarg factories +---------------------------------------------------- + +If during implementing your tests you realize that you +want to use a factory from multiple test files you can move it +to a :ref:`conftest.py ` file or even separately installable +:ref:`plugins ` without changing test code. The discovery of +funcarg factories starts at test classes, then test modules, then +``conftest.py`` files and finally builtin and 3-rd party plugins. + +Creating and using a session-shared funcarg +----------------------------------------------------------------- + +.. regendoc:wipe + +.. versionadded:: 2.3 + +The `@pytest.factory`_ marker allows to + +* mark a function as a factory for resources, useable by test and setup + functions + +* define parameters in order to run tests multiple times with + different resource instances + +* declare a scope which determines the level of caching, i.e. how often + the factory will be called. Valid scopes are ``session``, ``module``, + ``class`` and ``function``. + +Here is a simple example of a factory creating a shared ``smtplib.SMTP`` +connection resource which test functions then may use across the whole +test session:: + + # content of conftest.py + import pytest + import smtplib + + @pytest.factory(scope="session") + def smtp(testcontext): + return smtplib.SMTP("merlinux.eu") + +The name of the factory is ``smtp`` (the factory function name) +and you can access its result by listing the name ``smtp`` as +an input parameter in any test or setup function:: + + # content of test_module.py + def test_ehlo(smtp): + response = smtp.ehlo() + assert response[0] == 250 + assert "merlinux" in response[1] + assert 0 # for demo purposes + + def test_noop(smtp): + response = smtp.noop() + assert response[0] == 250 + assert 0 # for demo purposes + +We deliberately insert failing ``assert 0`` statements in order to +inspect what is going on and can now run the tests:: + + $ py.test -q test_module.py + collecting ... collected 2 items + FF + ================================= FAILURES ================================= + ________________________________ test_ehlo _________________________________ + + smtp = + + def test_ehlo(smtp): + response = smtp.ehlo() + assert response[0] == 250 + assert "merlinux" in response[1] + > assert 0 # for demo purposes + E assert 0 + + test_module.py:5: AssertionError + ________________________________ test_noop _________________________________ + + smtp = + + def test_noop(smtp): + response = smtp.noop() + assert response[0] == 250 + > assert 0 # for demo purposes + E assert 0 + + test_module.py:10: AssertionError + 2 failed in 0.20 seconds + +you see the two ``assert 0`` failing and can also see that +the same (session-scoped) object was passed into the two test functions. -.. _`parametrizing tests, generalized`: http://tetamap.wordpress.com/2009/05/13/parametrizing-python-tests-generalized/ +Parametrizing a session-shared funcarg resource +----------------------------------------------------------------- -.. _`blog post about the monkeypatch funcarg`: http://tetamap.wordpress.com/2009/03/03/monkeypatching-in-unit-tests-done-right/ -.. _`xUnit style`: xunit_setup.html +Extending the previous example, we can flag the factory to create +two ``smtp`` values which will cause all tests using it to +run twice with two different values. The factory function gets +access to each parameter through the special `testcontext`_ object:: + + # content of conftest.py + import pytest + import smtplib + + @pytest.factory(scope="session", + params=["merlinux.eu", "mail.python.org"]) + def smtp(testcontext): + return smtplib.SMTP(testcontext.param) + +The main change is the definition of a ``params`` list in the +``factory``-marker and the ``testcontext.param`` access within the +factory function. No test function code needs to change. +So let's just do another run:: + + $ py.test -q + collecting ... collected 4 items + FFFF + ================================= FAILURES ================================= + __________________________ test_ehlo[merlinux.eu] __________________________ + + smtp = + + def test_ehlo(smtp): + response = smtp.ehlo() + assert response[0] == 250 + assert "merlinux" in response[1] + > assert 0 # for demo purposes + E assert 0 + + test_module.py:5: AssertionError + __________________________ test_noop[merlinux.eu] __________________________ + + smtp = + + def test_noop(smtp): + response = smtp.noop() + assert response[0] == 250 + > assert 0 # for demo purposes + E assert 0 + + test_module.py:10: AssertionError + ________________________ test_ehlo[mail.python.org] ________________________ + + smtp = + + def test_ehlo(smtp): + response = smtp.ehlo() + assert response[0] == 250 + > assert "merlinux" in response[1] + E assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN' + + test_module.py:4: AssertionError + ________________________ test_noop[mail.python.org] ________________________ + + smtp = + + def test_noop(smtp): + response = smtp.noop() + assert response[0] == 250 + > assert 0 # for demo purposes + E assert 0 + + test_module.py:10: AssertionError + 4 failed in 6.00 seconds + +We get four failures because we are running the two tests twice with +different ``smtp`` instantiations as defined on the factory. +Note that with the ``mail.python.org`` connection the second test +fails in ``test_ehlo`` because it expects a specific server string. + +Adding a finalizer to the parametrized resource +-------------------------------------------------------- + +Further extending the ``smtp`` example, we now want to properly +close a smtp server connection after the last test using it +has been run. We can do this by calling the ``testcontext.addfinalizer()`` +helper:: + + # content of conftest.py + import pytest + import smtplib + + @pytest.factory(scope="session", + params=["merlinux.eu", "mail.python.org"]) + def smtp(testcontext): + smtp = smtplib.SMTP(testcontext.param) + def fin(): + print ("finalizing %s" % smtp) + smtp.close() + testcontext.addfinalizer(fin) + return smtp + +We also add a print call and then run py.test without the default +output capturing and disabled traceback reporting:: + + $ py.test -s -q --tb=no + collecting ... collected 4 items + FFFF + 4 failed in 5.62 seconds + finalizing + finalizing + +We see that the two ``smtp`` instances are finalized appropriately. + +Looking at test collection without running tests +------------------------------------------------------ + +You can also look at what tests pytest collects without running them:: + + $ py.test --collectonly + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 + plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov + collecting ... collected 4 items + + + + + + + ============================= in 0.02 seconds ============================= + +Note that pytest orders your test run by resource usage, minimizing +the number of active resources at any given time. + + +.. _`interdependent resources`: + +Interdepdendent resources +---------------------------------------------------------- + +You can not only use funcargs in test functions but also in their factories +themselves. Extending the previous example we can instantiate another +object ``app`` and stick the ``smtp`` resource into it like this:: + + # content of test_appsetup.py + + import pytest + + class App: + def __init__(self, smtp): + self.smtp = smtp + + @pytest.factory(scope="module") + def app(smtp): + return App(smtp) + + def test_exists(app): + assert app.smtp + +Here we define the factory local to the test module and make it access +the ``smtp`` resource by listing it as an input parameter. Let's run this:: + + $ py.test -v test_appsetup.py + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 -- /home/hpk/venv/1/bin/python + cachedir: /home/hpk/tmp/doc-exec-414/.cache + plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov + collecting ... collected 2 items + + test_appsetup.py:12: test_exists[merlinux.eu] PASSED + test_appsetup.py:12: test_exists[mail.python.org] PASSED + + ========================= 2 passed in 5.37 seconds ========================= + +Due to the parametrization of ``smtp`` the test will run twice with two +different ``App`` instances and respective smtp servers. There is no +need for the ``app`` factory to be aware of the parametrization. Note +that the ``app`` factory has a scope of ``module`` whereas it uses the +session-scoped ``smtp`` object: it is fine for factories to use +"broader" scoped resources but not the other way round. A +session-scoped resource could not use a module-scoped resource in a +meaningful way. + +.. _`automatic per-resource grouping`: + +Grouping tests by resource parameters +---------------------------------------------------------- + +.. regendoc: wipe + +pytest minimizes the number of active resources during test runs. +If you have a parametrized resource, then all the tests using one +resource instance will execute one after another. Then any finalizers +are called for that resource instance and then the next parametrized +resource instance is created and its tests are run. Among other things, +this eases testing of applications which create and use global state. + +The following example uses two parametrized funcargs, one of which is +scoped on a per-module basis, and all the functions perform ``print`` call +it to document the flow of calls:: + + # content of test_module.py + import pytest + + @pytest.factory(scope="module", params=["mod1", "mod2"]) + def modarg(testcontext): + param = testcontext.param + print "create", param + def fin(): + print "fin", param + testcontext.addfinalizer(fin) + return param + + @pytest.factory(scope="function", params=[1,2]) + def otherarg(testcontext): + return testcontext.param + + def test_0(otherarg): + print " test0", otherarg + def test_1(modarg): + print " test1", modarg + def test_2(otherarg, modarg): + print " test2", otherarg, modarg + +Let's run the tests in verbose mode and with looking at the print-output:: + + $ py.test -v -s test_module.py + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 -- /home/hpk/venv/1/bin/python + cachedir: /home/hpk/tmp/doc-exec-414/.cache + plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov + collecting ... collected 8 items + + test_module.py:16: test_0[1] PASSED + test_module.py:16: test_0[2] PASSED + test_module.py:18: test_1[mod1] PASSED + test_module.py:20: test_2[1-mod1] PASSED + test_module.py:20: test_2[2-mod1] PASSED + test_module.py:18: test_1[mod2] PASSED + test_module.py:20: test_2[1-mod2] PASSED + test_module.py:20: test_2[2-mod2] PASSED + + ========================= 8 passed in 0.02 seconds ========================= + test0 1 + test0 2 + create mod1 + test1 mod1 + test2 1 mod1 + test2 2 mod1 + fin mod1 + create mod2 + test1 mod2 + test2 1 mod2 + test2 2 mod2 + fin mod2 + +You can see that the parametrized module-scoped ``modarg`` resource caused +a re-ordering of test execution. The finalizer for the ``mod1`` parametrized +resource was executed before the ``mod2`` resource was setup. + +.. currentmodule:: _pytest.python +.. _`testcontext`: + +``testcontext``: interacting with test context +--------------------------------------------------- + +The ``testcontext`` object may be used by `@pytest.factory`_ or +:ref:`@pytest.setup ` marked functions. It contains +information relating to the test context within which the function +executes. Moreover, you can call +``testcontext.addfinalizer(myfinalizer)`` in order to trigger a call to +``myfinalizer`` after the last test in the test context has executed. +If passed to a parametrized factory ``testcontext.param`` will contain a +parameter (one value out of the ``params`` list specified with the +`@pytest.factory`_ marker). + +.. autoclass:: _pytest.python.TestContext() + :members: -.. _`funcarg factory`: -.. _factory: .. _`test generators`: .. _`parametrizing-tests`: .. _`parametrized test functions`: -Parametrizing multiple calls to a test function -=========================================================== +Parametrizing test functions +========================================================================== -You can parametrize multiple runs of the same test -function by adding new test function calls with different -function argument values. Let's look at a simple self-contained -example: +While the `@pytest.factory`_ decorator allows to define parametrization +of funcarg resources at the factory-level, there are also means to +define parametrization at test functions directly: -Basic generated test example ----------------------------- +* `@pytest.mark.parametrize`_ to provide multiple argument sets + for a particular test function or class. -Let's consider a test module which uses the ``pytest_generate_tests`` -hook to generate several calls to the same test function:: +* `pytest_generate_tests`_ to implement your own custom parametrization + scheme or extensions. + +.. _`@pytest.mark.parametrize`: + +``@pytest.mark.parametrize``: parametrizing test functions +--------------------------------------------------------------------- + +.. regendoc: wipe + +.. versionadded:: 2.2 + +The builtin ``pytest.mark.parametrize`` decorator enables +parametrization of arguments for a test function. Here is a typical example +of a test function that wants check for expected output given a certain input:: + + # content of test_expectation.py + import pytest + @pytest.mark.parametrize(("input", "expected"), [ + ("3+5", 8), + ("2+4", 6), + ("6*9", 42), + ]) + def test_eval(input, expected): + assert eval(input) == expected + +The ``@parametrize`` decorator defines three different argument sets for the +two ``(input, output)`` arguments of ``test_eval`` function so the latter +will be run three times:: + + $ py.test -q + + collecting ... collected 13 items + ....F........ + ================================= FAILURES ================================= + ____________________________ test_eval[6*9-42] _____________________________ + + input = '6*9', expected = 42 + + @pytest.mark.parametrize(("input", "expected"), [ + ("3+5", 8), + ("2+4", 6), + ("6*9", 42), + ]) + def test_eval(input, expected): + > assert eval(input) == expected + E assert 54 == 42 + E + where 54 = eval('6*9') + + test_expectation.py:8: AssertionError + 1 failed, 12 passed in 5.76 seconds + +As expected only one pair of input/output values fails the simple test function. +As usual you can see the ``input`` and ``output`` values in the traceback. + +Note that there are various ways how you can mark groups of functions, +see :ref:`mark`. + + +.. _`pytest_generate_tests`: + +Basic ``pytest_generate_tests`` example +--------------------------------------------- + +The ``pytest_generate_tests`` hook is typically used if you want +to go beyond what ``@pytest.mark.parametrize`` offers. For example, +let's say we want to execute a test with different computation +parameters and the parameter range shall be determined by a command +line argument. Let's first write a simple (do-nothing) computation test:: + + # content of test_compute.py + + def test_compute(param1): + assert param1 < 4 + +Now we add a ``conftest.py`` file containing the addition of a +command line option and the generation of tests depending on +that option:: + + # content of conftest.py + + def pytest_addoption(parser): + parser.addoption("--all", action="store_true", + help="run all combinations") - # content of test_example.py def pytest_generate_tests(metafunc): - if "numiter" in metafunc.funcargnames: - metafunc.parametrize("numiter", range(10)) + if 'param1' in metafunc.funcargnames: + if metafunc.config.option.all: + end = 5 + else: + end = 2 + metafunc.parametrize("param1", range(end)) - def test_func(numiter): - assert numiter < 9 +This means that we only run two tests if no option is passed:: -Running this will generate ten invocations of ``test_func`` passing in each of the items in the list of ``range(10)``:: + $ py.test -q test_compute.py + collecting ... collected 2 items + .. + 2 passed in 0.02 seconds - $ py.test test_example.py - =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2 - plugins: xdist, bugzilla, cache, oejskit, pep8, cov - collecting ... collected 10 items - - test_example.py .........F - +And we run five tests if we add the ``--all`` option:: + + $ py.test -q --all + collecting ... collected 5 items + ....F ================================= FAILURES ================================= - _______________________________ test_func[9] _______________________________ + _____________________________ test_compute[4] ______________________________ - numiter = 9 + param1 = 4 - def test_func(numiter): - > assert numiter < 9 - E assert 9 < 9 + def test_compute(param1): + > assert param1 < 4 + E assert 4 < 4 - test_example.py:6: AssertionError - ==================== 1 failed, 9 passed in 0.03 seconds ==================== + test_compute.py:3: AssertionError + 1 failed, 4 passed in 0.03 seconds -Obviously, only when ``numiter`` has the value of ``9`` does the test fail. Note that the ``pytest_generate_tests(metafunc)`` hook is called during -the test collection phase which is separate from the actual test running. -Let's just look at what is collected:: - - $ py.test --collectonly test_example.py - =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2 - plugins: xdist, bugzilla, cache, oejskit, pep8, cov - collecting ... collected 10 items - - - - - - - - - - - - - ============================= in 0.02 seconds ============================= - -If you want to select only the run with the value ``7`` you could do:: - - $ py.test -v -k 7 test_example.py # or -k test_func[7] - =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2 -- /home/hpk/venv/1/bin/python - cachedir: /home/hpk/tmp/doc-exec-271/.cache - plugins: xdist, bugzilla, cache, oejskit, pep8, cov - collecting ... collected 10 items - - test_example.py:5: test_func[0] PASSED - test_example.py:5: test_func[1] PASSED - test_example.py:5: test_func[2] PASSED - test_example.py:5: test_func[3] PASSED - test_example.py:5: test_func[4] PASSED - test_example.py:5: test_func[5] PASSED - test_example.py:5: test_func[6] PASSED - test_example.py:5: test_func[7] PASSED - test_example.py:5: test_func[8] PASSED - test_example.py:5: test_func[9] FAILED - - ================================= FAILURES ================================= - _______________________________ test_func[9] _______________________________ - - numiter = 9 - - def test_func(numiter): - > assert numiter < 9 - E assert 9 < 9 - - test_example.py:6: AssertionError - ==================== 1 failed, 9 passed in 0.03 seconds ==================== +As expected when running the full range of ``param1`` values +we'll get an error on the last one. You might want to look at :ref:`more parametrization examples `. + .. _`metafunc object`: The **metafunc** object @@ -260,3 +680,52 @@ in the class or module where a test function is defined: .. automethod:: Metafunc.parametrize .. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists) + + +.. regendoc:wipe + + +.. _`compatibility notes`: + +.. _`funcargscompat`: + +Compatibility notes +============================================================ + +**Funcargs** were originally introduced to pytest-2.0. In pytest-2.3 +the mechanism were extended and refined. Here are some related notes: + +* previously funcarg factories were specified with a special + ``pytest_funcarg__NAME`` prefix instead of using the + ``@pytest.factory`` decorator. + +* Factories received a `request`_ object which managed caching through + ``request.cached_setup()`` calls and allowed using other funcargs via + ``request.getfuncargvalue()`` calls. These intricate APIs made it hard + to do proper parametrization and implement resource caching. The + new ``@pytest.factory`` decorator allows to simply declare the scope + and let pytest figure things out for you. + +* if you used parametrization and funcarg factories which made use of + ``request.cached_setup()`` it is recommeneded to invest a few minutes + and simplify your funcarg factory code to use the `@pytest.factory`_ + decorator instead. This will also allow to take advantage of + the `automatic per-resource grouping`_ of tests. + +.. note:: + + Throughout the pytest documents the ``pytest_funcarg__NAME`` way of + defining a funcarg factory is often termed "old-style". Their + use remains fully supported and existing code using it should run + unmodified. + +.. _request: + +The request object passed to old-style factories +----------------------------------------------------------------- + +Old-style funcarg factory definitions can receive a :py:class:`~_pytest.python.FuncargRequest` object which +provides methods to manage caching and finalization in the context of the +test invocation as well as several attributes of the the underlying test item. + + diff --git a/doc/en/index.txt b/doc/en/index.txt index ee0774040..49e142282 100644 --- a/doc/en/index.txt +++ b/doc/en/index.txt @@ -27,7 +27,7 @@ Welcome to pytest! - (new in 2.3) :ref:`easy test resource management, scoping and parametrization ` - - (new in 2.3) :ref:`xunitsetup`. + - (new in 2.3) :ref:`setup functions`. - (new in 2.2) :ref:`durations` - (much improved in 2.2) :ref:`marking and test selection ` - (improved in 2.2) :ref:`parametrized test functions ` diff --git a/doc/en/resources.txt b/doc/en/resources.txt deleted file mode 100644 index 97f2805de..000000000 --- a/doc/en/resources.txt +++ /dev/null @@ -1,487 +0,0 @@ - -.. _resources: - -======================================================= -test resource injection and parametrization -======================================================= - -.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection - -.. versionadded: 2.3 - -pytest offers very flexible means for managing test resources and -test parametrization. - -The pytest resource management mechanism is an example of `Dependency -Injection`_ because test and :ref:`setup functions ` receive -resources simply by stating them as an input argument. Therefore and -also for historic reasons, they are often called **funcargs**. At test -writing time you typically do not need to care for the details of how -your required resources are constructed, if they live through a -function, class, module or session scope or if the test will be called -multiple times with different resource instances. - -To create a value with which to call a test function a resource factory -function is called which gets full access to the test context and can -register finalizers which are to be run after the last test in that context -finished. Resource factories can be implemented in same test class or -test module, in a per-directory ``conftest.py`` file or in an external -plugin. This allows **total de-coupling of test and setup code**, -lowering the cost of refactoring. - -A test function may be invoked multiple times in which case we -speak of parametrization. You can parametrize resources or parametrize -test function arguments directly or even implement your own parametrization -scheme through a plugin hook. - -A resource has a **name** under which test and setup functions -can access it by listing it as an input argument. Due to this and -also for historic reasons, resources are often called **funcargs**. -A resource is created by a factory which can be flagged with a **scope** -to only create resources on a per-class/per-module/per-session basis -instead of the default per-function scope. - -Concretely, there are three means of resource and parametrization management: - -* a `@pytest.factory`_ marker to define resource factories, - their scoping and parametrization. Factories can themselves - receive resources through their function arguments, easing - the setup of interdependent resources. They can also use - the special `testcontext`_ object to access details n which - the factory/setup is called and for registering finalizers. - -* a `@pytest.mark.parametrize`_ marker for executing test functions - multiple times with different parameter sets - -* a `pytest_generate_tests`_ plugin hook marker for implementing - your parametrization for a test function which may depend on - command line options, class/module attributes etc. - -Finally, pytest comes with some :ref:`builtinresources` which -you can use without defining them yourself. Moreover, third-party -plugins offer their own resources so that after installation -you can simply use them in your test and setup functions. - -.. _`@pytest.factory`: - -``@pytest.factory``: Creating parametrized, scoped resources ------------------------------------------------------------------ - -.. regendoc:wipe - -.. versionadded:: 2.3 - -The `@pytest.factory`_ marker allows to - -* mark a function as a factory for resources, useable by test and setup functions -* define parameters in order to run tests multiple times with different - resource instances -* set a scope which determines the level of caching, i.e. how often - the factory will be called. Valid scopes are ``session``, ``module``, - ``class`` and ``function``. - -Here is a simple example of a factory creating a shared ``smtplib.SMTP`` -connection resource which test functions then may use across the whole -test session:: - - # content of conftest.py - import pytest - import smtplib - - @pytest.factory(scope="session") - def smtp(testcontext): - smtp = smtplib.SMTP("merlinux.eu") - testcontext.addfinalizer(smtp.close) - return smtp - -The name of the resource is ``smtp`` (the factory function name) -and you can now access the ``smtp`` resource by listing it as -an input parameter in any test function below the directory where -``conftest.py`` is located:: - - # content of test_module.py - def test_ehlo(smtp): - response = smtp.ehlo() - assert response[0] == 250 - assert "merlinux" in response[1] - assert 0 # for demo purposes - - def test_noop(smtp): - response = smtp.noop() - assert response[0] == 250 - assert 0 # for demo purposes - -If you run the tests:: - - $ py.test -q - collecting ... collected 2 items - FF - ================================= FAILURES ================================= - ________________________________ test_ehlo _________________________________ - - smtp = - - def test_ehlo(smtp): - response = smtp.ehlo() - assert response[0] == 250 - assert "merlinux" in response[1] - > assert 0 # for demo purposes - E assert 0 - - test_module.py:5: AssertionError - ________________________________ test_noop _________________________________ - - smtp = - - def test_noop(smtp): - response = smtp.noop() - assert response[0] == 250 - > assert 0 # for demo purposes - E assert 0 - - test_module.py:10: AssertionError - 2 failed in 0.18 seconds - -you will see the two ``assert 0`` failing and can see that -the same (session-scoped) object was passed into the two test functions. - -If you now want to test multiple servers you can simply parametrize -the ``smtp`` factory:: - - # content of conftest.py - import pytest - import smtplib - - @pytest.factory(scope="session", - params=["merlinux.eu", "mail.python.org"]) - def smtp(testcontext): - smtp = smtplib.SMTP(testcontext.param) - def fin(): - smtp.close() - testcontext.addfinalizer(fin) - return smtp - -The main change is the definition of a ``params`` list in the -``factory``-marker and the ``testcontext.param`` access within the -factory function. No test code needs to change. So let's just do another -run:: - - $ py.test -q - collecting ... collected 4 items - FFFF - ================================= FAILURES ================================= - __________________________ test_ehlo[merlinux.eu] __________________________ - - smtp = - - def test_ehlo(smtp): - response = smtp.ehlo() - assert response[0] == 250 - assert "merlinux" in response[1] - > assert 0 # for demo purposes - E assert 0 - - test_module.py:5: AssertionError - __________________________ test_noop[merlinux.eu] __________________________ - - smtp = - - def test_noop(smtp): - response = smtp.noop() - assert response[0] == 250 - > assert 0 # for demo purposes - E assert 0 - - test_module.py:10: AssertionError - ________________________ test_ehlo[mail.python.org] ________________________ - - smtp = - - def test_ehlo(smtp): - response = smtp.ehlo() - assert response[0] == 250 - > assert "merlinux" in response[1] - E assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN' - - test_module.py:4: AssertionError - ________________________ test_noop[mail.python.org] ________________________ - - smtp = - - def test_noop(smtp): - response = smtp.noop() - assert response[0] == 250 - > assert 0 # for demo purposes - E assert 0 - - test_module.py:10: AssertionError - 4 failed in 6.42 seconds - -We get four failures because we are running the two tests twice with -different ``smtp`` instantiations as defined on the factory. -Note that with the ``mail.python.org`` connection the second test -fails in ``test_ehlo`` because it expects a specific server string. - -You can also look at what tests pytest collects without running them:: - - $ py.test --collectonly - =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 - plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov - collecting ... collected 4 items - - - - - - - ============================= in 0.02 seconds ============================= - -Note that pytest orders your test run by resource usage, minimizing -the number of active resources at any given time. - - -Interdepdendent resources ----------------------------------------------------------- - -You can not only use resources in test functions but also in resource factories -themselves. Extending the previous example we can instantiate an application -object by sticking the ``smtp`` resource into it:: - - # content of test_appsetup.py - - import pytest - - class App: - def __init__(self, smtp): - self.smtp = smtp - - @pytest.factory(scope="module") - def app(smtp): - return App(smtp) - - def test_exists(app): - assert app.smtp - -Let's run this:: - - $ py.test -v test_appsetup.py - =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 -- /home/hpk/venv/1/bin/python - cachedir: /home/hpk/tmp/doc-exec-398/.cache - plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov - collecting ... collected 2 items - - test_appsetup.py:12: test_exists[merlinux.eu] PASSED - test_appsetup.py:12: test_exists[mail.python.org] PASSED - - ========================= 2 passed in 5.96 seconds ========================= - -Due to the parametrization of ``smtp`` the test will -run twice with two different ``App`` instances and respective smtp servers. -There is no need for the ``app`` factory to be aware of the parametrization. - - - -Grouping tests by resource parameters ----------------------------------------------------------- - -.. regendoc: wipe - -pytest minimizes the number of active resources during test runs. -If you have a parametrized resource, then all the tests using one -resource instance will execute one after another. Then any finalizers -are called for that resource instance and then the next parametrized -resource instance is created and its tests are run. Among other things, -this eases testing of applications which create and use global state. - -The following example uses two parametrized funcargs, one of which is -scoped on a per-module basis:: - - # content of test_module.py - import pytest - - @pytest.factory(scope="module", params=["mod1", "mod2"]) - def modarg(testcontext): - param = testcontext.param - print "create", param - def fin(): - print "fin", param - testcontext.addfinalizer(fin) - return param - - @pytest.factory(scope="function", params=[1,2]) - def otherarg(testcontext): - return testcontext.param - - def test_0(otherarg): - print " test0", otherarg - def test_1(modarg): - print " test1", modarg - def test_2(otherarg, modarg): - print " test2", otherarg, modarg - -Let's run the tests in verbose mode and with looking at the print-output:: - - $ py.test -v -s - =========================== test session starts ============================ - platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 -- /home/hpk/venv/1/bin/python - cachedir: /home/hpk/tmp/doc-exec-398/.cache - plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov - collecting ... collected 8 items - - test_module.py:16: test_0[1] PASSED - test_module.py:16: test_0[2] PASSED - test_module.py:18: test_1[mod1] PASSED - test_module.py:20: test_2[1-mod1] PASSED - test_module.py:20: test_2[2-mod1] PASSED - test_module.py:18: test_1[mod2] PASSED - test_module.py:20: test_2[1-mod2] PASSED - test_module.py:20: test_2[2-mod2] PASSED - - ========================= 8 passed in 0.03 seconds ========================= - test0 1 - test0 2 - create mod1 - test1 mod1 - test2 1 mod1 - test2 2 mod1 - fin mod1 - create mod2 - test1 mod2 - test2 1 mod2 - test2 2 mod2 - fin mod2 - -You can see that the parametrized module-scoped ``modarg`` resource caused -a re-ordering of test execution. The finalizer for the ``mod1`` parametrized -resource was executed before the ``mod2`` resource was setup. - -.. currentmodule:: _pytest.python -.. _`testcontext`: - -``testcontext``: interacting with test context ---------------------------------------------------- - -The ``testcontext`` object may be received by `@pytest.factory`_ or -`@pytest.setup`_ marked functions. It contains information relating -to the test context within which the function executes. Moreover, you -can call ``testcontext.addfinalizer(myfinalizer)`` in order to trigger -a call to ``myfinalizer`` after the last test in the test context has executed. -If passed to a parametrized factory ``testcontext.param`` will contain -a parameter (one value out of the ``params`` list specified with the -`@pytest.factory`_ marker). - -.. autoclass:: _pytest.python.TestContext() - :members: - -.. _`@pytest.mark.parametrize`: - -``@pytest.mark.parametrize``: directly parametrizing test functions ----------------------------------------------------------------------------- - -.. versionadded:: 2.2 - -The builtin ``pytest.mark.parametrize`` decorator enables -parametrization of arguments for a test function. Here is an example -of a test function that wants check for expected output given a certain input:: - - # content of test_expectation.py - import pytest - @pytest.mark.parametrize(("input", "expected"), [ - ("3+5", 8), - ("2+4", 6), - ("6*9", 42), - ]) - def test_eval(input, expected): - assert eval(input) == expected - -we parametrize two arguments of the test function so that the test -function is called three times. Let's run it:: - - $ py.test -q - collecting ... collected 11 items - ..F........ - ================================= FAILURES ================================= - ____________________________ test_eval[6*9-42] _____________________________ - - input = '6*9', expected = 42 - - @pytest.mark.parametrize(("input", "expected"), [ - ("3+5", 8), - ("2+4", 6), - ("6*9", 42), - ]) - def test_eval(input, expected): - > assert eval(input) == expected - E assert 54 == 42 - E + where 54 = eval('6*9') - - test_expectation.py:8: AssertionError - 1 failed, 10 passed in 0.04 seconds - -As expected only one pair of input/output values fails the simple test function. - -Note that there are various ways how you can mark groups of functions, -see :ref:`mark`. - - -.. _`pytest_generate_tests`: - -``pytest_generate_test``: implementing your own parametrization scheme ----------------------------------------------------------------------------- - -.. regendoc:wipe - -Let's say we want to execute a test with different computation -parameters and the parameter range shall be determined by a command -line argument. Let's first write a simple (do-nothing) computation test:: - - # content of test_compute.py - - def test_compute(param1): - assert param1 < 4 - -Now we add a test configuration like this:: - - # content of conftest.py - - def pytest_addoption(parser): - parser.addoption("--all", action="store_true", - help="run all combinations") - - def pytest_generate_tests(metafunc): - if 'param1' in metafunc.funcargnames: - if metafunc.config.option.all: - end = 5 - else: - end = 2 - metafunc.parametrize("param1", range(end)) - -This means that we only run 2 tests if we do not pass ``--all``:: - - $ py.test -q test_compute.py - collecting ... collected 2 items - .. - 2 passed in 0.03 seconds - -We run only two computations, so we see two dots. -let's run the full monty:: - - $ py.test -q --all - collecting ... collected 5 items - ....F - ================================= FAILURES ================================= - _____________________________ test_compute[4] ______________________________ - - param1 = 4 - - def test_compute(param1): - > assert param1 < 4 - E assert 4 < 4 - - test_compute.py:3: AssertionError - 1 failed, 4 passed in 0.03 seconds - -As expected when running the full range of ``param1`` values -we'll get an error on the last one. diff --git a/doc/en/setup.txt b/doc/en/setup.txt index 71ff7bff0..d86c24f93 100644 --- a/doc/en/setup.txt +++ b/doc/en/setup.txt @@ -1,5 +1,7 @@ .. _xunitsetup: .. _setup: +.. _`setup functions`: +.. _`@pytest.setup`: ``@setup`` functions or: xunit on steroids ======================================================== @@ -18,19 +20,20 @@ it has finished. Unlike :ref:`injected resources ` setup functions work indirectly by causing global side effects or setting test case attributes which test methods can then access. -pytest originally introduced in 2005 a fine-grained model of detecting +pytest originally introduced in 2005 a scope-specific model of detecting setup and teardown functions on a per-module, class or function basis. -The Python unittest module and nose have subsequently incorporated them. -This model remains supported as :ref:`old-style xunit`. +The Python unittest package and nose have subsequently incorporated them. +This model remains supported by pytest as :ref:`old-style xunit`. -With pytest-2.3 a new ``pytest.setup()`` decorator is introduced -to mark functions as setup functions which: +Moreover, pytest-2.3 introduces a new ``pytest.setup()`` decorator +to mark functions as setup functions which allow to implement everything +you can do with the old-style and much more. Specifically setup functions: -- can receive resources through funcargs, +- can receive :ref:`resources through funcargs `, - fully interoperate with parametrized resources, -- can be defined in a plugin or conftest.py file and get called +- can be defined in a plugin or :ref:`conftest.py` file and get called on a per-session, per-module, per-class or per-function basis, -- can access the full :ref:`testcontext` for which the setup is called, +- can access the :ref:`testcontext ` for which the setup is called, - can precisely control teardown by registering one or multiple teardown functions as soon as they have performed some actions which need undoing, eliminating the no need for a separate @@ -75,7 +78,7 @@ Our ``mydir`` setup function is executed on a per-function basis, the default scope used by the ``pytest.setup`` decorator. It accesses the ``tmpdir`` resource which provides a new empty directory path object. The ``test_function2`` here checks that -it executes with a fresh directory and specifically +it executes with a fresh directory and that it does not see the previously created ``anotherfile``. We can thus expect two passing tests:: @@ -115,6 +118,11 @@ empty directory ahead of running a test. test modules accessing a global resource ------------------------------------------------------- +.. note:: + + Relying on `global state is considered bad programming practise `_ but when you work with an application + that relies on it you often have no choice. + If you want test modules to access a global resource, you can stick the resource to the module globals in a per-module setup function. We use a :ref:`resource factory diff --git a/doc/en/xunit_old.txt b/doc/en/xunit_setup.txt similarity index 87% rename from doc/en/xunit_old.txt rename to doc/en/xunit_setup.txt index 01f92bd8e..69dbc916f 100644 --- a/doc/en/xunit_old.txt +++ b/doc/en/xunit_setup.txt @@ -4,10 +4,13 @@ Old-style xunit-style setup ======================================== -This section describes the old way how you can implement setup and -teardown on a per-module/class/function basis. It remains fully -supported but it is recommended to rather use :ref:`@setup functions ` -or :ref:`injected resources ` for implementing your setup needs. +.. note:: + + This section describes the old way how you can implement setup and + teardown on a per-module/class/function basis. It remains fully + supported but it is recommended to rather use :ref:`@setup functions + ` or :ref:`injected resources ` for implementing your + setup needs. Module level setup/teardown --------------------------------------