219 lines
8.6 KiB
ReStructuredText
219 lines
8.6 KiB
ReStructuredText
:orphan:
|
|
|
|
.. _`funcargcompare`:
|
|
|
|
pytest-2.3: reasoning for fixture/funcarg evolution
|
|
=============================================================
|
|
|
|
**Target audience**: Reading this document requires basic knowledge of
|
|
python testing, xUnit setup methods and the (previous) basic pytest
|
|
funcarg mechanism, see http://pytest.org/2.2.4/funcargs.html
|
|
If you are new to pytest, then you can simply ignore this
|
|
section and read the other sections.
|
|
|
|
.. currentmodule:: _pytest
|
|
|
|
Shortcomings of the previous ``pytest_funcarg__`` mechanism
|
|
--------------------------------------------------------------
|
|
|
|
The pre pytest-2.3 funcarg mechanism calls a factory each time a
|
|
funcarg for a test function is required. If a factory wants to
|
|
re-use a resource across different scopes, it often used
|
|
the ``request.cached_setup()`` helper to manage caching of
|
|
resources. Here is a basic example how we could implement
|
|
a per-session Database object::
|
|
|
|
# content of conftest.py
|
|
class Database:
|
|
def __init__(self):
|
|
print ("database instance created")
|
|
def destroy(self):
|
|
print ("database instance destroyed")
|
|
|
|
def pytest_funcarg__db(request):
|
|
return request.cached_setup(setup=DataBase,
|
|
teardown=lambda db: db.destroy,
|
|
scope="session")
|
|
|
|
There are several limitations and difficulties with this approach:
|
|
|
|
1. Scoping funcarg resource creation is not straight forward, instead one must
|
|
understand the intricate cached_setup() method mechanics.
|
|
|
|
2. parametrizing the "db" resource is not straight forward:
|
|
you need to apply a "parametrize" decorator or implement a
|
|
:py:func:`~hookspec.pytest_generate_tests` hook
|
|
calling :py:func:`~python.Metafunc.parametrize` which
|
|
performs parametrization at the places where the resource
|
|
is used. Moreover, you need to modify the factory to use an
|
|
``extrakey`` parameter containing ``request.param`` to the
|
|
:py:func:`~python.Request.cached_setup` call.
|
|
|
|
3. Multiple parametrized session-scoped resources will be active
|
|
at the same time, making it hard for them to affect global state
|
|
of the application under test.
|
|
|
|
4. there is no way how you can make use of funcarg factories
|
|
in xUnit setup methods.
|
|
|
|
5. A non-parametrized fixture function cannot use a parametrized
|
|
funcarg resource if it isn't stated in the test function signature.
|
|
|
|
All of these limitations are addressed with pytest-2.3 and its
|
|
improved :ref:`fixture mechanism <fixture>`.
|
|
|
|
|
|
Direct scoping of fixture/funcarg factories
|
|
--------------------------------------------------------
|
|
|
|
Instead of calling cached_setup() with a cache scope, you can use the
|
|
:ref:`@pytest.fixture <pytest.fixture>` decorator and directly state
|
|
the scope::
|
|
|
|
@pytest.fixture(scope="session")
|
|
def db(request):
|
|
# factory will only be invoked once per session -
|
|
db = DataBase()
|
|
request.addfinalizer(db.destroy) # destroy when session is finished
|
|
return db
|
|
|
|
This factory implementation does not need to call ``cached_setup()`` anymore
|
|
because it will only be invoked once per session. Moreover, the
|
|
``request.addfinalizer()`` registers a finalizer according to the specified
|
|
resource scope on which the factory function is operating.
|
|
|
|
|
|
Direct parametrization of funcarg resource factories
|
|
----------------------------------------------------------
|
|
|
|
Previously, funcarg factories could not directly cause parametrization.
|
|
You needed to specify a ``@parametrize`` decorator on your test function
|
|
or implement a ``pytest_generate_tests`` hook to perform
|
|
parametrization, i.e. calling a test multiple times with different value
|
|
sets. pytest-2.3 introduces a decorator for use on the factory itself::
|
|
|
|
@pytest.fixture(params=["mysql", "pg"])
|
|
def db(request):
|
|
... # use request.param
|
|
|
|
Here the factory will be invoked twice (with the respective "mysql"
|
|
and "pg" values set as ``request.param`` attributes) and all of
|
|
the tests requiring "db" will run twice as well. The "mysql" and
|
|
"pg" values will also be used for reporting the test-invocation variants.
|
|
|
|
This new way of parametrizing funcarg factories should in many cases
|
|
allow to re-use already written factories because effectively
|
|
``request.param`` was already used when test functions/classes were
|
|
parametrized via
|
|
:py:func:`~_pytest.python.Metafunc.parametrize(indirect=True)` calls.
|
|
|
|
Of course it's perfectly fine to combine parametrization and scoping::
|
|
|
|
@pytest.fixture(scope="session", params=["mysql", "pg"])
|
|
def db(request):
|
|
if request.param == "mysql":
|
|
db = MySQL()
|
|
elif request.param == "pg":
|
|
db = PG()
|
|
request.addfinalizer(db.destroy) # destroy when session is finished
|
|
return db
|
|
|
|
This would execute all tests requiring the per-session "db" resource twice,
|
|
receiving the values created by the two respective invocations to the
|
|
factory function.
|
|
|
|
|
|
No ``pytest_funcarg__`` prefix when using @fixture decorator
|
|
-------------------------------------------------------------------
|
|
|
|
When using the ``@fixture`` decorator the name of the function
|
|
denotes the name under which the resource can be accessed as a function
|
|
argument::
|
|
|
|
@pytest.fixture()
|
|
def db(request):
|
|
...
|
|
|
|
The name under which the funcarg resource can be requested is ``db``.
|
|
|
|
You can still use the "old" non-decorator way of specifying funcarg factories
|
|
aka::
|
|
|
|
def pytest_funcarg__db(request):
|
|
...
|
|
|
|
|
|
But it is then not possible to define scoping and parametrization.
|
|
It is thus recommended to use the factory decorator.
|
|
|
|
|
|
solving per-session setup / autouse fixtures
|
|
--------------------------------------------------------------
|
|
|
|
pytest for a long time offered a pytest_configure and a pytest_sessionstart
|
|
hook which are often used to setup global resources. This suffers from
|
|
several problems:
|
|
|
|
1. in distributed testing the master process would setup test resources
|
|
that are never needed because it only co-ordinates the test run
|
|
activities of the slave processes.
|
|
|
|
2. if you only perform a collection (with "--collect-only")
|
|
resource-setup will still be executed.
|
|
|
|
3. If a pytest_sessionstart is contained in some subdirectories
|
|
conftest.py file, it will not be called. This stems from the
|
|
fact that this hook is actually used for reporting, in particular
|
|
the test-header with platform/custom information.
|
|
|
|
Moreover, it was not easy to define a scoped setup from plugins or
|
|
conftest files other than to implement a ``pytest_runtest_setup()`` hook
|
|
and caring for scoping/caching yourself. And it's virtually impossible
|
|
to do this with parametrization as ``pytest_runtest_setup()`` is called
|
|
during test execution and parametrization happens at collection time.
|
|
|
|
It follows that pytest_configure/session/runtest_setup are often not
|
|
appropriate for implementing common fixture needs. Therefore,
|
|
pytest-2.3 introduces :ref:`autouse fixtures` which fully
|
|
integrate with the generic :ref:`fixture mechanism <fixture>`
|
|
and obsolete many prior uses of pytest hooks.
|
|
|
|
funcargs/fixture discovery now happens at collection time
|
|
---------------------------------------------------------------------
|
|
|
|
Since pytest-2.3, discovery of fixture/funcarg factories are taken care of
|
|
at collection time. This is more efficient especially for large test suites.
|
|
Moreover, a call to "pytest --collect-only" should be able to in the future
|
|
show a lot of setup-information and thus presents a nice method to get an
|
|
overview of fixture management in your project.
|
|
|
|
.. _`compatibility notes`:
|
|
|
|
.. _`funcargscompat`:
|
|
|
|
Conclusion and compatibility notes
|
|
---------------------------------------------------------
|
|
|
|
**funcargs** were originally introduced to pytest-2.0. In pytest-2.3
|
|
the mechanism was extended and refined and is now described as
|
|
fixtures:
|
|
|
|
* previously funcarg factories were specified with a special
|
|
``pytest_funcarg__NAME`` prefix instead of using the
|
|
``@pytest.fixture`` decorator.
|
|
|
|
* Factories received a ``request`` object which managed caching through
|
|
``request.cached_setup()`` calls and allowed using other funcargs via
|
|
``request.getfuncargvalue()`` calls. These intricate APIs made it hard
|
|
to do proper parametrization and implement resource caching. The
|
|
new :py:func:`pytest.fixture` decorator allows to declare the scope
|
|
and let pytest figure things out for you.
|
|
|
|
* if you used parametrization and funcarg factories which made use of
|
|
``request.cached_setup()`` it is recommended to invest a few minutes
|
|
and simplify your fixture function code to use the :ref:`@pytest.fixture`
|
|
decorator instead. This will also allow to take advantage of
|
|
the automatic per-resource grouping of tests.
|
|
|
|
|