Moved more sections from reference to how-to.
* re-ordered and grouped how-to guides in how-to landing page * retitled moved documents appropriately * separated writing plugins/writing hook functions into two documents
This commit is contained in:
parent
63bc49de70
commit
d171a3f52d
|
@ -24,18 +24,30 @@ How-to guides
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
how-to/usage
|
how-to/usage
|
||||||
how-to/existingtestsuite
|
|
||||||
how-to/assert
|
how-to/assert
|
||||||
how-to/mark
|
|
||||||
how-to/monkeypatch
|
|
||||||
how-to/tmpdir
|
|
||||||
how-to/capture
|
|
||||||
how-to/skipping
|
|
||||||
how-to/parametrize
|
|
||||||
how-to/plugins
|
|
||||||
how-to/nose
|
|
||||||
how-to/bash-completion
|
|
||||||
how-to/fixtures
|
how-to/fixtures
|
||||||
|
how-to/mark
|
||||||
|
how-to/parametrize
|
||||||
|
how-to/tmpdir
|
||||||
|
how-to/monkeypatch
|
||||||
|
how-to/doctest
|
||||||
|
how-to/cache
|
||||||
|
|
||||||
|
how-to/logging
|
||||||
|
how-to/capture-stdout-stderr
|
||||||
|
how-to/capture-warnings
|
||||||
|
how-to/skipping
|
||||||
|
|
||||||
|
how-to/plugins
|
||||||
|
how-to/writing_plugins
|
||||||
|
how-to/writing_hook_functions
|
||||||
|
|
||||||
|
how-to/existingtestsuite
|
||||||
|
how-to/unittest
|
||||||
|
how-to/nose
|
||||||
|
how-to/xunit_setup
|
||||||
|
|
||||||
|
how-to/bash-completion
|
||||||
|
|
||||||
|
|
||||||
Reference guides
|
Reference guides
|
||||||
|
@ -45,14 +57,7 @@ Reference guides
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
reference/fixtures
|
reference/fixtures
|
||||||
reference/warnings
|
|
||||||
reference/doctest
|
|
||||||
reference/cache
|
|
||||||
reference/unittest
|
|
||||||
reference/xunit_setup
|
|
||||||
reference/plugin_list
|
reference/plugin_list
|
||||||
reference/writing_plugins
|
|
||||||
reference/logging
|
|
||||||
reference/customize
|
reference/customize
|
||||||
reference/reference
|
reference/reference
|
||||||
|
|
||||||
|
|
|
@ -6,6 +6,7 @@ About fixtures
|
||||||
.. seealso:: :ref:`how-to-fixtures`
|
.. seealso:: :ref:`how-to-fixtures`
|
||||||
.. seealso:: :ref:`Fixtures reference <reference-fixtures>`
|
.. seealso:: :ref:`Fixtures reference <reference-fixtures>`
|
||||||
|
|
||||||
|
pytest fixtures are designed to be explicit, modular and scalable.
|
||||||
|
|
||||||
What fixtures are
|
What fixtures are
|
||||||
-----------------
|
-----------------
|
||||||
|
|
|
@ -1,16 +1,14 @@
|
||||||
|
.. _`assert`:
|
||||||
|
|
||||||
How to write and report assertions in tests
|
How to write and report assertions in tests
|
||||||
==================================================
|
==================================================
|
||||||
|
|
||||||
.. _`assertfeedback`:
|
|
||||||
.. _`assert with the assert statement`:
|
.. _`assert with the assert statement`:
|
||||||
.. _`assert`:
|
|
||||||
|
|
||||||
|
|
||||||
Asserting with the ``assert`` statement
|
Asserting with the ``assert`` statement
|
||||||
---------------------------------------------------------
|
---------------------------------------------------------
|
||||||
|
|
||||||
``pytest`` allows you to use the standard python ``assert`` for verifying
|
``pytest`` allows you to use the standard Python ``assert`` for verifying
|
||||||
expectations and values in Python tests. For example, you can write the
|
expectations and values in Python tests. For example, you can write the
|
||||||
following:
|
following:
|
||||||
|
|
||||||
|
|
|
@ -2,8 +2,8 @@
|
||||||
.. _cache:
|
.. _cache:
|
||||||
|
|
||||||
|
|
||||||
Cache: working with cross-testrun state
|
How to re-run failed tests and maintain state between test runs
|
||||||
=======================================
|
===============================================================
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
.. _`warnings`:
|
.. _`warnings`:
|
||||||
|
|
||||||
Warnings Capture
|
How to capture warnings
|
||||||
================
|
=======================
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -1,6 +1,6 @@
|
||||||
.. _doctest:
|
.. _doctest:
|
||||||
|
|
||||||
Doctest integration for modules and test files
|
How to run doctests
|
||||||
=========================================================
|
=========================================================
|
||||||
|
|
||||||
By default, all files matching the ``test*.txt`` pattern will
|
By default, all files matching the ``test*.txt`` pattern will
|
|
@ -3,19 +3,60 @@
|
||||||
How-to guides
|
How-to guides
|
||||||
================
|
================
|
||||||
|
|
||||||
|
Core pytest functionality
|
||||||
|
-------------------------
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|
||||||
usage
|
usage
|
||||||
existingtestsuite
|
|
||||||
assert
|
assert
|
||||||
mark
|
|
||||||
monkeypatch
|
|
||||||
tmpdir
|
|
||||||
capture
|
|
||||||
skipping
|
|
||||||
parametrize
|
|
||||||
plugins
|
|
||||||
nose
|
|
||||||
bash-completion
|
|
||||||
fixtures
|
fixtures
|
||||||
|
mark
|
||||||
|
parametrize
|
||||||
|
tmpdir
|
||||||
|
monkeypatch
|
||||||
|
doctest
|
||||||
|
cache
|
||||||
|
|
||||||
|
test output and outcomes
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
logging
|
||||||
|
capture-stdout-stderr
|
||||||
|
capture-warnings
|
||||||
|
skipping
|
||||||
|
|
||||||
|
Plugins
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
plugins
|
||||||
|
writing_plugins
|
||||||
|
writing_hook_functions
|
||||||
|
|
||||||
|
pytest and other test systems
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
existingtestsuite
|
||||||
|
unittest
|
||||||
|
nose
|
||||||
|
xunit_setup
|
||||||
|
|
||||||
|
pytest development environment
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 1
|
||||||
|
|
||||||
|
bash-completion
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -1,10 +1,7 @@
|
||||||
.. _logging:
|
.. _logging:
|
||||||
|
|
||||||
Logging
|
How to manage logging
|
||||||
-------
|
---------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
pytest captures log messages of level ``WARNING`` or above automatically and displays them in their own section
|
pytest captures log messages of level ``WARNING`` or above automatically and displays them in their own section
|
||||||
for each failed test in the same manner as captured stdout and stderr.
|
for each failed test in the same manner as captured stdout and stderr.
|
|
@ -2,8 +2,8 @@
|
||||||
.. _`unittest.TestCase`:
|
.. _`unittest.TestCase`:
|
||||||
.. _`unittest`:
|
.. _`unittest`:
|
||||||
|
|
||||||
unittest.TestCase Support
|
How to use ``unittest``-based tests with pytest
|
||||||
=========================
|
===============================================
|
||||||
|
|
||||||
``pytest`` supports running Python ``unittest``-based tests out of the box.
|
``pytest`` supports running Python ``unittest``-based tests out of the box.
|
||||||
It's meant for leveraging existing ``unittest``-based test suites
|
It's meant for leveraging existing ``unittest``-based test suites
|
|
@ -0,0 +1,313 @@
|
||||||
|
.. _`writinghooks`:
|
||||||
|
|
||||||
|
Writing hook functions
|
||||||
|
======================
|
||||||
|
|
||||||
|
|
||||||
|
.. _validation:
|
||||||
|
|
||||||
|
hook function validation and execution
|
||||||
|
--------------------------------------
|
||||||
|
|
||||||
|
pytest calls hook functions from registered plugins for any
|
||||||
|
given hook specification. Let's look at a typical hook function
|
||||||
|
for the ``pytest_collection_modifyitems(session, config,
|
||||||
|
items)`` hook which pytest calls after collection of all test items is
|
||||||
|
completed.
|
||||||
|
|
||||||
|
When we implement a ``pytest_collection_modifyitems`` function in our plugin
|
||||||
|
pytest will during registration verify that you use argument
|
||||||
|
names which match the specification and bail out if not.
|
||||||
|
|
||||||
|
Let's look at a possible implementation:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
def pytest_collection_modifyitems(config, items):
|
||||||
|
# called after collection is completed
|
||||||
|
# you can modify the ``items`` list
|
||||||
|
...
|
||||||
|
|
||||||
|
Here, ``pytest`` will pass in ``config`` (the pytest config object)
|
||||||
|
and ``items`` (the list of collected test items) but will not pass
|
||||||
|
in the ``session`` argument because we didn't list it in the function
|
||||||
|
signature. This dynamic "pruning" of arguments allows ``pytest`` to
|
||||||
|
be "future-compatible": we can introduce new hook named parameters without
|
||||||
|
breaking the signatures of existing hook implementations. It is one of
|
||||||
|
the reasons for the general long-lived compatibility of pytest plugins.
|
||||||
|
|
||||||
|
Note that hook functions other than ``pytest_runtest_*`` are not
|
||||||
|
allowed to raise exceptions. Doing so will break the pytest run.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. _firstresult:
|
||||||
|
|
||||||
|
firstresult: stop at first non-None result
|
||||||
|
-------------------------------------------
|
||||||
|
|
||||||
|
Most calls to ``pytest`` hooks result in a **list of results** which contains
|
||||||
|
all non-None results of the called hook functions.
|
||||||
|
|
||||||
|
Some hook specifications use the ``firstresult=True`` option so that the hook
|
||||||
|
call only executes until the first of N registered functions returns a
|
||||||
|
non-None result which is then taken as result of the overall hook call.
|
||||||
|
The remaining hook functions will not be called in this case.
|
||||||
|
|
||||||
|
.. _`hookwrapper`:
|
||||||
|
|
||||||
|
hookwrapper: executing around other hooks
|
||||||
|
-------------------------------------------------
|
||||||
|
|
||||||
|
.. currentmodule:: _pytest.core
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
pytest plugins can implement hook wrappers which wrap the execution
|
||||||
|
of other hook implementations. A hook wrapper is a generator function
|
||||||
|
which yields exactly once. When pytest invokes hooks it first executes
|
||||||
|
hook wrappers and passes the same arguments as to the regular hooks.
|
||||||
|
|
||||||
|
At the yield point of the hook wrapper pytest will execute the next hook
|
||||||
|
implementations and return their result to the yield point in the form of
|
||||||
|
a :py:class:`Result <pluggy._Result>` instance which encapsulates a result or
|
||||||
|
exception info. The yield point itself will thus typically not raise
|
||||||
|
exceptions (unless there are bugs).
|
||||||
|
|
||||||
|
Here is an example definition of a hook wrapper:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.hookimpl(hookwrapper=True)
|
||||||
|
def pytest_pyfunc_call(pyfuncitem):
|
||||||
|
do_something_before_next_hook_executes()
|
||||||
|
|
||||||
|
outcome = yield
|
||||||
|
# outcome.excinfo may be None or a (cls, val, tb) tuple
|
||||||
|
|
||||||
|
res = outcome.get_result() # will raise if outcome was exception
|
||||||
|
|
||||||
|
post_process_result(res)
|
||||||
|
|
||||||
|
outcome.force_result(new_res) # to override the return value to the plugin system
|
||||||
|
|
||||||
|
Note that hook wrappers don't return results themselves, they merely
|
||||||
|
perform tracing or other side effects around the actual hook implementations.
|
||||||
|
If the result of the underlying hook is a mutable object, they may modify
|
||||||
|
that result but it's probably better to avoid it.
|
||||||
|
|
||||||
|
For more information, consult the
|
||||||
|
:ref:`pluggy documentation about hookwrappers <pluggy:hookwrappers>`.
|
||||||
|
|
||||||
|
.. _plugin-hookorder:
|
||||||
|
|
||||||
|
Hook function ordering / call example
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
For any given hook specification there may be more than one
|
||||||
|
implementation and we thus generally view ``hook`` execution as a
|
||||||
|
``1:N`` function call where ``N`` is the number of registered functions.
|
||||||
|
There are ways to influence if a hook implementation comes before or
|
||||||
|
after others, i.e. the position in the ``N``-sized list of functions:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# Plugin 1
|
||||||
|
@pytest.hookimpl(tryfirst=True)
|
||||||
|
def pytest_collection_modifyitems(items):
|
||||||
|
# will execute as early as possible
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
# Plugin 2
|
||||||
|
@pytest.hookimpl(trylast=True)
|
||||||
|
def pytest_collection_modifyitems(items):
|
||||||
|
# will execute as late as possible
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
# Plugin 3
|
||||||
|
@pytest.hookimpl(hookwrapper=True)
|
||||||
|
def pytest_collection_modifyitems(items):
|
||||||
|
# will execute even before the tryfirst one above!
|
||||||
|
outcome = yield
|
||||||
|
# will execute after all non-hookwrappers executed
|
||||||
|
|
||||||
|
Here is the order of execution:
|
||||||
|
|
||||||
|
1. Plugin3's pytest_collection_modifyitems called until the yield point
|
||||||
|
because it is a hook wrapper.
|
||||||
|
|
||||||
|
2. Plugin1's pytest_collection_modifyitems is called because it is marked
|
||||||
|
with ``tryfirst=True``.
|
||||||
|
|
||||||
|
3. Plugin2's pytest_collection_modifyitems is called because it is marked
|
||||||
|
with ``trylast=True`` (but even without this mark it would come after
|
||||||
|
Plugin1).
|
||||||
|
|
||||||
|
4. Plugin3's pytest_collection_modifyitems then executing the code after the yield
|
||||||
|
point. The yield receives a :py:class:`Result <pluggy._Result>` instance which encapsulates
|
||||||
|
the result from calling the non-wrappers. Wrappers shall not modify the result.
|
||||||
|
|
||||||
|
It's possible to use ``tryfirst`` and ``trylast`` also in conjunction with
|
||||||
|
``hookwrapper=True`` in which case it will influence the ordering of hookwrappers
|
||||||
|
among each other.
|
||||||
|
|
||||||
|
|
||||||
|
Declaring new hooks
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
This is a quick overview on how to add new hooks and how they work in general, but a more complete
|
||||||
|
overview can be found in `the pluggy documentation <https://pluggy.readthedocs.io/en/latest/>`__.
|
||||||
|
|
||||||
|
.. currentmodule:: _pytest.hookspec
|
||||||
|
|
||||||
|
Plugins and ``conftest.py`` files may declare new hooks that can then be
|
||||||
|
implemented by other plugins in order to alter behaviour or interact with
|
||||||
|
the new plugin:
|
||||||
|
|
||||||
|
.. autofunction:: pytest_addhooks
|
||||||
|
:noindex:
|
||||||
|
|
||||||
|
Hooks are usually declared as do-nothing functions that contain only
|
||||||
|
documentation describing when the hook will be called and what return values
|
||||||
|
are expected. The names of the functions must start with `pytest_` otherwise pytest won't recognize them.
|
||||||
|
|
||||||
|
Here's an example. Let's assume this code is in the ``sample_hook.py`` module.
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
def pytest_my_hook(config):
|
||||||
|
"""
|
||||||
|
Receives the pytest config and does things with it
|
||||||
|
"""
|
||||||
|
|
||||||
|
To register the hooks with pytest they need to be structured in their own module or class. This
|
||||||
|
class or module can then be passed to the ``pluginmanager`` using the ``pytest_addhooks`` function
|
||||||
|
(which itself is a hook exposed by pytest).
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
def pytest_addhooks(pluginmanager):
|
||||||
|
""" This example assumes the hooks are grouped in the 'sample_hook' module. """
|
||||||
|
from my_app.tests import sample_hook
|
||||||
|
|
||||||
|
pluginmanager.add_hookspecs(sample_hook)
|
||||||
|
|
||||||
|
For a real world example, see `newhooks.py`_ from `xdist <https://github.com/pytest-dev/pytest-xdist>`_.
|
||||||
|
|
||||||
|
.. _`newhooks.py`: https://github.com/pytest-dev/pytest-xdist/blob/974bd566c599dc6a9ea291838c6f226197208b46/xdist/newhooks.py
|
||||||
|
|
||||||
|
Hooks may be called both from fixtures or from other hooks. In both cases, hooks are called
|
||||||
|
through the ``hook`` object, available in the ``config`` object. Most hooks receive a
|
||||||
|
``config`` object directly, while fixtures may use the ``pytestconfig`` fixture which provides the same object.
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def my_fixture(pytestconfig):
|
||||||
|
# call the hook called "pytest_my_hook"
|
||||||
|
# 'result' will be a list of return values from all registered functions.
|
||||||
|
result = pytestconfig.hook.pytest_my_hook(config=pytestconfig)
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
Hooks receive parameters using only keyword arguments.
|
||||||
|
|
||||||
|
Now your hook is ready to be used. To register a function at the hook, other plugins or users must
|
||||||
|
now simply define the function ``pytest_my_hook`` with the correct signature in their ``conftest.py``.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
def pytest_my_hook(config):
|
||||||
|
"""
|
||||||
|
Print all active hooks to the screen.
|
||||||
|
"""
|
||||||
|
print(config.hook)
|
||||||
|
|
||||||
|
|
||||||
|
.. _`addoptionhooks`:
|
||||||
|
|
||||||
|
|
||||||
|
Using hooks in pytest_addoption
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
Occasionally, it is necessary to change the way in which command line options
|
||||||
|
are defined by one plugin based on hooks in another plugin. For example,
|
||||||
|
a plugin may expose a command line option for which another plugin needs
|
||||||
|
to define the default value. The pluginmanager can be used to install and
|
||||||
|
use hooks to accomplish this. The plugin would define and add the hooks
|
||||||
|
and use pytest_addoption as follows:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# contents of hooks.py
|
||||||
|
|
||||||
|
# Use firstresult=True because we only want one plugin to define this
|
||||||
|
# default value
|
||||||
|
@hookspec(firstresult=True)
|
||||||
|
def pytest_config_file_default_value():
|
||||||
|
""" Return the default value for the config file command line option. """
|
||||||
|
|
||||||
|
|
||||||
|
# contents of myplugin.py
|
||||||
|
|
||||||
|
|
||||||
|
def pytest_addhooks(pluginmanager):
|
||||||
|
""" This example assumes the hooks are grouped in the 'hooks' module. """
|
||||||
|
from . import hook
|
||||||
|
|
||||||
|
pluginmanager.add_hookspecs(hook)
|
||||||
|
|
||||||
|
|
||||||
|
def pytest_addoption(parser, pluginmanager):
|
||||||
|
default_value = pluginmanager.hook.pytest_config_file_default_value()
|
||||||
|
parser.addoption(
|
||||||
|
"--config-file",
|
||||||
|
help="Config file to use, defaults to %(default)s",
|
||||||
|
default=default_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
The conftest.py that is using myplugin would simply define the hook as follows:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
def pytest_config_file_default_value():
|
||||||
|
return "config.yaml"
|
||||||
|
|
||||||
|
|
||||||
|
Optionally using hooks from 3rd party plugins
|
||||||
|
---------------------------------------------
|
||||||
|
|
||||||
|
Using new hooks from plugins as explained above might be a little tricky
|
||||||
|
because of the standard :ref:`validation mechanism <validation>`:
|
||||||
|
if you depend on a plugin that is not installed, validation will fail and
|
||||||
|
the error message will not make much sense to your users.
|
||||||
|
|
||||||
|
One approach is to defer the hook implementation to a new plugin instead of
|
||||||
|
declaring the hook functions directly in your plugin module, for example:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# contents of myplugin.py
|
||||||
|
|
||||||
|
|
||||||
|
class DeferPlugin:
|
||||||
|
"""Simple plugin to defer pytest-xdist hook functions."""
|
||||||
|
|
||||||
|
def pytest_testnodedown(self, node, error):
|
||||||
|
"""standard xdist hook function."""
|
||||||
|
|
||||||
|
|
||||||
|
def pytest_configure(config):
|
||||||
|
if config.pluginmanager.hasplugin("xdist"):
|
||||||
|
config.pluginmanager.register(DeferPlugin())
|
||||||
|
|
||||||
|
This has the added benefit of allowing you to conditionally install hooks
|
||||||
|
depending on which plugins are installed.
|
|
@ -454,320 +454,3 @@ Additionally it is possible to copy examples for an example folder before runnin
|
||||||
For more information about the result object that ``runpytest()`` returns, and
|
For more information about the result object that ``runpytest()`` returns, and
|
||||||
the methods that it provides please check out the :py:class:`RunResult
|
the methods that it provides please check out the :py:class:`RunResult
|
||||||
<_pytest.pytester.RunResult>` documentation.
|
<_pytest.pytester.RunResult>` documentation.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. _`writinghooks`:
|
|
||||||
|
|
||||||
Writing hook functions
|
|
||||||
======================
|
|
||||||
|
|
||||||
|
|
||||||
.. _validation:
|
|
||||||
|
|
||||||
hook function validation and execution
|
|
||||||
--------------------------------------
|
|
||||||
|
|
||||||
pytest calls hook functions from registered plugins for any
|
|
||||||
given hook specification. Let's look at a typical hook function
|
|
||||||
for the ``pytest_collection_modifyitems(session, config,
|
|
||||||
items)`` hook which pytest calls after collection of all test items is
|
|
||||||
completed.
|
|
||||||
|
|
||||||
When we implement a ``pytest_collection_modifyitems`` function in our plugin
|
|
||||||
pytest will during registration verify that you use argument
|
|
||||||
names which match the specification and bail out if not.
|
|
||||||
|
|
||||||
Let's look at a possible implementation:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
def pytest_collection_modifyitems(config, items):
|
|
||||||
# called after collection is completed
|
|
||||||
# you can modify the ``items`` list
|
|
||||||
...
|
|
||||||
|
|
||||||
Here, ``pytest`` will pass in ``config`` (the pytest config object)
|
|
||||||
and ``items`` (the list of collected test items) but will not pass
|
|
||||||
in the ``session`` argument because we didn't list it in the function
|
|
||||||
signature. This dynamic "pruning" of arguments allows ``pytest`` to
|
|
||||||
be "future-compatible": we can introduce new hook named parameters without
|
|
||||||
breaking the signatures of existing hook implementations. It is one of
|
|
||||||
the reasons for the general long-lived compatibility of pytest plugins.
|
|
||||||
|
|
||||||
Note that hook functions other than ``pytest_runtest_*`` are not
|
|
||||||
allowed to raise exceptions. Doing so will break the pytest run.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
.. _firstresult:
|
|
||||||
|
|
||||||
firstresult: stop at first non-None result
|
|
||||||
-------------------------------------------
|
|
||||||
|
|
||||||
Most calls to ``pytest`` hooks result in a **list of results** which contains
|
|
||||||
all non-None results of the called hook functions.
|
|
||||||
|
|
||||||
Some hook specifications use the ``firstresult=True`` option so that the hook
|
|
||||||
call only executes until the first of N registered functions returns a
|
|
||||||
non-None result which is then taken as result of the overall hook call.
|
|
||||||
The remaining hook functions will not be called in this case.
|
|
||||||
|
|
||||||
.. _`hookwrapper`:
|
|
||||||
|
|
||||||
hookwrapper: executing around other hooks
|
|
||||||
-------------------------------------------------
|
|
||||||
|
|
||||||
.. currentmodule:: _pytest.core
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
pytest plugins can implement hook wrappers which wrap the execution
|
|
||||||
of other hook implementations. A hook wrapper is a generator function
|
|
||||||
which yields exactly once. When pytest invokes hooks it first executes
|
|
||||||
hook wrappers and passes the same arguments as to the regular hooks.
|
|
||||||
|
|
||||||
At the yield point of the hook wrapper pytest will execute the next hook
|
|
||||||
implementations and return their result to the yield point in the form of
|
|
||||||
a :py:class:`Result <pluggy._Result>` instance which encapsulates a result or
|
|
||||||
exception info. The yield point itself will thus typically not raise
|
|
||||||
exceptions (unless there are bugs).
|
|
||||||
|
|
||||||
Here is an example definition of a hook wrapper:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.hookimpl(hookwrapper=True)
|
|
||||||
def pytest_pyfunc_call(pyfuncitem):
|
|
||||||
do_something_before_next_hook_executes()
|
|
||||||
|
|
||||||
outcome = yield
|
|
||||||
# outcome.excinfo may be None or a (cls, val, tb) tuple
|
|
||||||
|
|
||||||
res = outcome.get_result() # will raise if outcome was exception
|
|
||||||
|
|
||||||
post_process_result(res)
|
|
||||||
|
|
||||||
outcome.force_result(new_res) # to override the return value to the plugin system
|
|
||||||
|
|
||||||
Note that hook wrappers don't return results themselves, they merely
|
|
||||||
perform tracing or other side effects around the actual hook implementations.
|
|
||||||
If the result of the underlying hook is a mutable object, they may modify
|
|
||||||
that result but it's probably better to avoid it.
|
|
||||||
|
|
||||||
For more information, consult the
|
|
||||||
:ref:`pluggy documentation about hookwrappers <pluggy:hookwrappers>`.
|
|
||||||
|
|
||||||
.. _plugin-hookorder:
|
|
||||||
|
|
||||||
Hook function ordering / call example
|
|
||||||
-------------------------------------
|
|
||||||
|
|
||||||
For any given hook specification there may be more than one
|
|
||||||
implementation and we thus generally view ``hook`` execution as a
|
|
||||||
``1:N`` function call where ``N`` is the number of registered functions.
|
|
||||||
There are ways to influence if a hook implementation comes before or
|
|
||||||
after others, i.e. the position in the ``N``-sized list of functions:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
# Plugin 1
|
|
||||||
@pytest.hookimpl(tryfirst=True)
|
|
||||||
def pytest_collection_modifyitems(items):
|
|
||||||
# will execute as early as possible
|
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
# Plugin 2
|
|
||||||
@pytest.hookimpl(trylast=True)
|
|
||||||
def pytest_collection_modifyitems(items):
|
|
||||||
# will execute as late as possible
|
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
# Plugin 3
|
|
||||||
@pytest.hookimpl(hookwrapper=True)
|
|
||||||
def pytest_collection_modifyitems(items):
|
|
||||||
# will execute even before the tryfirst one above!
|
|
||||||
outcome = yield
|
|
||||||
# will execute after all non-hookwrappers executed
|
|
||||||
|
|
||||||
Here is the order of execution:
|
|
||||||
|
|
||||||
1. Plugin3's pytest_collection_modifyitems called until the yield point
|
|
||||||
because it is a hook wrapper.
|
|
||||||
|
|
||||||
2. Plugin1's pytest_collection_modifyitems is called because it is marked
|
|
||||||
with ``tryfirst=True``.
|
|
||||||
|
|
||||||
3. Plugin2's pytest_collection_modifyitems is called because it is marked
|
|
||||||
with ``trylast=True`` (but even without this mark it would come after
|
|
||||||
Plugin1).
|
|
||||||
|
|
||||||
4. Plugin3's pytest_collection_modifyitems then executing the code after the yield
|
|
||||||
point. The yield receives a :py:class:`Result <pluggy._Result>` instance which encapsulates
|
|
||||||
the result from calling the non-wrappers. Wrappers shall not modify the result.
|
|
||||||
|
|
||||||
It's possible to use ``tryfirst`` and ``trylast`` also in conjunction with
|
|
||||||
``hookwrapper=True`` in which case it will influence the ordering of hookwrappers
|
|
||||||
among each other.
|
|
||||||
|
|
||||||
|
|
||||||
Declaring new hooks
|
|
||||||
------------------------
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
This is a quick overview on how to add new hooks and how they work in general, but a more complete
|
|
||||||
overview can be found in `the pluggy documentation <https://pluggy.readthedocs.io/en/latest/>`__.
|
|
||||||
|
|
||||||
.. currentmodule:: _pytest.hookspec
|
|
||||||
|
|
||||||
Plugins and ``conftest.py`` files may declare new hooks that can then be
|
|
||||||
implemented by other plugins in order to alter behaviour or interact with
|
|
||||||
the new plugin:
|
|
||||||
|
|
||||||
.. autofunction:: pytest_addhooks
|
|
||||||
:noindex:
|
|
||||||
|
|
||||||
Hooks are usually declared as do-nothing functions that contain only
|
|
||||||
documentation describing when the hook will be called and what return values
|
|
||||||
are expected. The names of the functions must start with `pytest_` otherwise pytest won't recognize them.
|
|
||||||
|
|
||||||
Here's an example. Let's assume this code is in the ``sample_hook.py`` module.
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
def pytest_my_hook(config):
|
|
||||||
"""
|
|
||||||
Receives the pytest config and does things with it
|
|
||||||
"""
|
|
||||||
|
|
||||||
To register the hooks with pytest they need to be structured in their own module or class. This
|
|
||||||
class or module can then be passed to the ``pluginmanager`` using the ``pytest_addhooks`` function
|
|
||||||
(which itself is a hook exposed by pytest).
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
def pytest_addhooks(pluginmanager):
|
|
||||||
""" This example assumes the hooks are grouped in the 'sample_hook' module. """
|
|
||||||
from my_app.tests import sample_hook
|
|
||||||
|
|
||||||
pluginmanager.add_hookspecs(sample_hook)
|
|
||||||
|
|
||||||
For a real world example, see `newhooks.py`_ from `xdist <https://github.com/pytest-dev/pytest-xdist>`_.
|
|
||||||
|
|
||||||
.. _`newhooks.py`: https://github.com/pytest-dev/pytest-xdist/blob/974bd566c599dc6a9ea291838c6f226197208b46/xdist/newhooks.py
|
|
||||||
|
|
||||||
Hooks may be called both from fixtures or from other hooks. In both cases, hooks are called
|
|
||||||
through the ``hook`` object, available in the ``config`` object. Most hooks receive a
|
|
||||||
``config`` object directly, while fixtures may use the ``pytestconfig`` fixture which provides the same object.
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
@pytest.fixture()
|
|
||||||
def my_fixture(pytestconfig):
|
|
||||||
# call the hook called "pytest_my_hook"
|
|
||||||
# 'result' will be a list of return values from all registered functions.
|
|
||||||
result = pytestconfig.hook.pytest_my_hook(config=pytestconfig)
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Hooks receive parameters using only keyword arguments.
|
|
||||||
|
|
||||||
Now your hook is ready to be used. To register a function at the hook, other plugins or users must
|
|
||||||
now simply define the function ``pytest_my_hook`` with the correct signature in their ``conftest.py``.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
def pytest_my_hook(config):
|
|
||||||
"""
|
|
||||||
Print all active hooks to the screen.
|
|
||||||
"""
|
|
||||||
print(config.hook)
|
|
||||||
|
|
||||||
|
|
||||||
.. _`addoptionhooks`:
|
|
||||||
|
|
||||||
|
|
||||||
Using hooks in pytest_addoption
|
|
||||||
-------------------------------
|
|
||||||
|
|
||||||
Occasionally, it is necessary to change the way in which command line options
|
|
||||||
are defined by one plugin based on hooks in another plugin. For example,
|
|
||||||
a plugin may expose a command line option for which another plugin needs
|
|
||||||
to define the default value. The pluginmanager can be used to install and
|
|
||||||
use hooks to accomplish this. The plugin would define and add the hooks
|
|
||||||
and use pytest_addoption as follows:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
# contents of hooks.py
|
|
||||||
|
|
||||||
# Use firstresult=True because we only want one plugin to define this
|
|
||||||
# default value
|
|
||||||
@hookspec(firstresult=True)
|
|
||||||
def pytest_config_file_default_value():
|
|
||||||
""" Return the default value for the config file command line option. """
|
|
||||||
|
|
||||||
|
|
||||||
# contents of myplugin.py
|
|
||||||
|
|
||||||
|
|
||||||
def pytest_addhooks(pluginmanager):
|
|
||||||
""" This example assumes the hooks are grouped in the 'hooks' module. """
|
|
||||||
from . import hook
|
|
||||||
|
|
||||||
pluginmanager.add_hookspecs(hook)
|
|
||||||
|
|
||||||
|
|
||||||
def pytest_addoption(parser, pluginmanager):
|
|
||||||
default_value = pluginmanager.hook.pytest_config_file_default_value()
|
|
||||||
parser.addoption(
|
|
||||||
"--config-file",
|
|
||||||
help="Config file to use, defaults to %(default)s",
|
|
||||||
default=default_value,
|
|
||||||
)
|
|
||||||
|
|
||||||
The conftest.py that is using myplugin would simply define the hook as follows:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
def pytest_config_file_default_value():
|
|
||||||
return "config.yaml"
|
|
||||||
|
|
||||||
|
|
||||||
Optionally using hooks from 3rd party plugins
|
|
||||||
---------------------------------------------
|
|
||||||
|
|
||||||
Using new hooks from plugins as explained above might be a little tricky
|
|
||||||
because of the standard :ref:`validation mechanism <validation>`:
|
|
||||||
if you depend on a plugin that is not installed, validation will fail and
|
|
||||||
the error message will not make much sense to your users.
|
|
||||||
|
|
||||||
One approach is to defer the hook implementation to a new plugin instead of
|
|
||||||
declaring the hook functions directly in your plugin module, for example:
|
|
||||||
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
# contents of myplugin.py
|
|
||||||
|
|
||||||
|
|
||||||
class DeferPlugin:
|
|
||||||
"""Simple plugin to defer pytest-xdist hook functions."""
|
|
||||||
|
|
||||||
def pytest_testnodedown(self, node, error):
|
|
||||||
"""standard xdist hook function."""
|
|
||||||
|
|
||||||
|
|
||||||
def pytest_configure(config):
|
|
||||||
if config.pluginmanager.hasplugin("xdist"):
|
|
||||||
config.pluginmanager.register(DeferPlugin())
|
|
||||||
|
|
||||||
This has the added benefit of allowing you to conditionally install hooks
|
|
||||||
depending on which plugins are installed.
|
|
|
@ -2,7 +2,7 @@
|
||||||
.. _`classic xunit`:
|
.. _`classic xunit`:
|
||||||
.. _xunitsetup:
|
.. _xunitsetup:
|
||||||
|
|
||||||
classic xunit-style setup
|
How to implement xunit-style set-up
|
||||||
========================================
|
========================================
|
||||||
|
|
||||||
This section describes a classic and popular way how you can implement
|
This section describes a classic and popular way how you can implement
|
|
@ -5,7 +5,7 @@
|
||||||
.. _`pytest.fixture`:
|
.. _`pytest.fixture`:
|
||||||
|
|
||||||
|
|
||||||
pytest fixtures: explicit, modular, scalable
|
Fixtures reference
|
||||||
========================================================
|
========================================================
|
||||||
|
|
||||||
.. seealso:: :ref:`about-fixtures`
|
.. seealso:: :ref:`about-fixtures`
|
||||||
|
|
|
@ -7,13 +7,6 @@ Reference guides
|
||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|
||||||
fixtures
|
fixtures
|
||||||
warnings
|
|
||||||
doctest
|
|
||||||
cache
|
|
||||||
unittest
|
|
||||||
xunit_setup
|
|
||||||
plugin_list
|
plugin_list
|
||||||
writing_plugins
|
|
||||||
logging
|
|
||||||
customize
|
customize
|
||||||
reference
|
reference
|
||||||
|
|
Loading…
Reference in New Issue