diff --git a/doc/test/examples.txt b/doc/test/examples.txt index a685399ce..722222e23 100644 --- a/doc/test/examples.txt +++ b/doc/test/examples.txt @@ -1,3 +1,23 @@ +Learning by examples +===================== + +adding custom options +---------------------- + +py.test supports adding of standard optparse_ Options. +A plugin may implement the ``addoption`` hook for registering +custom options:: + + def pytest_addoption(parser): + parser.addoption("-M", "--myopt", action="store", + help="specify string to set myopt") + + def pytest_configure(config): + if config.option.myopt: + # do action based on option value + # + +.. _optparse: http://docs.python.org/library/optparse.html Working Examples ================ diff --git a/doc/test/ext.txt b/doc/test/ext.txt deleted file mode 100644 index eff736c93..000000000 --- a/doc/test/ext.txt +++ /dev/null @@ -1,127 +0,0 @@ -====================================== -Writing plugins and extensions -====================================== - - -.. _`local plugin`: - -Local Plugins -================================== - -You can easily specify a project-specific or "local" -plugin by defining a ``ConftestPlugin`` in a ``conftest.py`` -file like this:: - - class ConftestPlugin: - """ my local plugin. """ - - -Learning by examples -===================== - -XXX - -adding custom options ----------------------- - -py.test supports adding of standard optparse_ Options. -A plugin may implement the ``addoption`` hook for registering -custom options:: - - class ConftestPlugin: - def pytest_addoption(self, parser): - parser.addoption("-M", "--myopt", action="store", - help="specify string to set myopt") - - def pytest_configure(self, config): - if config.option.myopt: - # do action based on option value - -.. _optparse: http://docs.python.org/library/optparse.html - -Setting default values for test options -======================================= - -You can see all available command line options by running:: - - py.test -h - -py.test will lookup values of options in this order: - -* option value supplied at command line -* content of environment variable ``PYTEST_OPTION_NAME=...`` -* ``name = ...`` setting in the nearest ``conftest.py`` file. - -The name of an option usually is the one you find -in the longform of the option, i.e. the name -behind the ``--`` double-dash. - -IOW, you can set default values for options per project, per -home-directoray, per shell session or per test-run. - -.. _`collection process`: - -Test Collection process -====================================================== - -The collecting process is iterative so that distribution -and execution of tests can start as soon as the first test -item is collected. Collection nodes with children are -called "Collectors" and terminal nodes are called "Items". -Here is an example of such a tree, generated with the -command ``py.test --collectonly py/xmlobj``:: - - - - - - - - - - - - - - - - -By default all directories not starting with a dot are traversed, -looking for ``test_*.py`` and ``*_test.py`` files. Those Python -files are imported under their `package name`_. - -The Module collector looks for test functions -and test classes and methods. Test functions and methods -are prefixed ``test`` by default. Test classes must -start with a capitalized ``Test`` prefix. - -.. _`package name`: - -constructing the package name for test modules -------------------------------------------------- - -Test modules are imported under their fully qualified -name. Given a filesystem ``fspath`` it is constructed as follows: - -* walk the directories up to the last one that contains - an ``__init__.py`` file. - -* perform ``sys.path.insert(0, basedir)``. - -* import the root package as ``root`` - -* determine the fully qualified name for ``fspath`` by either: - - * calling ``root.__pkg__.getimportname(fspath)`` if the - ``__pkg__`` exists.` or - - * otherwise use the relative path of the module path to - the base dir and turn slashes into dots and strike - the trailing ``.py``. - - - -Plugin hooks and events -======================================= - -XXX diff --git a/doc/test/extend.txt b/doc/test/extend.txt new file mode 100644 index 000000000..9ede74163 --- /dev/null +++ b/doc/test/extend.txt @@ -0,0 +1,174 @@ +================================================ +Extending and customizating py.test +================================================ + +.. _`original definition of the hook`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/api.py + +.. _`local plugin`: + +py.test implements much of its functionality by calling `well specified +hooks`_. Hook functions are defined in local or global plugins. +By default local plugins are the ``conftest.py`` modules in your project. +Global plugins are python modules or packages with a ``pytest_`` prefixed name. + + +Loading plugins and specifying dependencies +============================================ + +py.test loads plugin modules at tool startup in the following ways: + +* by reading the ``PYTEST_PLUGINS`` environment variable + and importing the comma-separated list of plugin names. + +* by pre-scanning the command line for the ``-p name`` option + and loading the specified plugin before actual command line parsing. + +* by loading all plugins specified by the ``pytest_plugins`` + variable in a ``conftest.py`` file or test modules. + +Note that at tool startup only ``conftest.py`` files in +the directory of the specified test modules (or the current dir if None) +or any of the parent directories are found. There is no try to +pre-scan all subdirectories to find ``conftest.py`` files or test +modules. Each plugins may specify its dependencies via +``pytest_plugins`` definition recursively. + +.. _`well specified hooks`: + +Available py.test hooks +==================================== + +A py.test hook is nothing more than a python function with +a ``pytest_`` prefixed name taking a number of arguments. +When loading a plugin module which contains hooks py.test performs +strict checking on all hook functions. Function and argument names need +to match exactly the `original definition of the hook`_. This allows +for early mismatch reporting and minimizes version incompatibilites. + +"runtest" hooks +------------------- + +Each test item is usually executed by calling the following three hooks:: + + pytest_runtest_setup(item) + pytest_runtest_call(item) + pytest_runtest_teardown(item) + +For each of the three invocations a `call object`_ encapsulates +information about the outcome of the call and is subsequently used +to make a report object:: + + report = hook.pytest_runtest_makereport(item, call) + +For example, the `pytest_pdb plugin`_ uses this hook to activate +interactive debugging on failures when ``--pdb`` is specified on the +command line. + +Usually three reports will be generated for a single test item. However, +if the ``pytest_runtest_setup`` fails no call or teardown hooks +will be called and only one report will be created. + +Each of the up to three reports is eventually fed to the logreport hook:: + + pytest_runtest_logreport(report) + +A ``report`` object contains status and reporting information:: + + report.longrepr = string/lines/object to print + report.when = "setup", "call" or "teardown" + report.shortrepr = letter for progress-report + report.passed = True or False + report.failed = True or False + report.skipped = True or False + +The `pytest_terminal plugin`_ uses this hook to print information +about a test run. + +The protocol described here is implemented via this hook:: + + pytest_runtest_protocol(item) -> True + +.. _`call object`: + +The call object contains information about a performed call:: + + call.excinfo = ExceptionInfo object or None + call.when = "setup", "call" or "teardown" + call.outerr = None or tuple of strings representing captured stdout/stderr + +.. _`pytest_pdb plugin`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/pytest_pdb.py +.. _`pytest_terminal plugin`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/pytest_terminal.py + + +Included default plugins +============================= + +You can find the source code of all default plugins in + + http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/ + +Additionally you can check out some more contributed plugins here + + http://bitbucket.org/hpk42/py-trunk/src/tip/contrib/ + + +.. _`collection process`: + +Test Collection process +====================================================== + +The collecting process is iterative so that distribution +and execution of tests can start as soon as the first test +item is collected. Collection nodes with children are +called "Collectors" and terminal nodes are called "Items". +Here is an example of such a tree, generated with the +command ``py.test --collectonly py/xmlobj``:: + + + + + + + + + + + + + + + + +By default all directories not starting with a dot are traversed, +looking for ``test_*.py`` and ``*_test.py`` files. Those Python +files are imported under their `package name`_. + +The Module collector looks for test functions +and test classes and methods. Test functions and methods +are prefixed ``test`` by default. Test classes must +start with a capitalized ``Test`` prefix. + +.. _`package name`: + +constructing the package name for test modules +------------------------------------------------- + +Test modules are imported under their fully qualified +name. Given a filesystem ``fspath`` it is constructed as follows: + +* walk the directories up to the last one that contains + an ``__init__.py`` file. + +* perform ``sys.path.insert(0, basedir)``. + +* import the root package as ``root`` + +* determine the fully qualified name for ``fspath`` by either: + + * calling ``root.__pkg__.getimportname(fspath)`` if the + ``__pkg__`` exists.` or + + * otherwise use the relative path of the module path to + the base dir and turn slashes into dots and strike + the trailing ``.py``. + diff --git a/doc/test/features.txt b/doc/test/features.txt index 0ef9c8353..7cd9babba 100644 --- a/doc/test/features.txt +++ b/doc/test/features.txt @@ -30,7 +30,7 @@ modules a leading ``test_`` or trailing ``_test`` filename. From each test module every function with a leading ``test_`` or class with a leading ``Test`` name is collected. -.. _`collection process`: ext.html#collection-process +.. _`collection process`: extend.html#collection-process funcargs and xUnit style setups @@ -287,19 +287,19 @@ To make it easier to distinguish the generated tests it is possible to specify a for x in (42,17,49): yield "case %d" % x, check, x -extensible plugin system +easy to extend ========================================= -py.test itself consists of many plugins -and you can easily write new `py.test plugins`_ -for these purposes: +Since 1.0 py.test has advanced `extension mechanisms`_. +One can can easily modify or add aspects for for +purposes such as: * reporting extensions -* customizing collection and run of tests +* customizing collection and execution of tests * running non-python tests -* managing test state setup +* managing custom test state setup -.. _`py.test plugins`: plugins.html +.. _`extension mechanisms`: extend.html .. _`reStructured Text`: http://docutils.sourceforge.net .. _`Python debugger`: http://docs.python.org/lib/module-pdb.html diff --git a/doc/test/funcargs.txt b/doc/test/funcargs.txt index da58b059e..f3dd40fc5 100644 --- a/doc/test/funcargs.txt +++ b/doc/test/funcargs.txt @@ -402,7 +402,7 @@ confused as to what the concrete question or answers actually mean, please see here_ :) Otherwise proceed to step 2. .. _here: http://uncyclopedia.wikia.com/wiki/The_Hitchhiker's_Guide_to_the_Galaxy -.. _`local plugin`: ext.html#local-plugin +.. _`local plugin`: extend.html#local-plugin step 2: adding a command line option diff --git a/doc/test/plugins.txt b/doc/test/plugins.txt deleted file mode 100644 index a17954cf7..000000000 --- a/doc/test/plugins.txt +++ /dev/null @@ -1,110 +0,0 @@ -========================== -hooks and plugins -========================== - -py.test implements much of its functionality by calling **hooks**. -A hook is a function with a ``pytest_`` prefixed name. Hook functions -are usually defined in plugins. A plugin is a regular python module or -package that makes hook functions available. - -When loading a plugin module (which needs to have a ``pytest_`` prefix as well) -py.test performs strict checking on the function signature. Function -and argument names need to match exactly the `original definition of the hook`_. -This allows for early mismatch reporting and minimizes version incompatibilites. - -.. _`original definition of the hook`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/api.py - -Loading plugins and specifying dependencies -============================================ - -py.test loads and configures plugins at tool startup: - -* by reading the ``PYTEST_PLUGINS`` environment variable - and importing the comma-separated list of plugin names. - -* by pre-scanning the command line for the ``-p name`` option - and loading the specified plugin *before actual command line parsing*. - -* by loading all plugins specified by the ``pytest_plugins`` - variable in a ``conftest.py`` file or test modules. - -Specifying a plugin in a test module or ``conftest.py`` will -only lead to activitation when ``py.test`` actually sees the -directory and the file during the collection process. This happens -already after command line parsing and there is no try to do -a "pre-scan of all subdirs" as this would mean a potentially -very large delay. As long as you don't add command line -options this detail does not need to worry you. - -A plugin module may specify its dependencies via -another ``pytest_plugins`` definition. - -Included plugins -================ - -You can find the source code of all default plugins in - - http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/ - -Additionally you can check out some more contributed plugins here - - http://bitbucket.org/hpk42/py-trunk/src/tip/contrib/ - - -Overview on available hooks -==================================== - -"runtest" hooks -------------------- - -Each test item is usually executed by calling the following three hooks:: - - pytest_runtest_setup(item) - pytest_runtest_call(item) - pytest_runtest_teardown(item) - -For each of the three invocations a `call object`_ encapsulates -information about the outcome of the call and is subsequently used -to make a report object:: - - report = hook.pytest_runtest_makereport(item, call) - -For example, the `pytest_pdb plugin`_ uses this hook to activate -interactive debugging on failures when ``--pdb`` is specified on the -command line. - -Usually three reports will be generated for a single test item. However, -if the ``pytest_runtest_setup`` fails no call or teardown hooks -will be called and only one report will be created. - -Each of the up to three reports is eventually fed to the logreport hook:: - - pytest_runtest_logreport(report) - -A ``report`` object contains status and reporting information:: - - report.longrepr = string/lines/object to print - report.when = "setup", "call" or "teardown" - report.shortrepr = letter for progress-report - report.passed = True or False - report.failed = True or False - report.skipped = True or False - -The `pytest_terminal plugin`_ uses this hook to print information -about a test run. - -The protocol described here is implemented via this hook:: - - pytest_runtest_protocol(item) -> True - -.. _`call object`: - -The call object contains information about a performed call:: - - call.excinfo = ExceptionInfo object or None - call.when = "setup", "call" or "teardown" - call.outerr = None or tuple of strings representing captured stdout/stderr - -.. _`pytest_pdb plugin`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/pytest_pdb.py -.. _`pytest_terminal plugin`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/pytest_terminal.py - diff --git a/doc/test/test.txt b/doc/test/test.txt index e7ade9f80..e9e9ecdd2 100644 --- a/doc/test/test.txt +++ b/doc/test/test.txt @@ -15,16 +15,13 @@ funcargs_: powerful parametrized test function setup `distributed testing`_: distribute test runs to other machines and platforms. -plugins_: using available plugins. - -extend_: writing plugins and advanced configuration. +extend_: easily write per-project hooks or global plugins .. _quickstart: quickstart.html .. _features: features.html .. _funcargs: funcargs.html -.. _plugins: plugins.html -.. _extend: ext.html +.. _extend: extend.html .. _`distributed testing`: dist.html