floraachy
/
pytest2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
pytest2/py/doc/_build/html/impl-test.html

342 lines
22 KiB
HTML
Raw Blame History

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>1.8. Implementation and Customization of py.test &mdash; py lib v1.0.0b1 documentation</title>
<link rel="stylesheet" href="_static/default.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: '',
VERSION: '1.0.0b1',
COLLAPSE_MODINDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="top" title="py lib v1.0.0b1 documentation" href="index.html" />
<link rel="up" title="1. py.test" href="test.html" />
<link rel="next" title="2. py.execnet" href="execnet.html" />
<link rel="prev" title="1.7. Working Examples" href="test-examples.html" />
</head>
<body>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="execnet.html" title="2. py.execnet"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="test-examples.html" title="1.7. Working Examples"
accesskey="P">previous</a> |</li>
<li><a href="index.html">py lib v1.0.0b1 documentation</a> &raquo;</li>
<li><a href="test.html" accesskey="U">1. py.test</a> &raquo;</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body">
<div class="section" id="implementation-and-customization-of-py-test">
<h1>1.8. Implementation and Customization of <tt class="docutils literal"><span class="pre">py.test</span></tt><a class="headerlink" href="#implementation-and-customization-of-py-test" title="Permalink to this headline"></a></h1>
<div class="section" id="collecting-and-running-tests-implementation-remarks">
<span id="basicpicture"></span><h2>1.8.1. Collecting and running tests / implementation remarks<a class="headerlink" href="#collecting-and-running-tests-implementation-remarks" title="Permalink to this headline"></a></h2>
<p>In order to customize <tt class="docutils literal"><span class="pre">py.test</span></tt> it&#8217;s good to understand
its basic architure (WARNING: these are not guaranteed
yet to stay the way they are now!):</p>
<div class="highlight-python"><pre> ___________________
| |
| Collector |
|___________________|
/ \
| Item.run()
| ^
receive test Items /
| /execute test Item
| /
___________________/
| |
| Session |
|___________________|
.............................
. conftest.py configuration .
. cmdline options .
.............................</pre>
</div>
<p>The <em>Session</em> basically receives test <em>Items</em> from a <em>Collector</em>,
and executes them via the <tt class="docutils literal"><span class="pre">Item.run()</span></tt> method. It monitors
the outcome of the test and reports about failures and successes.</p>
<div class="section" id="collectors-and-the-test-collection-process">
<span id="collection-process"></span><h3>1.8.1.1. Collectors and the test collection process<a class="headerlink" href="#collectors-and-the-test-collection-process" title="Permalink to this headline"></a></h3>
<p>The collecting process is iterative, i.e. the session
traverses and generates a <em>collector tree</em>. Here is an example of such
a tree, generated with the command <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">--collectonly</span> <span class="pre">py/xmlobj</span></tt>:</p>
<div class="highlight-python"><pre>&lt;Directory 'xmlobj'&gt;
&lt;Directory 'testing'&gt;
&lt;Module 'test_html.py' (py.__.xmlobj.testing.test_html)&gt;
&lt;Function 'test_html_name_stickyness'&gt;
&lt;Function 'test_stylenames'&gt;
&lt;Function 'test_class_None'&gt;
&lt;Function 'test_alternating_style'&gt;
&lt;Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)&gt;
&lt;Function 'test_tag_with_text'&gt;
&lt;Function 'test_class_identity'&gt;
&lt;Function 'test_tag_with_text_and_attributes'&gt;
&lt;Function 'test_tag_with_subclassed_attr_simple'&gt;
&lt;Function 'test_tag_nested'&gt;
&lt;Function 'test_tag_xmlname'&gt;</pre>
</div>
<p>By default all directories not starting with a dot are traversed,
looking for <tt class="docutils literal"><span class="pre">test_*.py</span></tt> and <tt class="docutils literal"><span class="pre">*_test.py</span></tt> files. Those files
are imported under their <a class="reference internal" href="#package-name">package name</a>.</p>
<p>The Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed <tt class="docutils literal"><span class="pre">test</span></tt> by default. Test classes must
start with a capitalized <tt class="docutils literal"><span class="pre">Test</span></tt> prefix.</p>
</div>
<div class="section" id="test-items-are-collectors-as-well">
<span id="collector-api"></span><h3>1.8.1.2. test items are collectors as well<a class="headerlink" href="#test-items-are-collectors-as-well" title="Permalink to this headline"></a></h3>
<p>To make the reporting life simple for the session object
items offer a <tt class="docutils literal"><span class="pre">run()</span></tt> method as well. In fact the session
distinguishes &#8220;collectors&#8221; from &#8220;items&#8221; solely by interpreting
their return value. If it is a list, then we recurse into
it, otherwise we consider the &#8220;test&#8221; as passed.</p>
</div>
<div class="section" id="constructing-the-package-name-for-test-modules">
<span id="package-name"></span><h3>1.8.1.3. constructing the package name for test modules<a class="headerlink" href="#constructing-the-package-name-for-test-modules" title="Permalink to this headline"></a></h3>
<p>Test modules are imported under their fully qualified
name. Given a filesystem <tt class="docutils literal"><span class="pre">fspath</span></tt> it is constructed as follows:</p>
<ul class="simple">
<li>walk the directories up to the last one that contains
an <tt class="docutils literal"><span class="pre">__init__.py</span></tt> file.</li>
<li>perform <tt class="docutils literal"><span class="pre">sys.path.insert(0,</span> <span class="pre">basedir)</span></tt>.</li>
<li>import the root package as <tt class="docutils literal"><span class="pre">root</span></tt></li>
<li>determine the fully qualified name for <tt class="docutils literal"><span class="pre">fspath</span></tt> by either:<ul>
<li>calling <tt class="docutils literal"><span class="pre">root.__pkg__.getimportname(fspath)</span></tt> if the
<tt class="docutils literal"><span class="pre">__pkg__</span></tt> exists.` or</li>
<li>otherwise use the relative path of the module path to
the base dir and turn slashes into dots and strike
the trailing <tt class="docutils literal"><span class="pre">.py</span></tt>.</li>
</ul>
</li>
</ul>
</div>
</div>
<div class="section" id="customizing-the-testing-process">
<h2>1.8.2. Customizing the testing process<a class="headerlink" href="#customizing-the-testing-process" title="Permalink to this headline"></a></h2>
<div class="section" id="writing-conftest-py-files">
<h3>1.8.2.1. writing conftest.py files<a class="headerlink" href="#writing-conftest-py-files" title="Permalink to this headline"></a></h3>
<p>You may put conftest.py files containing project-specific
configuration in your project&#8217;s root directory, it&#8217;s usually
best to put it just into the same directory level as your
topmost <tt class="docutils literal"><span class="pre">__init__.py</span></tt>. In fact, <tt class="docutils literal"><span class="pre">py.test</span></tt> performs
an &#8220;upwards&#8221; search starting from the directory that you specify
to be tested and will lookup configuration values right-to-left.
You may have options that reside e.g. in your home directory
but note that project specific settings will be considered
first. There is a flag that helps you debugging your
conftest.py configurations:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="n">py</span><span class="o">.</span><span class="n">test</span> <span class="o">--</span><span class="n">traceconfig</span>
</pre></div>
</div>
</div>
<div class="section" id="customizing-the-collecting-and-running-process">
<h3>1.8.2.2. customizing the collecting and running process<a class="headerlink" href="#customizing-the-collecting-and-running-process" title="Permalink to this headline"></a></h3>
<p>To introduce different test items you can create
one or more <tt class="docutils literal"><span class="pre">conftest.py</span></tt> files in your project.
When the collection process traverses directories
and modules the default collectors will produce
custom Collectors and Items if they are found
in a local <tt class="docutils literal"><span class="pre">conftest.py</span></tt> file.</p>
<div class="section" id="example-perform-additional-rest-checks">
<h4>1.8.2.2.1. example: perform additional ReST checks<a class="headerlink" href="#example-perform-additional-rest-checks" title="Permalink to this headline"></a></h4>
<p>With your custom collectors or items you can completely
derive from the standard way of collecting and running
tests in a localized manner. Let&#8217;s look at an example.
If you invoke <tt class="docutils literal"><span class="pre">py.test</span> <span class="pre">--collectonly</span> <span class="pre">py/documentation</span></tt>
then you get:</p>
<div class="highlight-python"><pre>&lt;DocDirectory 'documentation'&gt;
&lt;DocDirectory 'example'&gt;
&lt;DocDirectory 'pytest'&gt;
&lt;Module 'test_setup_flow_example.py' (test_setup_flow_example)&gt;
&lt;Class 'TestStateFullThing'&gt;
&lt;Instance '()'&gt;
&lt;Function 'test_42'&gt;
&lt;Function 'test_23'&gt;
&lt;ReSTChecker 'TODO.txt'&gt;
&lt;ReSTSyntaxTest 'TODO.txt'&gt;
&lt;LinkCheckerMaker 'checklinks'&gt;
&lt;ReSTChecker 'api.txt'&gt;
&lt;ReSTSyntaxTest 'api.txt'&gt;
&lt;LinkCheckerMaker 'checklinks'&gt;
&lt;CheckLink 'getting-started.html'&gt;
...</pre>
</div>
<p>In <tt class="docutils literal"><span class="pre">py/documentation/conftest.py</span></tt> you find the following
customization:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">class</span> <span class="nc">DocDirectory</span><span class="p">(</span><span class="n">py</span><span class="o">.</span><span class="n">test</span><span class="o">.</span><span class="n">collect</span><span class="o">.</span><span class="n">Directory</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">results</span> <span class="o">=</span> <span class="nb">super</span><span class="p">(</span><span class="n">DocDirectory</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">fspath</span><span class="o">.</span><span class="n">listdir</span><span class="p">(</span><span class="s">&#39;*.txt&#39;</span><span class="p">,</span> <span class="n">sort</span><span class="o">=</span><span class="bp">True</span><span class="p">):</span>
<span class="n">results</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">x</span><span class="o">.</span><span class="n">basename</span><span class="p">)</span>
<span class="k">return</span> <span class="n">results</span>
<span class="k">def</span> <span class="nf">join</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">name</span><span class="p">):</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">name</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s">&#39;.txt&#39;</span><span class="p">):</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">(</span><span class="n">DocDirectory</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="n">p</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fspath</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="k">if</span> <span class="n">p</span><span class="o">.</span><span class="n">check</span><span class="p">(</span><span class="nb">file</span><span class="o">=</span><span class="mf">1</span><span class="p">):</span>
<span class="k">return</span> <span class="n">ReSTChecker</span><span class="p">(</span><span class="n">p</span><span class="p">,</span> <span class="n">parent</span><span class="o">=</span><span class="bp">self</span><span class="p">)</span>
<span class="n">Directory</span> <span class="o">=</span> <span class="n">DocDirectory</span>
</pre></div>
</div>
<p>The existence of the &#8216;Directory&#8217; name in the
<tt class="docutils literal"><span class="pre">pypy/documentation/conftest.py</span></tt> module makes the collection
process defer to our custom &#8220;DocDirectory&#8221; collector. We extend
the set of collected test items by <tt class="docutils literal"><span class="pre">ReSTChecker</span></tt> instances
which themselves create <tt class="docutils literal"><span class="pre">ReSTSyntaxTest</span></tt> and <tt class="docutils literal"><span class="pre">LinkCheckerMaker</span></tt>
items. All of this instances (need to) follow the <a class="reference internal" href="#collector-api">collector API</a>.</p>
</div>
</div>
<div class="section" id="customizing-the-reporting-of-test-failures">
<h3>1.8.2.3. Customizing the reporting of Test Failures<a class="headerlink" href="#customizing-the-reporting-of-test-failures" title="Permalink to this headline"></a></h3>
<p>XXX implement Item.repr_run and Item.repr_path for your test items</p>
</div>
<div class="section" id="writing-new-assertion-methods">
<h3>1.8.2.4. Writing new assertion methods<a class="headerlink" href="#writing-new-assertion-methods" title="Permalink to this headline"></a></h3>
<p>XXX __tracebackhide__, and use &#8220;print&#8221;</p>
</div>
<div class="section" id="customizing-the-collection-process-in-a-module">
<h3>1.8.2.5. Customizing the collection process in a module<a class="headerlink" href="#customizing-the-collection-process-in-a-module" title="Permalink to this headline"></a></h3>
<blockquote>
REPEATED WARNING: details of the collection and running process are
still subject to refactorings and thus details will change.
If you are customizing py.test at &#8220;Item&#8221; level then you
definitely want to be subscribed to the <a class="reference external" href="http://codespeak.net/mailman/listinfo/py-dev">py-dev mailing list</a>
to follow ongoing development.</blockquote>
<p>If you have a module where you want to take responsibility for
collecting your own test Items and possibly even for executing
a test then you can provide <a class="reference external" href="test-features.html#generative-tests">generative tests</a> that yield
callables and possibly arguments as a tuple. This is especially
useful for calling application test machinery with different
parameter sets but counting each of the calls as a separate
tests.</p>
<p>The other extension possibility is about
specifying a custom test <tt class="docutils literal"><span class="pre">Item</span></tt> class which
is responsible for setting up and executing an underlying
test. Or you can extend the collection process for a whole
directory tree by putting Items in a <tt class="docutils literal"><span class="pre">conftest.py</span></tt> configuration file.
The collection process dynamically consults the <em>chain of conftest.py</em>
modules to determine collectors and items at <tt class="docutils literal"><span class="pre">Directory</span></tt>, <tt class="docutils literal"><span class="pre">Module</span></tt>,
<tt class="docutils literal"><span class="pre">Class</span></tt>, <tt class="docutils literal"><span class="pre">Function</span></tt> or <tt class="docutils literal"><span class="pre">Generator</span></tt> level respectively.</p>
</div>
<div class="section" id="customizing-execution-of-functions">
<h3>1.8.2.6. Customizing execution of Functions<a class="headerlink" href="#customizing-execution-of-functions" title="Permalink to this headline"></a></h3>
<ul class="simple">
<li><tt class="docutils literal"><span class="pre">py.test.collect.Function</span></tt> test items control execution
of a test function. <tt class="docutils literal"><span class="pre">function.run()</span></tt> will get called by the
session in order to actually run a test. The method is responsible
for performing proper setup/teardown (&#8220;Test Fixtures&#8221;) for a
Function test.</li>
<li><tt class="docutils literal"><span class="pre">Function.execute(target,</span> <span class="pre">*args)</span></tt> methods are invoked by
the default <tt class="docutils literal"><span class="pre">Function.run()</span></tt> to actually execute a python
function with the given (usually empty set of) arguments.</li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar">
<div class="sphinxsidebarwrapper">
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference external" href="">1.8. Implementation and Customization of <tt class="docutils literal"><span class="pre">py.test</span></tt></a><ul>
<li><a class="reference external" href="#collecting-and-running-tests-implementation-remarks">1.8.1. Collecting and running tests / implementation remarks</a><ul>
<li><a class="reference external" href="#collectors-and-the-test-collection-process">1.8.1.1. Collectors and the test collection process</a></li>
<li><a class="reference external" href="#test-items-are-collectors-as-well">1.8.1.2. test items are collectors as well</a></li>
<li><a class="reference external" href="#constructing-the-package-name-for-test-modules">1.8.1.3. constructing the package name for test modules</a></li>
</ul>
</li>
<li><a class="reference external" href="#customizing-the-testing-process">1.8.2. Customizing the testing process</a><ul>
<li><a class="reference external" href="#writing-conftest-py-files">1.8.2.1. writing conftest.py files</a></li>
<li><a class="reference external" href="#customizing-the-collecting-and-running-process">1.8.2.2. customizing the collecting and running process</a><ul>
<li><a class="reference external" href="#example-perform-additional-rest-checks">1.8.2.2.1. example: perform additional ReST checks</a></li>
</ul>
</li>
<li><a class="reference external" href="#customizing-the-reporting-of-test-failures">1.8.2.3. Customizing the reporting of Test Failures</a></li>
<li><a class="reference external" href="#writing-new-assertion-methods">1.8.2.4. Writing new assertion methods</a></li>
<li><a class="reference external" href="#customizing-the-collection-process-in-a-module">1.8.2.5. Customizing the collection process in a module</a></li>
<li><a class="reference external" href="#customizing-execution-of-functions">1.8.2.6. Customizing execution of Functions</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="test-examples.html"
title="previous chapter">1.7. Working Examples</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="execnet.html"
title="next chapter">2. py.execnet</a></p>
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/impl-test.txt"
rel="nofollow">Show Source</a></li>
</ul>
<div id="searchbox" style="display: none">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<input type="text" name="q" size="18" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
<p class="searchtip" style="font-size: 90%">
Enter search terms or a module, class or function name.
</p>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="execnet.html" title="2. py.execnet"
>next</a> |</li>
<li class="right" >
<a href="test-examples.html" title="1.7. Working Examples"
>previous</a> |</li>
<li><a href="index.html">py lib v1.0.0b1 documentation</a> &raquo;</li>
<li><a href="test.html" >1. py.test</a> &raquo;</li>
</ul>
</div>
<div class="footer">
&copy; Copyright 2009, Holger Krekel.
Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.7.
</div>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink