floraachy/pytest2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/master' into merge-master-into-features

Browse Source 
# Conflicts:
#	_pytest/capture.py
#	_pytest/compat.py
#	_pytest/python.py
#	testing/python/collect.py
#	testing/test_mark.py
This commit is contained in:
Bruno Oliveira
2017-05-03 19:04:53 -03:00
parent cccb2cc92b 3fc917a261
commit f3b359f5b8
23 changed files with 327 additions and 99 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
doc/en/assert.rst
View File

@@ -270,12 +270,21 @@ supporting modules which are not themselves test modules will not be rewritten.
.. note::
``pytest`` rewrites test modules on import. It does this by using an import
hook to write new pyc files. Most of the time this works transparently.
``pytest`` rewrites test modules on import by using an import
hook to write new ``pyc`` files. Most of the time this works transparently.
However, if you are messing with import yourself, the import hook may
interfere. If this is the case, use ``--assert=plain``. Additionally,
rewriting will fail silently if it cannot write new pycs, i.e. in a read-only
filesystem or a zipfile.
interfere.
If this is the case you have two options:
* Disable rewriting for a specific module by adding the string
``PYTEST_DONT_REWRITE`` to its docstring.
* Disable rewriting for all modules by using ``--assert=plain``.
Additionally, rewriting will fail silently if it cannot write new ``.pyc`` files,
i.e. in a read-only filesystem or a zipfile.
For further information, Benjamin Peterson wrote up `Behind the scenes of pytest's new assertion rewriting <http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html>`_.

1
doc/en/contents.rst
View File

@@ -12,6 +12,7 @@ Full pytest documentation
getting-started
usage
existingtestsuite
assert
builtin
fixture

6
doc/en/customize.rst
View File

@@ -45,11 +45,11 @@ Here is the algorithm which finds the rootdir from ``args``:
matched, it becomes the ini-file and its directory becomes the rootdir.
- if no ini-file was found, use the already determined common ancestor as root
directory. This allows to work with pytest in structures that are not part of
directory. This allows the use of pytest in structures that are not part of
a package and don't have any particular ini-file configuration.
If no ``args`` are given, pytest collects test below the current working
directory and also starts determining the rootdir from there.
directory and also starts determining the rootdir from there.
:warning: custom pytest plugin commandline arguments may include a path, as in
``pytest --log-output ../../test.log args``. Then ``args`` is mandatory,
@@ -97,6 +97,8 @@ check for ini-files as follows::
.. _`how to change command line options defaults`:
.. _`adding default options`:
How to change command line options defaults
------------------------------------------------

34
doc/en/existingtestsuite.rst Normal file
View File

@@ -0,0 +1,34 @@
.. _existingtestsuite:
Using pytest with an existing test suite
===========================================
Pytest can be used with most existing test suites, but its
behavior differs from other test runners such as :ref:`nose <noseintegration>` or
Python's default unittest framework.
Before using this section you will want to :ref:`install pytest <getstarted>`.
Running an existing test suite with pytest
---------------------------------------------
Say you want to contribute to an existing repository somewhere.
After pulling the code into your development space using some
flavor of version control and (optionally) setting up a virtualenv
you will want to run::
cd <repository>
pip install -e . # Environment dependent alternatives include
# 'python setup.py develop' and 'conda develop'
in your project root. This will set up a symlink to your code in
site-packages, allowing you to edit your code while your tests
run against it as if it were installed.
Setting up your project in development mode lets you avoid having to
reinstall every time you want to run your tests, and is less brittle than
mucking about with sys.path to point your tests at local code.
Also consider using :ref:`tox <use tox>`.
.. include:: links.inc

4
doc/en/fixture.rst
View File

@@ -253,7 +253,7 @@ the code after the *yield* statement serves as the teardown code:
import pytest
@pytest.fixture(scope="module")
def smtp(request):
def smtp():
smtp = smtplib.SMTP("smtp.gmail.com")
yield smtp # provide the fixture value
print("teardown smtp")
@@ -287,7 +287,7 @@ Note that we can also seamlessly use the ``yield`` syntax with ``with`` statemen
import pytest
@pytest.fixture(scope="module")
def smtp(request):
def smtp():
with smtplib.SMTP("smtp.gmail.com") as smtp:
yield smtp # provide the fixture value

21
doc/en/getting-started.rst
View File

@@ -49,17 +49,17 @@ That's it. You can execute the test function now::
platform linux -- Python 3.5.2, pytest-3.0.7, py-1.4.32, pluggy-0.4.0
rootdir: $REGENDOC_TMPDIR, inifile:
collected 1 items
test_sample.py F
======= FAILURES ========
_______ test_answer ________
def test_answer():
> assert func(3) == 5
E assert 4 == 5
E + where 4 = func(3)
test_sample.py:5: AssertionError
======= 1 failed in 0.12 seconds ========
@@ -128,15 +128,15 @@ run the module by passing its filename::
.F
======= FAILURES ========
_______ TestClass.test_two ________
self = <test_class.TestClass object at 0xdeadbeef>
def test_two(self):
x = "hello"
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_class.py:8: AssertionError
1 failed, 1 passed in 0.12 seconds
@@ -165,14 +165,14 @@ before performing the test function call. Let's just run it::
F
======= FAILURES ========
_______ test_needsfiles ________
tmpdir = local('PYTEST_TMPDIR/test_needsfiles0')
def test_needsfiles(tmpdir):
print (tmpdir)
> assert 0
E assert 0
test_tmpdir.py:3: AssertionError
--------------------------- Captured stdout call ---------------------------
PYTEST_TMPDIR/test_needsfiles0
@@ -192,6 +192,7 @@ Here are a few suggestions where to go next:
* :ref:`cmdline` for command line invocation examples
* :ref:`good practices <goodpractices>` for virtualenv, test layout
* :ref:`existingtestsuite` for working with pre-existing tests
* :ref:`fixtures` for providing a functional baseline to your tests
* :ref:`plugins` managing and writing plugins

133
doc/en/goodpractices.rst
View File

@@ -30,68 +30,106 @@ Within Python modules, ``pytest`` also discovers tests using the standard
Choosing a test layout / import rules
------------------------------------------
-------------------------------------
``pytest`` supports two common test layouts:
* putting tests into an extra directory outside your actual application
code, useful if you have many functional tests or for other reasons
want to keep tests separate from actual application code (often a good
idea)::
Tests outside application code
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
setup.py # your setuptools Python package metadata
Putting tests into an extra directory outside your actual application code
might be useful if you have many functional tests or for other reasons want
to keep tests separate from actual application code (often a good idea)::
setup.py
mypkg/
__init__.py
appmodule.py
app.py
view.py
tests/
test_app.py
test_view.py
...
This way your tests can run easily against an installed version
of ``mypkg``.
* inlining test directories into your application package, useful if you
have direct relation between (unit-)test and application modules and
want to distribute your tests along with your application::
Note that using this scheme your test files must have **unique names**, because
``pytest`` will import them as *top-level* modules since there are no packages
to derive a full package name from. In other words, the test files in the example above will
be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to
``sys.path``.
setup.py # your setuptools Python package metadata
If you need to have test modules with the same name, you might add ``__init__.py`` files to your
``tests`` folder and subfolders, changing them to packages::
setup.py
mypkg/
...
tests/
__init__.py
foo/
__init__.py
test_view.py
bar/
__init__.py
test_view.py
Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing
you to have modules with the same name. But now this introduces a subtle problem: in order to load
the test modules from the ``tests`` directory, pytest prepends the root of the repository to
``sys.path``, which adds the side-effect that now ``mypkg`` is also importable.
This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment,
because you want to test the *installed* version of your package, not the local code from the repository.
In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
sub-directory of your root::
setup.py
src/
mypkg/
__init__.py
app.py
view.py
tests/
__init__.py
foo/
__init__.py
test_view.py
bar/
__init__.py
test_view.py
This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent
`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_.
Tests as part of application code
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Inlining test directories into your application package
is useful if you have direct relation between tests and application modules and
want to distribute them along with your application::
setup.py
mypkg/
__init__.py
appmodule.py
...
app.py
view.py
test/
__init__.py
test_app.py
test_view.py
...
Important notes relating to both schemes:
In this scheme, it is easy to your run tests using the ``--pyargs`` option::
- **make sure that "mypkg" is importable**, for example by typing once::
pytest --pyargs mypkg
pip install -e . # install package using setup.py in editable mode
``pytest`` will discover where ``mypkg`` is installed and collect tests from there.
- **avoid "__init__.py" files in your test directories**.
This way your tests can run easily against an installed version
of ``mypkg``, independently from the installed package if it contains
the tests or not.
Note that this layout also works in conjunction with the ``src`` layout mentioned in the previous section.
- With inlined tests you might put ``__init__.py`` into test
directories and make them installable as part of your application.
Using the ``pytest --pyargs mypkg`` invocation pytest will
discover where mypkg is installed and collect tests from there.
With the "external" test you can still distribute tests but they
will not be installed or become importable.
Typically you can run tests by pointing to test directories or modules::
pytest tests/test_app.py # for external test dirs
pytest mypkg/test/test_app.py # for inlined test dirs
pytest mypkg # run tests in all below test directories
pytest # run all tests below current dir
...
Because of the above ``editable install`` mode you can change your
source code (both tests and the app) and rerun tests at will.
Once you are done with your work, you can `use tox`_ to make sure
that the package is really correct and tests pass in all
required configurations.
.. note::
@@ -144,7 +182,15 @@ for installing your application and any dependencies
as well as the ``pytest`` package itself. This ensures your code and
dependencies are isolated from the system Python installation.
If you frequently release code and want to make sure that your actual
You can then install your package in "editable" mode::
pip install -e .
which lets you change your source code (both tests and application) and rerun tests at will.
This is similar to running `python setup.py develop` or `conda develop` in that it installs
your package using a symlink to your development code.
Once you are done with your work and want to make sure that your actual
package passes all tests you may want to look into `tox`_, the
virtualenv test automation tool and its `pytest support
<https://tox.readthedocs.io/en/latest/example/pytest.html>`_.
@@ -154,11 +200,6 @@ options. It will run tests against the installed package and not
against your source code checkout, helping to detect packaging
glitches.
Continuous integration services such as Jenkins_ can make use of the
``--junitxml=PATH`` option to create a JUnitXML file and generate reports (e.g.
by publishing the results in a nice format with the `Jenkins xUnit Plugin
<https://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin>`_).
Integrating with setuptools / ``python setup.py test`` / ``pytest-runner``
--------------------------------------------------------------------------

13
doc/en/nose.rst
View File

@@ -47,9 +47,19 @@ Unsupported idioms / known issues
``tests.test_mod``) but different file system paths
(e.g. ``tests/test_mode.py`` and ``other/tests/test_mode.py``)
by extending sys.path/import semantics. pytest does not do that
but there is discussion in `issue268 <https://github.com/pytest-dev/pytest/issues/268>`_ for adding some support. Note that
but there is discussion in `#268 <https://github.com/pytest-dev/pytest/issues/268>`_ for adding some support. Note that
`nose2 choose to avoid this sys.path/import hackery <https://nose2.readthedocs.io/en/latest/differences.html#test-discovery-and-loading>`_.
If you place a conftest.py file in the root directory of your project
(as determined by pytest) pytest will run tests "nose style" against
the code below that directory by adding it to your ``sys.path`` instead of
running against your installed code.
You may find yourself wanting to do this if you ran ``python setup.py install``
to set up your project, as opposed to ``python setup.py develop`` or any of
the package manager equivalents. Installing with develop in a
virtual environment like Tox is recommended over this pattern.
- nose-style doctests are not collected and executed correctly,
also doctest fixtures don't work.
@@ -62,3 +72,4 @@ Unsupported idioms / known issues
being the recommended alternative.

3
doc/en/requirements.txt Normal file
View File

@@ -0,0 +1,3 @@
# pinning sphinx to 1.4.* due to search issues with rtd:
# https://github.com/rtfd/readthedocs-sphinx-ext/issues/25
sphinx ==1.4.*

12
doc/en/usage.rst
View File

@@ -19,6 +19,18 @@ You can invoke testing through the Python interpreter from the command line::
This is almost equivalent to invoking the command line script ``pytest [...]``
directly, except that python will also add the current directory to ``sys.path``.
Possible exit codes
--------------------------------------------------------------
Running ``pytest`` can result in six different exit codes:
:Exit code 0: All tests were collected and passed successfully
:Exit code 1: Tests were collected and run but some of the tests failed
:Exit code 2: Test execution was interrupted by the user
:Exit code 3: Internal error happened while executing tests
:Exit code 4: pytest command line usage error
:Exit code 5: No tests were collected
Getting help on version, option names, environment variables
--------------------------------------------------------------