287 lines
9.2 KiB
Plaintext
287 lines
9.2 KiB
Plaintext
|
|
Scoping and parametrizing Funcarg factories
|
|
---------------------------------------------------
|
|
|
|
.. regendoc:wipe
|
|
|
|
.. versionadded:: 2.3
|
|
|
|
The ``@pytest.mark.funcarg`` marker allows
|
|
|
|
* to mark a function without a ``pytest_funcarg__`` as a factory
|
|
* to cause parametrization and run all tests multiple times
|
|
with the multiple created resources
|
|
* to set a scope which determines the level of caching
|
|
|
|
Here is a simple example for defining a SMTPServer server
|
|
object with a session scope::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
import smtplib
|
|
|
|
@pytest.mark.funcarg(scope="session")
|
|
def smtp(request):
|
|
smtp = smtplib.SMTP("merlinux.eu")
|
|
request.addfinalizer(smtp.close)
|
|
return smtp
|
|
|
|
You can now use this server connection from your tests::
|
|
|
|
# content of test_module.py
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
assert "merlinux" in response[1]
|
|
assert 0 # for demo purposes
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
assert 0 # for demo purposes
|
|
|
|
If you run the tests::
|
|
|
|
$ py.test -q
|
|
collecting ... collected 2 items
|
|
FF
|
|
================================= FAILURES =================================
|
|
________________________________ test_ehlo _________________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x20ba7e8>
|
|
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
assert "merlinux" in response[1]
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:5: AssertionError
|
|
________________________________ test_noop _________________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x20ba7e8>
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:10: AssertionError
|
|
2 failed in 0.27 seconds
|
|
|
|
you will see the two ``assert 0`` failing and can see that
|
|
the same (session-scoped) object was passed into the two test functions.
|
|
|
|
If you now want to test multiple servers you can simply parametrize
|
|
the ``smtp`` factory::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
import smtplib
|
|
|
|
@pytest.mark.funcarg(scope="session",
|
|
params=["merlinux.eu", "mail.python.org"])
|
|
def smtp(request):
|
|
smtp = smtplib.SMTP(request.param)
|
|
def fin():
|
|
print "closing", smtp
|
|
smtp.close()
|
|
request.addfinalizer(fin)
|
|
return smtp
|
|
|
|
Only two lines changed and no test code needs to change. Let's do
|
|
another run::
|
|
|
|
$ py.test -q
|
|
collecting ... collected 4 items
|
|
FFFF
|
|
================================= FAILURES =================================
|
|
__________________________ test_ehlo[merlinux.eu] __________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x2a51830>
|
|
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
assert "merlinux" in response[1]
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:5: AssertionError
|
|
________________________ test_ehlo[mail.python.org] ________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x2a56c20>
|
|
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
> assert "merlinux" in response[1]
|
|
E assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
|
|
|
|
test_module.py:4: AssertionError
|
|
__________________________ test_noop[merlinux.eu] __________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x2a51830>
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:10: AssertionError
|
|
________________________ test_noop[mail.python.org] ________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x2a56c20>
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:10: AssertionError
|
|
4 failed in 6.91 seconds
|
|
closing <smtplib.SMTP instance at 0x2a56c20>
|
|
closing <smtplib.SMTP instance at 0x2a51830>
|
|
|
|
We get four failures because we are running the two tests twice with
|
|
different ``smtp`` instantiations as defined on the factory.
|
|
Note that with the ``mail.python.org`` connection the second tests
|
|
fails already in ``test_ehlo`` because it wrongly expects a specific
|
|
server string.
|
|
|
|
You can look at what tests pytest collects without running them::
|
|
|
|
$ py.test --collectonly
|
|
=========================== test session starts ============================
|
|
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev4
|
|
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
|
|
collecting ... collected 4 items
|
|
<Module 'test_module.py'>
|
|
<Function 'test_ehlo[merlinux.eu]'>
|
|
<Function 'test_ehlo[mail.python.org]'>
|
|
<Function 'test_noop[merlinux.eu]'>
|
|
<Function 'test_noop[mail.python.org]'>
|
|
|
|
============================= in 0.02 seconds =============================
|
|
|
|
And you can run without output capturing and minimized failure reporting to check that the ``smtp`` objects are finalized at session end::
|
|
|
|
$ py.test --tb=line -q -s
|
|
collecting ... collected 4 items
|
|
FFFF
|
|
================================= FAILURES =================================
|
|
/home/hpk/tmp/doc-exec-361/test_module.py:5: assert 0
|
|
/home/hpk/tmp/doc-exec-361/test_module.py:4: assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
|
|
/home/hpk/tmp/doc-exec-361/test_module.py:10: assert 0
|
|
/home/hpk/tmp/doc-exec-361/test_module.py:10: assert 0
|
|
4 failed in 6.83 seconds
|
|
closing <smtplib.SMTP instance at 0x236da28>
|
|
closing <smtplib.SMTP instance at 0x23687e8>
|
|
|
|
.. _`new_setup`:
|
|
|
|
``@pytest.mark.setup``: xUnit on steroids
|
|
--------------------------------------------------------------------
|
|
|
|
.. regendoc:wipe
|
|
|
|
.. versionadded:: 2.3
|
|
|
|
The ``@pytest.mark.setup`` marker allows
|
|
|
|
* to mark a function as a setup/fixture method; the function can itself
|
|
receive funcargs
|
|
* to set a scope which determines the level of caching and how often
|
|
the setup function is going to be called.
|
|
|
|
Here is a simple example which configures a global funcarg without
|
|
the test needing to have it in its signature::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
|
|
@pytest.mark.funcarg(scope="module")
|
|
def resource(request, tmpdir):
|
|
def fin():
|
|
print "finalize", tmpdir
|
|
request.addfinalizer(fin)
|
|
print "created resource", tmpdir
|
|
return tmpdir
|
|
|
|
And the test file contains a setup function using this resource::
|
|
|
|
# content of test_module.py
|
|
import pytest
|
|
|
|
@pytest.mark.setup(scope="function")
|
|
def setresource(resource):
|
|
global myresource
|
|
myresource = resource
|
|
|
|
def test_1():
|
|
assert myresource
|
|
print "using myresource", myresource
|
|
|
|
def test_2():
|
|
assert myresource
|
|
print "using myresource", myresource
|
|
|
|
Let's run this module::
|
|
|
|
$ py.test -qs
|
|
collecting ... collected 2 items
|
|
..
|
|
2 passed in 0.24 seconds
|
|
created resource /home/hpk/tmp/pytest-3715/test_10
|
|
using myresource /home/hpk/tmp/pytest-3715/test_10
|
|
using myresource /home/hpk/tmp/pytest-3715/test_10
|
|
finalize /home/hpk/tmp/pytest-3715/test_10
|
|
|
|
The two test functions will see the same resource instance because it has
|
|
a module life cycle or scope.
|
|
|
|
The resource funcarg can later add parametrization without any test
|
|
or setup code needing to change::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
|
|
@pytest.mark.funcarg(scope="module", params=["aaa", "bbb"])
|
|
def resource(request, tmpdir):
|
|
newtmp = tmpdir.join(request.param)
|
|
def fin():
|
|
print "finalize", newtmp
|
|
request.addfinalizer(fin)
|
|
print "created resource", newtmp
|
|
return newtmp
|
|
|
|
Running this will run four tests::
|
|
|
|
$ py.test -qs
|
|
collecting ... collected 4 items
|
|
....
|
|
4 passed in 0.24 seconds
|
|
created resource /home/hpk/tmp/pytest-3716/test_1_aaa_0/aaa
|
|
using myresource /home/hpk/tmp/pytest-3716/test_1_aaa_0/aaa
|
|
created resource /home/hpk/tmp/pytest-3716/test_1_bbb_0/bbb
|
|
using myresource /home/hpk/tmp/pytest-3716/test_1_bbb_0/bbb
|
|
using myresource /home/hpk/tmp/pytest-3716/test_1_aaa_0/aaa
|
|
using myresource /home/hpk/tmp/pytest-3716/test_1_bbb_0/bbb
|
|
finalize /home/hpk/tmp/pytest-3716/test_1_bbb_0/bbb
|
|
finalize /home/hpk/tmp/pytest-3716/test_1_aaa_0/aaa
|
|
|
|
Each parameter causes the creation of a respective resource and the
|
|
unchanged test module uses it in its ``@setup`` decorated method.
|
|
|
|
.. note::
|
|
|
|
Currently, parametrized tests are sorted by test function location
|
|
so a test function will execute multiple times with different parametrized
|
|
funcargs. If you have class/module/session scoped funcargs and
|
|
they cause global side effects this can cause problems because the
|
|
code under test may not be prepared to deal with it.
|