Compare commits

..

29 Commits

Author SHA1 Message Date
pytest bot
3a2fd96305 Prepare release version 6.2.3 2021-04-03 21:41:18 +00:00
Ran Benita
138b19a930 Merge pull request #8517 from bluetech/backport-mktmp
[6.2.x] Fix minor temporary directory security issue
2021-04-04 00:34:15 +03:00
Ran Benita
822686e880 tmpdir: prevent using a non-private root temp directory
pytest uses a root temp directory named `/tmp/pytest-of-<username>`. The
name is predictable, and the directory might already exists from a
previous run, so that's allowed.

This makes it possible for my_user to pre-create
`/tmp/pytest-of-another_user`, thus giving my_user control of
another_user's tempdir.

Prevent this scenario by adding a couple of safety checks. I believe
they are sufficient.

Testing the first check requires changing the owner, which requires
root permissions, so can't be unit-tested easily, but I checked it
manually.
2021-04-04 00:04:50 +03:00
Ran Benita
9dc54f79b0 tmpdir: fix temporary directories created with world-readable permissions
(Written for a Unix system, but might be applicable to Windows as well).

pytest creates a root temporary directory under /tmp, named
`pytest-of-<username>`, and creates tmp_path's and other under it.
/tmp is shared between all users of the system.

This root temporary directory was created with 0o777&~umask permissions,
which usually becomes 0o755, meaning any user in the system could list
and read the files, which is undesirable.

Use 0o700 permissions instead. Also for subdirectories, because the root
dir is adjustable.
2021-04-04 00:00:42 +03:00
Ran Benita
93dbae24e1 pathlib: inline ensure_reset_dir()
This is only used in TempPathFactory.getbasetemp(). We'll be wanting
further control/care there, so move it into there.
2021-04-03 23:39:37 +03:00
Ran Benita
02fdbe2e76 pathlib: remove useless temporary variable 2021-04-03 23:39:32 +03:00
Bruno Oliveira
12e7db85af Merge pull request #8285 from nicoddemus/backport-8280
[6.2.x] Doc: Move the module declaration to index.rst
2021-01-27 09:13:27 -03:00
Bruno Oliveira
56e4392444 Merge pull request #8280 from xuhdev/module
Doc: Move the module declaration to index.rst
2021-01-27 09:07:33 -03:00
Bruno Oliveira
8220eca963 Merge pull request #8275 from pytest-dev/release-6.2.2
Prepare release 6.2.2
2021-01-25 11:52:23 -03:00
pytest bot
b9c98762f5 Prepare release version 6.2.2 2021-01-25 12:30:53 +00:00
Bruno Oliveira
8003fd23b9 Merge pull request #8259 from nicoddemus/backport-8250
[6.2.x] Fix faulthandler for Twisted Logger when used with "--capture=no"
2021-01-20 10:14:34 -03:00
Bruno Oliveira
8d605b9b26 Merge pull request #8250 from daq-tools/fix-twisted-capture 2021-01-20 09:47:10 -03:00
Bruno Oliveira
14e0c3e105 Merge pull request #8225 from The-Compiler/training-update (#8226)
doc: Add note about training early bird discount
2021-01-05 20:49:50 +01:00
Bruno Oliveira
45facc16c8 Merge pull request #8224 from nicoddemus/backport-8220
[6.2.x] DOC: Mark pytest module
2021-01-05 13:31:19 -03:00
Bruno Oliveira
99fe887d7c Merge pull request #8220 from xuhdev/module-doc
DOC: Mark pytest module
2021-01-05 13:26:19 -03:00
Bruno Oliveira
8dbf9dc1aa Merge pull request #8167 from nicoddemus/backport-8166
[6.2.x] Add Changelog to setup.cfg (#8166)
2020-12-17 13:59:14 -03:00
Adam Johnson
baaee2148d Add Changelog to setup.cfg (#8166)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
2020-12-17 13:56:18 -03:00
Bruno Oliveira
f7d1ab870f Merge pull request #8163 from bluetech/backport-8152
[6.2.x] terminal: fix "(<Skipped instance>)" skip reason in test status line
2020-12-17 08:31:53 -03:00
Ran Benita
b8201c280e Merge pull request #8152 from bluetech/empty-skip
terminal: fix "(<Skipped instance>)" skip reason in test status line
(cherry picked from commit 02e69e5cdc)
2020-12-17 12:59:01 +02:00
Bruno Oliveira
1f0c50b475 Merge pull request #8160 from nicoddemus/backport-7381
[6.2.x] Clarify fixture execution order and provide visual aids (#7381)
2020-12-16 14:02:02 -03:00
Chris NeJame
da82e1853c Clarify fixture execution order and provide visual aids (#7381)
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
2020-12-16 13:54:52 -03:00
Bruno Oliveira
a566eb9c70 Merge pull request #8149 from pytest-dev/release-6.2.1
Prepare release 6.2.1
2020-12-15 12:39:06 -03:00
pytest bot
d3971c30f4 Prepare release version 6.2.1 2020-12-15 13:06:34 +00:00
Bruno Oliveira
780044b64a Merge pull request #8147 from nicoddemus/backport-8137
[6.2.x] python_api: handle array-like args in approx() #8137
2020-12-15 09:08:58 -03:00
Jakob van Santen
8354995abc python_api: handle array-like args in approx() (#8137) 2020-12-15 08:50:11 -03:00
Bruno Oliveira
8b8b1214f4 Merge pull request #8135 from nicoddemus/backport-8123
[6.2] Merge pull request #8123 from nicoddemus/import-mismatch-unc
2020-12-13 10:50:49 -03:00
Bruno Oliveira
f854cf66f4 Merge pull request #8123 from nicoddemus/import-mismatch-unc
Compare also paths on Windows when considering ImportPathMismatchError
2020-12-13 10:35:59 -03:00
Ran Benita
c475106f12 Merge pull request #8130 from pytest-dev/release-6.2.0
Prepare release 6.2.0
2020-12-12 23:21:28 +02:00
pytest bot
e7073afe6e Prepare release version 6.2.0 2020-12-12 22:45:09 +02:00
289 changed files with 13714 additions and 36030 deletions

View File

@@ -25,7 +25,6 @@ exclude_lines =
^\s*raise NotImplementedError\b
^\s*return NotImplemented\b
^\s*assert False(,|$)
^\s*assert_never\(
^\s*if TYPE_CHECKING:
^\s*@overload( |$)

View File

@@ -13,7 +13,7 @@ If this change fixes an issue, please:
Unless your change is trivial or a small documentation fix (e.g., a typo or reword of a small section) please:
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/main/changelog/README.rst) for details.
- [ ] Create a new changelog file in the `changelog` folder, with a name like `<ISSUE NUMBER>.<TYPE>.rst`. See [changelog/README.rst](https://github.com/pytest-dev/pytest/blob/master/changelog/README.rst) for details.
Write sentences in the **past or present tense**, examples:

View File

@@ -9,9 +9,3 @@ updates:
allow:
- dependency-type: direct
- dependency-type: indirect
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
time: "03:00"
open-pull-requests-limit: 10

View File

@@ -1,51 +0,0 @@
name: backport
on:
# Note that `pull_request_target` has security implications:
# https://securitylab.github.com/research/github-actions-preventing-pwn-requests/
# In particular:
# - Only allow triggers that can be used only be trusted users
# - Don't execute any code from the target branch
# - Don't use cache
pull_request_target:
types: [labeled]
# Set permissions at the job level.
permissions: {}
jobs:
backport:
if: startsWith(github.event.label.name, 'backport ') && github.event.pull_request.merged
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
persist-credentials: true
- name: Create backport PR
run: |
set -eux
git config --global user.name "pytest bot"
git config --global user.email "pytestbot@gmail.com"
label='${{ github.event.label.name }}'
target_branch="${label#backport }"
backport_branch=backport-${{ github.event.number }}-to-"${target_branch}"
subject="[$target_branch] $(gh pr view --json title -q .title ${{ github.event.number }})"
git checkout origin/"${target_branch}" -b "${backport_branch}"
git cherry-pick -x --mainline 1 ${{ github.event.pull_request.merge_commit_sha }}
git commit --amend --message "$subject"
git push --set-upstream origin --force-with-lease "${backport_branch}"
gh pr create \
--base "${target_branch}" \
--title "${subject}" \
--body "Backport of PR #${{ github.event.number }} to $target_branch branch. PR created by backport workflow."
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,60 +0,0 @@
name: deploy
on:
push:
tags:
# These tags are protected, see:
# https://github.com/pytest-dev/pytest/settings/tag_protection
- "[0-9]+.[0-9]+.[0-9]+"
- "[0-9]+.[0-9]+.[0-9]+rc[0-9]+"
# Set permissions at the job level.
permissions: {}
jobs:
deploy:
if: github.repository == 'pytest-dev/pytest'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
persist-credentials: false
- name: Build and Check Package
uses: hynek/build-and-inspect-python-package@v1.5
- name: Download Package
uses: actions/download-artifact@v3
with:
name: Packages
path: dist
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.pypi_token }}
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.7"
- name: Install tox
run: |
python -m pip install --upgrade pip
pip install --upgrade tox
- name: Publish GitHub release notes
env:
GH_RELEASE_NOTES_TOKEN: ${{ github.token }}
run: |
sudo apt-get install pandoc
tox -e publish-gh-release-notes

View File

@@ -1,62 +1,43 @@
name: test
name: main
on:
push:
branches:
- main
- master
- "[0-9]+.[0-9]+.x"
- "test-me-*"
tags:
- "[0-9]+.[0-9]+.[0-9]+"
- "[0-9]+.[0-9]+.[0-9]+rc[0-9]+"
pull_request:
branches:
- main
- master
- "[0-9]+.[0-9]+.x"
env:
PYTEST_ADDOPTS: "--color=yes"
# Cancel running jobs for the same workflow and branch.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
# Set permissions at the job level.
permissions: {}
jobs:
build:
runs-on: ${{ matrix.os }}
timeout-minutes: 45
permissions:
contents: read
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
name: [
"windows-py36",
"windows-py37",
"windows-py37-pluggy",
"windows-py38",
"windows-py39",
"windows-py310",
"windows-py311",
"ubuntu-py36",
"ubuntu-py37",
"ubuntu-py37-pluggy",
"ubuntu-py37-freeze",
"ubuntu-py38",
"ubuntu-py39",
"ubuntu-py310",
"ubuntu-py311",
"ubuntu-pypy3",
"macos-py37",
"macos-py38",
"macos-py39",
"macos-py310",
"docs",
"doctesting",
@@ -64,6 +45,10 @@ jobs:
]
include:
- name: "windows-py36"
python: "3.6"
os: windows-latest
tox_env: "py36-xdist"
- name: "windows-py37"
python: "3.7"
os: windows-latest
@@ -71,25 +56,17 @@ jobs:
- name: "windows-py37-pluggy"
python: "3.7"
os: windows-latest
tox_env: "py37-pluggymain-pylib-xdist"
tox_env: "py37-pluggymaster-xdist"
- name: "windows-py38"
python: "3.8"
os: windows-latest
tox_env: "py38-unittestextras"
use_coverage: true
- name: "windows-py39"
python: "3.9"
os: windows-latest
tox_env: "py39-xdist"
- name: "windows-py310"
python: "3.10"
os: windows-latest
tox_env: "py310-xdist"
- name: "windows-py311"
python: "3.11-dev"
os: windows-latest
tox_env: "py311"
- name: "ubuntu-py36"
python: "3.6"
os: ubuntu-latest
tox_env: "py36-xdist"
- name: "ubuntu-py37"
python: "3.7"
os: ubuntu-latest
@@ -98,7 +75,7 @@ jobs:
- name: "ubuntu-py37-pluggy"
python: "3.7"
os: ubuntu-latest
tox_env: "py37-pluggymain-pylib-xdist"
tox_env: "py37-pluggymaster-xdist"
- name: "ubuntu-py37-freeze"
python: "3.7"
os: ubuntu-latest
@@ -111,17 +88,8 @@ jobs:
python: "3.9"
os: ubuntu-latest
tox_env: "py39-xdist"
- name: "ubuntu-py310"
python: "3.10"
os: ubuntu-latest
tox_env: "py310-xdist"
- name: "ubuntu-py311"
python: "3.11-dev"
os: ubuntu-latest
tox_env: "py311"
use_coverage: true
- name: "ubuntu-pypy3"
python: "pypy-3.7"
python: "pypy3"
os: ubuntu-latest
tox_env: "pypy3-xdist"
@@ -134,17 +102,9 @@ jobs:
os: macos-latest
tox_env: "py38-xdist"
use_coverage: true
- name: "macos-py39"
python: "3.9"
os: macos-latest
tox_env: "py39-xdist"
- name: "macos-py310"
python: "3.10"
os: macos-latest
tox_env: "py310-xdist"
- name: "plugins"
python: "3.9"
python: "3.7"
os: ubuntu-latest
tox_env: "plugins"
@@ -159,16 +119,13 @@ jobs:
use_coverage: true
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v2
with:
fetch-depth: 0
persist-credentials: false
- name: Set up Python ${{ matrix.python }}
uses: actions/setup-python@v4
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
@@ -180,24 +137,71 @@ jobs:
- name: Test with coverage
if: "matrix.use_coverage"
run: "tox -e ${{ matrix.tox_env }}-coverage"
env:
_PYTEST_TOX_COVERAGE_RUN: "coverage run -m"
COVERAGE_PROCESS_START: ".coveragerc"
_PYTEST_TOX_EXTRA_DEP: "coverage-enable-subprocess"
run: "tox -e ${{ matrix.tox_env }}"
- name: Generate coverage report
if: "matrix.use_coverage"
run: python -m coverage xml
- name: Prepare coverage token
if: (matrix.use_coverage && ( github.repository == 'pytest-dev/pytest' || github.event_name == 'pull_request' ))
run: |
python scripts/append_codecov_token.py
- name: Upload coverage to Codecov
if: "matrix.use_coverage"
uses: codecov/codecov-action@v3
continue-on-error: true
with:
fail_ci_if_error: true
files: ./coverage.xml
verbose: true
- name: Report coverage
if: (matrix.use_coverage)
env:
CODECOV_NAME: ${{ matrix.name }}
run: bash scripts/report-coverage.sh -F GHA,${{ runner.os }}
check-package:
linting:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Build and Check Package
uses: hynek/build-and-inspect-python-package@v1.5
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
- name: set PY
run: echo "name=PY::$(python -c 'import hashlib, sys;print(hashlib.sha256(sys.version.encode()+sys.executable.encode()).hexdigest())')" >> $GITHUB_ENV
- uses: actions/cache@v2
with:
path: ~/.cache/pre-commit
key: pre-commit|${{ env.PY }}|${{ hashFiles('.pre-commit-config.yaml') }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install tox
- run: tox -e linting
deploy:
if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags') && github.repository == 'pytest-dev/pytest'
runs-on: ubuntu-latest
timeout-minutes: 30
needs: [build]
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.7"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install --upgrade wheel setuptools tox
- name: Build package
run: |
python setup.py sdist bdist_wheel
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@master
with:
user: __token__
password: ${{ secrets.pypi_token }}
- name: Publish GitHub release notes
env:
GH_RELEASE_NOTES_TOKEN: ${{ secrets.release_notes }}
run: |
sudo apt-get install pandoc
tox -e publish-gh-release-notes

View File

@@ -1,52 +0,0 @@
name: prepare release pr
on:
workflow_dispatch:
inputs:
branch:
description: 'Branch to base the release from'
required: true
default: ''
major:
description: 'Major release? (yes/no)'
required: true
default: 'no'
prerelease:
description: 'Prerelease (ex: rc1). Leave empty if not a pre-release.'
required: false
default: ''
# Set permissions at the job level.
permissions: {}
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.8"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install --upgrade setuptools tox
- name: Prepare release PR (minor/patch release)
if: github.event.inputs.major == 'no'
run: |
tox -e prepare-release-pr -- ${{ github.event.inputs.branch }} ${{ github.token }} --prerelease='${{ github.event.inputs.prerelease }}'
- name: Prepare release PR (major release)
if: github.event.inputs.major == 'yes'
run: |
tox -e prepare-release-pr -- ${{ github.event.inputs.branch }} ${{ github.token }} --major --prerelease='${{ github.event.inputs.prerelease }}'

View File

@@ -0,0 +1,31 @@
# part of our release process, see `release-on-comment.py`
name: release on comment
on:
issues:
types: [opened, edited]
issue_comment:
types: [created, edited]
jobs:
build:
runs-on: ubuntu-latest
if: (github.event.comment && startsWith(github.event.comment.body, '@pytestbot please')) || (github.event.issue && !github.event.comment && startsWith(github.event.issue.body, '@pytestbot please'))
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.8"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install --upgrade setuptools tox
- name: Prepare release
run: |
tox -e release-on-comment -- $GITHUB_EVENT_PATH ${{ secrets.chatops }}

View File

@@ -1,49 +0,0 @@
name: Update Plugin List
on:
schedule:
# At 00:00 on Sunday.
# https://crontab.guru
- cron: '0 0 * * 0'
workflow_dispatch:
# Set permissions at the job level.
permissions: {}
jobs:
update-plugin-list:
if: github.repository_owner == 'pytest-dev'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install packaging requests tabulate[widechars] tqdm
- name: Update Plugin List
run: python scripts/update-plugin-list.py
- name: Create Pull Request
uses: peter-evans/create-pull-request@38e0b6e68b4c852a5500a94740f0e535e0d7ba54
with:
commit-message: '[automated] Update plugin list'
author: 'pytest bot <pytestbot@users.noreply.github.com>'
branch: update-plugin-list/patch
delete-branch: true
branch-suffix: short-commit-hash
title: '[automated] Update plugin list'
body: '[automated] Update plugin list'

4
.gitignore vendored
View File

@@ -50,10 +50,6 @@ coverage.xml
.project
.settings
.vscode
__pycache__/
# generated by pip
pip-wheel-metadata/
# pytest debug logs generated via --debug
pytestdebug.log

View File

@@ -1,18 +1,16 @@
default_language_version:
python: "3.10"
repos:
- repo: https://github.com/psf/black
rev: 23.3.0
rev: 19.10b0
hooks:
- id: black
args: [--safe, --quiet]
- repo: https://github.com/asottile/blacken-docs
rev: 1.13.0
rev: v1.8.0
hooks:
- id: blacken-docs
additional_dependencies: [black==23.1.0]
additional_dependencies: [black==19.10b0]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
@@ -22,56 +20,45 @@ repos:
- id: debug-statements
exclude: _pytest/(debugging|hookspec).py
language_version: python3
- repo: https://github.com/PyCQA/autoflake
rev: v2.0.2
hooks:
- id: autoflake
name: autoflake
args: ["--in-place", "--remove-unused-variables", "--remove-all-unused-imports"]
language: python
files: \.py$
- repo: https://github.com/PyCQA/flake8
rev: 6.0.0
- repo: https://gitlab.com/pycqa/flake8
rev: 3.8.3
hooks:
- id: flake8
language_version: python3
additional_dependencies:
- flake8-typing-imports==1.12.0
- flake8-typing-imports==1.9.0
- flake8-docstrings==1.5.0
- repo: https://github.com/asottile/reorder_python_imports
rev: v3.9.0
rev: v2.3.5
hooks:
- id: reorder-python-imports
args: ['--application-directories=.:src', --py37-plus]
args: ['--application-directories=.:src', --py36-plus]
- repo: https://github.com/asottile/pyupgrade
rev: v3.3.1
rev: v2.7.2
hooks:
- id: pyupgrade
args: [--py37-plus]
args: [--py36-plus]
- repo: https://github.com/asottile/setup-cfg-fmt
rev: v2.2.0
rev: v1.11.0
hooks:
- id: setup-cfg-fmt
args: ["--max-py-version=3.11", "--include-version-classifiers"]
# TODO: when upgrading setup-cfg-fmt this can be removed
args: [--max-py-version=3.9]
- repo: https://github.com/pre-commit/pygrep-hooks
rev: v1.10.0
rev: v1.6.0
hooks:
- id: python-use-type-annotations
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.1.1
rev: v0.790
hooks:
- id: mypy
files: ^(src/|testing/)
args: []
additional_dependencies:
- iniconfig>=1.1.0
- py>=1.8.2
- attrs>=19.2.0
- packaging
- tomli
- types-pkg_resources
# for mypy running on python>=3.11 since exceptiongroup is only a dependency
# on <3.11
- exceptiongroup>=1.0.0rc8
- repo: local
hooks:
- id: rst
@@ -102,9 +89,3 @@ repos:
xml\.
)
types: [python]
- id: py-path-deprecated
name: py.path usage is deprecated
exclude: docs|src/_pytest/deprecated.py|testing/deprecated_test.py|src/_pytest/legacypath.py
language: pygrep
entry: \bpy\.path\.local
types: [python]

View File

@@ -1,22 +1,12 @@
version: 2
python:
version: 3.7
install:
# Install pytest first, then doc/en/requirements.txt.
# This order is important to honor any pins in doc/en/requirements.txt
# when the pinned library is also a dependency of pytest.
- method: pip
path: .
- requirements: doc/en/requirements.txt
build:
os: ubuntu-20.04
tools:
python: "3.9"
apt_packages:
- inkscape
- requirements: doc/en/requirements.txt
- method: pip
path: .
formats:
- epub
- pdf
- htmlzip

73
AUTHORS
View File

@@ -5,18 +5,14 @@ Contributors include::
Aaron Coleman
Abdeali JK
Abdelrahman Elbehery
Abhijeet Kasurde
Adam Johnson
Adam Uhlir
Ahn Ki-Wook
Akiomi Kamakura
Alan Velasco
Alessio Izzo
Alexander Johnson
Alexander King
Alexei Kozlenok
Alice Purcell
Allan Feldman
Aly Sivji
Amir Elkess
@@ -27,7 +23,6 @@ Andras Tim
Andrea Cimatoribus
Andreas Motl
Andreas Zeidler
Andrew Shapton
Andrey Paramonov
Andrzej Klajnert
Andrzej Ostrowski
@@ -35,21 +30,16 @@ Andy Freeland
Anthon van der Neut
Anthony Shaw
Anthony Sottile
Anton Grinevich
Anton Lodder
Antony Lee
Arel Cordero
Arias Emmanuel
Ariel Pillemer
Armin Rigo
Aron Coyle
Aron Curzon
Ashish Kurmi
Aviral Verma
Aviv Palivoda
Babak Keyvani
Barney Gale
Ben Gartner
Ben Webb
Benjamin Peterson
Bernard Pratz
@@ -59,7 +49,6 @@ Brian Maissy
Brian Okken
Brianna Laugher
Bruno Oliveira
Cal Jacobson
Cal Leeming
Carl Friedrich Bolz
Carlos Jenkins
@@ -67,11 +56,8 @@ Ceridwen
Charles Cloud
Charles Machalow
Charnjit SiNGH (CCSJ)
Cheuk Ting Ho
Chris Lamb
Chris NeJame
Chris Rose
Chris Wheeler
Christian Boelsen
Christian Fetzer
Christian Neumüller
@@ -84,16 +70,12 @@ Christopher Gilling
Claire Cecil
Claudio Madotto
CrazyMerlyn
Cristian Vera
Cyrus Maden
Damian Skrzypczak
Daniel Grana
Daniel Hahler
Daniel Nuri
Daniel Sánchez Castelló
Daniel Valenzuela Zenteno
Daniel Wandschneider
Daniele Procida
Danielle Jenkins
Daniil Galiev
Dave Hunt
@@ -105,7 +87,6 @@ David Vierra
Daw-Ran Liou
Debi Mishra
Denis Kirisov
Denivy Braiam Rück
Dhiren Serai
Diego Russo
Dmitry Dygalo
@@ -114,35 +95,28 @@ Dominic Mortlock
Duncan Betts
Edison Gustavo Muenz
Edoardo Batini
Edson Tadeu M. Manoel
Eduardo Schettino
Eli Boyarski
Elizaveta Shashkova
Éloi Rivard
Endre Galaczi
Eric Hunsberger
Eric Liu
Eric Siegerman
Erik Aronesty
Erik M. Bray
Evan Kepner
Fabien Zarifian
Fabio Zadrozny
Felix Hofstätter
Felix Nieuwenhuizen
Feng Ma
Florian Bruhin
Florian Dahlitz
Floris Bruynooghe
Gabriel Landau
Gabriel Reis
Garvit Shubham
Gene Wood
George Kussumoto
Georgy Dyuldin
Gergely Kalmár
Gleb Nikonorov
Graeme Smecher
Graham Horler
Greg Price
Gregory Lee
@@ -151,7 +125,6 @@ Grigorii Eremeev (budulianin)
Guido Wesdorp
Guoqiang Zhang
Harald Armin Massa
Harshna
Henk-Jaap Wagenaar
Holger Kohr
Hugo van Kemenade
@@ -160,13 +133,10 @@ Ian Bicking
Ian Lesperance
Ilya Konstantinov
Ionuț Turturică
Itxaso Aizpurua
Iwan Briquemont
Jaap Broekhuizen
Jake VanderPlas
Jakob van Santen
Jakub Mitoraj
James Bourbeau
Jan Balster
Janne Vanhala
Jason R. Coombs
@@ -176,9 +146,7 @@ Jeff Rackauckas
Jeff Widman
Jenni Rinker
John Eddie Ayson
John Litborn
John Towler
Jon Parise
Jon Sonesen
Jonas Obrist
Jordan Guymon
@@ -188,7 +156,6 @@ Joseph Hunkeler
Josh Karpel
Joshua Bronson
Jurko Gospodnetić
Justice Ndou
Justyna Janczyszyn
Kale Kundert
Kamran Ahmad
@@ -198,14 +165,9 @@ Katarzyna Jachim
Katarzyna Król
Katerina Koukiou
Keri Volans
Kevin C
Kevin Cox
Kevin Hierro Carrasco
Kevin J. Foley
Kian Eliasi
Kian-Meng Ang
Kodi B. Arfer
Kojo Idrissa
Kostis Anagnostopoulos
Kristoffer Nordström
Kyle Altendorf
@@ -228,7 +190,6 @@ Marcin Bachry
Marco Gorelli
Mark Abramowitz
Mark Dickinson
Marko Pacak
Markus Unterwaditzer
Martijn Faassen
Martin Altmayer
@@ -242,6 +203,7 @@ Matthias Hafner
Maxim Filipenko
Maximilian Cosmo Sitter
mbyt
Mickey Pashov
Michael Aquilina
Michael Birtwell
Michael Droettboom
@@ -249,8 +211,6 @@ Michael Goerz
Michael Krebs
Michael Seifert
Michal Wajszczuk
Michał Zięba
Mickey Pashov
Mihai Capotă
Mike Hoyle (hoylemd)
Mike Lundy
@@ -264,26 +224,20 @@ Nicholas Murphy
Niclas Olofsson
Nicolas Delaby
Nikolay Kondratyev
Nipunn Koorapati
Oleg Pidsadnyi
Oleg Sushchenko
Olga Matoula
Oliver Bestwalter
Omar Kohl
Omer Hadari
Ondřej Súkup
Oscar Benjamin
Parth Patel
Patrick Hayes
Paul Müller
Paul Reece
Pauli Virtanen
Pavel Karateev
Paweł Adamczak
Pedro Algarvio
Petter Strandmark
Philipp Loose
Pierre Sassoulas
Pieter Mulder
Piotr Banaszkiewicz
Piotr Helm
@@ -293,14 +247,12 @@ Prashant Sharma
Pulkit Goyal
Punyashloka Biswal
Quentin Pradet
q0w
Ralf Schmitt
Ralph Giles
Ram Rachum
Ralph Giles
Ran Benita
Raphael Castaneda
Raphael Pierzina
Rafal Semik
Raquel Alegre
Ravi Chandra
Robert Holt
@@ -313,28 +265,22 @@ Ross Lawley
Ruaridh Williamson
Russel Winder
Ryan Wooden
Saiprasad Kale
Samuel Colvin
Samuel Dion-Girardeau
Samuel Searles-Bryant
Samuele Pedroni
Sanket Duthade
Sankt Petersbug
Saravanan Padmanaban
Segev Finer
Serhii Mozghovyi
Seth Junot
Shantanu Jain
Shubham Adep
Simon Gomizelj
Simon Holesch
Simon Kerr
Skylar Downes
Srinivas Reddy Thatiparthy
Stefan Farmbauer
Stefan Scherfke
Stefan Zimmermann
Stefanie Molin
Stefano Taschini
Steffen Allner
Stephan Obermann
@@ -342,36 +288,28 @@ Sven-Hendrik Haase
Sylvain Marié
Tadek Teleżyński
Takafumi Arakaki
Taneli Hukkinen
Tanvi Mehta
Tarcisio Fischer
Tareq Alayan
Tatiana Ovary
Ted Xiao
Terje Runde
Thomas Grainger
Thomas Hisch
Tim Hoffmann
Tim Strazny
TJ Bruno
Tobias Diez
Tom Dalton
Tom Viner
Tomáš Gavenčiak
Tomer Keren
Tony Narlock
Tor Colvin
Trevor Bekolay
Tyler Goodlet
Tzu-ping Chung
Vasily Kuznetsov
Victor Maryama
Victor Rodriguez
Victor Uriarte
Vidar T. Fauske
Virgil Dupras
Vitaly Lashmanov
Vivaan Verma
Vlad Dragos
Vlad Radziuk
Vladyslav Rachek
@@ -384,14 +322,7 @@ Wouter van Ackooy
Xixi Zhao
Xuan Luong
Xuecong Liao
Yannick Péroux
Yoav Caspi
Yuliang Shao
Yusuke Kadowaki
Yuval Shimon
Zac Hatfield-Dodds
Zachary Kneupper
Zachary OBrien
Zhouxin Qiu
Zoltán Máté
Zsolt Cserna

View File

@@ -4,4 +4,4 @@ Changelog
The pytest CHANGELOG is located `here <https://docs.pytest.org/en/stable/changelog.html>`__.
The source document can be found at: https://github.com/pytest-dev/pytest/blob/main/doc/en/changelog.rst
The source document can be found at: https://github.com/pytest-dev/pytest/blob/master/doc/en/changelog.rst

View File

@@ -50,8 +50,6 @@ Fix bugs
--------
Look through the `GitHub issues for bugs <https://github.com/pytest-dev/pytest/labels/type:%20bug>`_.
See also the `"status: easy" issues <https://github.com/pytest-dev/pytest/labels/status%3A%20easy>`_
that are friendly to new contributors.
:ref:`Talk <contact>` to developers to find out how you can fix specific bugs. To indicate that you are going
to work on a particular issue, add a comment to that effect on the specific issue.
@@ -162,7 +160,7 @@ the following:
- an issue tracker for bug reports and enhancement requests.
- a `changelog <https://keepachangelog.com/>`_.
- a `changelog <http://keepachangelog.com/>`_.
If no contributor strongly objects and two agree, the repository can then be
transferred to the ``pytest-dev`` organisation.
@@ -223,7 +221,7 @@ changes you want to review and merge. Pull requests are stored on
Once you send a pull request, we can discuss its potential modifications and
even add more commits to it later on. There's an excellent tutorial on how Pull
Requests work in the
`GitHub Help Center <https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>`_.
`GitHub Help Center <https://help.github.com/articles/using-pull-requests/>`_.
Here is a simple overview, with pytest-specific bits:
@@ -236,19 +234,14 @@ Here is a simple overview, with pytest-specific bits:
$ git clone git@github.com:YOUR_GITHUB_USERNAME/pytest.git
$ cd pytest
# now, create your own branch off "main":
# now, create your own branch off "master":
$ git checkout -b your-bugfix-branch-name main
$ git checkout -b your-bugfix-branch-name master
Given we have "major.minor.micro" version numbers, bug fixes will usually
be released in micro releases whereas features will be released in
minor releases and incompatible changes in major releases.
You will need the tags to test locally, so be sure you have the tags from the main repository. If you suspect you don't, set the main repository as upstream and fetch the tags::
$ git remote add upstream https://github.com/pytest-dev/pytest
$ git fetch upstream --tags
If you need some help with Git, follow this quick start
guide: https://git.wiki.kernel.org/index.php/QuickStart
@@ -266,7 +259,7 @@ Here is a simple overview, with pytest-specific bits:
Tox is used to run all the tests and will automatically setup virtualenvs
to run the tests in.
(will implicitly use https://virtualenv.pypa.io/en/latest/)::
(will implicitly use http://www.virtualenv.org/en/latest/)::
$ pip install tox
@@ -325,26 +318,26 @@ Here is a simple overview, with pytest-specific bits:
compare: your-branch-name
base-fork: pytest-dev/pytest
base: main
base: master
Writing Tests
~~~~~~~~~~~~~
Writing tests for plugins or for pytest itself is often done using the `pytester fixture <https://docs.pytest.org/en/stable/reference/reference.html#pytester>`_, as a "black-box" test.
Writing tests for plugins or for pytest itself is often done using the `testdir fixture <https://docs.pytest.org/en/stable/reference.html#testdir>`_, as a "black-box" test.
For example, to ensure a simple test passes you can write:
.. code-block:: python
def test_true_assertion(pytester):
pytester.makepyfile(
def test_true_assertion(testdir):
testdir.makepyfile(
"""
def test_foo():
assert True
"""
)
result = pytester.runpytest()
result = testdir.runpytest()
result.assert_outcomes(failed=0, passed=1)
@@ -353,14 +346,14 @@ Alternatively, it is possible to make checks based on the actual output of the t
.. code-block:: python
def test_true_assertion(pytester):
pytester.makepyfile(
def test_true_assertion(testdir):
testdir.makepyfile(
"""
def test_foo():
assert False
"""
)
result = pytester.runpytest()
result = testdir.runpytest()
result.stdout.fnmatch_lines(["*assert False*", "*1 failed*"])
When choosing a file where to write a new test, take a look at the existing files and see if there's
@@ -385,7 +378,7 @@ them.
Backporting bug fixes for the next patch release
------------------------------------------------
Pytest makes a feature release every few weeks or months. In between, patch releases
Pytest makes feature release every few weeks or months. In between, patch releases
are made to the previous feature release, containing bug fixes only. The bug fixes
usually fix regressions, but may be any change that should reach users before the
next feature release.
@@ -394,22 +387,15 @@ Suppose for example that the latest release was 1.2.3, and you want to include
a bug fix in 1.2.4 (check https://github.com/pytest-dev/pytest/releases for the
actual latest release). The procedure for this is:
#. First, make sure the bug is fixed in the ``main`` branch, with a regular pull
#. First, make sure the bug is fixed the ``master`` branch, with a regular pull
request, as described above. An exception to this is if the bug fix is not
applicable to ``main`` anymore.
applicable to ``master`` anymore.
Automatic method:
Add a ``backport 1.2.x`` label to the PR you want to backport. This will create
a backport PR against the ``1.2.x`` branch.
Manual method:
#. ``git checkout origin/1.2.x -b backport-XXXX`` # use the main PR number here
#. ``git checkout origin/1.2.x -b backport-XXXX`` # use the master PR number here
#. Locate the merge commit on the PR, in the *merged* message, for example:
nicoddemus merged commit 0f8b462 into pytest-dev:main
nicoddemus merged commit 0f8b462 into pytest-dev:master
#. ``git cherry-pick -x -m1 REVISION`` # use the revision you found above (``0f8b462``).
@@ -422,8 +408,8 @@ Manual method:
Who does the backporting
~~~~~~~~~~~~~~~~~~~~~~~~
As mentioned above, bugs should first be fixed on ``main`` (except in rare occasions
that a bug only happens in a previous release). So, who should do the backport procedure described
As mentioned above, bugs should first be fixed on ``master`` (except in rare occasions
that a bug only happens in a previous release). So who should do the backport procedure described
above?
1. If the bug was fixed by a core developer, it is the main responsibility of that core developer
@@ -431,8 +417,8 @@ above?
2. However, often the merge is done by another maintainer, in which case it is nice of them to
do the backport procedure if they have the time.
3. For bugs submitted by non-maintainers, it is expected that a core developer will to do
the backport, normally the one that merged the PR on ``main``.
4. If a non-maintainers notices a bug which is fixed on ``main`` but has not been backported
the backport, normally the one that merged the PR on ``master``.
4. If a non-maintainers notices a bug which is fixed on ``master`` but has not been backported
(due to maintainers forgetting to apply the *needs backport* label, or just plain missing it),
they are also welcome to open a PR with the backport. The procedure is simple and really
helps with the maintenance of the project.
@@ -461,7 +447,7 @@ can always reopen the issue/pull request in their own time later if it makes sen
When to close
~~~~~~~~~~~~~
Here are a few general rules the maintainers use deciding when to close issues/PRs because
Here are a few general rules the maintainers use to decide when to close issues/PRs because
of lack of inactivity:
* Issues labeled ``question`` or ``needs information``: closed after 14 days inactive.
@@ -473,15 +459,15 @@ The above are **not hard rules**, but merely **guidelines**, and can be (and oft
Closing pull requests
~~~~~~~~~~~~~~~~~~~~~
When closing a Pull Request, it needs to be acknowledging the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss a stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
When closing a Pull Request, it needs to be acknowledge the time, effort, and interest demonstrated by the person which submitted it. As mentioned previously, it is not the intent of the team to dismiss stalled pull request entirely but to merely to clear up our queue, so a message like the one below is warranted when closing a pull request that went stale:
Hi <contributor>,
First of all, we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
First of all we would like to thank you for your time and effort on working on this, the pytest team deeply appreciates it.
We noticed it has been awhile since you have updated this PR, however. pytest is a high activity project, with many issues/PRs being opened daily, so it is hard for us maintainers to track which PRs are ready for merging, for review, or need more attention.
So for those reasons we, think it is best to close the PR for now, but with the only intention to clean up our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
So for those reasons we think it is best to close the PR for now, but with the only intention to cleanup our queue, it is by no means a rejection of your changes. We still encourage you to re-open this PR (it is just a click of a button away) when you are ready to get back to it.
Again we appreciate your time for working on this, and hope you might get back to this at a later time!

View File

@@ -1,6 +1,6 @@
The MIT License (MIT)
Copyright (c) 2004 Holger Krekel and others
Copyright (c) 2004-2020 Holger Krekel and others
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in

View File

@@ -1,7 +1,6 @@
.. image:: https://github.com/pytest-dev/pytest/raw/main/doc/en/img/pytest_logo_curves.svg
.. image:: https://docs.pytest.org/en/stable/_static/pytest1.png
:target: https://docs.pytest.org/en/stable/
:align: center
:height: 200
:alt: pytest
@@ -16,16 +15,15 @@
.. image:: https://img.shields.io/pypi/pyversions/pytest.svg
:target: https://pypi.org/project/pytest/
.. image:: https://codecov.io/gh/pytest-dev/pytest/branch/main/graph/badge.svg
.. image:: https://codecov.io/gh/pytest-dev/pytest/branch/master/graph/badge.svg
:target: https://codecov.io/gh/pytest-dev/pytest
:alt: Code coverage Status
.. image:: https://github.com/pytest-dev/pytest/workflows/test/badge.svg
:target: https://github.com/pytest-dev/pytest/actions?query=workflow%3Atest
.. image:: https://travis-ci.org/pytest-dev/pytest.svg?branch=master
:target: https://travis-ci.org/pytest-dev/pytest
.. image:: https://results.pre-commit.ci/badge/github/pytest-dev/pytest/main.svg
:target: https://results.pre-commit.ci/latest/github/pytest-dev/pytest/main
:alt: pre-commit.ci status
.. image:: https://github.com/pytest-dev/pytest/workflows/main/badge.svg
:target: https://github.com/pytest-dev/pytest/actions?query=workflow%3Amain
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/psf/black
@@ -37,15 +35,6 @@
:target: https://pytest.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. image:: https://img.shields.io/badge/Discord-pytest--dev-blue
:target: https://discord.com/invite/pytest-dev
:alt: Discord
.. image:: https://img.shields.io/badge/Libera%20chat-%23pytest-orange
:target: https://web.libera.chat/#pytest
:alt: Libera chat
The ``pytest`` framework makes it easy to write small tests, yet
scales to support complex functional testing for applications and libraries.
@@ -88,21 +77,21 @@ Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` stat
Features
--------
- Detailed info on failing `assert statements <https://docs.pytest.org/en/stable/how-to/assert.html>`_ (no need to remember ``self.assert*`` names)
- Detailed info on failing `assert statements <https://docs.pytest.org/en/stable/assert.html>`_ (no need to remember ``self.assert*`` names)
- `Auto-discovery
<https://docs.pytest.org/en/stable/explanation/goodpractices.html#python-test-discovery>`_
<https://docs.pytest.org/en/stable/goodpractices.html#python-test-discovery>`_
of test modules and functions
- `Modular fixtures <https://docs.pytest.org/en/stable/explanation/fixtures.html>`_ for
- `Modular fixtures <https://docs.pytest.org/en/stable/fixture.html>`_ for
managing small or parametrized long-lived test resources
- Can run `unittest <https://docs.pytest.org/en/stable/how-to/unittest.html>`_ (or trial),
`nose <https://docs.pytest.org/en/stable/how-to/nose.html>`_ test suites out of the box
- Can run `unittest <https://docs.pytest.org/en/stable/unittest.html>`_ (or trial),
`nose <https://docs.pytest.org/en/stable/nose.html>`_ test suites out of the box
- Python 3.7+ or PyPy3
- Python 3.6+ and PyPy3
- Rich plugin architecture, with over 850+ `external plugins <https://docs.pytest.org/en/latest/reference/plugin_list.html>`_ and thriving community
- Rich plugin architecture, with over 850+ `external plugins <http://plugincompat.herokuapp.com>`_ and thriving community
Documentation
@@ -160,8 +149,8 @@ Tidelift will coordinate the fix and disclosure.
License
-------
Copyright Holger Krekel and others, 2004.
Copyright Holger Krekel and others, 2004-2020.
Distributed under the terms of the `MIT`_ license, pytest is free and open source software.
.. _`MIT`: https://github.com/pytest-dev/pytest/blob/main/LICENSE
.. _`MIT`: https://github.com/pytest-dev/pytest/blob/master/LICENSE

View File

@@ -14,89 +14,59 @@ Preparing: Automatic Method
~~~~~~~~~~~~~~~~~~~~~~~~~~~
We have developed an automated workflow for releases, that uses GitHub workflows and is triggered
by `manually running <https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow>`__
the `prepare-release-pr workflow <https://github.com/pytest-dev/pytest/actions/workflows/prepare-release-pr.yml>`__
on GitHub Actions.
by opening an issue.
The automation will decide the new version number based on the following criteria:
Bug-fix releases
^^^^^^^^^^^^^^^^
- If the "major release" input is set to "yes", release a new major release
(e.g. 7.0.0 -> 8.0.0)
- If there are any ``.feature.rst`` or ``.breaking.rst`` files in the
``changelog`` directory, release a new minor release (e.g. 7.0.0 -> 7.1.0)
- Otherwise, release a bugfix release (e.g. 7.0.0 -> 7.0.1)
- If the "prerelease" input is set, append the string to the version number
(e.g. 7.0.0 -> 8.0.0rc1), if "major" is set, and "prerelease" is set to `rc1`)
A bug-fix release is always done from a maintenance branch, so for example to release bug-fix
``5.1.2``, open a new issue and add this comment to the body::
Bug-fix and minor releases
^^^^^^^^^^^^^^^^^^^^^^^^^^
@pytestbot please prepare release from 5.1.x
Bug-fix and minor releases are always done from a maintenance branch. First,
consider double-checking the ``changelog`` directory to see if there are any
breaking changes or new features.
Where ``5.1.x`` is the maintenance branch for the ``5.1`` series.
For a new minor release, first create a new maintenance branch from ``main``::
The automated workflow will publish a PR for a branch ``release-5.1.2``
and notify it as a comment in the issue.
git fetch upstream
git branch 7.1.x upstream/main
git push upstream 7.1.x
Then, trigger the workflow with the following inputs:
- branch: **7.1.x**
- major release: **no**
- prerelease: empty
Or via the commandline using `GitHub's cli <https://github.com/cli/cli>`__::
gh workflow run prepare-release-pr.yml -f branch=7.1.x -f major=no -f prerelease=
Where ``7.1.x`` is the maintenance branch for the ``7.1`` series. The automated
workflow will publish a PR for a branch ``release-7.1.0``.
Similarly, for a bug-fix release, use the existing maintenance branch and
trigger the workflow with e.g. ``branch: 7.0.x`` to get a new ``release-7.0.1``
PR.
Major releases
Minor releases
^^^^^^^^^^^^^^
1. Create a new maintenance branch from ``main``::
1. Create a new maintenance branch from ``master``::
git fetch upstream
git branch 8.0.x upstream/main
git push upstream 8.0.x
git fetch --all
git branch 5.2.x upstream/master
git push upstream 5.2.x
2. Trigger the workflow with the following inputs:
2. Open a new issue and add this comment to the body::
- branch: **8.0.x**
- major release: **yes**
- prerelease: empty
@pytestbot please prepare release from 5.2.x
Or via the commandline::
The automated workflow will publish a PR for a branch ``release-5.2.0`` and
notify it as a comment in the issue.
gh workflow run prepare-release-pr.yml -f branch=8.0.x -f major=yes -f prerelease=
Major and release candidates
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The automated workflow will publish a PR for a branch ``release-8.0.0``.
1. Create a new maintenance branch from ``master``::
git fetch --all
git branch 6.0.x upstream/master
git push upstream 6.0.x
2. For a **major release**, open a new issue and add this comment in the body::
@pytestbot please prepare major release from 6.0.x
For a **release candidate**, the comment must be (TODO: `#7551 <https://github.com/pytest-dev/pytest/issues/7551>`__)::
@pytestbot please prepare release candidate from 6.0.x
The automated workflow will publish a PR for a branch ``release-6.0.0`` and
notify it as a comment in the issue.
At this point on, this follows the same workflow as other maintenance branches: bug-fixes are merged
into ``main`` and ported back to the maintenance branch, even for release candidates.
Release candidates
^^^^^^^^^^^^^^^^^^
To release a release candidate, set the "prerelease" input to the version number
suffix to use. To release a ``8.0.0rc1``, proceed like under "major releases", but set:
- branch: 8.0.x
- major release: yes
- prerelease: **rc1**
Or via the commandline::
gh workflow run prepare-release-pr.yml -f branch=8.0.x -f major=yes -f prerelease=rc1
The automated workflow will publish a PR for a branch ``release-8.0.0rc1``.
into ``master`` and ported back to the maintenance branch, even for release candidates.
**A note about release candidates**
@@ -113,7 +83,7 @@ to be executed on that platform.
To release a version ``MAJOR.MINOR.PATCH``, follow these steps:
#. For major and minor releases, create a new branch ``MAJOR.MINOR.x`` from
``upstream/main`` and push it to ``upstream``.
``upstream/master`` and push it to ``upstream``.
#. Create a branch ``release-MAJOR.MINOR.PATCH`` from the ``MAJOR.MINOR.x`` branch.
@@ -136,31 +106,29 @@ Both automatic and manual processes described above follow the same steps from t
#. After all tests pass and the PR has been approved, tag the release commit
in the ``release-MAJOR.MINOR.PATCH`` branch and push it. This will publish to PyPI::
git fetch upstream
git fetch --all
git tag MAJOR.MINOR.PATCH upstream/release-MAJOR.MINOR.PATCH
git push upstream MAJOR.MINOR.PATCH
git push git@github.com:pytest-dev/pytest.git MAJOR.MINOR.PATCH
Wait for the deploy to complete, then make sure it is `available on PyPI <https://pypi.org/project/pytest>`_.
#. Merge the PR. **Make sure it's not squash-merged**, so that the tagged commit ends up in the main branch.
#. Merge the PR.
#. Cherry-pick the CHANGELOG / announce files to the ``main`` branch::
#. Cherry-pick the CHANGELOG / announce files to the ``master`` branch::
git fetch upstream
git checkout upstream/main -b cherry-pick-release
git fetch --all --prune
git checkout origin/master -b cherry-pick-release
git cherry-pick -x -m1 upstream/MAJOR.MINOR.x
#. Open a PR for ``cherry-pick-release`` and merge it once CI passes. No need to wait for approvals if there were no conflicts on the previous step.
#. For major and minor releases (or the first prerelease of it), tag the release cherry-pick merge commit in main with
#. For major and minor releases, tag the release cherry-pick merge commit in master with
a dev tag for the next feature release::
git checkout main
git checkout master
git pull
git tag MAJOR.{MINOR+1}.0.dev0
git push upstream MAJOR.{MINOR+1}.0.dev0
#. For major and minor releases, change the default version in the `Read the Docs Settings <https://readthedocs.org/dashboard/pytest/advanced/>`_ to the new branch.
git push git@github.com:pytest-dev/pytest.git MAJOR.{MINOR+1}.0.dev0
#. Send an email announcement with the contents from::

View File

@@ -25,7 +25,6 @@ The current list of contributors receiving funding are:
* `@asottile`_
* `@nicoddemus`_
* `@The-Compiler`_
Contributors interested in receiving a part of the funds just need to submit a PR adding their
name to the list. Contributors that want to stop receiving the funds should also submit a PR
@@ -57,4 +56,3 @@ funds. Just drop a line to one of the `@pytest-dev/tidelift-admins`_ or use the
.. _`@asottile`: https://github.com/asottile
.. _`@nicoddemus`: https://github.com/nicoddemus
.. _`@The-Compiler`: https://github.com/The-Compiler

View File

@@ -34,10 +34,6 @@ REGENDOC_ARGS := \
regen: REGENDOC_FILES:=*.rst */*.rst
regen:
# need to reset cachedir to the non-tox default
PYTHONDONTWRITEBYTECODE=1 \
PYTEST_ADDOPTS="-pno:hypothesis -p no:hypothesispytest -Wignore::pytest.PytestUnknownMarkWarning -o cache_dir=.pytest_cache" \
COLUMNS=76 \
regendoc --update ${REGENDOC_FILES} ${REGENDOC_ARGS}
PYTHONDONTWRITEBYTECODE=1 PYTEST_ADDOPTS="-pno:hypothesis -Wignore::pytest.PytestUnknownMarkWarning" COLUMNS=76 regendoc --update ${REGENDOC_FILES} ${REGENDOC_ARGS}
.PHONY: regen

View File

@@ -1,22 +1,16 @@
<h3>Contents</h3>
<h3><a href="{{ pathto(master_doc) }}">{{ _('Table Of Contents') }}</a></h3>
<ul>
<li><a href="{{ pathto('index') }}">Home</a></li>
<li><a href="{{ pathto('getting-started') }}">Get started</a></li>
<li><a href="{{ pathto('how-to/index') }}">How-to guides</a></li>
<li><a href="{{ pathto('reference/index') }}">Reference guides</a></li>
<li><a href="{{ pathto('explanation/index') }}">Explanation</a></li>
<li><a href="{{ pathto('contents') }}">Complete table of contents</a></li>
<li><a href="{{ pathto('example/index') }}">Library of examples</a></li>
</ul>
<h3>About the project</h3>
<ul>
<li><a href="{{ pathto('getting-started') }}">Install</a></li>
<li><a href="{{ pathto('contents') }}">Contents</a></li>
<li><a href="{{ pathto('reference') }}">API Reference</a></li>
<li><a href="{{ pathto('example/index') }}">Examples</a></li>
<li><a href="{{ pathto('customize') }}">Customize</a></li>
<li><a href="{{ pathto('changelog') }}">Changelog</a></li>
<li><a href="{{ pathto('contributing') }}">Contributing</a></li>
<li><a href="{{ pathto('backwards-compatibility') }}">Backwards Compatibility</a></li>
<li><a href="{{ pathto('py27-py34-deprecation') }}">Python 2.7 and 3.4 Support</a></li>
<li><a href="{{ pathto('sponsor') }}">Sponsor</a></li>
<li><a href="{{ pathto('tidelift') }}">pytest for Enterprise</a></li>
<li><a href="{{ pathto('license') }}">License</a></li>
@@ -29,3 +23,5 @@
{%- endif %}
<hr>
<a href="{{ pathto('genindex') }}">Index</a>
<hr>

View File

@@ -2,6 +2,7 @@
<ul>
<li><a href="https://pypi.org/project/pytest/">pytest @ PyPI</a></li>
<li><a href="https://github.com/pytest-dev/pytest/">pytest @ GitHub</a></li>
<li><a href="http://plugincompat.herokuapp.com/">3rd party plugins</a></li>
<li><a href="https://github.com/pytest-dev/pytest/issues">Issue Tracker</a></li>
<li><a href="https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf">PDF Documentation</a>
</ul>

View File

@@ -10,9 +10,10 @@ Are you an enthusiastic pytest user, the local testing guru in your workplace? O
We will pair experienced pytest users with open source projects, for a month's effort of getting new development teams started with pytest.
In 2015 we are trying this for the first time. In February and March 2015 we will gather volunteers on both sides, in April we will do the work, and in May we will evaluate how it went. This effort is being coordinated by Brianna Laugher. If you have any questions or comments, you can raise them on the `@pytestdotorg twitter account <https://twitter.com/pytestdotorg>`_\, the :issue:`issue tracker <676>` or the `pytest-dev mailing list`_.
In 2015 we are trying this for the first time. In February and March 2015 we will gather volunteers on both sides, in April we will do the work, and in May we will evaluate how it went. This effort is being coordinated by Brianna Laugher. If you have any questions or comments, you can raise them on the `@pytestdotorg twitter account <https://twitter.com/pytestdotorg>`_ the `issue tracker`_ or the `pytest-dev mailing list`_.
.. _`issue tracker`: https://github.com/pytest-dev/pytest/issues/676
.. _`pytest-dev mailing list`: https://mail.python.org/mailman/listinfo/pytest-dev

View File

@@ -6,19 +6,6 @@ Release announcements
:maxdepth: 2
release-7.3.0
release-7.2.2
release-7.2.1
release-7.2.0
release-7.1.3
release-7.1.2
release-7.1.1
release-7.1.0
release-7.0.1
release-7.0.0
release-7.0.0rc1
release-6.2.5
release-6.2.4
release-6.2.3
release-6.2.2
release-6.2.1

View File

@@ -36,12 +36,12 @@ New Features
import pytest ; pytest.main(arglist, pluginlist)
see http://pytest.org/en/stable/how-to/usage.html for details.
see http://pytest.org/en/stable/usage.html for details.
- new and better reporting information in assert expressions
if comparing lists, sequences or strings.
see http://pytest.org/en/stable/how-to/assert.html#newreport
see http://pytest.org/en/stable/assert.html#newreport
- new configuration through ini-files (setup.cfg or tox.ini recognized),
for example::
@@ -50,7 +50,7 @@ New Features
norecursedirs = .hg data* # don't ever recurse in such dirs
addopts = -x --pyargs # add these command line options by default
see http://pytest.org/en/stable/reference/customize.html
see http://pytest.org/en/stable/customize.html
- improved standard unittest support. In general py.test should now
better be able to run custom unittest.TestCases like twisted trial

View File

@@ -57,7 +57,7 @@ Changes between 2.0.0 and 2.0.1
- refinements to "collecting" output on non-ttys
- refine internal plugin registration and --traceconfig output
- introduce a mechanism to prevent/unregister plugins from the
command line, see http://pytest.org/en/stable/how-to/plugins.html#cmdunregister
command line, see http://pytest.org/en/stable/plugins.html#cmdunregister
- activate resultlog plugin by default
- fix regression wrt yielded tests which due to the
collection-before-running semantics were not

View File

@@ -12,7 +12,7 @@ courtesy of Benjamin Peterson. You can now safely use ``assert``
statements in test modules without having to worry about side effects
or python optimization ("-OO") options. This is achieved by rewriting
assert statements in test modules upon import, using a PEP302 hook.
See https://docs.pytest.org/en/stable/how-to/assert.html for
See https://docs.pytest.org/en/stable/assert.html for
detailed information. The work has been partly sponsored by my company,
merlinux GmbH.
@@ -24,7 +24,7 @@ If you want to install or upgrade pytest, just type one of::
easy_install -U pytest
best,
holger krekel / https://merlinux.eu/
holger krekel / http://merlinux.eu
Changes between 2.0.3 and 2.1.0
----------------------------------------------

View File

@@ -20,7 +20,7 @@ If you want to install or upgrade pytest, just type one of::
easy_install -U pytest
best,
holger krekel / https://merlinux.eu/
holger krekel / http://merlinux.eu
Changes between 2.1.0 and 2.1.1
----------------------------------------------

View File

@@ -19,7 +19,7 @@ If you want to install or upgrade pytest, just type one of::
easy_install -U pytest
best,
holger krekel / https://merlinux.eu/
holger krekel / http://merlinux.eu
Changes between 2.1.1 and 2.1.2
----------------------------------------

View File

@@ -9,7 +9,7 @@ with these improvements:
- new @pytest.mark.parametrize decorator to run tests with different arguments
- new metafunc.parametrize() API for parametrizing arguments independently
- see examples at http://pytest.org/en/stable/example/how-to/parametrize.html
- see examples at http://pytest.org/en/stable/example/parametrize.html
- NOTE that parametrize() related APIs are still a bit experimental
and might change in future releases.
@@ -78,7 +78,7 @@ Changes between 2.1.3 and 2.2.0
or through plugin hooks. Also introduce a "--strict" option which
will treat unregistered markers as errors
allowing to avoid typos and maintain a well described set of markers
for your test suite. See examples at http://pytest.org/en/stable/how-to/mark.html
for your test suite. See examples at http://pytest.org/en/stable/mark.html
and its links.
- issue50: introduce "-m marker" option to select tests based on markers
(this is a stricter and more predictable version of "-k" in that "-m"

View File

@@ -13,12 +13,12 @@ re-usable fixture design.
For detailed info and tutorial-style examples, see:
http://pytest.org/en/stable/explanation/fixtures.html
http://pytest.org/en/stable/fixture.html
Moreover, there is now support for using pytest fixtures/funcargs with
unittest-style suites, see here for examples:
http://pytest.org/en/stable/how-to/unittest.html
http://pytest.org/en/stable/unittest.html
Besides, more unittest-test suites are now expected to "simply work"
with pytest.

View File

@@ -16,7 +16,7 @@ comes with the following fixes and features:
- yielded test functions will now have autouse-fixtures active but
cannot accept fixtures as funcargs - it's anyway recommended to
rather use the post-2.0 parametrize features instead of yield, see:
http://pytest.org/en/stable/example/how-to/parametrize.html
http://pytest.org/en/stable/example/parametrize.html
- fix autouse-issue where autouse-fixtures would not be discovered
if defined in an a/conftest.py file and tests in a/tests/test_some.py
- fix issue226 - LIFO ordering for fixture teardowns

View File

@@ -23,13 +23,14 @@ a full list of details. A few feature highlights:
called if the corresponding setup method succeeded.
- integrate tab-completion on command line options if you
have :pypi:`argcomplete` configured.
have `argcomplete <https://pypi.org/project/argcomplete/>`_
configured.
- allow boolean expression directly with skipif/xfail
if a "reason" is also specified.
- a new hook ``pytest_load_initial_conftests`` allows plugins like
:pypi:`pytest-django` to
`pytest-django <https://pypi.org/project/pytest-django/>`_ to
influence the environment before conftest files import ``django``.
- reporting: color the last line red or green depending if

View File

@@ -11,7 +11,7 @@ clear information about the circumstances and a simple example which
reproduces the problem.
The issue tracker is of course not empty now. We have many remaining
"enhancement" issues which we'll hopefully can tackle in 2014 with your
"enhacement" issues which we'll hopefully can tackle in 2014 with your
help.
For those who use older Python versions, please note that pytest is not

View File

@@ -45,29 +45,29 @@ The py.test Development Team
**New Features**
* New ``pytest.mark.skip`` mark, which unconditionally skips marked tests.
Thanks :user:`MichaelAquilina` for the complete PR (:pull:`1040`).
Thanks `@MichaelAquilina`_ for the complete PR (`#1040`_).
* ``--doctest-glob`` may now be passed multiple times in the command-line.
Thanks :user:`jab` and :user:`nicoddemus` for the PR.
Thanks `@jab`_ and `@nicoddemus`_ for the PR.
* New ``-rp`` and ``-rP`` reporting options give the summary and full output
of passing tests, respectively. Thanks to :user:`codewarrior0` for the PR.
of passing tests, respectively. Thanks to `@codewarrior0`_ for the PR.
* ``pytest.mark.xfail`` now has a ``strict`` option which makes ``XPASS``
tests to fail the test suite, defaulting to ``False``. There's also a
``xfail_strict`` ini option that can be used to configure it project-wise.
Thanks :user:`rabbbit` for the request and :user:`nicoddemus` for the PR (:issue:`1355`).
Thanks `@rabbbit`_ for the request and `@nicoddemus`_ for the PR (`#1355`_).
* ``Parser.addini`` now supports options of type ``bool``. Thanks
:user:`nicoddemus` for the PR.
`@nicoddemus`_ for the PR.
* New ``ALLOW_BYTES`` doctest option strips ``b`` prefixes from byte strings
in doctest output (similar to ``ALLOW_UNICODE``).
Thanks :user:`jaraco` for the request and :user:`nicoddemus` for the PR (:issue:`1287`).
Thanks `@jaraco`_ for the request and `@nicoddemus`_ for the PR (`#1287`_).
* give a hint on KeyboardInterrupt to use the --fulltrace option to show the errors,
this fixes :issue:`1366`.
Thanks to :user:`hpk42` for the report and :user:`RonnyPfannschmidt` for the PR.
this fixes `#1366`_.
Thanks to `@hpk42`_ for the report and `@RonnyPfannschmidt`_ for the PR.
* catch IndexError exceptions when getting exception source location. This fixes
pytest internal error for dynamically generated code (fixtures and tests)
@@ -91,44 +91,69 @@ The py.test Development Team
`pylib <https://pylib.readthedocs.io/en/stable/>`_.
* ``pytest_enter_pdb`` now optionally receives the pytest config object.
Thanks :user:`nicoddemus` for the PR.
Thanks `@nicoddemus`_ for the PR.
* Removed code and documentation for Python 2.5 or lower versions,
including removal of the obsolete ``_pytest.assertion.oldinterpret`` module.
Thanks :user:`nicoddemus` for the PR (:issue:`1226`).
Thanks `@nicoddemus`_ for the PR (`#1226`_).
* Comparisons now always show up in full when ``CI`` or ``BUILD_NUMBER`` is
found in the environment, even when -vv isn't used.
Thanks :user:`The-Compiler` for the PR.
Thanks `@The-Compiler`_ for the PR.
* ``--lf`` and ``--ff`` now support long names: ``--last-failed`` and
``--failed-first`` respectively.
Thanks :user:`MichaelAquilina` for the PR.
Thanks `@MichaelAquilina`_ for the PR.
* Added expected exceptions to pytest.raises fail message
* Collection only displays progress ("collecting X items") when in a terminal.
This avoids cluttering the output when using ``--color=yes`` to obtain
colors in CI integrations systems (:issue:`1397`).
colors in CI integrations systems (`#1397`_).
**Bug Fixes**
* The ``-s`` and ``-c`` options should now work under ``xdist``;
``Config.fromdictargs`` now represents its input much more faithfully.
Thanks to :user:`bukzor` for the complete PR (:issue:`680`).
Thanks to `@bukzor`_ for the complete PR (`#680`_).
* Fix (:issue:`1290`): support Python 3.5's ``@`` operator in assertion rewriting.
Thanks :user:`Shinkenjoe` for report with test case and :user:`tomviner` for the PR.
* Fix (`#1290`_): support Python 3.5's ``@`` operator in assertion rewriting.
Thanks `@Shinkenjoe`_ for report with test case and `@tomviner`_ for the PR.
* Fix formatting utf-8 explanation messages (:issue:`1379`).
Thanks :user:`biern` for the PR.
* Fix formatting utf-8 explanation messages (`#1379`_).
Thanks `@biern`_ for the PR.
* Fix `traceback style docs`_ to describe all of the available options
(auto/long/short/line/native/no), with ``auto`` being the default since v2.6.
Thanks :user:`hackebrot` for the PR.
Thanks `@hackebrot`_ for the PR.
* Fix (:issue:`1422`): junit record_xml_property doesn't allow multiple records
* Fix (`#1422`_): junit record_xml_property doesn't allow multiple records
with same name.
.. _`traceback style docs`: https://pytest.org/en/stable/how-to/output.html#modifying-python-traceback-printing
.. _`traceback style docs`: https://pytest.org/en/stable/usage.html#modifying-python-traceback-printing
.. _#1422: https://github.com/pytest-dev/pytest/issues/1422
.. _#1379: https://github.com/pytest-dev/pytest/issues/1379
.. _#1366: https://github.com/pytest-dev/pytest/issues/1366
.. _#1040: https://github.com/pytest-dev/pytest/pull/1040
.. _#680: https://github.com/pytest-dev/pytest/issues/680
.. _#1287: https://github.com/pytest-dev/pytest/pull/1287
.. _#1226: https://github.com/pytest-dev/pytest/pull/1226
.. _#1290: https://github.com/pytest-dev/pytest/pull/1290
.. _#1355: https://github.com/pytest-dev/pytest/pull/1355
.. _#1397: https://github.com/pytest-dev/pytest/issues/1397
.. _@biern: https://github.com/biern
.. _@MichaelAquilina: https://github.com/MichaelAquilina
.. _@bukzor: https://github.com/bukzor
.. _@hpk42: https://github.com/hpk42
.. _@nicoddemus: https://github.com/nicoddemus
.. _@jab: https://github.com/jab
.. _@codewarrior0: https://github.com/codewarrior0
.. _@jaraco: https://github.com/jaraco
.. _@The-Compiler: https://github.com/The-Compiler
.. _@Shinkenjoe: https://github.com/Shinkenjoe
.. _@tomviner: https://github.com/tomviner
.. _@RonnyPfannschmidt: https://github.com/RonnyPfannschmidt
.. _@rabbbit: https://github.com/rabbbit
.. _@hackebrot: https://github.com/hackebrot

View File

@@ -37,21 +37,31 @@ The py.test Development Team
**Bug Fixes**
* Improve error message when a plugin fails to load.
Thanks :user:`nicoddemus` for the PR.
Thanks `@nicoddemus`_ for the PR.
* Fix (:issue:`1178`):
* Fix (`#1178 <https://github.com/pytest-dev/pytest/issues/1178>`_):
``pytest.fail`` with non-ascii characters raises an internal pytest error.
Thanks :user:`nicoddemus` for the PR.
Thanks `@nicoddemus`_ for the PR.
* Fix (:issue:`469`): junit parses report.nodeid incorrectly, when params IDs
contain ``::``. Thanks :user:`tomviner` for the PR (:pull:`1431`).
* Fix (`#469`_): junit parses report.nodeid incorrectly, when params IDs
contain ``::``. Thanks `@tomviner`_ for the PR (`#1431`_).
* Fix (:issue:`578`): SyntaxErrors
* Fix (`#578 <https://github.com/pytest-dev/pytest/issues/578>`_): SyntaxErrors
containing non-ascii lines at the point of failure generated an internal
py.test error.
Thanks :user:`asottile` for the report and :user:`nicoddemus` for the PR.
Thanks `@asottile`_ for the report and `@nicoddemus`_ for the PR.
* Fix (:issue:`1437`): When passing in a bytestring regex pattern to parameterize
* Fix (`#1437`_): When passing in a bytestring regex pattern to parameterize
attempt to decode it as utf-8 ignoring errors.
* Fix (:issue:`649`): parametrized test nodes cannot be specified to run on the command line.
* Fix (`#649`_): parametrized test nodes cannot be specified to run on the command line.
.. _#1437: https://github.com/pytest-dev/pytest/issues/1437
.. _#469: https://github.com/pytest-dev/pytest/issues/469
.. _#1431: https://github.com/pytest-dev/pytest/pull/1431
.. _#649: https://github.com/pytest-dev/pytest/issues/649
.. _@asottile: https://github.com/asottile
.. _@nicoddemus: https://github.com/nicoddemus
.. _@tomviner: https://github.com/tomviner

View File

@@ -39,27 +39,40 @@ The py.test Development Team
**Bug Fixes**
* fix :issue:`510`: skip tests where one parameterize dimension was empty
thanks Alex Stapleton for the Report and :user:`RonnyPfannschmidt` for the PR
* fix `#510`_: skip tests where one parameterize dimension was empty
thanks Alex Stapleton for the Report and `@RonnyPfannschmidt`_ for the PR
* Fix Xfail does not work with condition keyword argument.
Thanks :user:`astraw38` for reporting the issue (:issue:`1496`) and :user:`tomviner`
for PR the (:pull:`1524`).
Thanks `@astraw38`_ for reporting the issue (`#1496`_) and `@tomviner`_
for PR the (`#1524`_).
* Fix win32 path issue when putting custom config file with absolute path
in ``pytest.main("-c your_absolute_path")``.
* Fix maximum recursion depth detection when raised error class is not aware
of unicode/encoded bytes.
Thanks :user:`prusse-martin` for the PR (:pull:`1506`).
Thanks `@prusse-martin`_ for the PR (`#1506`_).
* Fix ``pytest.mark.skip`` mark when used in strict mode.
Thanks :user:`pquentin` for the PR and :user:`RonnyPfannschmidt` for
Thanks `@pquentin`_ for the PR and `@RonnyPfannschmidt`_ for
showing how to fix the bug.
* Minor improvements and fixes to the documentation.
Thanks :user:`omarkohl` for the PR.
Thanks `@omarkohl`_ for the PR.
* Fix ``--fixtures`` to show all fixture definitions as opposed to just
one per fixture name.
Thanks to :user:`hackebrot` for the PR.
Thanks to `@hackebrot`_ for the PR.
.. _#510: https://github.com/pytest-dev/pytest/issues/510
.. _#1506: https://github.com/pytest-dev/pytest/pull/1506
.. _#1496: https://github.com/pytest-dev/pytest/issues/1496
.. _#1524: https://github.com/pytest-dev/pytest/pull/1524
.. _@astraw38: https://github.com/astraw38
.. _@hackebrot: https://github.com/hackebrot
.. _@omarkohl: https://github.com/omarkohl
.. _@pquentin: https://github.com/pquentin
.. _@prusse-martin: https://github.com/prusse-martin
.. _@RonnyPfannschmidt: https://github.com/RonnyPfannschmidt
.. _@tomviner: https://github.com/tomviner

View File

@@ -1,22 +0,0 @@
pytest-6.2.4
=======================================
pytest 6.2.4 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Anthony Sottile
* Bruno Oliveira
* Christian Maurer
* Florian Bruhin
* Ran Benita
Happy testing,
The pytest Development Team

View File

@@ -1,30 +0,0 @@
pytest-6.2.5
=======================================
pytest 6.2.5 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Anthony Sottile
* Bruno Oliveira
* Brylie Christopher Oxley
* Daniel Asztalos
* Florian Bruhin
* Jason Haugen
* MapleCCC
* Michał Górny
* Miro Hrončok
* Ran Benita
* Ronny Pfannschmidt
* Sylvain Bellemare
* Thomas Güttler
Happy testing,
The pytest Development Team

View File

@@ -1,74 +0,0 @@
pytest-7.0.0
=======================================
The pytest team is proud to announce the 7.0.0 release!
This release contains new features, improvements, bug fixes, and breaking changes, so users
are encouraged to take a look at the CHANGELOG carefully:
https://docs.pytest.org/en/stable/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/stable/
As usual, you can upgrade from PyPI via:
pip install -U pytest
Thanks to all of the contributors to this release:
* Adam J. Stewart
* Alexander King
* Amin Alaee
* Andrew Neitsch
* Anthony Sottile
* Ben Davies
* Bernát Gábor
* Brian Okken
* Bruno Oliveira
* Cristian Vera
* Dan Alvizu
* David Szotten
* Eddie
* Emmanuel Arias
* Emmanuel Meric de Bellefon
* Eric Liu
* Florian Bruhin
* GergelyKalmar
* Graeme Smecher
* Harshna
* Hugo van Kemenade
* Jakub Kulík
* James Myatt
* Jeff Rasley
* Kale Kundert
* Kian Meng, Ang
* Miro Hrončok
* Naveen-Pratap
* Oleg Höfling
* Olga Matoula
* Ran Benita
* Ronny Pfannschmidt
* Simon K
* Srip
* Sören Wegener
* Taneli Hukkinen
* Terje Runde
* Thomas Grainger
* Thomas Hisch
* William Jamir Silva
* Yuval Shimon
* Zac Hatfield-Dodds
* andrewdotn
* denivyruck
* ericluoliu
* oleg.hoefling
* symonk
* ziebam
* Éloi Rivard
* Éric
Happy testing,
The pytest Development Team

View File

@@ -1,74 +0,0 @@
pytest-7.0.0rc1
=======================================
The pytest team is proud to announce the 7.0.0rc1 prerelease!
This is a prerelease, not intended for production use, but to test the upcoming features and improvements
in order to catch any major problems before the final version is released to the major public.
We appreciate your help testing this out before the final release, making sure to report any
regressions to our issue tracker:
https://github.com/pytest-dev/pytest/issues
When doing so, please include the string ``[prerelease]`` in the title.
You can upgrade from PyPI via:
pip install pytest==7.0.0rc1
Users are encouraged to take a look at the CHANGELOG carefully:
https://docs.pytest.org/en/7.0.x/changelog.html
Thanks to all the contributors to this release:
* Adam J. Stewart
* Alexander King
* Amin Alaee
* Andrew Neitsch
* Anthony Sottile
* Ben Davies
* Bernát Gábor
* Brian Okken
* Bruno Oliveira
* Cristian Vera
* David Szotten
* Eddie
* Emmanuel Arias
* Emmanuel Meric de Bellefon
* Eric Liu
* Florian Bruhin
* GergelyKalmar
* Graeme Smecher
* Harshna
* Hugo van Kemenade
* Jakub Kulík
* James Myatt
* Jeff Rasley
* Kale Kundert
* Miro Hrončok
* Naveen-Pratap
* Oleg Höfling
* Ran Benita
* Ronny Pfannschmidt
* Simon K
* Srip
* Sören Wegener
* Taneli Hukkinen
* Terje Runde
* Thomas Grainger
* Thomas Hisch
* William Jamir Silva
* Zac Hatfield-Dodds
* andrewdotn
* denivyruck
* ericluoliu
* oleg.hoefling
* symonk
* ziebam
* Éloi Rivard
Happy testing,
The pytest Development Team

View File

@@ -1,20 +0,0 @@
pytest-7.0.1
=======================================
pytest 7.0.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Anthony Sottile
* Bruno Oliveira
* Ran Benita
Happy testing,
The pytest Development Team

View File

@@ -1,48 +0,0 @@
pytest-7.1.0
=======================================
The pytest team is proud to announce the 7.1.0 release!
This release contains new features, improvements, and bug fixes,
the full list of changes is available in the changelog:
https://docs.pytest.org/en/stable/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/stable/
As usual, you can upgrade from PyPI via:
pip install -U pytest
Thanks to all of the contributors to this release:
* Akuli
* Andrew Svetlov
* Anthony Sottile
* Brett Holman
* Bruno Oliveira
* Chris NeJame
* Dan Alvizu
* Elijah DeLee
* Emmanuel Arias
* Fabian Egli
* Florian Bruhin
* Gabor Szabo
* Hasan Ramezani
* Hugo van Kemenade
* Kian Meng, Ang
* Kojo Idrissa
* Masaru Tsuchiyama
* Olga Matoula
* P. L. Lim
* Ran Benita
* Tobias Deiminger
* Yuval Shimon
* eduardo naufel schettino
* Éric
Happy testing,
The pytest Development Team

View File

@@ -1,18 +0,0 @@
pytest-7.1.1
=======================================
pytest 7.1.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Ran Benita
Happy testing,
The pytest Development Team

View File

@@ -1,23 +0,0 @@
pytest-7.1.2
=======================================
pytest 7.1.2 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Anthony Sottile
* Bruno Oliveira
* Hugo van Kemenade
* Kian Eliasi
* Ran Benita
* Zac Hatfield-Dodds
Happy testing,
The pytest Development Team

View File

@@ -1,28 +0,0 @@
pytest-7.1.3
=======================================
pytest 7.1.3 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Anthony Sottile
* Bruno Oliveira
* Gergely Kalmár
* Nipunn Koorapati
* Pax
* Sviatoslav Sydorenko
* Tim Hoffmann
* Tony Narlock
* Wolfremium
* Zach OBrien
* aizpurua23a
Happy testing,
The pytest Development Team

View File

@@ -1,93 +0,0 @@
pytest-7.2.0
=======================================
The pytest team is proud to announce the 7.2.0 release!
This release contains new features, improvements, and bug fixes,
the full list of changes is available in the changelog:
https://docs.pytest.org/en/stable/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/stable/
As usual, you can upgrade from PyPI via:
pip install -U pytest
Thanks to all of the contributors to this release:
* Aaron Berdy
* Adam Turner
* Albert Villanova del Moral
* Alice Purcell
* Anthony Sottile
* Anton Yakutovich
* Babak Keyvani
* Brandon Chinn
* Bruno Oliveira
* Chanvin Xiao
* Cheuk Ting Ho
* Chris Wheeler
* EmptyRabbit
* Ezio Melotti
* Florian Best
* Florian Bruhin
* Fredrik Berndtsson
* Gabriel Landau
* Gergely Kalmár
* Hugo van Kemenade
* James Gerity
* John Litborn
* Jon Parise
* Kevin C
* Kian Eliasi
* MatthewFlamm
* Miro Hrončok
* Nate Meyvis
* Neil Girdhar
* Nhieuvu1802
* Nipunn Koorapati
* Ofek Lev
* Paul Müller
* Paul Reece
* Pax
* Pete Baughman
* Peyman Salehi
* Philipp A
* Ran Benita
* Robert O'Shea
* Ronny Pfannschmidt
* Rowin
* Ruth Comer
* Samuel Colvin
* Samuel Gaist
* Sandro Tosi
* Shantanu
* Simon K
* Stephen Rosen
* Sviatoslav Sydorenko
* Tatiana Ovary
* Thierry Moisan
* Thomas Grainger
* Tim Hoffmann
* Tobias Diez
* Tony Narlock
* Vivaan Verma
* Wolfremium
* Zac Hatfield-Dodds
* Zach OBrien
* aizpurua23a
* gresm
* holesch
* itxasos23
* johnkangw
* skhomuti
* sommersoft
* wodny
* zx.qiu
Happy testing,
The pytest Development Team

View File

@@ -1,25 +0,0 @@
pytest-7.2.1
=======================================
pytest 7.2.1 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Anthony Sottile
* Bruno Oliveira
* Daniel Valenzuela
* Kadino
* Prerak Patel
* Ronny Pfannschmidt
* Santiago Castro
* s-padmanaban
Happy testing,
The pytest Development Team

View File

@@ -1,25 +0,0 @@
pytest-7.2.2
=======================================
pytest 7.2.2 has just been released to PyPI.
This is a bug-fix release, being a drop-in replacement. To upgrade::
pip install --upgrade pytest
The full changelog is available at https://docs.pytest.org/en/stable/changelog.html.
Thanks to all of the contributors to this release:
* Bruno Oliveira
* Garvit Shubham
* Mahesh Vashishtha
* Ramsey
* Ronny Pfannschmidt
* Teejay
* q0w
* vin01
Happy testing,
The pytest Development Team

View File

@@ -1,130 +0,0 @@
pytest-7.3.0
=======================================
The pytest team is proud to announce the 7.3.0 release!
This release contains new features, improvements, and bug fixes,
the full list of changes is available in the changelog:
https://docs.pytest.org/en/stable/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/stable/
As usual, you can upgrade from PyPI via:
pip install -U pytest
Thanks to all of the contributors to this release:
* Aaron Berdy
* Adam Turner
* Albert Villanova del Moral
* Alessio Izzo
* Alex Hadley
* Alice Purcell
* Anthony Sottile
* Anton Yakutovich
* Ashish Kurmi
* Babak Keyvani
* Billy
* Brandon Chinn
* Bruno Oliveira
* Cal Jacobson
* Chanvin Xiao
* Cheuk Ting Ho
* Chris Wheeler
* Daniel Garcia Moreno
* Daniel Scheffler
* Daniel Valenzuela
* EmptyRabbit
* Ezio Melotti
* Felix Hofstätter
* Florian Best
* Florian Bruhin
* Fredrik Berndtsson
* Gabriel Landau
* Garvit Shubham
* Gergely Kalmár
* HTRafal
* Hugo van Kemenade
* Ilya Konstantinov
* Itxaso Aizpurua
* James Gerity
* Jay
* John Litborn
* Jon Parise
* Jouke Witteveen
* Kadino
* Kevin C
* Kian Eliasi
* Klaus Rettinghaus
* Kodi Arfer
* Mahesh Vashishtha
* Manuel Jacob
* Marko Pacak
* MatthewFlamm
* Miro Hrončok
* Nate Meyvis
* Neil Girdhar
* Nhieuvu1802
* Nipunn Koorapati
* Ofek Lev
* Paul Kehrer
* Paul Müller
* Paul Reece
* Pax
* Pete Baughman
* Peyman Salehi
* Philipp A
* Pierre Sassoulas
* Prerak Patel
* Ramsey
* Ran Benita
* Robert O'Shea
* Ronny Pfannschmidt
* Rowin
* Ruth Comer
* Samuel Colvin
* Samuel Gaist
* Sandro Tosi
* Santiago Castro
* Shantanu
* Simon K
* Stefanie Molin
* Stephen Rosen
* Sviatoslav Sydorenko
* Tatiana Ovary
* Teejay
* Thierry Moisan
* Thomas Grainger
* Tim Hoffmann
* Tobias Diez
* Tony Narlock
* Vivaan Verma
* Wolfremium
* Yannick PÉROUX
* Yusuke Kadowaki
* Zac Hatfield-Dodds
* Zach OBrien
* aizpurua23a
* bitzge
* bluthej
* gresm
* holesch
* itxasos23
* johnkangw
* q0w
* rdb
* s-padmanaban
* skhomuti
* sommersoft
* vin01
* wim glenn
* wodny
* zx.qiu
Happy testing,
The pytest Development Team

View File

@@ -1,14 +1,16 @@
.. _`assert`:
How to write and report assertions in tests
The writing and reporting of assertions in tests
==================================================
.. _`assertfeedback`:
.. _`assert with the assert statement`:
.. _`assert`:
Asserting with the ``assert`` statement
---------------------------------------------------------
``pytest`` allows you to use the standard Python ``assert`` for verifying
``pytest`` allows you to use the standard python ``assert`` for verifying
expectations and values in Python tests. For example, you can write the
following:
@@ -29,8 +31,9 @@ you will see the return value of the function call:
$ pytest test_assert1.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_assert1.py F [100%]
@@ -95,13 +98,13 @@ and if you need to have access to the actual exception info you may use:
f()
assert "maximum recursion" in str(excinfo.value)
``excinfo`` is an :class:`~pytest.ExceptionInfo` instance, which is a wrapper around
``excinfo`` is an ``ExceptionInfo`` instance, which is a wrapper around
the actual exception raised. The main attributes of interest are
``.type``, ``.value`` and ``.traceback``.
You can pass a ``match`` keyword parameter to the context-manager to test
that a regular expression matches on the string representation of an exception
(similar to the ``TestCase.assertRaisesRegex`` method from ``unittest``):
(similar to the ``TestCase.assertRaisesRegexp`` method from ``unittest``):
.. code-block:: python
@@ -172,6 +175,8 @@ when it encounters comparisons. For example:
.. code-block:: python
# content of test_assert2.py
def test_set_comparison():
set1 = set("1308")
set2 = set("8035")
@@ -183,8 +188,9 @@ if you run this module:
$ pytest test_assert2.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_assert2.py F [100%]
@@ -201,9 +207,9 @@ if you run this module:
E '1'
E Extra items in the right set:
E '5'
E Use -v to get more diff
E Use -v to get the full diff
test_assert2.py:4: AssertionError
test_assert2.py:6: AssertionError
========================= short test summary info ==========================
FAILED test_assert2.py::test_set_comparison - AssertionError: assert {'0'...
============================ 1 failed in 0.12s =============================
@@ -238,7 +244,7 @@ file which provides an alternative explanation for ``Foo`` objects:
if isinstance(left, Foo) and isinstance(right, Foo) and op == "==":
return [
"Comparing Foo instances:",
f" vals: {left.val} != {right.val}",
" vals: {} != {}".format(left.val, right.val),
]
now, given this test module:
@@ -295,7 +301,7 @@ modules directly discovered by its test collection process, so **asserts in
supporting modules which are not themselves test modules will not be rewritten**.
You can manually enable assertion rewriting for an imported module by calling
:ref:`register_assert_rewrite <assertion-rewriting>`
`register_assert_rewrite <https://docs.pytest.org/en/stable/writing_plugins.html#assertion-rewriting>`_
before you import it (a good place to do that is in your root ``conftest.py``).
For further information, Benjamin Peterson wrote up `Behind the scenes of pytest's new assertion rewriting <http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html>`_.

View File

@@ -22,9 +22,7 @@ b) transitional: the old and new API don't conflict
We will only start the removal of deprecated functionality in major releases (e.g. if we deprecate something in 3.0 we will start to remove it in 4.0), and keep it around for at least two minor releases (e.g. if we deprecate something in 3.9 and 4.0 is the next release, we start to remove it in 5.0, not in 4.0).
A deprecated feature scheduled to be removed in major version X will use the warning class `PytestRemovedInXWarning` (a subclass of :class:`~pytest.PytestDeprecationwarning`).
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn `PytestRemovedInXWarning` (e.g. `PytestRemovedIn4Warning`) into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
When the deprecation expires (e.g. 4.0 is released), we won't remove the deprecated functionality immediately, but will use the standard warning filters to turn them into **errors** by default. This approach makes it explicit that removal is imminent, and still gives you time to turn the deprecated feature into a warning instead of an error so it can be dealt with in your own time. In the next minor release (e.g. 4.1), the feature will be effectively removed.
c) true breakage: should only be considered when normal transition is unreasonably unsustainable and would offset important development/features by years.
@@ -32,15 +30,15 @@ c) true breakage: should only be considered when normal transition is unreasonab
Examples for such upcoming changes:
* removal of ``pytest_runtest_protocol/nextitem`` - :issue:`895`
* removal of ``pytest_runtest_protocol/nextitem`` - `#895`_
* rearranging of the node tree to include ``FunctionDefinition``
* rearranging of ``SetupState`` :issue:`895`
* rearranging of ``SetupState`` `#895`_
True breakages must be announced first in an issue containing:
* Detailed description of the change
* Rationale
* Expected impact on users and plugin authors (example in :issue:`895`)
* Expected impact on users and plugin authors (example in `#895`_)
After there's no hard *-1* on the issue it should be followed up by an initial proof-of-concept Pull Request.
@@ -79,16 +77,4 @@ Features currently deprecated and removed in previous releases can be found in :
We track future deprecation and removal of features using milestones and the `deprecation <https://github.com/pytest-dev/pytest/issues?q=label%3A%22type%3A+deprecation%22>`_ and `removal <https://github.com/pytest-dev/pytest/labels/type%3A%20removal>`_ labels on GitHub.
Python version support
======================
Released pytest versions support all Python versions that are actively maintained at the time of the release:
============== ===================
pytest version min. Python version
============== ===================
7.1+ 3.7+
6.2 - 7.0 3.6+
5.0 - 6.1 3.5+
3.3 - 4.6 2.7, 3.4+
============== ===================
.. _`#895`: https://github.com/pytest-dev/pytest/issues/895

View File

@@ -1,11 +1,11 @@
.. _bash_completion:
How to set up bash completion
=============================
Setting up bash completion
==========================
When using bash as your shell, ``pytest`` can use argcomplete
(https://kislyuk.github.io/argcomplete/) for auto-completion.
(https://argcomplete.readthedocs.io/) for auto-completion.
For this ``argcomplete`` needs to be installed **and** enabled.
Install argcomplete using:

View File

@@ -6,7 +6,7 @@ Pytest API and builtin fixtures
================================================
Most of the information of this page has been moved over to :ref:`api-reference`.
Most of the information of this page has been moved over to :ref:`reference`.
For information on plugin hooks and objects, see :ref:`plugins`.
@@ -16,13 +16,8 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
.. code-block:: pytest
$ pytest --fixtures -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
collected 0 items
cache -- .../_pytest/cacheprovider.py:510
$ pytest -q --fixtures
cache
Return a cache object that can persist state between testing sessions.
cache.get(key, default)
@@ -33,95 +28,40 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
Values can be any object handled by the json stdlib module.
capsysbinary -- .../_pytest/capture.py:1001
Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``.
The captured output is made available via ``capsysbinary.readouterr()``
method calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``bytes`` objects.
Returns an instance of :class:`CaptureFixture[bytes] <pytest.CaptureFixture>`.
Example:
.. code-block:: python
def test_output(capsysbinary):
print("hello")
captured = capsysbinary.readouterr()
assert captured.out == b"hello\n"
capfd -- .../_pytest/capture.py:1029
Enable text capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``text`` objects.
Returns an instance of :class:`CaptureFixture[str] <pytest.CaptureFixture>`.
Example:
.. code-block:: python
def test_system_echo(capfd):
os.system('echo "hello"')
captured = capfd.readouterr()
assert captured.out == "hello\n"
capfdbinary -- .../_pytest/capture.py:1057
Enable bytes capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``byte`` objects.
Returns an instance of :class:`CaptureFixture[bytes] <pytest.CaptureFixture>`.
Example:
.. code-block:: python
def test_system_echo(capfdbinary):
os.system('echo "hello"')
captured = capfdbinary.readouterr()
assert captured.out == b"hello\n"
capsys -- .../_pytest/capture.py:973
capsys
Enable text capturing of writes to ``sys.stdout`` and ``sys.stderr``.
The captured output is made available via ``capsys.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``text`` objects.
Returns an instance of :class:`CaptureFixture[str] <pytest.CaptureFixture>`.
capsysbinary
Enable bytes capturing of writes to ``sys.stdout`` and ``sys.stderr``.
Example:
The captured output is made available via ``capsysbinary.readouterr()``
method calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``bytes`` objects.
.. code-block:: python
capfd
Enable text capturing of writes to file descriptors ``1`` and ``2``.
def test_output(capsys):
print("hello")
captured = capsys.readouterr()
assert captured.out == "hello\n"
The captured output is made available via ``capfd.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``text`` objects.
doctest_namespace [session scope] -- .../_pytest/doctest.py:737
capfdbinary
Enable bytes capturing of writes to file descriptors ``1`` and ``2``.
The captured output is made available via ``capfd.readouterr()`` method
calls, which return a ``(out, err)`` namedtuple.
``out`` and ``err`` will be ``byte`` objects.
doctest_namespace [session scope]
Fixture that returns a :py:class:`dict` that will be injected into the
namespace of doctests.
Usually this fixture is used in conjunction with another ``autouse`` fixture:
.. code-block:: python
@pytest.fixture(autouse=True)
def add_np(doctest_namespace):
doctest_namespace["np"] = numpy
For more details: :ref:`doctest_namespace`.
pytestconfig [session scope] -- .../_pytest/fixtures.py:1360
Session-scoped fixture that returns the session's :class:`pytest.Config`
object.
pytestconfig [session scope]
Session-scoped fixture that returns the :class:`_pytest.config.Config` object.
Example::
@@ -129,7 +69,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
if pytestconfig.getoption("verbose") > 0:
...
record_property -- .../_pytest/junitxml.py:282
record_property
Add extra properties to the calling test.
User properties become part of the test report and are available to the
@@ -143,13 +83,13 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
def test_function(record_property):
record_property("example_key", 1)
record_xml_attribute -- .../_pytest/junitxml.py:305
record_xml_attribute
Add extra xml attributes to the tag for the calling test.
The fixture is callable with ``name, value``. The value is
automatically XML-encoded.
record_testsuite_property [session scope] -- .../_pytest/junitxml.py:343
record_testsuite_property [session scope]
Record a new ``<property>`` tag as child of the root ``<testsuite>``.
This is suitable to writing global information regarding the entire test
@@ -163,40 +103,15 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
record_testsuite_property("ARCH", "PPC")
record_testsuite_property("STORAGE_TYPE", "CEPH")
:param name:
The property name.
:param value:
The property value. Will be converted to a string.
``name`` must be a string, ``value`` will be converted to a string and properly xml-escaped.
.. warning::
Currently this fixture **does not work** with the
`pytest-xdist <https://github.com/pytest-dev/pytest-xdist>`__ plugin. See
:issue:`7767` for details.
`pytest-xdist <https://github.com/pytest-dev/pytest-xdist>`__ plugin. See issue
`#7767 <https://github.com/pytest-dev/pytest/issues/7767>`__ for details.
tmpdir_factory [session scope] -- .../_pytest/legacypath.py:302
Return a :class:`pytest.TempdirFactory` instance for the test session.
tmpdir -- .../_pytest/legacypath.py:309
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging. If
``--basetemp`` is used then it is cleared each session. See :ref:`base
temporary directory`.
The returned object is a `legacy_path`_ object.
.. note::
These days, it is preferred to use ``tmp_path``.
:ref:`About the tmpdir and tmpdir_factory fixtures<tmpdir and tmpdir_factory>`.
.. _legacy_path: https://py.readthedocs.io/en/latest/path.html
caplog -- .../_pytest/logging.py:498
caplog
Access and control log capturing.
Captured logs are available through the following properties/methods::
@@ -207,55 +122,65 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
* caplog.record_tuples -> list of (logger_name, level, message) tuples
* caplog.clear() -> clear captured records and formatted log output string
monkeypatch -- .../_pytest/monkeypatch.py:29
monkeypatch
A convenient fixture for monkey-patching.
The fixture provides these methods to modify objects, dictionaries, or
:data:`os.environ`:
The fixture provides these methods to modify objects, dictionaries or
os.environ::
* :meth:`monkeypatch.setattr(obj, name, value, raising=True) <pytest.MonkeyPatch.setattr>`
* :meth:`monkeypatch.delattr(obj, name, raising=True) <pytest.MonkeyPatch.delattr>`
* :meth:`monkeypatch.setitem(mapping, name, value) <pytest.MonkeyPatch.setitem>`
* :meth:`monkeypatch.delitem(obj, name, raising=True) <pytest.MonkeyPatch.delitem>`
* :meth:`monkeypatch.setenv(name, value, prepend=None) <pytest.MonkeyPatch.setenv>`
* :meth:`monkeypatch.delenv(name, raising=True) <pytest.MonkeyPatch.delenv>`
* :meth:`monkeypatch.syspath_prepend(path) <pytest.MonkeyPatch.syspath_prepend>`
* :meth:`monkeypatch.chdir(path) <pytest.MonkeyPatch.chdir>`
* :meth:`monkeypatch.context() <pytest.MonkeyPatch.context>`
monkeypatch.setattr(obj, name, value, raising=True)
monkeypatch.delattr(obj, name, raising=True)
monkeypatch.setitem(mapping, name, value)
monkeypatch.delitem(obj, name, raising=True)
monkeypatch.setenv(name, value, prepend=False)
monkeypatch.delenv(name, raising=True)
monkeypatch.syspath_prepend(path)
monkeypatch.chdir(path)
All modifications will be undone after the requesting test function or
fixture has finished. The ``raising`` parameter determines if a :class:`KeyError`
or :class:`AttributeError` will be raised if the set/deletion operation does not have the
specified target.
fixture has finished. The ``raising`` parameter determines if a KeyError
or AttributeError will be raised if the set/deletion operation has no target.
To undo modifications done by the fixture in a contained scope,
use :meth:`context() <pytest.MonkeyPatch.context>`.
recwarn -- .../_pytest/recwarn.py:30
recwarn
Return a :class:`WarningsRecorder` instance that records all warnings emitted by test functions.
See https://docs.pytest.org/en/latest/how-to/capture-warnings.html for information
See http://docs.python.org/library/warnings.html for information
on warning categories.
tmp_path_factory [session scope] -- .../_pytest/tmpdir.py:245
Return a :class:`pytest.TempPathFactory` instance for the test session.
tmpdir_factory [session scope]
Return a :class:`_pytest.tmpdir.TempdirFactory` instance for the test session.
tmp_path -- .../_pytest/tmpdir.py:260
tmp_path_factory [session scope]
Return a :class:`_pytest.tmpdir.TempPathFactory` instance for the test session.
tmpdir
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging.
This behavior can be configured with :confval:`tmp_path_retention_count` and
:confval:`tmp_path_retention_policy`.
If ``--basetemp`` is used then it is cleared each session. See :ref:`base
and old bases are removed after 3 sessions, to aid in debugging. If
``--basetemp`` is used then it is cleared each session. See :ref:`base
temporary directory`.
The returned object is a `py.path.local`_ path object.
.. _`py.path.local`: https://py.readthedocs.io/en/latest/path.html
tmp_path
Return a temporary directory path object which is unique to each test
function invocation, created as a sub directory of the base temporary
directory.
By default, a new base temporary directory is created each test session,
and old bases are removed after 3 sessions, to aid in debugging. If
``--basetemp`` is used then it is cleared each session. See :ref:`base
temporary directory`.
The returned object is a :class:`pathlib.Path` object.
========================== no tests ran in 0.12s ===========================
no tests ran in 0.12s
You can also interactively ask for help, e.g. by typing on the Python interactive prompt something like:

View File

@@ -2,8 +2,8 @@
.. _cache:
How to re-run failed tests and maintain state between test runs
===============================================================
Cache: working with cross-testrun state
=======================================
@@ -86,8 +86,9 @@ If you then run it with ``--lf``:
$ pytest --lf
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
run-last-failure: rerun previous 2 failures
@@ -132,8 +133,9 @@ of ``FF`` and dots):
$ pytest --ff
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 50 items
run-last-failure: rerun previous 2 failures first
@@ -199,6 +201,7 @@ across pytest invocations:
# content of test_caching.py
import pytest
import time
def expensive_computation():
@@ -233,7 +236,7 @@ If you run this command for the first time, you can see the print statement:
> assert mydata == 23
E assert 42 == 23
test_caching.py:19: AssertionError
test_caching.py:20: AssertionError
-------------------------- Captured stdout setup ---------------------------
running expensive computation...
========================= short test summary info ==========================
@@ -256,7 +259,7 @@ the cache and nothing will be printed:
> assert mydata == 23
E assert 42 == 23
test_caching.py:19: AssertionError
test_caching.py:20: AssertionError
========================= short test summary info ==========================
FAILED test_caching.py::test_function - assert 42 == 23
1 failed in 0.12s
@@ -274,14 +277,73 @@ You can always peek at the content of the cache using the
$ pytest --cache-show
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
cachedir: /home/sweet/project/.pytest_cache
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
--------------------------- cache values for '*' ---------------------------
cache/lastfailed contains:
{'test_caching.py::test_function': True}
{'test_50.py::test_num[17]': True,
'test_50.py::test_num[25]': True,
'test_assert1.py::test_function': True,
'test_assert2.py::test_set_comparison': True,
'test_caching.py::test_function': True,
'test_foocompare.py::test_compare': True}
cache/nodeids contains:
['test_caching.py::test_function']
['test_50.py::test_num[0]',
'test_50.py::test_num[10]',
'test_50.py::test_num[11]',
'test_50.py::test_num[12]',
'test_50.py::test_num[13]',
'test_50.py::test_num[14]',
'test_50.py::test_num[15]',
'test_50.py::test_num[16]',
'test_50.py::test_num[17]',
'test_50.py::test_num[18]',
'test_50.py::test_num[19]',
'test_50.py::test_num[1]',
'test_50.py::test_num[20]',
'test_50.py::test_num[21]',
'test_50.py::test_num[22]',
'test_50.py::test_num[23]',
'test_50.py::test_num[24]',
'test_50.py::test_num[25]',
'test_50.py::test_num[26]',
'test_50.py::test_num[27]',
'test_50.py::test_num[28]',
'test_50.py::test_num[29]',
'test_50.py::test_num[2]',
'test_50.py::test_num[30]',
'test_50.py::test_num[31]',
'test_50.py::test_num[32]',
'test_50.py::test_num[33]',
'test_50.py::test_num[34]',
'test_50.py::test_num[35]',
'test_50.py::test_num[36]',
'test_50.py::test_num[37]',
'test_50.py::test_num[38]',
'test_50.py::test_num[39]',
'test_50.py::test_num[3]',
'test_50.py::test_num[40]',
'test_50.py::test_num[41]',
'test_50.py::test_num[42]',
'test_50.py::test_num[43]',
'test_50.py::test_num[44]',
'test_50.py::test_num[45]',
'test_50.py::test_num[46]',
'test_50.py::test_num[47]',
'test_50.py::test_num[48]',
'test_50.py::test_num[49]',
'test_50.py::test_num[4]',
'test_50.py::test_num[5]',
'test_50.py::test_num[6]',
'test_50.py::test_num[7]',
'test_50.py::test_num[8]',
'test_50.py::test_num[9]',
'test_assert1.py::test_function',
'test_assert2.py::test_set_comparison',
'test_caching.py::test_function',
'test_foocompare.py::test_compare']
cache/stepwise contains:
[]
example/value contains:
@@ -296,9 +358,10 @@ filtering:
$ pytest --cache-show example/*
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
cachedir: /home/sweet/project/.pytest_cache
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
cachedir: $PYTHON_PREFIX/.pytest_cache
----------------------- cache values for 'example/*' -----------------------
example/value contains:
42
@@ -320,9 +383,7 @@ servers where isolation and correctness is more important
than speed.
.. _cache stepwise:
Stepwise
--------
As an alternative to ``--lf -x``, especially for cases where you expect a large part of the test suite will fail, ``--sw``, ``--stepwise`` allows you to fix them one at a time. The test suite will run until the first failure and then stop. At the next invocation, tests will continue from the last failing test and then run until the next failing test. You may use the ``--stepwise-skip`` option to ignore one failing test and stop the test execution on the second failing test instead. This is useful if you get stuck on a failing test and just want to ignore it until later. Providing ``--stepwise-skip`` will also enable ``--stepwise`` implicitly.
As an alternative to ``--lf -x``, especially for cases where you expect a large part of the test suite will fail, ``--sw``, ``--stepwise`` allows you to fix them one at a time. The test suite will run until the first failure and then stop. At the next invocation, tests will continue from the last failing test and then run until the next failing test. You may use the ``--stepwise-skip`` option to ignore one failing test and stop the test execution on the second failing test instead. This is useful if you get stuck on a failing test and just want to ignore it until later.

View File

@@ -1,7 +1,7 @@
.. _`captures`:
How to capture stdout/stderr output
Capturing of the stdout/stderr output
=========================================================
Default stdout/stderr/stdin capturing behaviour
@@ -83,8 +83,9 @@ of the failing function and hide the other one:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
test_module.py .F [100%]
@@ -98,7 +99,7 @@ of the failing function and hide the other one:
test_module.py:12: AssertionError
-------------------------- Captured stdout setup ---------------------------
setting up <function test_func2 at 0xdeadbeef0001>
setting up <function test_func2 at 0xdeadbeef>
========================= short test summary info ==========================
FAILED test_module.py::test_func2 - assert False
======================= 1 failed, 1 passed in 0.12s ========================

File diff suppressed because it is too large Load Diff

View File

@@ -17,9 +17,7 @@
# The short X.Y version.
import ast
import os
import shutil
import sys
from textwrap import dedent
from typing import List
from typing import TYPE_CHECKING
@@ -37,27 +35,8 @@ release = ".".join(version.split(".")[:2])
# sys.path.insert(0, os.path.abspath('.'))
autodoc_member_order = "bysource"
autodoc_typehints = "description"
autodoc_typehints_description_target = "documented"
todo_include_todos = 1
latex_engine = "lualatex"
latex_elements = {
"preamble": dedent(
r"""
\directlua{
luaotfload.add_fallback("fallbacks", {
"Noto Serif CJK SC:style=Regular;",
"Symbola:Style=Regular;"
})
}
\setmainfont{FreeSerif}[RawFeature={fallback=fallbacks}]
"""
)
}
# -- General configuration -----------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
@@ -70,7 +49,6 @@ extensions = [
"pygments_pytest",
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"sphinx.ext.extlinks",
"sphinx.ext.intersphinx",
"sphinx.ext.todo",
"sphinx.ext.viewcode",
@@ -78,13 +56,6 @@ extensions = [
"sphinxcontrib_trio",
]
# Building PDF docs on readthedocs requires inkscape for svg to pdf
# conversion. The relevant plugin is not useful for normal HTML builds, but
# it still raises warnings and fails CI if inkscape is not available. So
# only use the plugin if inkscape is actually available.
if shutil.which("inkscape"):
extensions.append("sphinxcontrib.inkscapeconverter")
# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
@@ -99,7 +70,7 @@ master_doc = "contents"
# General information about the project.
project = "pytest"
copyright = "2015, holger krekel and pytest-dev team"
copyright = "20152020, holger krekel and pytest-dev team"
# The language for content autogenerated by Sphinx. Refer to documentation
@@ -151,6 +122,7 @@ pygments_style = "sphinx"
# A list of regular expressions that match URIs that should not be checked when
# doing a linkcheck.
linkcheck_ignore = [
"https://github.com/numpy/numpy/blob/master/doc/release/1.16.0-notes.rst#new-deprecations",
"https://blogs.msdn.microsoft.com/bharry/2017/06/28/testing-in-a-cloud-delivery-cadence/",
"http://pythontesting.net/framework/pytest-introduction/",
r"https://github.com/pytest-dev/pytest/issues/\d+",
@@ -161,16 +133,6 @@ linkcheck_ignore = [
linkcheck_workers = 5
_repo = "https://github.com/pytest-dev/pytest"
extlinks = {
"bpo": ("https://bugs.python.org/issue%s", "bpo-%s"),
"pypi": ("https://pypi.org/project/%s/", "%s"),
"issue": (f"{_repo}/issues/%s", "issue #%s"),
"pull": (f"{_repo}/pull/%s", "pull request #%s"),
"user": ("https://github.com/%s", "@%s"),
}
# -- Options for HTML output ---------------------------------------------------
sys.path.append(os.path.abspath("_themes"))
@@ -197,7 +159,7 @@ html_short_title = "pytest-%s" % release
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "img/pytest_logo_curves.svg"
html_logo = "img/pytest1.png"
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
@@ -248,7 +210,7 @@ html_sidebars = {
html_domain_indices = True
# If false, no index is generated.
html_use_index = False
html_use_index = True
# If true, the index is split into individual pages for each letter.
# html_split_index = False
@@ -289,7 +251,7 @@ latex_documents = [
"contents",
"pytest.tex",
"pytest Documentation",
"holger krekel, trainer and consultant, https://merlinux.eu/",
"holger krekel, trainer and consultant, http://merlinux.eu",
"manual",
)
]
@@ -321,9 +283,7 @@ latex_domain_indices = False
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
("how-to/usage", "pytest", "pytest usage", ["holger krekel at merlinux eu"], 1)
]
man_pages = [("usage", "pytest", "pytest usage", ["holger krekel at merlinux eu"], 1)]
# -- Options for Epub output ---------------------------------------------------
@@ -332,7 +292,7 @@ man_pages = [
epub_title = "pytest"
epub_author = "holger krekel at merlinux eu"
epub_publisher = "holger krekel at merlinux eu"
epub_copyright = "2013, holger krekel et alii"
epub_copyright = "2013-2020, holger krekel et alii"
# The language of the text. It defaults to the language option
# or en if the language is not set.
@@ -385,15 +345,10 @@ texinfo_documents = [
]
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {
"pluggy": ("https://pluggy.readthedocs.io/en/stable", None),
"pluggy": ("https://pluggy.readthedocs.io/en/latest", None),
"python": ("https://docs.python.org/3", None),
"numpy": ("https://numpy.org/doc/stable", None),
"pip": ("https://pip.pypa.io/en/stable", None),
"tox": ("https://tox.wiki/en/stable", None),
"virtualenv": ("https://virtualenv.pypa.io/en/stable", None),
"setuptools": ("https://setuptools.pypa.io/en/stable", None),
"packaging": ("https://packaging.python.org/en/latest", None),
}
@@ -421,6 +376,8 @@ def configure_logging(app: "sphinx.application.Sphinx") -> None:
def setup(app: "sphinx.application.Sphinx") -> None:
# from sphinx.ext.autodoc import cut_lines
# app.connect('autodoc-process-docstring', cut_lines(4, what=['module']))
app.add_crossref_type(
"fixture",
"fixture",
@@ -442,13 +399,6 @@ def setup(app: "sphinx.application.Sphinx") -> None:
indextemplate="pair: %s; global variable interpreted by pytest",
)
app.add_crossref_type(
directivename="hook",
rolename="hook",
objname="pytest hook",
indextemplate="pair: %s; hook",
)
configure_logging(app)
# Make Sphinx mark classes with "final" when decorated with @final.
@@ -469,7 +419,3 @@ def setup(app: "sphinx.application.Sphinx") -> None:
)
sphinx.pycode.parser.VariableCommentPicker.is_final = patched_is_final
# legacypath.py monkey-patches pytest.Testdir in. Import the file so
# that autodoc can discover references to it.
import _pytest.legacypath # noqa: F401

View File

@@ -7,24 +7,21 @@ Contact channels
- `pytest issue tracker`_ to report bugs or suggest features (for version
2.0 and above).
- `pytest discussions`_ at github for general questions.
- `pytest discord server <https://discord.com/invite/pytest-dev>`_
for pytest development visibility and general assistance.
- `pytest on stackoverflow.com <http://stackoverflow.com/search?q=pytest>`_
to post precise questions with the tag ``pytest``. New Questions will usually
to post questions with the tag ``pytest``. New Questions will usually
be seen by pytest users or developers and answered quickly.
- `Testing In Python`_: a mailing list for Python testing tools and discussion.
- `pytest-dev at python.org (mailing list)`_ pytest specific announcements and discussions.
- `pytest-commit at python.org (mailing list)`_: for commits and new issues
- :doc:`contribution guide <contributing>` for help on submitting pull
requests to GitHub.
- ``#pytest`` `on irc.libera.chat <ircs://irc.libera.chat:6697/#pytest>`_ IRC
channel for random questions (using an IRC client, `via webchat
<https://web.libera.chat/#pytest>`_, or `via Matrix
<https://matrix.to/#/%23pytest:libera.chat>`_).
- ``#pylib`` on irc.freenode.net IRC channel for random questions.
- private mail to Holger.Krekel at gmail com if you want to communicate sensitive issues
@@ -33,21 +30,19 @@ Contact channels
consulting.
.. _`pytest issue tracker`: https://github.com/pytest-dev/pytest/issues
.. _`old issue tracker`: https://bitbucket.org/hpk42/py-trunk/issues/
.. _`old issue tracker`: http://bitbucket.org/hpk42/py-trunk/issues/
.. _`pytest discussions`: https://github.com/pytest-dev/pytest/discussions
.. _`merlinux.eu`: https://merlinux.eu/
.. _`merlinux.eu`: http://merlinux.eu
.. _`get an account`:
.. _tetamap: https://tetamap.wordpress.com/
.. _tetamap: http://tetamap.wordpress.com
.. _`@pylibcommit`: https://twitter.com/pylibcommit
.. _`@pylibcommit`: http://twitter.com/pylibcommit
.. _`Testing in Python`: http://lists.idyll.org/listinfo/testing-in-python
.. _FOAF: https://en.wikipedia.org/wiki/FOAF
.. _FOAF: http://en.wikipedia.org/wiki/FOAF
.. _`py-dev`:
.. _`development mailing list`:
.. _`pytest-dev at python.org (mailing list)`: http://mail.python.org/mailman/listinfo/pytest-dev

View File

@@ -7,84 +7,41 @@ Full pytest documentation
.. `Download latest version as EPUB <http://media.readthedocs.org/epub/pytest/latest/pytest.epub>`_
Start here
-----------
.. toctree::
:maxdepth: 2
getting-started
usage
existingtestsuite
assert
fixture
mark
monkeypatch
tmpdir
capture
warnings
doctest
skipping
parametrize
cache
unittest
nose
xunit_setup
plugins
writing_plugins
logging
reference
How-to guides
-------------
.. toctree::
:maxdepth: 2
how-to/usage
how-to/assert
how-to/fixtures
how-to/mark
how-to/parametrize
how-to/tmp_path
how-to/monkeypatch
how-to/doctest
how-to/cache
how-to/logging
how-to/capture-stdout-stderr
how-to/capture-warnings
how-to/skipping
how-to/plugins
how-to/writing_plugins
how-to/writing_hook_functions
how-to/existingtestsuite
how-to/unittest
how-to/nose
how-to/xunit_setup
how-to/bash-completion
Reference guides
-----------------
.. toctree::
:maxdepth: 2
reference/fixtures
reference/plugin_list
reference/customize
reference/reference
Explanation
-----------------
.. toctree::
:maxdepth: 2
explanation/anatomy
explanation/fixtures
explanation/goodpractices
explanation/flaky
explanation/pythonpath
Further topics
-----------------
.. toctree::
:maxdepth: 2
goodpractices
flaky
pythonpath
customize
example/index
bash-completion
backwards-compatibility
deprecations
py27-py34-deprecation
contributing
development_guide
@@ -94,9 +51,9 @@ Further topics
license
contact
history
historical-notes
talks
projects
.. only:: html

View File

@@ -20,7 +20,8 @@ Configuration file formats
--------------------------
Many :ref:`pytest settings <ini options ref>` can be set in a *configuration file*, which
by convention resides in the root directory of your repository.
by convention resides on the root of your repository or in your
tests folder.
A quick example of the configuration files supported by pytest:
@@ -29,11 +30,9 @@ pytest.ini
``pytest.ini`` files take precedence over other files, even when empty.
Alternatively, the hidden version ``.pytest.ini`` can be used.
.. code-block:: ini
# pytest.ini or .pytest.ini
# pytest.ini
[pytest]
minversion = 6.0
addopts = -ra -q
@@ -90,7 +89,7 @@ and can also be used to hold pytest configuration if they have a ``[pytest]`` se
setup.cfg
~~~~~~~~~
``setup.cfg`` files are general purpose configuration files, used originally by :doc:`distutils <python:distutils/configfile>`, and can also be used to hold pytest configuration
``setup.cfg`` files are general purpose configuration files, used originally by `distutils <https://docs.python.org/3/distutils/configfile.html>`__, and can also be used to hold pytest configuration
if they have a ``[tool:pytest]`` section.
.. code-block:: ini
@@ -146,8 +145,6 @@ Finding the ``rootdir``
Here is the algorithm which finds the rootdir from ``args``:
- If ``-c`` is passed in the command-line, use that as configuration file, and its directory as ``rootdir``.
- Determine the common ancestor directory for the specified ``args`` that are
recognised as paths that exist in the file system. If no such paths are
found, the common ancestor directory is set to the current working directory.
@@ -163,7 +160,7 @@ Here is the algorithm which finds the rootdir from ``args``:
``setup.cfg`` in each of the specified ``args`` and upwards. If one is
matched, it becomes the ``configfile`` and its directory becomes the ``rootdir``.
- If no ``configfile`` was found and no configuration argument is passed, use the already determined common ancestor as root
- If no ``configfile`` was found, use the already determined common ancestor as root
directory. This allows the use of pytest in structures that are not part of
a package and don't have any particular configuration file.
@@ -180,12 +177,12 @@ Files will only be matched for configuration if:
The files are considered in the order above. Options from multiple ``configfiles`` candidates
are never merged - the first match wins.
The :class:`Config <pytest.Config>` object (accessible via hooks or through the :fixture:`pytestconfig` fixture)
The internal :class:`Config <_pytest.config.Config>` object (accessible via hooks or through the :fixture:`pytestconfig` fixture)
will subsequently carry these attributes:
- :attr:`config.rootpath <pytest.Config.rootpath>`: the determined root directory, guaranteed to exist.
- :attr:`config.rootpath <_pytest.config.Config.rootpath>`: the determined root directory, guaranteed to exist.
- :attr:`config.inipath <pytest.Config.inipath>`: the determined ``configfile``, may be ``None``
- :attr:`config.inipath <_pytest.config.Config.inipath>`: the determined ``configfile``, may be ``None``
(it is named ``inipath`` for historical reasons).
.. versionadded:: 6.1
@@ -227,7 +224,7 @@ check for configuration files as follows:
Custom pytest plugin commandline arguments may include a path, as in
``pytest --log-output ../../test.log args``. Then ``args`` is mandatory,
otherwise pytest uses the folder of test.log for rootdir determination
(see also :issue:`1435`).
(see also `issue 1435 <https://github.com/pytest-dev/pytest/issues/1435>`_).
A dot ``.`` for referencing to the current working directory is also
possible.
@@ -240,11 +237,3 @@ Builtin configuration file options
----------------------------------------------
For the full list of options consult the :ref:`reference documentation <ini options ref>`.
Syntax highlighting theme customization
---------------------------------------
The syntax highlighting themes used by pytest can be customized using two environment variables:
- :envvar:`PYTEST_THEME` sets a `pygment style <https://pygments.org/docs/styles/>`_ to use.
- :envvar:`PYTEST_THEME_MODE` sets this style to *light* or *dark*.

View File

@@ -16,433 +16,7 @@ Deprecated Features
-------------------
Below is a complete list of all pytest features which are considered deprecated. Using those features will issue
:class:`~pytest.PytestWarning` or subclasses, which can be filtered using :ref:`standard warning filters <warnings>`.
.. _nose-deprecation:
Support for tests written for nose
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.2
Support for running tests written for `nose <https://nose.readthedocs.io/en/latest/>`__ is now deprecated.
``nose`` has been in maintenance mode-only for years, and maintaining the plugin is not trivial as it spills
over the code base (see :issue:`9886` for more details).
setup/teardown
^^^^^^^^^^^^^^
One thing that might catch users by surprise is that plain ``setup`` and ``teardown`` methods are not pytest native,
they are in fact part of the ``nose`` support.
.. code-block:: python
class Test:
def setup(self):
self.resource = make_resource()
def teardown(self):
self.resource.close()
def test_foo(self):
...
def test_bar(self):
...
Native pytest support uses ``setup_method`` and ``teardown_method`` (see :ref:`xunit-method-setup`), so the above should be changed to:
.. code-block:: python
class Test:
def setup_method(self):
self.resource = make_resource()
def teardown_method(self):
self.resource.close()
def test_foo(self):
...
def test_bar(self):
...
This is easy to do in an entire code base by doing a simple find/replace.
@with_setup
^^^^^^^^^^^
Code using `@with_setup <with-setup-nose>`_ such as this:
.. code-block:: python
from nose.tools import with_setup
def setup_some_resource():
...
def teardown_some_resource():
...
@with_setup(setup_some_resource, teardown_some_resource)
def test_foo():
...
Will also need to be ported to a supported pytest style. One way to do it is using a fixture:
.. code-block:: python
import pytest
def setup_some_resource():
...
def teardown_some_resource():
...
@pytest.fixture
def some_resource():
setup_some_resource()
yield
teardown_some_resource()
def test_foo(some_resource):
...
.. _`with-setup-nose`: https://nose.readthedocs.io/en/latest/testing_tools.html?highlight=with_setup#nose.tools.with_setup
.. _instance-collector-deprecation:
The ``pytest.Instance`` collector
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. versionremoved:: 7.0
The ``pytest.Instance`` collector type has been removed.
Previously, Python test methods were collected as :class:`~pytest.Class` -> ``Instance`` -> :class:`~pytest.Function`.
Now :class:`~pytest.Class` collects the test methods directly.
Most plugins which reference ``Instance`` do so in order to ignore or skip it,
using a check such as ``if isinstance(node, Instance): return``.
Such plugins should simply remove consideration of ``Instance`` on pytest>=7.
However, to keep such uses working, a dummy type has been instanted in ``pytest.Instance`` and ``_pytest.python.Instance``,
and importing it emits a deprecation warning. This will be removed in pytest 8.
.. _node-ctor-fspath-deprecation:
``fspath`` argument for Node constructors replaced with ``pathlib.Path``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
In order to support the transition from ``py.path.local`` to :mod:`pathlib`,
the ``fspath`` argument to :class:`~_pytest.nodes.Node` constructors like
:func:`pytest.Function.from_parent()` and :func:`pytest.Class.from_parent()`
is now deprecated.
Plugins which construct nodes should pass the ``path`` argument, of type
:class:`pathlib.Path`, instead of the ``fspath`` argument.
Plugins which implement custom items and collectors are encouraged to replace
``fspath`` parameters (``py.path.local``) with ``path`` parameters
(``pathlib.Path``), and drop any other usage of the ``py`` library if possible.
If possible, plugins with custom items should use :ref:`cooperative
constructors <uncooperative-constructors-deprecated>` to avoid hardcoding
arguments they only pass on to the superclass.
.. note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes (the
new attribute being ``path``) is **the opposite** of the situation for
hooks, :ref:`outlined below <legacy-path-hooks-deprecated>` (the old
argument being ``path``).
This is an unfortunate artifact due to historical reasons, which should be
resolved in future versions as we slowly get rid of the :pypi:`py`
dependency (see :issue:`9283` for a longer discussion).
Due to the ongoing migration of methods like :meth:`~_pytest.Item.reportinfo`
which still is expected to return a ``py.path.local`` object, nodes still have
both ``fspath`` (``py.path.local``) and ``path`` (``pathlib.Path``) attributes,
no matter what argument was used in the constructor. We expect to deprecate the
``fspath`` attribute in a future release.
.. _legacy-path-hooks-deprecated:
Configuring hook specs/impls using markers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Before pluggy, pytest's plugin library, was its own package and had a clear API,
pytest just used ``pytest.mark`` to configure hooks.
The :py:func:`pytest.hookimpl` and :py:func:`pytest.hookspec` decorators
have been available since years and should be used instead.
.. code-block:: python
@pytest.mark.tryfirst
def pytest_runtest_call():
...
# or
def pytest_runtest_call():
...
pytest_runtest_call.tryfirst = True
should be changed to:
.. code-block:: python
@pytest.hookimpl(tryfirst=True)
def pytest_runtest_call():
...
Changed ``hookimpl`` attributes:
* ``tryfirst``
* ``trylast``
* ``optionalhook``
* ``hookwrapper``
Changed ``hookwrapper`` attributes:
* ``firstresult``
* ``historic``
``py.path.local`` arguments for hooks replaced with ``pathlib.Path``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
* :hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
* :hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
* :hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
* :hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
* :hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
.. note::
The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
:ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
being ``path``) is **the opposite** of the situation for hooks (the old
argument being ``path``).
This is an unfortunate artifact due to historical reasons, which should be
resolved in future versions as we slowly get rid of the :pypi:`py`
dependency (see :issue:`9283` for a longer discussion).
Directly constructing internal classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
Directly constructing the following classes is now deprecated:
- ``_pytest.mark.structures.Mark``
- ``_pytest.mark.structures.MarkDecorator``
- ``_pytest.mark.structures.MarkGenerator``
- ``_pytest.python.Metafunc``
- ``_pytest.runner.CallInfo``
- ``_pytest._code.ExceptionInfo``
- ``_pytest.config.argparsing.Parser``
- ``_pytest.config.argparsing.OptionGroup``
- ``_pytest.pytester.HookRecorder``
These constructors have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.
.. _cmdline-preparse-deprecated:
Passing ``msg=`` to ``pytest.skip``, ``pytest.fail`` or ``pytest.exit``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
Passing the keyword argument ``msg`` to :func:`pytest.skip`, :func:`pytest.fail` or :func:`pytest.exit`
is now deprecated and ``reason`` should be used instead. This change is to bring consistency between these
functions and the ``@pytest.mark.skip`` and ``@pytest.mark.xfail`` markers which already accept a ``reason`` argument.
.. code-block:: python
def test_fail_example():
# old
pytest.fail(msg="foo")
# new
pytest.fail(reason="bar")
def test_skip_example():
# old
pytest.skip(msg="foo")
# new
pytest.skip(reason="bar")
def test_exit_example():
# old
pytest.exit(msg="foo")
# new
pytest.exit(reason="bar")
Implementing the ``pytest_cmdline_preparse`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
Implementing the :hook:`pytest_cmdline_preparse` hook has been officially deprecated.
Implement the :hook:`pytest_load_initial_conftests` hook instead.
.. code-block:: python
def pytest_cmdline_preparse(config: Config, args: List[str]) -> None:
...
# becomes:
def pytest_load_initial_conftests(
early_config: Config, parser: Parser, args: List[str]
) -> None:
...
.. _diamond-inheritance-deprecated:
Diamond inheritance between :class:`pytest.Collector` and :class:`pytest.Item`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
Defining a custom pytest node type which is both an :class:`pytest.Item <Item>` and a :class:`pytest.Collector <Collector>` (e.g. :class:`pytest.File <File>`) now issues a warning.
It was never sanely supported and triggers hard to debug errors.
Some plugins providing linting/code analysis have been using this as a hack.
Instead, a separate collector node should be used, which collects the item. See
:ref:`non-python tests` for an example, as well as an `example pr fixing inheritance`_.
.. _example pr fixing inheritance: https://github.com/asmeurer/pytest-flakes/pull/40/files
.. _uncooperative-constructors-deprecated:
Constructors of custom :class:`pytest.Node` subclasses should take ``**kwargs``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
If custom subclasses of nodes like :class:`pytest.Item` override the
``__init__`` method, they should take ``**kwargs``. Thus,
.. code-block:: python
class CustomItem(pytest.Item):
def __init__(self, name, parent, additional_arg):
super().__init__(name, parent)
self.additional_arg = additional_arg
should be turned into:
.. code-block:: python
class CustomItem(pytest.Item):
def __init__(self, *, additional_arg, **kwargs):
super().__init__(**kwargs)
self.additional_arg = additional_arg
to avoid hard-coding the arguments pytest can pass to the superclass.
See :ref:`non-python tests` for a full example.
For cases without conflicts, no deprecation warning is emitted. For cases with
conflicts (such as :class:`pytest.File` now taking ``path`` instead of
``fspath``, as :ref:`outlined above <node-ctor-fspath-deprecation>`), a
deprecation warning is now raised.
Backward compatibilities in ``Parser.addoption``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 2.4
Several behaviors of :meth:`Parser.addoption <pytest.Parser.addoption>` are now
scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):
- ``parser.addoption(..., help=".. %default ..")`` - use ``%(default)s`` instead.
- ``parser.addoption(..., type="int/string/float/complex")`` - use ``type=int`` etc. instead.
Using ``pytest.warns(None)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.0
:func:`pytest.warns(None) <pytest.warns>` is now deprecated because it was frequently misused.
Its correct usage was checking that the code emits at least one warning of any type - like ``pytest.warns()``
or ``pytest.warns(Warning)``.
See :ref:`warns use cases` for examples.
Returning non-None value in test functions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 7.2
A :class:`pytest.PytestReturnNotNoneWarning` is now emitted if a test function returns something other than `None`.
This prevents a common mistake among beginners that expect that returning a `bool` would cause a test to pass or fail, for example:
.. code-block:: python
@pytest.mark.parametrize(
["a", "b", "result"],
[
[1, 2, 5],
[2, 3, 8],
[5, 3, 18],
],
)
def test_foo(a, b, result):
return foo(a, b) == result
Given that pytest ignores the return value, this might be surprising that it will never fail.
The proper fix is to change the `return` to an `assert`:
.. code-block:: python
@pytest.mark.parametrize(
["a", "b", "result"],
[
[1, 2, 5],
[2, 3, 8],
[5, 3, 18],
],
)
def test_foo(a, b, result):
assert foo(a, b) == result
:class:`PytestWarning` or subclasses, which can be filtered using :ref:`standard warning filters <warnings>`.
The ``--strict`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -467,42 +41,29 @@ The ``yield_fixture`` function/decorator
It has been so for a very long time, so can be search/replaced safely.
Removed Features
----------------
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
.. deprecated:: 6.0
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recored`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest.collect`` module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 6.0
.. versionremoved:: 7.0
The ``pytest.collect`` module is no longer part of the public API, all its names
should now be imported from ``pytest`` directly instead.
The ``pytest_warning_captured`` hook
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 6.0
.. versionremoved:: 7.0
This hook has an `item` parameter which cannot be serialized by ``pytest-xdist``.
Use the ``pytest_warning_recorded`` hook instead, which replaces the ``item`` parameter
by a ``nodeid`` parameter.
The ``pytest._fillfuncargs`` function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. deprecated:: 6.0
.. versionremoved:: 7.0
This function was kept for backward compatibility with an older plugin.
@@ -511,6 +72,12 @@ it, use `function._request._fillfixtures()` instead, though note this is not
a public API and may break in the future.
Removed Features
----------------
As stated in our :ref:`backwards-compatibility` policy, deprecated features are removed only in major releases after
an appropriate period of deprecation has passed.
``--no-print-logs`` command-line option
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -525,7 +92,6 @@ A ``--show-capture`` command-line option was added in ``pytest 3.5.0`` which all
display captured output when tests fail: ``no``, ``stdout``, ``stderr``, ``log`` or ``all`` (the default).
.. _resultlog deprecated:
Result log (``--result-log``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -548,8 +114,8 @@ at some point, depending on the plans for the plugins and number of users using
.. versionremoved:: 6.0
The ``pytest_collect_directory`` hook has not worked properly for years (it was called
but the results were ignored). Users may consider using :hook:`pytest_collection_modifyitems` instead.
The ``pytest_collect_directory`` has not worked properly for years (it was called
but the results were ignored). Users may consider using :func:`pytest_collection_modifyitems <_pytest.hookspec.pytest_collection_modifyitems>` instead.
TerminalReporter.writer
~~~~~~~~~~~~~~~~~~~~~~~
@@ -563,8 +129,6 @@ with ``py.io.TerminalWriter``.
Plugins that used ``TerminalReporter.writer`` directly should instead use ``TerminalReporter``
methods that provide the same functionality.
.. _junit-family changed default value:
``junit_family`` default value change to "xunit2"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -652,8 +216,6 @@ in places where we or plugin authors must distinguish between fixture names and
names supplied by non-fixture things such as ``pytest.mark.parametrize``.
.. _pytest.config global deprecated:
``pytest.config`` global
~~~~~~~~~~~~~~~~~~~~~~~~
@@ -698,7 +260,7 @@ Becomes:
If you still have concerns about this deprecation and future removal, please comment on
:issue:`3974`.
`issue #3974 <https://github.com/pytest-dev/pytest/issues/3974>`__.
.. _raises-warns-exec:
@@ -751,8 +313,6 @@ This issue should affect only advanced plugins who create new collection types,
message please contact the authors so they can change the code.
.. _marks in pytest.parametrize deprecated:
marks in ``pytest.mark.parametrize``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -801,8 +361,6 @@ To update the code, use ``pytest.param``:
...
.. _pytest_funcarg__ prefix deprecated:
``pytest_funcarg__`` prefix
~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -834,15 +392,13 @@ Switch over to the ``@pytest.fixture`` decorator:
to avoid conflicts with other distutils commands.
.. _metafunc.addcall deprecated:
Metafunc.addcall
~~~~~~~~~~~~~~~~
.. versionremoved:: 4.0
``Metafunc.addcall`` was a precursor to the current parametrized mechanism. Users should use
:meth:`pytest.Metafunc.parametrize` instead.
``_pytest.python.Metafunc.addcall`` was a precursor to the current parametrized mechanism. Users should use
:meth:`_pytest.python.Metafunc.parametrize` instead.
Example:
@@ -860,8 +416,6 @@ Becomes:
metafunc.parametrize("i", [1, 2], ids=["1", "2"])
.. _cached_setup deprecated:
``cached_setup``
~~~~~~~~~~~~~~~~
@@ -890,12 +444,10 @@ This should be updated to make use of standard fixture mechanisms:
session.close()
You can consult :std:doc:`funcarg comparison section in the docs <funcarg_compare>` for
You can consult `funcarg comparison section in the docs <https://docs.pytest.org/en/stable/funcarg_compare.html>`_ for
more information.
.. _pytest_plugins in non-top-level conftest files deprecated:
pytest_plugins in non-top-level conftest files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -906,8 +458,6 @@ files because they will activate referenced plugins *globally*, which is surpris
features ``conftest.py`` files are only *active* for tests at or below it.
.. _config.warn and node.warn deprecated:
``Config.warn`` and ``Node.warn``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -935,8 +485,6 @@ Becomes:
* ``node.warn("CI", "some message")``: this code/message form has been **removed** and should be converted to the warning instance form above.
.. _record_xml_property deprecated:
record_xml_property
~~~~~~~~~~~~~~~~~~~
@@ -960,8 +508,6 @@ Change to:
...
.. _passing command-line string to pytest.main deprecated:
Passing command-line string to ``pytest.main()``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -984,8 +530,6 @@ By passing a string, users expect that pytest will interpret that command-line u
on (for example ``bash`` or ``Powershell``), but this is very hard/impossible to do in a portable way.
.. _calling fixtures directly deprecated:
Calling fixtures directly
~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1039,8 +583,6 @@ with the ``name`` parameter:
return cell()
.. _yield tests deprecated:
``yield`` tests
~~~~~~~~~~~~~~~
@@ -1052,7 +594,7 @@ that are then turned into proper test methods. Example:
.. code-block:: python
def check(x, y):
assert x**x == y
assert x ** x == y
def test_squared():
@@ -1067,9 +609,7 @@ This form of test function doesn't support fixtures properly, and users should s
@pytest.mark.parametrize("x, y", [(2, 4), (3, 9)])
def test_squared(x, y):
assert x**x == y
.. _internal classes accessed through node deprecated:
assert x ** x == y
Internal classes accessed through ``Node``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1105,8 +645,6 @@ As part of a large :ref:`marker-revamp` we already deprecated using ``MarkInfo``
the only correct way to get markers of an element is via ``node.iter_markers(name)``.
.. _pytest.namespace deprecated:
``pytest_namespace``
~~~~~~~~~~~~~~~~~~~~

View File

@@ -4,4 +4,4 @@ Development Guide
The contributing guidelines are to be found :ref:`here <contributing>`.
The release procedure for pytest is documented on
`GitHub <https://github.com/pytest-dev/pytest/blob/main/RELEASING.rst>`_.
`GitHub <https://github.com/pytest-dev/pytest/blob/master/RELEASING.rst>`_.

View File

@@ -1,10 +1,9 @@
.. _doctest:
How to run doctests
Doctest integration for modules and test files
=========================================================
By default, all files matching the ``test*.txt`` pattern will
be run through the python standard :mod:`doctest` module. You
be run through the python standard ``doctest`` module. You
can change the pattern by issuing:
.. code-block:: bash
@@ -30,8 +29,9 @@ then you can just invoke ``pytest`` directly:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_example.txt . [100%]
@@ -48,7 +48,7 @@ and functions, including from test modules:
# content of mymodule.py
def something():
"""a doctest in a docstring
""" a doctest in a docstring
>>> something()
42
"""
@@ -58,8 +58,9 @@ and functions, including from test modules:
$ pytest --doctest-modules
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
mymodule.py . [ 50%]
@@ -90,12 +91,10 @@ that will be used for those doctest files using the
[pytest]
doctest_encoding = latin1
.. _using doctest options:
Using 'doctest' options
-----------------------
Python's standard :mod:`doctest` module provides some :ref:`options <python:option-flags-and-directives>`
Python's standard ``doctest`` module provides some `options <https://docs.python.org/3/library/doctest.html#option-flags>`__
to configure the strictness of doctest tests. In pytest, you can enable those flags using the
configuration file.
@@ -126,17 +125,14 @@ pytest also introduces new options:
in expected doctest output.
* ``NUMBER``: when enabled, floating-point numbers only need to match as far as
the precision you have written in the expected doctest output. The numbers are
compared using :func:`pytest.approx` with relative tolerance equal to the
precision. For example, the following output would only need to match to 2
decimal places when comparing ``3.14`` to
``pytest.approx(math.pi, rel=10**-2)``::
the precision you have written in the expected doctest output. For example,
the following output would only need to match to 2 decimal places::
>>> math.pi
3.14
If you wrote ``3.1416`` then the actual output would need to match to
approximately 4 decimal places; and so on.
If you wrote ``3.1416`` then the actual output would need to match to 4
decimal places; and so on.
This avoids false positives caused by limited floating-point precision, like
this::
@@ -197,7 +193,7 @@ It is possible to use fixtures using the ``getfixture`` helper:
.. code-block:: text
# content of example.rst
>>> tmp = getfixture('tmp_path')
>>> tmp = getfixture('tmpdir')
>>> ...
>>>
@@ -242,6 +238,7 @@ which can then be used in your doctests directly:
>>> len(a)
10
"""
pass
Note that like the normal ``conftest.py``, the fixtures are discovered in the directory tree conftest is in.
Meaning that if you put your doctest with your source code, the relevant conftest.py needs to be in the same directory tree.
@@ -254,7 +251,7 @@ For the same reasons one might want to skip normal tests, it is also possible to
tests inside doctests.
To skip a single check inside a doctest you can use the standard
:data:`doctest.SKIP` directive:
`doctest.SKIP <https://docs.python.org/3/library/doctest.html#doctest.SKIP>`__ directive:
.. code-block:: python

View File

@@ -5,9 +5,9 @@ failure_demo = os.path.join(os.path.dirname(__file__), "failure_demo.py")
pytest_plugins = ("pytester",)
def test_failure_demo_fails_properly(pytester):
target = pytester.path.joinpath(os.path.basename(failure_demo))
def test_failure_demo_fails_properly(testdir):
target = testdir.tmpdir.join(os.path.basename(failure_demo))
shutil.copy(failure_demo, target)
result = pytester.runpytest(target, syspathinsert=True)
result = testdir.runpytest(target, syspathinsert=True)
result.stdout.fnmatch_lines(["*44 failed*"])
assert result.ret != 0

View File

@@ -25,7 +25,7 @@ example: specifying and selecting acceptance tests
self.tmpdir = request.config.mktemp(request.function.__name__, numbered=True)
def run(self, *cmd):
"""called by test code to execute an acceptance test."""
""" called by test code to execute an acceptance test. """
self.tmpdir.chdir()
return subprocess.check_output(cmd).decode()

View File

@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="572" height="542">
<svg xmlns="http://www.w3.org/2000/svg" width="572" height="542">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
@@ -37,7 +37,7 @@
<path d="M 26,271 A 260 260 0 0 1 546 271" id="testp"/>
</defs>
<text class="package">
<textPath xlink:href="#testp" startOffset="50%">tests</textPath>
<textPath href="#testp" startOffset="50%">tests</textPath>
</text>
<!-- subpackage -->
@@ -47,7 +47,7 @@
<path d="M 56,271 A 130 130 0 0 1 316 271" id="subpackage"/>
</defs>
<text class="package">
<textPath xlink:href="#subpackage" startOffset="50%">subpackage</textPath>
<textPath href="#subpackage" startOffset="50%">subpackage</textPath>
</text>
<!-- test_subpackage.py -->
@@ -57,7 +57,7 @@
<path d="M 106,311 A 80 80 0 0 1 266 311" id="testSubpackage"/>
</defs>
<text class="module">
<textPath xlink:href="#testSubpackage" startOffset="50%">test_subpackage.py</textPath>
<textPath href="#testSubpackage" startOffset="50%">test_subpackage.py</textPath>
</text>
<!-- innermost -->
<line x1="186" x2="186" y1="271" y2="351"/>
@@ -102,7 +102,7 @@
<path d="M 366,271 A 75 75 0 0 1 516 271" id="testTop"/>
</defs>
<text class="module">
<textPath xlink:href="#testTop" startOffset="50%">test_top.py</textPath>
<textPath href="#testTop" startOffset="50%">test_top.py</textPath>
</text>
<!-- innermost -->
<line x1="441" x2="441" y1="306" y2="236"/>

Before

Width:  |  Height:  |  Size: 5.0 KiB

After

Width:  |  Height:  |  Size: 4.9 KiB

View File

@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="587" height="382">
<svg xmlns="http://www.w3.org/2000/svg" width="587" height="382">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
@@ -38,7 +38,7 @@
<path d="M 411,86 A 75 75 0 0 1 561 86" id="pluginA"/>
</defs>
<text class="plugin">
<textPath xlink:href="#pluginA" startOffset="50%">plugin_a</textPath>
<textPath href="#pluginA" startOffset="50%">plugin_a</textPath>
</text>
<!-- scope order number -->
<mask id="pluginAOrderMask">
@@ -55,7 +55,7 @@
<path d="M 411,296 A 75 75 0 0 1 561 296" id="pluginB"/>
</defs>
<text class="plugin">
<textPath xlink:href="#pluginB" startOffset="50%">plugin_b</textPath>
<textPath href="#pluginB" startOffset="50%">plugin_b</textPath>
</text>
<!-- scope order number -->
<mask id="pluginBOrderMask">
@@ -72,7 +72,7 @@
<path d="M 11,191 A 180 180 0 0 1 371 191" id="testp"/>
</defs>
<text class="package">
<textPath xlink:href="#testp" startOffset="50%">tests</textPath>
<textPath href="#testp" startOffset="50%">tests</textPath>
</text>
<!-- scope order number -->
<mask id="mainOrderMask">
@@ -89,7 +89,7 @@
<path d="M 61,231 A 130 130 0 0 1 321 231" id="subpackage"/>
</defs>
<text class="package">
<textPath xlink:href="#subpackage" startOffset="50%">subpackage</textPath>
<textPath href="#subpackage" startOffset="50%">subpackage</textPath>
</text>
<!-- scope order number -->
<mask id="subpackageOrderMask">
@@ -106,7 +106,7 @@
<path d="M 111,271 A 80 80 0 0 1 271 271" id="testSubpackage"/>
</defs>
<text class="module">
<textPath xlink:href="#testSubpackage" startOffset="50%">test_subpackage.py</textPath>
<textPath href="#testSubpackage" startOffset="50%">test_subpackage.py</textPath>
</text>
<!-- scope order number -->
<mask id="testSubpackageOrderMask">

Before

Width:  |  Height:  |  Size: 5.4 KiB

After

Width:  |  Height:  |  Size: 5.3 KiB

View File

@@ -1,56 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="112" height="682">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
dominant-baseline: middle;
text-anchor: middle;
fill: #062886;
font-size: medium;
}
ellipse.fixture, rect.test {
fill: #eeffcc;
stroke: #007020;
stroke-width: 2;
}
text.fixture {
color: #06287e;
}
circle.class {
fill: #c3e0ec;
stroke: #0e84b5;
stroke-width: 2;
}
text.class {
fill: #0e84b5;
}
path, line {
stroke: black;
stroke-width: 2;
fill: none;
}
rect.autouse {
fill: #ca7f3d;
}
</style>
<line x1="56" x2="56" y1="681" y2="26" />
<ellipse class="fixture" rx="50" ry="25" cx="56" cy="26" />
<text x="56" y="26">order</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="96" />
<text x="56" y="96">a</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="166" />
<text x="56" y="166">b</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="236" />
<text x="56" y="236">c</text>
<rect class="autouse" width="112" height="40" x="0" y="286" />
<text x="56" y="306">autouse</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="376" />
<text x="56" y="376">d</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="446" />
<text x="56" y="446">e</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="516" />
<text x="56" y="516">f</text>
<ellipse class="fixture" rx="25" ry="25" cx="56" cy="586" />
<text x="56" y="586">g</text>
<rect class="test" width="110" height="50" x="1" y="631" />
<text x="56" y="656">test_order</text>
</svg>

Before

Width:  |  Height:  |  Size: 1.9 KiB

View File

@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="862" height="402">
<svg xmlns="http://www.w3.org/2000/svg" width="862" height="402">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
@@ -48,7 +48,7 @@
<path d="M31,201 A 190 190 0 0 1 411 201" id="testClassWith"/>
</defs>
<text class="class">
<textPath xlink:href="#testClassWith" startOffset="50%">TestWithC1Request</textPath>
<textPath href="#testClassWith" startOffset="50%">TestWithC1Request</textPath>
</text>
<!-- TestWithoutC1Request -->
@@ -67,7 +67,7 @@
<path d="M451,201 A 190 190 0 0 1 831 201" id="testClassWithout"/>
</defs>
<text class="class">
<textPath xlink:href="#testClassWithout" startOffset="50%">TestWithoutC1Request</textPath>
<textPath href="#testClassWithout" startOffset="50%">TestWithoutC1Request</textPath>
</text>
<rect class="autouse" width="862" height="40" x="1" y="181" />

Before

Width:  |  Height:  |  Size: 2.6 KiB

After

Width:  |  Height:  |  Size: 2.6 KiB

View File

@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="862" height="502">
<svg xmlns="http://www.w3.org/2000/svg" width="862" height="502">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
@@ -39,7 +39,7 @@
<path d="M11,251 A 240 240 0 0 1 491 251" id="testClassWith"/>
</defs>
<text class="class">
<textPath xlink:href="#testClassWith" startOffset="50%">TestWithAutouse</textPath>
<textPath href="#testClassWith" startOffset="50%">TestWithAutouse</textPath>
</text>
<mask id="autouseScope">
<circle fill="white" r="249" cx="251" cy="251" />
@@ -79,7 +79,7 @@
<path d="M 531,251 A 160 160 0 0 1 851 251" id="testClassWithout"/>
</defs>
<text class="class">
<textPath xlink:href="#testClassWithout" startOffset="50%">TestWithoutAutouse</textPath>
<textPath href="#testClassWithout" startOffset="50%">TestWithoutAutouse</textPath>
</text>
<!-- TestWithoutAutouse.test_req -->

Before

Width:  |  Height:  |  Size: 3.6 KiB

After

Width:  |  Height:  |  Size: 3.5 KiB

View File

@@ -17,7 +17,7 @@ def b(a, order):
@pytest.fixture
def c(b, order):
def c(a, b, order):
order.append("c")

View File

@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="262" height="537">
<svg xmlns="http://www.w3.org/2000/svg" width="262" height="537">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
@@ -50,6 +50,6 @@
<path d="M131,526 A 120 120 0 0 1 136 286" id="testClass"/>
</defs>
<text class="class">
<textPath xlink:href="#testClass" startOffset="50%">TestClass</textPath>
<textPath href="#testClass" startOffset="50%">TestClass</textPath>
</text>
</svg>

Before

Width:  |  Height:  |  Size: 1.9 KiB

After

Width:  |  Height:  |  Size: 1.8 KiB

View File

@@ -1,4 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="562" height="532">
<svg xmlns="http://www.w3.org/2000/svg" width="562" height="532">
<style>
text {
font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace;
@@ -41,7 +41,7 @@
<path d="M 26,266 A 255 255 0 0 1 536 266" id="testModule"/>
</defs>
<text class="module">
<textPath xlink:href="#testModule" startOffset="50%">test_fixtures_request_different_scope.py</textPath>
<textPath href="#testModule" startOffset="50%">test_fixtures_request_different_scope.py</textPath>
</text>
<!-- TestOne -->
@@ -61,7 +61,7 @@
<path d="M 51,266 A 90 90 0 0 1 231 266" id="testOne"/>
</defs>
<text class="class">
<textPath xlink:href="#testOne" startOffset="50%">TestOne</textPath>
<textPath href="#testOne" startOffset="50%">TestOne</textPath>
</text>
<!-- scope order number -->
<mask id="testOneOrderMask">
@@ -95,7 +95,7 @@
<path d="M 331,266 A 90 90 0 0 1 511 266" id="testTwo"/>
</defs>
<text class="class">
<textPath xlink:href="#testTwo" startOffset="50%">TestTwo</textPath>
<textPath href="#testTwo" startOffset="50%">TestTwo</textPath>
</text>
<!-- scope order number -->
<mask id="testTwoOrderMask">

Before

Width:  |  Height:  |  Size: 4.3 KiB

After

Width:  |  Height:  |  Size: 4.2 KiB

View File

@@ -13,12 +13,12 @@ answers.
For basic examples, see
- :ref:`get-started` for basic introductory examples
- :doc:`../getting-started` for basic introductory examples
- :ref:`assert` for basic assertion examples
- :ref:`Fixtures <fixtures>` for basic fixture/setup examples
- :ref:`parametrize` for basic test function parametrization
- :ref:`unittest` for basic unittest integration
- :ref:`noseintegration` for basic nosetests integration
- :doc:`../unittest` for basic unittest integration
- :doc:`../nose` for basic nosetests integration
The following examples aim at various use cases you might encounter.

View File

@@ -45,9 +45,9 @@ You can then restrict a test run to only run tests marked with ``webtest``:
$ pytest -v -m webtest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 3 deselected / 1 selected
test_server.py::test_send_http PASSED [100%]
@@ -60,9 +60,9 @@ Or the inverse, running all tests except the webtest ones:
$ pytest -v -m "not webtest"
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 1 deselected / 3 selected
test_server.py::test_something_quick PASSED [ 33%]
@@ -82,9 +82,9 @@ tests based on their module, class, method, or function name:
$ pytest -v test_server.py::TestClass::test_method
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED [100%]
@@ -97,9 +97,9 @@ You can also select on the class:
$ pytest -v test_server.py::TestClass
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED [100%]
@@ -112,9 +112,9 @@ Or select multiple nodes:
$ pytest -v test_server.py::TestClass test_server.py::test_send_http
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 2 items
test_server.py::TestClass::test_method PASSED [ 50%]
@@ -156,9 +156,9 @@ The expression matching is now case-insensitive.
$ pytest -v -k http # running with the above defined example module
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 3 deselected / 1 selected
test_server.py::test_send_http PASSED [100%]
@@ -171,9 +171,9 @@ And you can also run all tests except the ones that match the keyword:
$ pytest -k "not send_http" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 1 deselected / 3 selected
test_server.py::test_something_quick PASSED [ 33%]
@@ -188,9 +188,9 @@ Or to select "http" and "quick" tests:
$ pytest -k "http or quick" -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 4 items / 2 deselected / 2 selected
test_server.py::test_send_http PASSED [ 50%]
@@ -234,21 +234,21 @@ You can ask which markers exist for your test suite - the list includes our just
@pytest.mark.slow: mark test as slow.
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-skipif
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-skipif
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-xfail
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-xfail
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/how-to/parametrize.html for more info and examples.
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/parametrize.html for more info and examples.
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/fixture.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
For an example on how to add and work with markers from a plugin, see
@@ -346,7 +346,7 @@ Custom marker and command line option to control test runs
Plugins can provide custom markers and implement specific behaviour
based on it. This is a self-contained example which adds a command
line option and a parametrized test function marker to run tests
specified via named environments:
specifies via named environments:
.. code-block:: python
@@ -375,7 +375,7 @@ specified via named environments:
envnames = [mark.args[0] for mark in item.iter_markers(name="env")]
if envnames:
if item.config.getoption("-E") not in envnames:
pytest.skip(f"test requires env in {envnames!r}")
pytest.skip("test requires env in {!r}".format(envnames))
A test file using this local plugin:
@@ -397,8 +397,9 @@ the test needs:
$ pytest -E stage2
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_someenv.py s [100%]
@@ -411,8 +412,9 @@ and here is one that specifies exactly the environment needed:
$ pytest -E stage1
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_someenv.py . [100%]
@@ -426,21 +428,21 @@ The ``--markers`` option always gives you a list of available markers:
$ pytest --markers
@pytest.mark.env(name): mark test to run only on named environment
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/how-to/capture-warnings.html#pytest-mark-filterwarnings
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/stable/warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-skipif
@pytest.mark.skipif(condition, ..., *, reason=...): skip the given test function if any of the conditions evaluate to True. Example: skipif(sys.platform == 'win32') skips the test if we are on the win32 platform. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-skipif
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference/reference.html#pytest-mark-xfail
@pytest.mark.xfail(condition, ..., *, reason=..., run=True, raises=None, strict=xfail_strict): mark the test function as an expected failure if any of the conditions evaluate to True. Optionally specify a reason for better reporting and run=False if you don't even want to execute the test function. If only specific exception(s) are expected, you can list them in raises, and if the test fails in other ways, it will be reported as a true failure. See https://docs.pytest.org/en/stable/reference.html#pytest-mark-xfail
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/how-to/parametrize.html for more info and examples.
@pytest.mark.parametrize(argnames, argvalues): call a test function multiple times passing in different arguments in turn. argvalues generally needs to be a list of values if argnames specifies only one name or a list of tuples of values if argnames specifies multiple names. Example: @parametrize('arg1', [1,2]) would lead to two calls of the decorated test function, one with arg1=1 and another with arg1=2.see https://docs.pytest.org/en/stable/parametrize.html for more info and examples.
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/explanation/fixtures.html#usefixtures
@pytest.mark.usefixtures(fixturename1, fixturename2, ...): mark tests as needing all of the specified fixtures. see https://docs.pytest.org/en/stable/fixture.html#usefixtures
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible. DEPRECATED, use @pytest.hookimpl(tryfirst=True) instead.
@pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible. DEPRECATED, use @pytest.hookimpl(trylast=True) instead.
@pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
.. _`passing callables to custom markers`:
@@ -486,7 +488,7 @@ The output is as follows:
.. code-block:: pytest
$ pytest -q -s
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef0001>,), kwargs={})
Mark(name='my_marker', args=(<function hello_world at 0xdeadbeef>,), kwargs={})
.
1 passed in 0.12s
@@ -528,7 +530,7 @@ test function. From a conftest file we can read it like this:
def pytest_runtest_setup(item):
for mark in item.iter_markers(name="glob"):
print(f"glob args={mark.args} kwargs={mark.kwargs}")
print("glob args={} kwargs={}".format(mark.args, mark.kwargs))
sys.stdout.flush()
Let's run this without capturing output and see what we get:
@@ -558,7 +560,6 @@ for your particular platform, you could use the following plugin:
# content of conftest.py
#
import sys
import pytest
ALL = set("darwin linux win32".split())
@@ -568,7 +569,7 @@ for your particular platform, you could use the following plugin:
supported_platforms = ALL.intersection(mark.name for mark in item.iter_markers())
plat = sys.platform
if supported_platforms and plat not in supported_platforms:
pytest.skip(f"cannot run on platform {plat}")
pytest.skip("cannot run on platform {}".format(plat))
then tests will be skipped if they were specified for a different platform.
Let's do a little test file to show how this looks like:
@@ -604,14 +605,15 @@ then you will see two tests skipped and two executed tests as expected:
$ pytest -rs # this option reports skip reasons
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
test_plat.py s.s. [100%]
========================= short test summary info ==========================
SKIPPED [2] conftest.py:13: cannot run on platform linux
SKIPPED [2] conftest.py:12: cannot run on platform linux
======================= 2 passed, 2 skipped in 0.12s =======================
Note that if you specify a platform via the marker-command line option like this:
@@ -620,8 +622,9 @@ Note that if you specify a platform via the marker-command line option like this
$ pytest -m linux
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items / 3 deselected / 1 selected
test_plat.py . [100%]
@@ -683,8 +686,9 @@ We can now use the ``-m option`` to select one set:
$ pytest -m interface --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items / 2 deselected / 2 selected
test_module.py FF [100%]
@@ -709,8 +713,9 @@ or to select both "event" and "interface" tests:
$ pytest -m "interface or event" --tb=short
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items / 1 deselected / 3 selected
test_module.py FFF [100%]

View File

@@ -12,8 +12,8 @@ pythonlist = ["python3.5", "python3.6", "python3.7"]
@pytest.fixture(params=pythonlist)
def python1(request, tmp_path):
picklefile = tmp_path / "data.pickle"
def python1(request, tmpdir):
picklefile = tmpdir.join("data.pickle")
return Python(request.param, picklefile)
@@ -30,8 +30,8 @@ class Python:
self.picklefile = picklefile
def dumps(self, obj):
dumpfile = self.picklefile.with_name("dump.py")
dumpfile.write_text(
dumpfile = self.picklefile.dirpath("dump.py")
dumpfile.write(
textwrap.dedent(
r"""
import pickle
@@ -46,8 +46,8 @@ class Python:
subprocess.check_call((self.pythonpath, str(dumpfile)))
def load_and_is_true(self, expression):
loadfile = self.picklefile.with_name("load.py")
loadfile.write_text(
loadfile = self.picklefile.dirpath("load.py")
loadfile.write(
textwrap.dedent(
r"""
import pickle

View File

@@ -9,7 +9,8 @@ Working with non-python tests
A basic example for specifying tests in Yaml files
--------------------------------------------------------------
.. _`pytest-yamlwsgi`: https://pypi.org/project/pytest-yamlwsgi/
.. _`pytest-yamlwsgi`: http://bitbucket.org/aafshar/pytest-yamlwsgi/src/tip/pytest_yamlwsgi.py
.. _`PyYAML`: https://pypi.org/project/PyYAML/
Here is an example ``conftest.py`` (extracted from Ali Afshar's special purpose `pytest-yamlwsgi`_ plugin). This ``conftest.py`` will collect ``test*.yaml`` files and will execute the yaml-formatted content as custom tests:
@@ -21,15 +22,16 @@ You can create a simple example file:
.. include:: nonpython/test_simple.yaml
:literal:
and if you installed :pypi:`PyYAML` or a compatible YAML-parser you can
and if you installed `PyYAML`_ or a compatible YAML-parser you can
now execute the test specification:
.. code-block:: pytest
nonpython $ pytest test_simple.yaml
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project/nonpython
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collected 2 items
test_simple.yaml F. [100%]
@@ -64,9 +66,9 @@ consulted when reporting in ``verbose`` mode:
nonpython $ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project/nonpython
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collecting ... collected 2 items
test_simple.yaml::hello FAILED [ 50%]
@@ -90,8 +92,9 @@ interesting to just look at the collection tree:
nonpython $ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project/nonpython
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/nonpython
collected 2 items
<Package nonpython>

View File

@@ -2,9 +2,9 @@
import pytest
def pytest_collect_file(parent, file_path):
if file_path.suffix == ".yaml" and file_path.name.startswith("test"):
return YamlFile.from_parent(parent, path=file_path)
def pytest_collect_file(parent, path):
if path.ext == ".yaml" and path.basename.startswith("test"):
return YamlFile.from_parent(parent, fspath=path)
class YamlFile(pytest.File):
@@ -12,14 +12,14 @@ class YamlFile(pytest.File):
# We need a yaml parser, e.g. PyYAML.
import yaml
raw = yaml.safe_load(self.path.open())
raw = yaml.safe_load(self.fspath.open())
for name, spec in sorted(raw.items()):
yield YamlItem.from_parent(self, name=name, spec=spec)
class YamlItem(pytest.Item):
def __init__(self, *, spec, **kwargs):
super().__init__(**kwargs)
def __init__(self, name, parent, spec):
super().__init__(name, parent)
self.spec = spec
def runtest(self):
@@ -40,7 +40,7 @@ class YamlItem(pytest.Item):
)
def reportinfo(self):
return self.path, 0, f"usecase: {self.name}"
return self.fspath, 0, f"usecase: {self.name}"
class YamlException(Exception):

View File

@@ -97,10 +97,10 @@ the argument name:
# content of test_time.py
from datetime import datetime, timedelta
import pytest
from datetime import datetime, timedelta
testdata = [
(datetime(2001, 12, 12), datetime(2001, 12, 11), timedelta(1)),
(datetime(2001, 12, 11), datetime(2001, 12, 12), timedelta(-1)),
@@ -160,8 +160,9 @@ objects, they are still using the default pytest representation:
$ pytest test_time.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 8 items
<Module test_time.py>
@@ -182,7 +183,9 @@ together with the actual data, instead of listing them separately.
A quick port of "testscenarios"
------------------------------------
Here is a quick port to run tests configured with :pypi:`testscenarios`,
.. _`test scenarios`: https://pypi.org/project/testscenarios/
Here is a quick port to run tests configured with `test scenarios`_,
an add-on from Robert Collins for the standard unittest framework. We
only have to work a bit to construct the correct arguments for pytest's
:py:func:`Metafunc.parametrize`:
@@ -222,8 +225,9 @@ this is a fully self-contained example which you can run with:
$ pytest test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
test_scenarios.py .... [100%]
@@ -236,16 +240,17 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
$ pytest --collect-only test_scenarios.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
<Module test_scenarios.py>
<Class TestSampleWithScenarios>
<Function test_demo1[basic]>
<Function test_demo2[basic]>
<Function test_demo1[advanced]>
<Function test_demo2[advanced]>
<Function test_demo1[basic]>
<Function test_demo2[basic]>
<Function test_demo1[advanced]>
<Function test_demo2[advanced]>
======================== 4 tests collected in 0.12s ========================
@@ -314,8 +319,9 @@ Let's first see how it looks like at collection time:
$ pytest test_backends.py --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
<Module test_backends.py>
@@ -333,7 +339,7 @@ And then when we run the test:
================================= FAILURES =================================
_________________________ test_db_initialized[d2] __________________________
db = <conftest.DB2 object at 0xdeadbeef0001>
db = <conftest.DB2 object at 0xdeadbeef>
def test_db_initialized(db):
# a dummy test
@@ -412,9 +418,9 @@ The result of this test will be successful:
$ pytest -v test_indirect_list.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 1 item
test_indirect_list.py::test_indirect[a-b] PASSED [100%]
@@ -472,7 +478,7 @@ argument sets to use for each test function. Let's run it:
================================= FAILURES =================================
________________________ TestClass.test_equals[1-2] ________________________
self = <test_parametrize.TestClass object at 0xdeadbeef0002>, a = 1, b = 2
self = <test_parametrize.TestClass object at 0xdeadbeef>, a = 1, b = 2
def test_equals(self, a, b):
> assert a == b
@@ -504,9 +510,9 @@ Running it results in some skips if we don't have all the python interpreters in
. $ pytest -rs -q multipython.py
sssssssssssssssssssssssssss [100%]
========================= short test summary info ==========================
SKIPPED [9] multipython.py:69: 'python3.5' not found
SKIPPED [9] multipython.py:69: 'python3.6' not found
SKIPPED [9] multipython.py:69: 'python3.7' not found
SKIPPED [9] multipython.py:29: 'python3.5' not found
SKIPPED [9] multipython.py:29: 'python3.6' not found
SKIPPED [9] multipython.py:29: 'python3.7' not found
27 skipped in 0.12s
Indirect parametrization of optional implementations/imports
@@ -567,14 +573,15 @@ If you run this with reporting for skips enabled:
$ pytest -rs test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
test_module.py .s [100%]
========================= short test summary info ==========================
SKIPPED [1] test_module.py:3: could not import 'opt2': No module named 'opt2'
SKIPPED [1] conftest.py:12: could not import 'opt2': No module named 'opt2'
======================= 1 passed, 1 skipped in 0.12s =======================
You'll see that we don't have an ``opt2`` module and thus the second test run
@@ -628,9 +635,9 @@ Then run ``pytest`` with verbose mode and with only the ``basic`` marker:
$ pytest -v -m basic
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collecting ... collected 24 items / 21 deselected / 3 selected
test_pytest_param_example.py::test_eval[1+7-8] PASSED [ 33%]
@@ -657,17 +664,20 @@ Use :func:`pytest.raises` with the
:ref:`pytest.mark.parametrize ref` decorator to write parametrized tests
in which some tests raise exceptions and others do not.
It may be helpful to use ``nullcontext`` as a complement to ``raises``.
For example:
It is helpful to define a no-op context manager ``does_not_raise`` to serve
as a complement to ``raises``. For example:
.. code-block:: python
from contextlib import nullcontext as does_not_raise
from contextlib import contextmanager
import pytest
@contextmanager
def does_not_raise():
yield
@pytest.mark.parametrize(
"example_input,expectation",
[
@@ -684,3 +694,22 @@ For example:
In the example above, the first three test cases should run unexceptionally,
while the fourth should raise ``ZeroDivisionError``.
If you're only supporting Python 3.7+, you can simply use ``nullcontext``
to define ``does_not_raise``:
.. code-block:: python
from contextlib import nullcontext as does_not_raise
Or, if you're supporting Python 3.3+ you can use:
.. code-block:: python
from contextlib import ExitStack as does_not_raise
Or, if desired, you can ``pip install contextlib2`` and use:
.. code-block:: python
from contextlib2 import nullcontext as does_not_raise

View File

@@ -147,15 +147,15 @@ The test collection would look like this:
$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
configfile: pytest.ini
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 2 items
<Module check_myapp.py>
<Class CheckMyApp>
<Function simple_check>
<Function complex_check>
<Function simple_check>
<Function complex_check>
======================== 2 tests collected in 0.12s ========================
@@ -209,16 +209,16 @@ You can always peek at the collection tree without running tests like this:
. $ pytest --collect-only pythoncollection.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
configfile: pytest.ini
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 3 items
<Module CWD/pythoncollection.py>
<Function test_function>
<Class TestClass>
<Function test_method>
<Function test_anothermethod>
<Function test_method>
<Function test_anothermethod>
======================== 3 tests collected in 0.12s ========================
@@ -291,9 +291,9 @@ file will be left out:
$ pytest --collect-only
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
configfile: pytest.ini
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, configfile: pytest.ini
collected 0 items
======================= no tests collected in 0.12s ========================

View File

@@ -9,8 +9,9 @@ Here is a nice run of several failures and how ``pytest`` presents things:
assertion $ pytest failure_demo.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project/assertion
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR/assertion
collected 44 items
failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [100%]
@@ -28,7 +29,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:19: AssertionError
_________________________ TestFailing.test_simple __________________________
self = <failure_demo.TestFailing object at 0xdeadbeef0001>
self = <failure_demo.TestFailing object at 0xdeadbeef>
def test_simple(self):
def f():
@@ -39,13 +40,13 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert f() == g()
E assert 42 == 43
E + where 42 = <function TestFailing.test_simple.<locals>.f at 0xdeadbeef0002>()
E + and 43 = <function TestFailing.test_simple.<locals>.g at 0xdeadbeef0003>()
E + where 42 = <function TestFailing.test_simple.<locals>.f at 0xdeadbeef>()
E + and 43 = <function TestFailing.test_simple.<locals>.g at 0xdeadbeef>()
failure_demo.py:30: AssertionError
____________________ TestFailing.test_simple_multiline _____________________
self = <failure_demo.TestFailing object at 0xdeadbeef0004>
self = <failure_demo.TestFailing object at 0xdeadbeef>
def test_simple_multiline(self):
> otherfunc_multi(42, 6 * 9)
@@ -62,7 +63,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:14: AssertionError
___________________________ TestFailing.test_not ___________________________
self = <failure_demo.TestFailing object at 0xdeadbeef0005>
self = <failure_demo.TestFailing object at 0xdeadbeef>
def test_not(self):
def f():
@@ -70,12 +71,12 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert not f()
E assert not 42
E + where 42 = <function TestFailing.test_not.<locals>.f at 0xdeadbeef0006>()
E + where 42 = <function TestFailing.test_not.<locals>.f at 0xdeadbeef>()
failure_demo.py:39: AssertionError
_________________ TestSpecialisedExplanations.test_eq_text _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0007>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_text(self):
> assert "spam" == "eggs"
@@ -86,7 +87,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:44: AssertionError
_____________ TestSpecialisedExplanations.test_eq_similar_text _____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0008>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_similar_text(self):
> assert "foo 1 bar" == "foo 2 bar"
@@ -99,7 +100,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:47: AssertionError
____________ TestSpecialisedExplanations.test_eq_multiline_text ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0009>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_multiline_text(self):
> assert "foo\nspam\nbar" == "foo\neggs\nbar"
@@ -112,7 +113,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:50: AssertionError
______________ TestSpecialisedExplanations.test_eq_long_text _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef000a>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_long_text(self):
a = "1" * 100 + "a" + "2" * 100
@@ -129,7 +130,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:55: AssertionError
_________ TestSpecialisedExplanations.test_eq_long_text_multiline __________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef000b>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_long_text_multiline(self):
a = "1\n" * 100 + "a" + "2\n" * 100
@@ -144,23 +145,23 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E 1
E 1...
E
E ...Full output truncated (6 lines hidden), use '-vv' to show
E ...Full output truncated (7 lines hidden), use '-vv' to show
failure_demo.py:60: AssertionError
_________________ TestSpecialisedExplanations.test_eq_list _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef000c>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_list(self):
> assert [0, 1, 2] == [0, 1, 3]
E assert [0, 1, 2] == [0, 1, 3]
E At index 2 diff: 2 != 3
E Use -v to get more diff
E Use -v to get the full diff
failure_demo.py:63: AssertionError
______________ TestSpecialisedExplanations.test_eq_list_long _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef000d>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_list_long(self):
a = [0] * 100 + [1] + [3] * 100
@@ -168,12 +169,12 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert a == b
E assert [0, 0, 0, 0, 0, 0, ...] == [0, 0, 0, 0, 0, 0, ...]
E At index 100 diff: 1 != 2
E Use -v to get more diff
E Use -v to get the full diff
failure_demo.py:68: AssertionError
_________________ TestSpecialisedExplanations.test_eq_dict _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef000e>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_dict(self):
> assert {"a": 0, "b": 1, "c": 0} == {"a": 0, "b": 2, "d": 0}
@@ -184,41 +185,43 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E Left contains 1 more item:
E {'c': 0}
E Right contains 1 more item:
E {'d': 0}
E Use -v to get more diff
E {'d': 0}...
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:71: AssertionError
_________________ TestSpecialisedExplanations.test_eq_set __________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef000f>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_set(self):
> assert {0, 10, 11, 12} == {0, 20, 21}
E assert {0, 10, 11, 12} == {0, 20, 21}
E AssertionError: assert {0, 10, 11, 12} == {0, 20, 21}
E Extra items in the left set:
E 10
E 11
E 12
E Extra items in the right set:
E 20
E 21
E Use -v to get more diff
E 21...
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:74: AssertionError
_____________ TestSpecialisedExplanations.test_eq_longer_list ______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0010>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_longer_list(self):
> assert [1, 2] == [1, 2, 3]
E assert [1, 2] == [1, 2, 3]
E Right contains one more item: 3
E Use -v to get more diff
E Use -v to get the full diff
failure_demo.py:77: AssertionError
_________________ TestSpecialisedExplanations.test_in_list _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0011>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_in_list(self):
> assert 1 in [0, 2, 3, 4, 5]
@@ -227,7 +230,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:80: AssertionError
__________ TestSpecialisedExplanations.test_not_in_text_multiline __________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0012>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_not_in_text_multiline(self):
text = "some multiline\ntext\nwhich\nincludes foo\nand a\ntail"
@@ -239,13 +242,14 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E which
E includes foo
E ? +++
E and a
E tail
E and a...
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:84: AssertionError
___________ TestSpecialisedExplanations.test_not_in_text_single ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0013>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_not_in_text_single(self):
text = "single foo line"
@@ -258,7 +262,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:88: AssertionError
_________ TestSpecialisedExplanations.test_not_in_text_single_long _________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0014>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_not_in_text_single_long(self):
text = "head " * 50 + "foo " + "tail " * 20
@@ -271,7 +275,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:92: AssertionError
______ TestSpecialisedExplanations.test_not_in_text_single_long_term _______
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0015>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_not_in_text_single_long_term(self):
text = "head " * 50 + "f" * 70 + "tail " * 20
@@ -284,7 +288,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:96: AssertionError
______________ TestSpecialisedExplanations.test_eq_dataclass _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0016>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_dataclass(self):
from dataclasses import dataclass
@@ -304,14 +308,14 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E ['b']
E
E Drill down into differing attribute b:
E b: 'b' != 'c'
E - c
E + b
E b: 'b' != 'c'...
E
E ...Full output truncated (3 lines hidden), use '-vv' to show
failure_demo.py:108: AssertionError
________________ TestSpecialisedExplanations.test_eq_attrs _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef0017>
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_attrs(self):
import attr
@@ -331,9 +335,9 @@ Here is a nice run of several failures and how ``pytest`` presents things:
E ['b']
E
E Drill down into differing attribute b:
E b: 'b' != 'c'
E - c
E + b
E b: 'b' != 'c'...
E
E ...Full output truncated (3 lines hidden), use '-vv' to show
failure_demo.py:120: AssertionError
______________________________ test_attribute ______________________________
@@ -345,7 +349,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
i = Foo()
> assert i.b == 2
E assert 1 == 2
E + where 1 = <failure_demo.test_attribute.<locals>.Foo object at 0xdeadbeef0018>.b
E + where 1 = <failure_demo.test_attribute.<locals>.Foo object at 0xdeadbeef>.b
failure_demo.py:128: AssertionError
_________________________ test_attribute_instance __________________________
@@ -356,8 +360,8 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert Foo().b == 2
E AssertionError: assert 1 == 2
E + where 1 = <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef0019>.b
E + where <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef0019> = <class 'failure_demo.test_attribute_instance.<locals>.Foo'>()
E + where 1 = <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef>.b
E + where <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef> = <class 'failure_demo.test_attribute_instance.<locals>.Foo'>()
failure_demo.py:135: AssertionError
__________________________ test_attribute_failure __________________________
@@ -375,7 +379,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:146:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <failure_demo.test_attribute_failure.<locals>.Foo object at 0xdeadbeef001a>
self = <failure_demo.test_attribute_failure.<locals>.Foo object at 0xdeadbeef>
def _get_b(self):
> raise Exception("Failed to get attrib")
@@ -393,15 +397,15 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert Foo().b == Bar().b
E AssertionError: assert 1 == 2
E + where 1 = <failure_demo.test_attribute_multiple.<locals>.Foo object at 0xdeadbeef001b>.b
E + where <failure_demo.test_attribute_multiple.<locals>.Foo object at 0xdeadbeef001b> = <class 'failure_demo.test_attribute_multiple.<locals>.Foo'>()
E + and 2 = <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef001c>.b
E + where <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef001c> = <class 'failure_demo.test_attribute_multiple.<locals>.Bar'>()
E + where 1 = <failure_demo.test_attribute_multiple.<locals>.Foo object at 0xdeadbeef>.b
E + where <failure_demo.test_attribute_multiple.<locals>.Foo object at 0xdeadbeef> = <class 'failure_demo.test_attribute_multiple.<locals>.Foo'>()
E + and 2 = <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef>.b
E + where <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef> = <class 'failure_demo.test_attribute_multiple.<locals>.Bar'>()
failure_demo.py:156: AssertionError
__________________________ TestRaises.test_raises __________________________
self = <failure_demo.TestRaises object at 0xdeadbeef001d>
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_raises(self):
s = "qwe"
@@ -411,7 +415,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:166: ValueError
______________________ TestRaises.test_raises_doesnt _______________________
self = <failure_demo.TestRaises object at 0xdeadbeef001e>
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_raises_doesnt(self):
> raises(OSError, int, "3")
@@ -420,7 +424,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:169: Failed
__________________________ TestRaises.test_raise ___________________________
self = <failure_demo.TestRaises object at 0xdeadbeef001f>
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_raise(self):
> raise ValueError("demo error")
@@ -429,7 +433,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:172: ValueError
________________________ TestRaises.test_tupleerror ________________________
self = <failure_demo.TestRaises object at 0xdeadbeef0020>
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_tupleerror(self):
> a, b = [1] # NOQA
@@ -438,7 +442,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:175: ValueError
______ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ______
self = <failure_demo.TestRaises object at 0xdeadbeef0021>
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_reinterpret_fails_with_print_for_the_fun_of_it(self):
items = [1, 2, 3]
@@ -451,7 +455,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
items is [1, 2, 3]
________________________ TestRaises.test_some_error ________________________
self = <failure_demo.TestRaises object at 0xdeadbeef0022>
self = <failure_demo.TestRaises object at 0xdeadbeef>
def test_some_error(self):
> if namenotexi: # NOQA
@@ -482,7 +486,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
abc-123:2: AssertionError
____________________ TestMoreErrors.test_complex_error _____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef0023>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_complex_error(self):
def f():
@@ -508,7 +512,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:6: AssertionError
___________________ TestMoreErrors.test_z1_unpack_error ____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef0024>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_z1_unpack_error(self):
items = []
@@ -518,7 +522,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:217: ValueError
____________________ TestMoreErrors.test_z2_type_error _____________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef0025>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_z2_type_error(self):
items = 3
@@ -528,20 +532,20 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:221: TypeError
______________________ TestMoreErrors.test_startswith ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef0026>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_startswith(self):
s = "123"
g = "456"
> assert s.startswith(g)
E AssertionError: assert False
E + where False = <built-in method startswith of str object at 0xdeadbeef0027>('456')
E + where <built-in method startswith of str object at 0xdeadbeef0027> = '123'.startswith
E + where False = <built-in method startswith of str object at 0xdeadbeef>('456')
E + where <built-in method startswith of str object at 0xdeadbeef> = '123'.startswith
failure_demo.py:226: AssertionError
__________________ TestMoreErrors.test_startswith_nested ___________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef0028>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_startswith_nested(self):
def f():
@@ -552,15 +556,15 @@ Here is a nice run of several failures and how ``pytest`` presents things:
> assert f().startswith(g())
E AssertionError: assert False
E + where False = <built-in method startswith of str object at 0xdeadbeef0027>('456')
E + where <built-in method startswith of str object at 0xdeadbeef0027> = '123'.startswith
E + where '123' = <function TestMoreErrors.test_startswith_nested.<locals>.f at 0xdeadbeef0029>()
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef002a>()
E + where False = <built-in method startswith of str object at 0xdeadbeef>('456')
E + where <built-in method startswith of str object at 0xdeadbeef> = '123'.startswith
E + where '123' = <function TestMoreErrors.test_startswith_nested.<locals>.f at 0xdeadbeef>()
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef>()
failure_demo.py:235: AssertionError
_____________________ TestMoreErrors.test_global_func ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002b>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_global_func(self):
> assert isinstance(globf(42), float)
@@ -571,18 +575,18 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:238: AssertionError
_______________________ TestMoreErrors.test_instance _______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002c>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_instance(self):
self.x = 6 * 7
> assert self.x != 42
E assert 42 != 42
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef002c>.x
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef>.x
failure_demo.py:242: AssertionError
_______________________ TestMoreErrors.test_compare ________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002d>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_compare(self):
> assert globf(10) < 5
@@ -592,7 +596,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:245: AssertionError
_____________________ TestMoreErrors.test_try_finally ______________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef002e>
self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
def test_try_finally(self):
x = 1
@@ -603,7 +607,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:250: AssertionError
___________________ TestCustomAssertMsg.test_single_line ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef002f>
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
def test_single_line(self):
class A:
@@ -618,7 +622,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:261: AssertionError
____________________ TestCustomAssertMsg.test_multiline ____________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef0030>
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
def test_multiline(self):
class A:
@@ -637,7 +641,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
failure_demo.py:268: AssertionError
___________________ TestCustomAssertMsg.test_custom_repr ___________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef0031>
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
def test_custom_repr(self):
class JSON:
@@ -670,7 +674,7 @@ Here is a nice run of several failures and how ``pytest`` presents things:
FAILED failure_demo.py::TestSpecialisedExplanations::test_eq_list - asser...
FAILED failure_demo.py::TestSpecialisedExplanations::test_eq_list_long - ...
FAILED failure_demo.py::TestSpecialisedExplanations::test_eq_dict - Asser...
FAILED failure_demo.py::TestSpecialisedExplanations::test_eq_set - assert...
FAILED failure_demo.py::TestSpecialisedExplanations::test_eq_set - Assert...
FAILED failure_demo.py::TestSpecialisedExplanations::test_eq_longer_list
FAILED failure_demo.py::TestSpecialisedExplanations::test_in_list - asser...
FAILED failure_demo.py::TestSpecialisedExplanations::test_not_in_text_multiline

View File

@@ -69,7 +69,7 @@ Here is a basic pattern to achieve this:
For this to work we need to add a command line option and
provide the ``cmdopt`` through a :ref:`fixture function <fixture>`:
provide the ``cmdopt`` through a :ref:`fixture function <fixture function>`:
.. code-block:: python
@@ -139,67 +139,10 @@ And now with supplying a command line option:
FAILED test_sample.py::test_answer - assert 0
1 failed in 0.12s
You can see that the command line option arrived in our test.
We could add simple validation for the input by listing the choices:
.. code-block:: python
# content of conftest.py
import pytest
def pytest_addoption(parser):
parser.addoption(
"--cmdopt",
action="store",
default="type1",
help="my option: type1 or type2",
choices=("type1", "type2"),
)
Now we'll get feedback on a bad argument:
.. code-block:: pytest
$ pytest -q --cmdopt=type3
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: argument --cmdopt: invalid choice: 'type3' (choose from 'type1', 'type2')
If you need to provide more detailed error messages, you can use the
``type`` parameter and raise ``pytest.UsageError``:
.. code-block:: python
# content of conftest.py
import pytest
def type_checker(value):
msg = "cmdopt must specify a numeric type as typeNNN"
if not value.startswith("type"):
raise pytest.UsageError(msg)
try:
int(value[4:])
except ValueError:
raise pytest.UsageError(msg)
return value
def pytest_addoption(parser):
parser.addoption(
"--cmdopt",
action="store",
default="type1",
help="my option: type1 or type2",
type=type_checker,
)
This completes the basic pattern. However, one often rather wants to
process command line options outside of the test and rather pass in
different or more complex objects.
You can see that the command line option arrived in our test. This
completes the basic pattern. However, one often rather wants to process
command line options outside of the test and rather pass in different or
more complex objects.
Dynamically adding command line options
--------------------------------------------------------------
@@ -223,7 +166,7 @@ the command line arguments before they get processed:
num = max(multiprocessing.cpu_count() / 2, 1)
args[:] = ["-n", str(num)] + args
If you have the :pypi:`xdist plugin <pytest-xdist>` installed
If you have the `xdist plugin <https://pypi.org/project/pytest-xdist/>`_ installed
you will now always perform test runs using a number
of subprocesses close to your CPU. Running in an empty
directory with the above conftest.py:
@@ -232,8 +175,9 @@ directory with the above conftest.py:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 0 items
========================== no tests ran in 0.12s ===========================
@@ -296,8 +240,9 @@ and when running it will see a skipped "slow" test:
$ pytest -rs # "-rs" means report details on the little 's'
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
test_module.py .s [100%]
@@ -312,8 +257,9 @@ Or run it including the ``slow`` marked test:
$ pytest --runslow
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
test_module.py .. [100%]
@@ -342,7 +288,7 @@ Example:
def checkconfig(x):
__tracebackhide__ = True
if not hasattr(x, "config"):
pytest.fail(f"not configured: {x}")
pytest.fail("not configured: {}".format(x))
def test_something():
@@ -376,7 +322,6 @@ this to make sure unexpected exception types aren't hidden:
.. code-block:: python
import operator
import pytest
@@ -387,7 +332,7 @@ this to make sure unexpected exception types aren't hidden:
def checkconfig(x):
__tracebackhide__ = operator.methodcaller("errisinstance", ConfigException)
if not hasattr(x, "config"):
raise ConfigException(f"not configured: {x}")
raise ConfigException("not configured: {}".format(x))
def test_something():
@@ -456,9 +401,10 @@ which will add the string to the test header accordingly:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
project deps: mylib-1.1
rootdir: /home/sweet/project
rootdir: $REGENDOC_TMPDIR
collected 0 items
========================== no tests ran in 0.12s ===========================
@@ -484,11 +430,11 @@ which will add info only when run with "--v":
$ pytest -v
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y -- $PYTHON_PREFIX/bin/python
cachedir: .pytest_cache
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
cachedir: $PYTHON_PREFIX/.pytest_cache
info1: did you know that ...
did you?
rootdir: /home/sweet/project
rootdir: $REGENDOC_TMPDIR
collecting ... collected 0 items
========================== no tests ran in 0.12s ===========================
@@ -499,8 +445,9 @@ and nothing when run plainly:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 0 items
========================== no tests ran in 0.12s ===========================
@@ -538,8 +485,9 @@ Now we can profile which test functions execute the slowest:
$ pytest --durations=3
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
test_some_are_slow.py ... [100%]
@@ -566,7 +514,6 @@ an ``incremental`` marker which is to be used on classes:
# content of conftest.py
from typing import Dict, Tuple
import pytest
# store history of failures per test class name and per index in parametrize (if parametrize used)
@@ -610,7 +557,7 @@ an ``incremental`` marker which is to be used on classes:
test_name = _test_failed_incremental[cls_name].get(parametrize_index, None)
# if name found, test has failed for the combination of class name & test name
if test_name is not None:
pytest.xfail(f"previous test failed ({test_name})")
pytest.xfail("previous test failed ({})".format(test_name))
These two hook implementations work together to abort incremental-marked
@@ -644,8 +591,9 @@ If we run this:
$ pytest -rx
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 4 items
test_step.py .Fx. [100%]
@@ -653,7 +601,7 @@ If we run this:
================================= FAILURES =================================
____________________ TestUserHandling.test_modification ____________________
self = <test_step.TestUserHandling object at 0xdeadbeef0001>
self = <test_step.TestUserHandling object at 0xdeadbeef>
def test_modification(self):
> assert 0
@@ -661,7 +609,8 @@ If we run this:
test_step.py:11: AssertionError
========================= short test summary info ==========================
XFAIL test_step.py::TestUserHandling::test_deletion - reason: previous test failed (test_modification)
XFAIL test_step.py::TestUserHandling::test_deletion
reason: previous test failed (test_modification)
================== 1 failed, 2 passed, 1 xfailed in 0.12s ==================
We'll see that ``test_deletion`` was not executed because ``test_modification``
@@ -672,7 +621,7 @@ Package/Directory-level fixtures (setups)
-------------------------------------------------------
If you have nested test directories, you can have per-directory fixture scopes
by placing fixture functions in a ``conftest.py`` file in that directory.
by placing fixture functions in a ``conftest.py`` file in that directory
You can use all types of fixtures including :ref:`autouse fixtures
<autouse fixtures>` which are the equivalent of xUnit's setup/teardown
concept. It's however recommended to have explicit fixture references in your
@@ -726,8 +675,9 @@ We can run this:
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 7 items
test_step.py .Fx. [ 57%]
@@ -737,17 +687,17 @@ We can run this:
================================== ERRORS ==================================
_______________________ ERROR at setup of test_root ________________________
file /home/sweet/project/b/test_error.py, line 1
file $REGENDOC_TMPDIR/b/test_error.py, line 1
def test_root(db): # no db here, will error out
E fixture 'db' not found
> available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
> use 'pytest --fixtures [testpath]' for help on them.
/home/sweet/project/b/test_error.py:1
$REGENDOC_TMPDIR/b/test_error.py:1
================================= FAILURES =================================
____________________ TestUserHandling.test_modification ____________________
self = <test_step.TestUserHandling object at 0xdeadbeef0002>
self = <test_step.TestUserHandling object at 0xdeadbeef>
def test_modification(self):
> assert 0
@@ -756,21 +706,21 @@ We can run this:
test_step.py:11: AssertionError
_________________________________ test_a1 __________________________________
db = <conftest.DB object at 0xdeadbeef0003>
db = <conftest.DB object at 0xdeadbeef>
def test_a1(db):
> assert 0, db # to show value
E AssertionError: <conftest.DB object at 0xdeadbeef0003>
E AssertionError: <conftest.DB object at 0xdeadbeef>
E assert 0
a/test_db.py:2: AssertionError
_________________________________ test_a2 __________________________________
db = <conftest.DB object at 0xdeadbeef0003>
db = <conftest.DB object at 0xdeadbeef>
def test_a2(db):
> assert 0, db # to show value
E AssertionError: <conftest.DB object at 0xdeadbeef0003>
E AssertionError: <conftest.DB object at 0xdeadbeef>
E assert 0
a/test_db2.py:2: AssertionError
@@ -803,9 +753,8 @@ case we just write some information out to a ``failures`` file:
# content of conftest.py
import os.path
import pytest
import os.path
@pytest.hookimpl(tryfirst=True, hookwrapper=True)
@@ -819,8 +768,8 @@ case we just write some information out to a ``failures`` file:
mode = "a" if os.path.exists("failures") else "w"
with open("failures", mode) as f:
# let's also access a fixture for the fun of it
if "tmp_path" in item.fixturenames:
extra = " ({})".format(item.funcargs["tmp_path"])
if "tmpdir" in item.fixturenames:
extra = " ({})".format(item.funcargs["tmpdir"])
else:
extra = ""
@@ -832,7 +781,7 @@ if you then have failing tests:
.. code-block:: python
# content of test_module.py
def test_fail1(tmp_path):
def test_fail1(tmpdir):
assert 0
@@ -845,8 +794,9 @@ and run them:
$ pytest test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
test_module.py FF [100%]
@@ -854,9 +804,9 @@ and run them:
================================= FAILURES =================================
________________________________ test_fail1 ________________________________
tmp_path = PosixPath('PYTEST_TMPDIR/test_fail10')
tmpdir = local('PYTEST_TMPDIR/test_fail10')
def test_fail1(tmp_path):
def test_fail1(tmpdir):
> assert 0
E assert 0
@@ -892,11 +842,8 @@ here is a little example implemented via a local plugin:
.. code-block:: python
# content of conftest.py
from typing import Dict
import pytest
from pytest import StashKey, CollectReport
phase_report_key = StashKey[Dict[str, CollectReport]]()
import pytest
@pytest.hookimpl(tryfirst=True, hookwrapper=True)
@@ -905,9 +852,10 @@ here is a little example implemented via a local plugin:
outcome = yield
rep = outcome.get_result()
# store test results for each phase of a call, which can
# set a report attribute for each phase of a call, which can
# be "setup", "call", "teardown"
item.stash.setdefault(phase_report_key, {})[rep.when] = rep
setattr(item, "rep_" + rep.when, rep)
@pytest.fixture
@@ -915,11 +863,11 @@ here is a little example implemented via a local plugin:
yield
# request.node is an "item" because we use the default
# "function" scope
report = request.node.stash[phase_report_key]
if report["setup"].failed:
print("setting up a test failed or skipped", request.node.nodeid)
elif ("call" not in report) or report["call"].failed:
print("executing test failed or skipped", request.node.nodeid)
if request.node.rep_setup.failed:
print("setting up a test failed!", request.node.nodeid)
elif request.node.rep_setup.passed:
if request.node.rep_call.failed:
print("executing test failed", request.node.nodeid)
if you then have failing tests:
@@ -953,12 +901,13 @@ and run it:
$ pytest -s test_module.py
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 3 items
test_module.py Esetting up a test failed or skipped test_module.py::test_setup_fails
Fexecuting test failed or skipped test_module.py::test_call_fails
test_module.py Esetting up a test failed! test_module.py::test_setup_fails
Fexecuting test failed test_module.py::test_call_fails
F
================================== ERRORS ==================================
@@ -1008,7 +957,8 @@ which test got stuck, for example if pytest was run in quiet mode (``-q``) or yo
output. This is particularly a problem if the problem happens only sporadically, the famous "flaky" kind of tests.
``pytest`` sets the :envvar:`PYTEST_CURRENT_TEST` environment variable when running tests, which can be inspected
by process monitoring utilities or libraries like :pypi:`psutil` to discover which test got stuck if necessary:
by process monitoring utilities or libraries like `psutil <https://pypi.org/project/psutil/>`_ to discover which
test got stuck if necessary:
.. code-block:: python
@@ -1062,7 +1012,7 @@ your frozen program work as the pytest runner by some clever
argument handling during program startup. This allows you to
have a single executable, which is usually more convenient.
Please note that the mechanism for plugin discovery used by pytest
(setuptools entry points) doesn't work with frozen executables so pytest
(setupttools entry points) doesn't work with frozen executables so pytest
can't find any third party plugins automatically. To include third party plugins
like ``pytest-timeout`` they must be imported explicitly and passed on to pytest.main.
@@ -1070,7 +1020,6 @@ like ``pytest-timeout`` they must be imported explicitly and passed on to pytest
# contents of app_main.py
import sys
import pytest_timeout # Third party plugin
if len(sys.argv) > 1 and sys.argv[1] == "--pytest":

View File

@@ -43,7 +43,7 @@ will be called ahead of running any tests:
print("test_method1 called")
def test_method2(self):
print("test_method2 called")
print("test_method1 called")
class TestOther:
@@ -77,7 +77,7 @@ If you run this without output capturing:
callme other called
SomeTest callme called
test_method1 called
.test_method2 called
.test_method1 called
.test other
.test_unit1 method called
.

View File

@@ -1,7 +1,7 @@
.. _existingtestsuite:
How to use pytest with an existing test suite
==============================================
Using pytest with an existing test suite
===========================================
Pytest can be used with most existing test suites, but its
behavior differs from other test runners such as :ref:`nose <noseintegration>` or

View File

@@ -1,46 +0,0 @@
.. _test-anatomy:
Anatomy of a test
=================
In the simplest terms, a test is meant to look at the result of a particular
behavior, and make sure that result aligns with what you would expect.
Behavior is not something that can be empirically measured, which is why writing
tests can be challenging.
"Behavior" is the way in which some system **acts in response** to a particular
situation and/or stimuli. But exactly *how* or *why* something is done is not
quite as important as *what* was done.
You can think of a test as being broken down into four steps:
1. **Arrange**
2. **Act**
3. **Assert**
4. **Cleanup**
**Arrange** is where we prepare everything for our test. This means pretty
much everything except for the "**act**". It's lining up the dominoes so that
the **act** can do its thing in one, state-changing step. This can mean
preparing objects, starting/killing services, entering records into a database,
or even things like defining a URL to query, generating some credentials for a
user that doesn't exist yet, or just waiting for some process to finish.
**Act** is the singular, state-changing action that kicks off the **behavior**
we want to test. This behavior is what carries out the changing of the state of
the system under test (SUT), and it's the resulting changed state that we can
look at to make a judgement about the behavior. This typically takes the form of
a function/method call.
**Assert** is where we look at that resulting state and check if it looks how
we'd expect after the dust has settled. It's where we gather evidence to say the
behavior does or does not aligns with what we expect. The ``assert`` in our test
is where we take that measurement/observation and apply our judgement to it. If
something should be green, we'd say ``assert thing == "green"``.
**Cleanup** is where the test picks up after itself, so other tests aren't being
accidentally influenced by it.
At its core, the test is ultimately the **act** and **assert** steps, with the
**arrange** step only providing the context. **Behavior** exists between **act**
and **assert**.

View File

@@ -1,174 +0,0 @@
.. _about-fixtures:
About fixtures
===============
.. seealso:: :ref:`how-to-fixtures`
.. seealso:: :ref:`Fixtures reference <reference-fixtures>`
pytest fixtures are designed to be explicit, modular and scalable.
What fixtures are
-----------------
In testing, a `fixture <https://en.wikipedia.org/wiki/Test_fixture#Software>`_
provides a defined, reliable and consistent context for the tests. This could
include environment (for example a database configured with known parameters)
or content (such as a dataset).
Fixtures define the steps and data that constitute the *arrange* phase of a
test (see :ref:`test-anatomy`). In pytest, they are functions you define that
serve this purpose. They can also be used to define a test's *act* phase; this
is a powerful technique for designing more complex tests.
The services, state, or other operating environments set up by fixtures are
accessed by test functions through arguments. For each fixture used by a test
function there is typically a parameter (named after the fixture) in the test
function's definition.
We can tell pytest that a particular function is a fixture by decorating it with
:py:func:`@pytest.fixture <pytest.fixture>`. Here's a simple example of
what a fixture in pytest might look like:
.. code-block:: python
import pytest
class Fruit:
def __init__(self, name):
self.name = name
def __eq__(self, other):
return self.name == other.name
@pytest.fixture
def my_fruit():
return Fruit("apple")
@pytest.fixture
def fruit_basket(my_fruit):
return [Fruit("banana"), my_fruit]
def test_my_fruit_in_basket(my_fruit, fruit_basket):
assert my_fruit in fruit_basket
Tests don't have to be limited to a single fixture, either. They can depend on
as many fixtures as you want, and fixtures can use other fixtures, as well. This
is where pytest's fixture system really shines.
Improvements over xUnit-style setup/teardown functions
-----------------------------------------------------------
pytest fixtures offer dramatic improvements over the classic xUnit
style of setup/teardown functions:
* fixtures have explicit names and are activated by declaring their use
from test functions, modules, classes or whole projects.
* fixtures are implemented in a modular manner, as each fixture name
triggers a *fixture function* which can itself use other fixtures.
* fixture management scales from simple unit to complex
functional testing, allowing to parametrize fixtures and tests according
to configuration and component options, or to re-use fixtures
across function, class, module or whole test session scopes.
* teardown logic can be easily, and safely managed, no matter how many fixtures
are used, without the need to carefully handle errors by hand or micromanage
the order that cleanup steps are added.
In addition, pytest continues to support :ref:`xunitsetup`. You can mix
both styles, moving incrementally from classic to new style, as you
prefer. You can also start out from existing :ref:`unittest.TestCase
style <unittest.TestCase>` or :ref:`nose based <nosestyle>` projects.
Fixture errors
--------------
pytest does its best to put all the fixtures for a given test in a linear order
so that it can see which fixture happens first, second, third, and so on. If an
earlier fixture has a problem, though, and raises an exception, pytest will stop
executing fixtures for that test and mark the test as having an error.
When a test is marked as having an error, it doesn't mean the test failed,
though. It just means the test couldn't even be attempted because one of the
things it depends on had a problem.
This is one reason why it's a good idea to cut out as many unnecessary
dependencies as possible for a given test. That way a problem in something
unrelated isn't causing us to have an incomplete picture of what may or may not
have issues.
Here's a quick example to help explain:
.. code-block:: python
import pytest
@pytest.fixture
def order():
return []
@pytest.fixture
def append_first(order):
order.append(1)
@pytest.fixture
def append_second(order, append_first):
order.extend([2])
@pytest.fixture(autouse=True)
def append_third(order, append_second):
order += [3]
def test_order(order):
assert order == [1, 2, 3]
If, for whatever reason, ``order.append(1)`` had a bug and it raises an exception,
we wouldn't be able to know if ``order.extend([2])`` or ``order += [3]`` would
also have problems. After ``append_first`` throws an exception, pytest won't run
any more fixtures for ``test_order``, and it won't even try to run
``test_order`` itself. The only things that would've run would be ``order`` and
``append_first``.
Sharing test data
-----------------
If you want to make test data from files available to your tests, a good way
to do this is by loading these data in a fixture for use by your tests.
This makes use of the automatic caching mechanisms of pytest.
Another good approach is by adding the data files in the ``tests`` folder.
There are also community plugins available to help to manage this aspect of
testing, e.g. :pypi:`pytest-datadir` and :pypi:`pytest-datafiles`.
.. _fixtures-signal-cleanup:
A note about fixture cleanup
----------------------------
pytest does not do any special processing for :data:`SIGTERM <signal.SIGTERM>` and
:data:`SIGQUIT <signal.SIGQUIT>` signals (:data:`SIGINT <signal.SIGINT>` is handled naturally
by the Python runtime via :class:`KeyboardInterrupt`), so fixtures that manage external resources which are important
to be cleared when the Python process is terminated (by those signals) might leak resources.
The reason pytest does not handle those signals to perform fixture cleanup is that signal handlers are global,
and changing them might interfere with the code under execution.
If fixtures in your suite need special care regarding termination in those scenarios,
see :issue:`this comment <5243#issuecomment-491522595>` in the issue
tracker for a possible workaround.

View File

@@ -1,15 +0,0 @@
:orphan:
.. _explanation:
Explanation
================
.. toctree::
:maxdepth: 1
anatomy
fixtures
goodpractices
flaky
pythonpath

File diff suppressed because it is too large Load Diff

View File

@@ -28,7 +28,7 @@ Flaky tests sometimes appear when a test suite is run in parallel (such as use o
Overly strict assertion
~~~~~~~~~~~~~~~~~~~~~~~
Overly strict assertions can cause problems with floating point comparison as well as timing issues. :func:`pytest.approx` is useful here.
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/stable/reference.html#pytest-approx>`_ is useful here.
Pytest features
@@ -94,7 +94,7 @@ Mark Lapierre discusses the `Pros and Cons of Quarantined Tests <https://dev.to/
CI tools that rerun on failure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/previous-versions/azure/devops/2017/dec-11-vsts?view=tfs-2017#identify-flaky-tests>`_ and rerun failed tests.
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/azure/devops/release-notes/2017/dec-11-vsts#identify-flaky-tests>`_ and rerun failed tests.
@@ -113,7 +113,7 @@ Resources
* `Eradicating Non-Determinism in Tests <https://martinfowler.com/articles/nonDeterminism.html>`_ by Martin Fowler, 2011
* `No more flaky tests on the Go team <https://www.thoughtworks.com/insights/blog/no-more-flaky-tests-go-team>`_ by Pavan Sudarshan, 2012
* `The Build That Cried Broken: Building Trust in your Continuous Integration Tests <https://www.youtube.com/embed/VotJqV4n8ig>`_ talk (video) by `Angie Jones <https://angiejones.tech/>`_ at SeleniumConf Austin 2017
* `The Build That Cried Broken: Building Trust in your Continuous Integration Tests <https://www.youtube.com/embed/VotJqV4n8ig>`_ talk (video) by `Angie Jones <http://angiejones.tech/>`_ at SeleniumConf Austin 2017
* `Test and Code Podcast: Flaky Tests and How to Deal with Them <https://testandcode.com/50>`_ by Brian Okken and Anthony Shaw, 2018
* Microsoft:

View File

@@ -7,7 +7,7 @@ pytest-2.3: reasoning for fixture/funcarg evolution
**Target audience**: Reading this document requires basic knowledge of
python testing, xUnit setup methods and the (previous) basic pytest
funcarg mechanism, see :ref:`historical funcargs and pytest.funcargs`.
funcarg mechanism, see https://docs.pytest.org/en/stable/historical-notes.html#funcargs-and-pytest-funcarg.
If you are new to pytest, then you can simply ignore this
section and read the other sections.
@@ -47,7 +47,7 @@ There are several limitations and difficulties with this approach:
2. parametrizing the "db" resource is not straight forward:
you need to apply a "parametrize" decorator or implement a
:py:func:`~hookspec.pytest_generate_tests` hook
calling :py:func:`~pytest.Metafunc.parametrize` which
calling :py:func:`~python.Metafunc.parametrize` which
performs parametrization at the places where the resource
is used. Moreover, you need to modify the factory to use an
``extrakey`` parameter containing ``request.param`` to the
@@ -113,7 +113,7 @@ This new way of parametrizing funcarg factories should in many cases
allow to re-use already written factories because effectively
``request.param`` was already used when test functions/classes were
parametrized via
:py:func:`metafunc.parametrize(indirect=True) <pytest.Metafunc.parametrize>` calls.
:py:func:`metafunc.parametrize(indirect=True) <_pytest.python.Metafunc.parametrize>` calls.
Of course it's perfectly fine to combine parametrization and scoping:
@@ -168,7 +168,7 @@ pytest for a long time offered a pytest_configure and a pytest_sessionstart
hook which are often used to setup global resources. This suffers from
several problems:
1. in distributed testing the managing process would setup test resources
1. in distributed testing the master process would setup test resources
that are never needed because it only co-ordinates the test run
activities of the worker processes.

View File

@@ -1,16 +1,22 @@
.. _get-started:
Get Started
Installation and Getting Started
===================================
**Pythons**: Python 3.6, 3.7, 3.8, 3.9, PyPy3
**Platforms**: Linux and Windows
**PyPI package name**: `pytest <https://pypi.org/project/pytest/>`_
**Documentation as PDF**: `download latest <https://media.readthedocs.org/pdf/pytest/latest/pytest.pdf>`_
``pytest`` is a framework that makes building simple and scalable tests easy. Tests are expressive and readable—no boilerplate code required. Get started in minutes with a small unit test or complex functional test for your application or library.
.. _`getstarted`:
.. _`installation`:
Install ``pytest``
----------------------------------------
``pytest`` requires: Python 3.7+ or PyPy3.
1. Run the following command in your command line:
.. code-block:: bash
@@ -22,14 +28,14 @@ Install ``pytest``
.. code-block:: bash
$ pytest --version
pytest 7.3.0
pytest 6.2.3
.. _`simpletest`:
Create your first test
----------------------------------------------------------
Create a new file called ``test_sample.py``, containing a function, and a test:
Create a simple test function with just four lines of code:
.. code-block:: python
@@ -41,14 +47,15 @@ Create a new file called ``test_sample.py``, containing a function, and a test:
def test_answer():
assert func(3) == 5
The test
Thats it. You can now execute the test function:
.. code-block:: pytest
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_sample.py F [100%]
@@ -70,7 +77,7 @@ The ``[100%]`` refers to the overall progress of running all test cases. After i
.. note::
You can use the ``assert`` statement to verify test expectations. pytests :ref:`Advanced assertion introspection <python:assert>` will intelligently report intermediate values of the assert expression so you can avoid the many names :ref:`of JUnit legacy methods <testcase-objects>`.
You can use the ``assert`` statement to verify test expectations. pytests `Advanced assertion introspection <http://docs.python.org/reference/simple_stmts.html#the-assert-statement>`_ will intelligently report intermediate values of the assert expression so you can avoid the many names `of JUnit legacy methods <http://docs.python.org/library/unittest.html#test-cases>`_.
Run multiple tests
----------------------------------------------------------
@@ -137,7 +144,7 @@ Once you develop multiple tests, you may want to group them into a class. pytest
================================= FAILURES =================================
____________________________ TestClass.test_two ____________________________
self = <test_class.TestClass object at 0xdeadbeef0001>
self = <test_class.TestClass object at 0xdeadbeef>
def test_two(self):
x = "hello"
@@ -168,73 +175,77 @@ This is outlined below:
# content of test_class_demo.py
class TestClassDemoInstance:
value = 0
def test_one(self):
self.value = 1
assert self.value == 1
assert 0
def test_two(self):
assert self.value == 1
assert 0
.. code-block:: pytest
$ pytest -k TestClassDemoInstance -q
.F [100%]
FF [100%]
================================= FAILURES =================================
______________________ TestClassDemoInstance.test_one ______________________
self = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef>
def test_one(self):
> assert 0
E assert 0
test_class_demo.py:3: AssertionError
______________________ TestClassDemoInstance.test_two ______________________
self = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef0002>
self = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef>
def test_two(self):
> assert self.value == 1
E assert 0 == 1
E + where 0 = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef0002>.value
> assert 0
E assert 0
test_class_demo.py:9: AssertionError
test_class_demo.py:6: AssertionError
========================= short test summary info ==========================
FAILED test_class_demo.py::TestClassDemoInstance::test_two - assert 0 == 1
1 failed, 1 passed in 0.12s
Note that attributes added at class level are *class attributes*, so they will be shared between tests.
FAILED test_class_demo.py::TestClassDemoInstance::test_one - assert 0
FAILED test_class_demo.py::TestClassDemoInstance::test_two - assert 0
2 failed in 0.12s
Request a unique temporary directory for functional tests
--------------------------------------------------------------
``pytest`` provides :std:doc:`Builtin fixtures/function arguments <builtin>` to request arbitrary resources, like a unique temporary directory:
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/stable/builtin.html>`_ to request arbitrary resources, like a unique temporary directory:
.. code-block:: python
# content of test_tmp_path.py
def test_needsfiles(tmp_path):
print(tmp_path)
# content of test_tmpdir.py
def test_needsfiles(tmpdir):
print(tmpdir)
assert 0
List the name ``tmp_path`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory:
List the name ``tmpdir`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory:
.. code-block:: pytest
$ pytest -q test_tmp_path.py
$ pytest -q test_tmpdir.py
F [100%]
================================= FAILURES =================================
_____________________________ test_needsfiles ______________________________
tmp_path = PosixPath('PYTEST_TMPDIR/test_needsfiles0')
tmpdir = local('PYTEST_TMPDIR/test_needsfiles0')
def test_needsfiles(tmp_path):
print(tmp_path)
def test_needsfiles(tmpdir):
print(tmpdir)
> assert 0
E assert 0
test_tmp_path.py:3: AssertionError
test_tmpdir.py:3: AssertionError
--------------------------- Captured stdout call ---------------------------
PYTEST_TMPDIR/test_needsfiles0
========================= short test summary info ==========================
FAILED test_tmp_path.py::test_needsfiles - assert 0
FAILED test_tmpdir.py::test_needsfiles - assert 0
1 failed in 0.12s
More info on temporary directory handling is available at :ref:`Temporary directories and files <tmp_path handling>`.
More info on tmpdir handling is available at :ref:`Temporary directories and files <tmpdir handling>`.
Find out what kind of builtin :ref:`pytest fixtures <fixtures>` exist with the command:
@@ -249,7 +260,7 @@ Continue reading
Check out additional pytest resources to help you customize tests for your unique workflow:
* ":ref:`usage`" for command line invocation examples
* ":ref:`cmdline`" for command line invocation examples
* ":ref:`existingtestsuite`" for working with pre-existing tests
* ":ref:`mark`" for information on the ``pytest.mark`` mechanism
* ":ref:`fixtures`" for providing a functional baseline to your tests

View File

@@ -7,34 +7,28 @@ Good Integration Practices
Install package with pip
-------------------------------------------------
For development, we recommend you use :mod:`venv` for virtual environments and
:doc:`pip:index` for installing your application and any dependencies,
For development, we recommend you use venv_ for virtual environments and
pip_ for installing your application and any dependencies,
as well as the ``pytest`` package itself.
This ensures your code and dependencies are isolated from your system Python installation.
Create a ``pyproject.toml`` file in the root of your repository as described in
:doc:`packaging:tutorials/packaging-projects`.
The first few lines should look like this:
Next, place a ``setup.py`` file in the root of your package with the following minimum content:
.. code-block:: toml
.. code-block:: python
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
from setuptools import setup, find_packages
[project]
name = "PACKAGENAME"
version = "PACKAGEVERSION"
setup(name="PACKAGENAME", packages=find_packages())
where ``PACKAGENAME`` and ``PACKAGEVERSION`` are the name and version of your package respectively.
You can then install your package in "editable" mode by running from the same directory:
Where ``PACKAGENAME`` is the name of your package. You can then install your package in "editable" mode by running from the same directory:
.. code-block:: bash
pip install -e .
pip install -e .
which lets you change your source code (both tests and application) and rerun tests at will.
This is similar to running ``python setup.py develop`` or ``conda develop`` in that it installs
your package using a symlink to your development code.
.. _`test discovery`:
.. _`Python test discovery`:
@@ -51,10 +45,10 @@ Conventions for Python test discovery
* In those directories, search for ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_.
* From those files, collect test items:
* ``test`` prefixed test functions or methods outside of class.
* ``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method). Methods decorated with ``@staticmethod`` and ``@classmethods`` are also considered.
* ``test`` prefixed test functions or methods outside of class
* ``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method)
For examples of how to customize your test discovery :doc:`/example/pythoncollection`.
For examples of how to customize your test discovery :doc:`example/pythoncollection`.
Within Python modules, ``pytest`` also discovers tests using the standard
:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.
@@ -74,12 +68,11 @@ to keep tests separate from actual application code (often a good idea):
.. code-block:: text
pyproject.toml
src/
mypkg/
__init__.py
app.py
view.py
setup.py
mypkg/
__init__.py
app.py
view.py
tests/
test_app.py
test_view.py
@@ -89,57 +82,82 @@ This has the following benefits:
* Your tests can run against an installed version after executing ``pip install .``.
* Your tests can run against the local copy with an editable install after executing ``pip install --editable .``.
For new projects, we recommend to use ``importlib`` :ref:`import mode <import-modes>`
(see which-import-mode_ for a detailed explanation).
To this end, add the following to your ``pyproject.toml``:
.. code-block:: toml
[tool.pytest.ini_options]
addopts = [
"--import-mode=importlib",
]
.. _src-layout:
Generally, but especially if you use the default import mode ``prepend``,
it is **strongly** suggested to use a ``src`` layout.
Here, your application root package resides in a sub-directory of your root,
i.e. ``src/mypkg/`` instead of ``mypkg``.
This layout prevents a lot of common pitfalls and has many benefits,
which are better explained in this excellent `blog post`_ by Ionel Cristian Mărieș.
.. _blog post: https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>
* If you don't have a ``setup.py`` file and are relying on the fact that Python by default puts the current
directory in ``sys.path`` to import your package, you can execute ``python -m pytest`` to execute the tests against the
local copy directly, without using ``pip``.
.. note::
If you do not use an editable install and use the ``src`` layout as above you need to extend the Python's
search path for module files to execute the tests against the local copy directly. You can do it in an
ad-hoc manner by setting the ``PYTHONPATH`` environment variable:
.. code-block:: bash
PYTHONPATH=src pytest
or in a permanent manner by using the :confval:`pythonpath` configuration variable and adding the
following to your ``pyproject.toml``:
.. code-block:: toml
[tool.pytest.ini_options]
pythonpath = "src"
.. note::
If you do not use an editable install and not use the ``src`` layout (``mypkg`` directly in the root
directory) you can rely on the fact that Python by default puts the current directory in ``sys.path`` to
import your package and run ``python -m pytest`` to execute the tests against the local copy directly.
See :ref:`pytest vs python -m pytest` for more information about the difference between calling ``pytest`` and
``python -m pytest``.
Note that this scheme has a drawback if you are using ``prepend`` :ref:`import mode <import-modes>`
(which is the default): your test files must have **unique names**, because
``pytest`` will import them as *top-level* modules since there are no packages
to derive a full package name from. In other words, the test files in the example above will
be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to
``sys.path``.
If you need to have test modules with the same name, you might add ``__init__.py`` files to your
``tests`` folder and subfolders, changing them to packages:
.. code-block:: text
setup.py
mypkg/
...
tests/
__init__.py
foo/
__init__.py
test_view.py
bar/
__init__.py
test_view.py
Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing
you to have modules with the same name. But now this introduces a subtle problem: in order to load
the test modules from the ``tests`` directory, pytest prepends the root of the repository to
``sys.path``, which adds the side-effect that now ``mypkg`` is also importable.
This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment,
because you want to test the *installed* version of your package, not the local code from the repository.
.. _`src-layout`:
In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
sub-directory of your root:
.. code-block:: text
setup.py
src/
mypkg/
__init__.py
app.py
view.py
tests/
__init__.py
foo/
__init__.py
test_view.py
bar/
__init__.py
test_view.py
This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent
`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_.
.. note::
The new ``--import-mode=importlib`` (see :ref:`import-modes`) doesn't have
any of the drawbacks above because ``sys.path`` and ``sys.modules`` are not changed when importing
test modules, so users that run
into this issue are strongly encouraged to try it and report if the new option works well for them.
The ``src`` directory layout is still strongly recommended however.
Tests as part of application code
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -149,12 +167,12 @@ want to distribute them along with your application:
.. code-block:: text
pyproject.toml
[src/]mypkg/
setup.py
mypkg/
__init__.py
app.py
view.py
tests/
test/
__init__.py
test_app.py
test_view.py
@@ -173,11 +191,11 @@ Note that this layout also works in conjunction with the ``src`` layout mentione
.. note::
You can use namespace packages (PEP420) for your application
You can use Python3 namespace packages (PEP420) for your application
but pytest will still perform `test package name`_ discovery based on the
presence of ``__init__.py`` files. If you use one of the
two recommended file system layouts above but leave away the ``__init__.py``
files from your directories, it should just work. From
files from your directories it should just work on Python3.3 and above. From
"inlined tests", however, you will need to use absolute imports for
getting at your application code.
@@ -212,85 +230,23 @@ Note that this layout also works in conjunction with the ``src`` layout mentione
much less surprising.
.. _which-import-mode:
Choosing an import mode
^^^^^^^^^^^^^^^^^^^^^^^
For historical reasons, pytest defaults to the ``prepend`` :ref:`import mode <import-modes>`
instead of the ``importlib`` import mode we recommend for new projects.
The reason lies in the way the ``prepend`` mode works:
Since there are no packages to derive a full package name from,
``pytest`` will import your test files as *top-level* modules.
The test files in the first example (:ref:`src layout <src-layout>`) would be imported as
``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to ``sys.path``.
This results in a drawback compared to the import mode ``importlib``:
your test files must have **unique names**.
If you need to have test modules with the same name,
as a workaround you might add ``__init__.py`` files to your ``tests`` folder and subfolders,
changing them to packages:
.. code-block:: text
pyproject.toml
mypkg/
...
tests/
__init__.py
foo/
__init__.py
test_view.py
bar/
__init__.py
test_view.py
Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``,
allowing you to have modules with the same name.
But now this introduces a subtle problem:
in order to load the test modules from the ``tests`` directory,
pytest prepends the root of the repository to ``sys.path``,
which adds the side-effect that now ``mypkg`` is also importable.
This is problematic if you are using a tool like tox_ to test your package in a virtual environment,
because you want to test the *installed* version of your package,
not the local code from the repository.
The ``importlib`` import mode does not have any of the drawbacks above,
because ``sys.path`` is not changed when importing test modules.
.. _`buildout`: http://www.buildout.org/en/latest/
.. _`virtualenv`: https://pypi.org/project/virtualenv/
.. _`buildout`: http://www.buildout.org/
.. _pip: https://pypi.org/project/pip/
.. _`use tox`:
tox
---
------
Once you are done with your work and want to make sure that your actual
package passes all tests you may want to look into :doc:`tox <tox:index>`, the
virtualenv test automation tool.
``tox`` helps you to setup virtualenv environments with pre-defined
package passes all tests you may want to look into `tox`_, the
virtualenv test automation tool and its `pytest support
<https://tox.readthedocs.io/en/latest/example/pytest.html>`_.
tox helps you to setup virtualenv environments with pre-defined
dependencies and then executing a pre-configured test command with
options. It will run tests against the installed package and not
against your source code checkout, helping to detect packaging
glitches.
Do not run via setuptools
-------------------------
Integration with setuptools is **not recommended**,
i.e. you should not be using ``python setup.py test`` or ``pytest-runner``,
and may stop working in the future.
This is deprecated since it depends on deprecated features of setuptools
and relies on features that break security mechanisms in pip.
For example 'setup_requires' and 'tests_require' bypass ``pip --require-hashes``.
For more information and migration instructions,
see the `pytest-runner notice <https://github.com/pytest-dev/pytest-runner#deprecation-notice>`_.
See also `pypa/setuptools#1684 <https://github.com/pypa/setuptools/issues/1684>`_.
setuptools intends to
`remove the test command <https://github.com/pypa/setuptools/issues/931>`_.
.. _`venv`: https://docs.python.org/3/library/venv.html

View File

@@ -76,38 +76,38 @@ order doesn't even matter. You probably want to think of your marks as a set her
If you are unsure or have any questions, please consider opening
:issue:`an issue <new>`.
`an issue <https://github.com/pytest-dev/pytest/issues>`_.
Related issues
~~~~~~~~~~~~~~
Here is a non-exhaustive list of issues fixed by the new implementation:
* Marks don't pick up nested classes (:issue:`199`).
* Marks don't pick up nested classes (`#199 <https://github.com/pytest-dev/pytest/issues/199>`_).
* Markers stain on all related classes (:issue:`568`).
* Markers stain on all related classes (`#568 <https://github.com/pytest-dev/pytest/issues/568>`_).
* Combining marks - args and kwargs calculation (:issue:`2897`).
* Combining marks - args and kwargs calculation (`#2897 <https://github.com/pytest-dev/pytest/issues/2897>`_).
* ``request.node.get_marker('name')`` returns ``None`` for markers applied in classes (:issue:`902`).
* ``request.node.get_marker('name')`` returns ``None`` for markers applied in classes (`#902 <https://github.com/pytest-dev/pytest/issues/902>`_).
* Marks applied in parametrize are stored as markdecorator (:issue:`2400`).
* Marks applied in parametrize are stored as markdecorator (`#2400 <https://github.com/pytest-dev/pytest/issues/2400>`_).
* Fix marker interaction in a backward incompatible way (:issue:`1670`).
* Fix marker interaction in a backward incompatible way (`#1670 <https://github.com/pytest-dev/pytest/issues/1670>`_).
* Refactor marks to get rid of the current "marks transfer" mechanism (:issue:`2363`).
* Refactor marks to get rid of the current "marks transfer" mechanism (`#2363 <https://github.com/pytest-dev/pytest/issues/2363>`_).
* Introduce FunctionDefinition node, use it in generate_tests (:issue:`2522`).
* Introduce FunctionDefinition node, use it in generate_tests (`#2522 <https://github.com/pytest-dev/pytest/issues/2522>`_).
* Remove named marker attributes and collect markers in items (:issue:`891`).
* Remove named marker attributes and collect markers in items (`#891 <https://github.com/pytest-dev/pytest/issues/891>`_).
* skipif mark from parametrize hides module level skipif mark (:issue:`1540`).
* skipif mark from parametrize hides module level skipif mark (`#1540 <https://github.com/pytest-dev/pytest/issues/1540>`_).
* skipif + parametrize not skipping tests (:issue:`1296`).
* skipif + parametrize not skipping tests (`#1296 <https://github.com/pytest-dev/pytest/issues/1296>`_).
* Marker transfer incompatible with inheritance (:issue:`535`).
* Marker transfer incompatible with inheritance (`#535 <https://github.com/pytest-dev/pytest/issues/535>`_).
More details can be found in the :pull:`original PR <3317>`.
More details can be found in the `original PR <https://github.com/pytest-dev/pytest/pull/3317>`_.
.. note::
@@ -125,7 +125,6 @@ as a third party plugin named ``pytest-cache``. The core plugin
is compatible regarding command line options and API usage except that you
can only store/receive data between test runs that is json-serializable.
.. _historical funcargs and pytest.funcargs:
funcargs and ``pytest_funcarg__``
---------------------------------

View File

@@ -1,145 +0,0 @@
History
=======
pytest has a long and interesting history. The `first commit
<https://github.com/pytest-dev/pytest/commit/5992a8ef21424d7571305a8d7e2a3431ee7e1e23>`__
in this repository is from January 2007, and even that commit alone already
tells a lot: The repository originally was from the :pypi:`py`
library (later split off to pytest), and it
originally was a SVN revision, migrated to Mercurial, and finally migrated to
git.
However, the commit says “create the new development trunk” and is
already quite big: *435 files changed, 58640 insertions(+)*. This is because
pytest originally was born as part of `PyPy <https://www.pypy.org/>`__, to make
it easier to write tests for it. Here's how it evolved from there to its own
project:
- Late 2002 / early 2003, `PyPy was
born <https://morepypy.blogspot.com/2018/09/the-first-15-years-of-pypy.html>`__.
- Like that blog post mentioned, from very early on, there was a big
focus on testing. There were various ``testsupport`` files on top of
unittest.py, and as early as June 2003, Holger Krekel (:user:`hpk42`)
`refactored <https://mail.python.org/pipermail/pypy-dev/2003-June/000787.html>`__
its test framework to clean things up (``pypy.tool.test``, but still
on top of ``unittest.py``, with nothing pytest-like yet).
- In December 2003, there was `another
iteration <https://foss.heptapod.net/pypy/pypy/-/commit/02752373e1b29d89c6bb0a97e5f940caa22bdd63>`__
at improving their testing situation, by Stefan Schwarzer, called
``pypy.tool.newtest``.
- However, it didnt seem to be around for long, as around June/July
2004, efforts started on a thing called ``utest``, offering plain
assertions. This seems like the start of something pytest-like, but
unfortunately, it's unclear where the test runner's code was at the time.
The closest thing still around is `this
file <https://foss.heptapod.net/pypy/pypy/-/commit/0735f9ed287ec20950a7dd0a16fc10810d4f6847>`__,
but that doesnt seem like a complete test runner at all. What can be seen
is that there were `various
efforts <https://foss.heptapod.net/pypy/pypy/-/commits/branch/default?utf8=%E2%9C%93&search=utest>`__
by Laura Creighton and Samuele Pedroni (:user:`pedronis`) at automatically
converting existing tests to the new ``utest`` framework.
- Around the same time, for Europython 2004, @hpk42 `started a
project <http://web.archive.org/web/20041020215353/http://codespeak.net/svn/user/hpk/talks/std-talk.txt>`__
originally called “std”, intended to be a “complementary standard
library” - already laying out the principles behind what later became
pytest:
- current “batteries included” are very useful, but
- some of them are written in a pretty much java-like style,
especially the unittest-framework
- […]
- the best API is one that doesnt exist
[…]
- a testing package should require as few boilerplate code as
possible and offer much flexibility
- it should provide premium quality tracebacks and debugging aid
[…]
- first of all … forget about limited “assertXYZ APIs” and use the
real thing, e.g.::
assert x == y
- this works with plain python but you get unhelpful “assertion
failed” errors with no information
- std.utest (magic!) actually reinterprets the assertion expression
and offers detailed information about underlying values
- In September 2004, the ``py-dev`` mailinglist gets born, which `is
now <https://mail.python.org/pipermail/pytest-dev/>`__ ``pytest-dev``,
but thankfully with all the original archives still intact.
- Around September/October 2004, the ``std`` project `was renamed
<https://mail.python.org/pipermail/pypy-dev/2004-September/001565.html>`__ to
``py`` and ``std.utest`` became ``py.test``. This is also the first time the
`entire source
code <https://foss.heptapod.net/pypy/pypy/-/commit/42cf50c412026028e20acd23d518bd92e623ac11>`__,
seems to be available, with much of the API still being around today:
- ``py.path.local``, which is being phased out of pytest (in favour of
pathlib) some 16-17 years later
- The idea of the collection tree, including ``Collector``,
``FSCollector``, ``Directory``, ``PyCollector``, ``Module``,
``Class``
- Arguments like ``-x`` / ``--exitfirst``, ``-l`` /
``--showlocals``, ``--fulltrace``, ``--pdb``, ``-S`` /
``--nocapture`` (``-s`` / ``--capture=off`` today),
``--collectonly`` (``--collect-only`` today)
- In the same month, the ``py`` library `gets split off
<https://foss.heptapod.net/pypy/pypy/-/commit/6bdafe9203ad92eb259270b267189141c53bce33>`__
from ``PyPy``
- It seemed to get rather quiet for a while, and little seemed to happen
between October 2004 (removing ``py`` from PyPy) and January
2007 (first commit in the now-pytest repository). However, there were
various discussions about features/ideas on the mailinglist, and
:pypi:`a couple of releases <py/0.8.0-alpha2/#history>` every
couple of months:
- March 2006: py 0.8.0-alpha2
- May 2007: py 0.9.0
- March 2008: py 0.9.1 (first release to be found `in the pytest
changelog <https://github.com/pytest-dev/pytest/blob/main/doc/en/changelog.rst#091>`__!)
- August 2008: py 0.9.2
- In August 2009, py 1.0.0 was released, `introducing a lot of
fundamental
features <https://holgerkrekel.net/2009/08/04/pylib-1-0-0-released-the-testing-with-python-innovations-continue/>`__:
- funcargs/fixtures
- A `plugin
architecture <http://web.archive.org/web/20090629032718/https://codespeak.net/py/dist/test/extend.html>`__
which still looks very much the same today!
- Various `default
plugins <http://web.archive.org/web/20091005181132/https://codespeak.net/py/dist/test/plugin/index.html>`__,
including
`monkeypatch <http://web.archive.org/web/20091012022829/http://codespeak.net/py/dist/test/plugin/how-to/monkeypatch.html>`__
- Even back there, the
`FAQ <http://web.archive.org/web/20091005222413/http://codespeak.net/py/dist/faq.html>`__
said:
Clearly, [a second standard library] was ambitious and the naming has
maybe haunted the project rather than helping it. There may be a
project name change and possibly a split up into different projects
sometime.
and that finally happened in November 2010, when pytest 2.0.0 `was
released <https://mail.python.org/pipermail/pytest-dev/2010-November/001687.html>`__
as a package separate from ``py`` (but still called ``py.test``).
- In August 2016, pytest 3.0.0 :std:ref:`was released <release-3.0.0>`,
which adds ``pytest`` (rather than ``py.test``) as the recommended
command-line entry point
Due to this history, it's difficult to answer the question when pytest was started.
It depends what point should really be seen as the start of it all. One
possible interpretation is to pick Europython 2004, i.e. around June/July
2004.

View File

@@ -1,160 +0,0 @@
.. _how-to-handle-failures:
How to handle test failures
=============================
.. _maxfail:
Stopping after the first (or N) failures
---------------------------------------------------
To stop the testing process after the first (N) failures:
.. code-block:: bash
pytest -x # stop after first failure
pytest --maxfail=2 # stop after two failures
.. _pdb-option:
Using :doc:`python:library/pdb` with pytest
-------------------------------------------
Dropping to :doc:`pdb <python:library/pdb>` on failures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python comes with a builtin Python debugger called :doc:`pdb <python:library/pdb>`. ``pytest``
allows one to drop into the :doc:`pdb <python:library/pdb>` prompt via a command line option:
.. code-block:: bash
pytest --pdb
This will invoke the Python debugger on every failure (or KeyboardInterrupt).
Often you might only want to do this for the first failing test to understand
a certain failure situation:
.. code-block:: bash
pytest -x --pdb # drop to PDB on first failure, then end test session
pytest --pdb --maxfail=3 # drop to PDB for first three failures
Note that on any failure the exception information is stored on
``sys.last_value``, ``sys.last_type`` and ``sys.last_traceback``. In
interactive use, this allows one to drop into postmortem debugging with
any debug tool. One can also manually access the exception information,
for example::
>>> import sys
>>> sys.last_traceback.tb_lineno
42
>>> sys.last_value
AssertionError('assert result == "ok"',)
.. _trace-option:
Dropping to :doc:`pdb <python:library/pdb>` at the start of a test
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``pytest`` allows one to drop into the :doc:`pdb <python:library/pdb>` prompt immediately at the start of each test via a command line option:
.. code-block:: bash
pytest --trace
This will invoke the Python debugger at the start of every test.
.. _breakpoints:
Setting breakpoints
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. versionadded: 2.4.0
To set a breakpoint in your code use the native Python ``import pdb;pdb.set_trace()`` call
in your code and pytest automatically disables its output capture for that test:
* Output capture in other tests is not affected.
* Any prior test output that has already been captured and will be processed as
such.
* Output capture gets resumed when ending the debugger session (via the
``continue`` command).
.. _`breakpoint-builtin`:
Using the builtin breakpoint function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python 3.7 introduces a builtin ``breakpoint()`` function.
Pytest supports the use of ``breakpoint()`` with the following behaviours:
- When ``breakpoint()`` is called and ``PYTHONBREAKPOINT`` is set to the default value, pytest will use the custom internal PDB trace UI instead of the system default ``Pdb``.
- When tests are complete, the system will default back to the system ``Pdb`` trace UI.
- With ``--pdb`` passed to pytest, the custom internal Pdb trace UI is used with both ``breakpoint()`` and failed tests/unhandled exceptions.
- ``--pdbcls`` can be used to specify a custom debugger class.
.. _faulthandler:
Fault Handler
-------------
.. versionadded:: 5.0
The :mod:`faulthandler` standard module
can be used to dump Python tracebacks on a segfault or after a timeout.
The module is automatically enabled for pytest runs, unless the ``-p no:faulthandler`` is given
on the command-line.
Also the :confval:`faulthandler_timeout=X<faulthandler_timeout>` configuration option can be used
to dump the traceback of all threads if a test takes longer than ``X``
seconds to finish (not available on Windows).
.. note::
This functionality has been integrated from the external
`pytest-faulthandler <https://github.com/pytest-dev/pytest-faulthandler>`__ plugin, with two
small differences:
* To disable it, use ``-p no:faulthandler`` instead of ``--no-faulthandler``: the former
can be used with any plugin, so it saves one option.
* The ``--faulthandler-timeout`` command-line option has become the
:confval:`faulthandler_timeout` configuration option. It can still be configured from
the command-line using ``-o faulthandler_timeout=X``.
.. _unraisable:
Warning about unraisable exceptions and unhandled thread exceptions
-------------------------------------------------------------------
.. versionadded:: 6.2
.. note::
These features only work on Python>=3.8.
Unhandled exceptions are exceptions that are raised in a situation in which
they cannot propagate to a caller. The most common case is an exception raised
in a :meth:`__del__ <object.__del__>` implementation.
Unhandled thread exceptions are exceptions raised in a :class:`~threading.Thread`
but not handled, causing the thread to terminate uncleanly.
Both types of exceptions are normally considered bugs, but may go unnoticed
because they don't cause the program itself to crash. Pytest detects these
conditions and issues a warning that is visible in the test run summary.
The plugins are automatically enabled for pytest runs, unless the
``-p no:unraisableexception`` (for unraisable exceptions) and
``-p no:threadexception`` (for thread exceptions) options are given on the
command-line.
The warnings may be silenced selectively using the :ref:`pytest.mark.filterwarnings ref`
mark. The warning categories are :class:`pytest.PytestUnraisableExceptionWarning` and
:class:`pytest.PytestUnhandledThreadExceptionWarning`.

View File

@@ -1,64 +0,0 @@
:orphan:
.. _how-to:
How-to guides
================
Core pytest functionality
-------------------------
.. toctree::
:maxdepth: 1
usage
assert
fixtures
mark
parametrize
tmp_path
monkeypatch
doctest
cache
Test output and outcomes
----------------------------
.. toctree::
:maxdepth: 1
failures
output
logging
capture-stdout-stderr
capture-warnings
skipping
Plugins
----------------------------
.. toctree::
:maxdepth: 1
plugins
writing_plugins
writing_hook_functions
pytest and other test systems
-----------------------------
.. toctree::
:maxdepth: 1
existingtestsuite
unittest
nose
xunit_setup
pytest development environment
------------------------------
.. toctree::
:maxdepth: 1
bash-completion

View File

@@ -1,215 +0,0 @@
.. _usage:
How to invoke pytest
==========================================
.. seealso:: :ref:`Complete pytest command-line flag reference <command-line-flags>`
In general, pytest is invoked with the command ``pytest`` (see below for :ref:`other ways to invoke pytest
<invoke-other>`). This will execute all tests in all files whose names follow the form ``test_*.py`` or ``\*_test.py``
in the current directory and its subdirectories. More generally, pytest follows :ref:`standard test discovery rules
<test discovery>`.
.. _select-tests:
Specifying which tests to run
------------------------------
Pytest supports several ways to run and select tests from the command-line.
**Run tests in a module**
.. code-block:: bash
pytest test_mod.py
**Run tests in a directory**
.. code-block:: bash
pytest testing/
**Run tests by keyword expressions**
.. code-block:: bash
pytest -k "MyClass and not method"
This will run tests which contain names that match the given *string expression* (case-insensitive),
which can include Python operators that use filenames, class names and function names as variables.
The example above will run ``TestMyClass.test_something`` but not ``TestMyClass.test_method_simple``.
.. _nodeids:
**Run tests by node ids**
Each collected test is assigned a unique ``nodeid`` which consist of the module filename followed
by specifiers like class names, function names and parameters from parametrization, separated by ``::`` characters.
To run a specific test within a module:
.. code-block:: bash
pytest test_mod.py::test_func
Another example specifying a test method in the command line:
.. code-block:: bash
pytest test_mod.py::TestClass::test_method
**Run tests by marker expressions**
.. code-block:: bash
pytest -m slow
Will run all tests which are decorated with the ``@pytest.mark.slow`` decorator.
For more information see :ref:`marks <mark>`.
**Run tests from packages**
.. code-block:: bash
pytest --pyargs pkg.testing
This will import ``pkg.testing`` and use its filesystem location to find and run tests from.
Getting help on version, option names, environment variables
--------------------------------------------------------------
.. code-block:: bash
pytest --version # shows where pytest was imported from
pytest --fixtures # show available builtin function arguments
pytest -h | --help # show help on command line and config file options
.. _durations:
Profiling test execution duration
-------------------------------------
.. versionchanged:: 6.0
To get a list of the slowest 10 test durations over 1.0s long:
.. code-block:: bash
pytest --durations=10 --durations-min=1.0
By default, pytest will not show test durations that are too small (<0.005s) unless ``-vv`` is passed on the command-line.
Managing loading of plugins
-------------------------------
Early loading plugins
~~~~~~~~~~~~~~~~~~~~~~~
You can early-load plugins (internal and external) explicitly in the command-line with the ``-p`` option::
pytest -p mypluginmodule
The option receives a ``name`` parameter, which can be:
* A full module dotted name, for example ``myproject.plugins``. This dotted name must be importable.
* The entry-point name of a plugin. This is the name passed to ``setuptools`` when the plugin is
registered. For example to early-load the :pypi:`pytest-cov` plugin you can use::
pytest -p pytest_cov
Disabling plugins
~~~~~~~~~~~~~~~~~~
To disable loading specific plugins at invocation time, use the ``-p`` option
together with the prefix ``no:``.
Example: to disable loading the plugin ``doctest``, which is responsible for
executing doctest tests from text files, invoke pytest like this:
.. code-block:: bash
pytest -p no:doctest
.. _invoke-other:
Other ways of calling pytest
-----------------------------------------------------
.. _invoke-python:
Calling pytest through ``python -m pytest``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can invoke testing through the Python interpreter from the command line:
.. code-block:: text
python -m pytest [...]
This is almost equivalent to invoking the command line script ``pytest [...]``
directly, except that calling via ``python`` will also add the current directory to ``sys.path``.
.. _`pytest.main-usage`:
Calling pytest from Python code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can invoke ``pytest`` from Python code directly:
.. code-block:: python
retcode = pytest.main()
this acts as if you would call "pytest" from the command line.
It will not raise :class:`SystemExit` but return the :ref:`exit code <exit-codes>` instead.
You can pass in options and arguments:
.. code-block:: python
retcode = pytest.main(["-x", "mytestdir"])
You can specify additional plugins to ``pytest.main``:
.. code-block:: python
# content of myinvoke.py
import sys
import pytest
class MyPlugin:
def pytest_sessionfinish(self):
print("*** test run reporting finishing")
if __name__ == "__main__":
sys.exit(pytest.main(["-qq"], plugins=[MyPlugin()]))
Running it will show that ``MyPlugin`` was added and its
hook was invoked:
.. code-block:: pytest
$ python myinvoke.py
*** test run reporting finishing
.. note::
Calling ``pytest.main()`` will result in importing your tests and any modules
that they import. Due to the caching mechanism of python's import system,
making subsequent calls to ``pytest.main()`` from the same process will not
reflect changes to those files between the calls. For this reason, making
multiple calls to ``pytest.main()`` from the same process (in order to re-run
tests, for example) is not recommended.

View File

@@ -1,353 +0,0 @@
.. _`writinghooks`:
Writing hook functions
======================
.. _validation:
hook function validation and execution
--------------------------------------
pytest calls hook functions from registered plugins for any
given hook specification. Let's look at a typical hook function
for the ``pytest_collection_modifyitems(session, config,
items)`` hook which pytest calls after collection of all test items is
completed.
When we implement a ``pytest_collection_modifyitems`` function in our plugin
pytest will during registration verify that you use argument
names which match the specification and bail out if not.
Let's look at a possible implementation:
.. code-block:: python
def pytest_collection_modifyitems(config, items):
# called after collection is completed
# you can modify the ``items`` list
...
Here, ``pytest`` will pass in ``config`` (the pytest config object)
and ``items`` (the list of collected test items) but will not pass
in the ``session`` argument because we didn't list it in the function
signature. This dynamic "pruning" of arguments allows ``pytest`` to
be "future-compatible": we can introduce new hook named parameters without
breaking the signatures of existing hook implementations. It is one of
the reasons for the general long-lived compatibility of pytest plugins.
Note that hook functions other than ``pytest_runtest_*`` are not
allowed to raise exceptions. Doing so will break the pytest run.
.. _firstresult:
firstresult: stop at first non-None result
-------------------------------------------
Most calls to ``pytest`` hooks result in a **list of results** which contains
all non-None results of the called hook functions.
Some hook specifications use the ``firstresult=True`` option so that the hook
call only executes until the first of N registered functions returns a
non-None result which is then taken as result of the overall hook call.
The remaining hook functions will not be called in this case.
.. _`hookwrapper`:
hookwrapper: executing around other hooks
-------------------------------------------------
.. currentmodule:: _pytest.core
pytest plugins can implement hook wrappers which wrap the execution
of other hook implementations. A hook wrapper is a generator function
which yields exactly once. When pytest invokes hooks it first executes
hook wrappers and passes the same arguments as to the regular hooks.
At the yield point of the hook wrapper pytest will execute the next hook
implementations and return their result to the yield point in the form of
a :py:class:`Result <pluggy._Result>` instance which encapsulates a result or
exception info. The yield point itself will thus typically not raise
exceptions (unless there are bugs).
Here is an example definition of a hook wrapper:
.. code-block:: python
import pytest
@pytest.hookimpl(hookwrapper=True)
def pytest_pyfunc_call(pyfuncitem):
do_something_before_next_hook_executes()
outcome = yield
# outcome.excinfo may be None or a (cls, val, tb) tuple
res = outcome.get_result() # will raise if outcome was exception
post_process_result(res)
outcome.force_result(new_res) # to override the return value to the plugin system
Note that hook wrappers don't return results themselves, they merely
perform tracing or other side effects around the actual hook implementations.
If the result of the underlying hook is a mutable object, they may modify
that result but it's probably better to avoid it.
For more information, consult the
:ref:`pluggy documentation about hookwrappers <pluggy:hookwrappers>`.
.. _plugin-hookorder:
Hook function ordering / call example
-------------------------------------
For any given hook specification there may be more than one
implementation and we thus generally view ``hook`` execution as a
``1:N`` function call where ``N`` is the number of registered functions.
There are ways to influence if a hook implementation comes before or
after others, i.e. the position in the ``N``-sized list of functions:
.. code-block:: python
# Plugin 1
@pytest.hookimpl(tryfirst=True)
def pytest_collection_modifyitems(items):
# will execute as early as possible
...
# Plugin 2
@pytest.hookimpl(trylast=True)
def pytest_collection_modifyitems(items):
# will execute as late as possible
...
# Plugin 3
@pytest.hookimpl(hookwrapper=True)
def pytest_collection_modifyitems(items):
# will execute even before the tryfirst one above!
outcome = yield
# will execute after all non-hookwrappers executed
Here is the order of execution:
1. Plugin3's pytest_collection_modifyitems called until the yield point
because it is a hook wrapper.
2. Plugin1's pytest_collection_modifyitems is called because it is marked
with ``tryfirst=True``.
3. Plugin2's pytest_collection_modifyitems is called because it is marked
with ``trylast=True`` (but even without this mark it would come after
Plugin1).
4. Plugin3's pytest_collection_modifyitems then executing the code after the yield
point. The yield receives a :py:class:`Result <pluggy._Result>` instance which encapsulates
the result from calling the non-wrappers. Wrappers shall not modify the result.
It's possible to use ``tryfirst`` and ``trylast`` also in conjunction with
``hookwrapper=True`` in which case it will influence the ordering of hookwrappers
among each other.
Declaring new hooks
------------------------
.. note::
This is a quick overview on how to add new hooks and how they work in general, but a more complete
overview can be found in `the pluggy documentation <https://pluggy.readthedocs.io/en/latest/>`__.
.. currentmodule:: _pytest.hookspec
Plugins and ``conftest.py`` files may declare new hooks that can then be
implemented by other plugins in order to alter behaviour or interact with
the new plugin:
.. autofunction:: pytest_addhooks
:noindex:
Hooks are usually declared as do-nothing functions that contain only
documentation describing when the hook will be called and what return values
are expected. The names of the functions must start with `pytest_` otherwise pytest won't recognize them.
Here's an example. Let's assume this code is in the ``sample_hook.py`` module.
.. code-block:: python
def pytest_my_hook(config):
"""
Receives the pytest config and does things with it
"""
To register the hooks with pytest they need to be structured in their own module or class. This
class or module can then be passed to the ``pluginmanager`` using the ``pytest_addhooks`` function
(which itself is a hook exposed by pytest).
.. code-block:: python
def pytest_addhooks(pluginmanager):
"""This example assumes the hooks are grouped in the 'sample_hook' module."""
from my_app.tests import sample_hook
pluginmanager.add_hookspecs(sample_hook)
For a real world example, see `newhooks.py`_ from `xdist <https://github.com/pytest-dev/pytest-xdist>`_.
.. _`newhooks.py`: https://github.com/pytest-dev/pytest-xdist/blob/974bd566c599dc6a9ea291838c6f226197208b46/xdist/newhooks.py
Hooks may be called both from fixtures or from other hooks. In both cases, hooks are called
through the ``hook`` object, available in the ``config`` object. Most hooks receive a
``config`` object directly, while fixtures may use the ``pytestconfig`` fixture which provides the same object.
.. code-block:: python
@pytest.fixture()
def my_fixture(pytestconfig):
# call the hook called "pytest_my_hook"
# 'result' will be a list of return values from all registered functions.
result = pytestconfig.hook.pytest_my_hook(config=pytestconfig)
.. note::
Hooks receive parameters using only keyword arguments.
Now your hook is ready to be used. To register a function at the hook, other plugins or users must
now simply define the function ``pytest_my_hook`` with the correct signature in their ``conftest.py``.
Example:
.. code-block:: python
def pytest_my_hook(config):
"""
Print all active hooks to the screen.
"""
print(config.hook)
.. _`addoptionhooks`:
Using hooks in pytest_addoption
-------------------------------
Occasionally, it is necessary to change the way in which command line options
are defined by one plugin based on hooks in another plugin. For example,
a plugin may expose a command line option for which another plugin needs
to define the default value. The pluginmanager can be used to install and
use hooks to accomplish this. The plugin would define and add the hooks
and use pytest_addoption as follows:
.. code-block:: python
# contents of hooks.py
# Use firstresult=True because we only want one plugin to define this
# default value
@hookspec(firstresult=True)
def pytest_config_file_default_value():
"""Return the default value for the config file command line option."""
# contents of myplugin.py
def pytest_addhooks(pluginmanager):
"""This example assumes the hooks are grouped in the 'hooks' module."""
from . import hooks
pluginmanager.add_hookspecs(hooks)
def pytest_addoption(parser, pluginmanager):
default_value = pluginmanager.hook.pytest_config_file_default_value()
parser.addoption(
"--config-file",
help="Config file to use, defaults to %(default)s",
default=default_value,
)
The conftest.py that is using myplugin would simply define the hook as follows:
.. code-block:: python
def pytest_config_file_default_value():
return "config.yaml"
Optionally using hooks from 3rd party plugins
---------------------------------------------
Using new hooks from plugins as explained above might be a little tricky
because of the standard :ref:`validation mechanism <validation>`:
if you depend on a plugin that is not installed, validation will fail and
the error message will not make much sense to your users.
One approach is to defer the hook implementation to a new plugin instead of
declaring the hook functions directly in your plugin module, for example:
.. code-block:: python
# contents of myplugin.py
class DeferPlugin:
"""Simple plugin to defer pytest-xdist hook functions."""
def pytest_testnodedown(self, node, error):
"""standard xdist hook function."""
def pytest_configure(config):
if config.pluginmanager.hasplugin("xdist"):
config.pluginmanager.register(DeferPlugin())
This has the added benefit of allowing you to conditionally install hooks
depending on which plugins are installed.
.. _plugin-stash:
Storing data on items across hook functions
-------------------------------------------
Plugins often need to store data on :class:`~pytest.Item`\s in one hook
implementation, and access it in another. One common solution is to just
assign some private attribute directly on the item, but type-checkers like
mypy frown upon this, and it may also cause conflicts with other plugins.
So pytest offers a better way to do this, :attr:`item.stash <_pytest.nodes.Node.stash>`.
To use the "stash" in your plugins, first create "stash keys" somewhere at the
top level of your plugin:
.. code-block:: python
been_there_key = pytest.StashKey[bool]()
done_that_key = pytest.StashKey[str]()
then use the keys to stash your data at some point:
.. code-block:: python
def pytest_runtest_setup(item: pytest.Item) -> None:
item.stash[been_there_key] = True
item.stash[done_that_key] = "no"
and retrieve them at another point:
.. code-block:: python
def pytest_runtest_teardown(item: pytest.Item) -> None:
if not item.stash[been_there_key]:
print("Oh?")
item.stash[done_that_key] = "yes!"
Stashes are available on all node types (like :class:`~pytest.Class`,
:class:`~pytest.Session`) and also on :class:`~pytest.Config`, if needed.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 5.9 KiB

Some files were not shown because too many files have changed in this diff Show More