diff --git a/doc/en/contents.rst b/doc/en/contents.rst index 49bb67de3..9883eaa64 100644 --- a/doc/en/contents.rst +++ b/doc/en/contents.rst @@ -33,11 +33,11 @@ Full pytest documentation reference goodpractices + flaky pythonpath customize example/index bash-completion - flaky backwards-compatibility deprecations diff --git a/doc/en/example/multipython.py b/doc/en/example/multipython.py index 21fdf6fc2..a5360ed5a 100644 --- a/doc/en/example/multipython.py +++ b/doc/en/example/multipython.py @@ -33,7 +33,7 @@ class Python(object): dumpfile = self.picklefile.dirpath("dump.py") dumpfile.write( textwrap.dedent( - """\ + r"""\ import pickle f = open({!r}, 'wb') s = pickle.dump({!r}, f, protocol=2) @@ -49,7 +49,7 @@ class Python(object): loadfile = self.picklefile.dirpath("load.py") loadfile.write( textwrap.dedent( - """\ + r"""\ import pickle f = open({!r}, 'rb') obj = pickle.load(f) diff --git a/doc/en/flaky.rst b/doc/en/flaky.rst index c0f06a585..734ca1cde 100644 --- a/doc/en/flaky.rst +++ b/doc/en/flaky.rst @@ -1,4 +1,3 @@ -.. _flaky: Flaky tests ----------- @@ -25,7 +24,7 @@ Flaky tests sometimes appear when a test suite is run in parallel (such as use o - The flaky test is reliant on data from a previous test that doesn't clean up after itself, and in parallel runs that previous test is not always present - Tests that modify global state typically cannot be run in parallel. - + Overly strict assertion ~~~~~~~~~~~~~~~~~~~~~~~ @@ -44,7 +43,7 @@ Xfail strict PYTEST_CURRENT_TEST ~~~~~~~~~~~~~~~~~~~ -:ref:`pytest current test env ref` may be useful for figuring out "which test got stuck". +:ref:`pytest current test env` may be useful for figuring out "which test got stuck". Plugins @@ -62,7 +61,7 @@ Plugins to deliberately randomize tests can help expose tests with state problem * `pytest-random-order `_ * `pytest-randomly `_ - + Other general strategies ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -81,7 +80,7 @@ For UI tests these are important for understanding what the state of the UI was Delete or rewrite the test ~~~~~~~~~~~~~~~~~~~~~~~~~~ -If the functionality is covered by other tests, perhaps the test can be removed. If not, perhaps it can be rewritten at a lower level which will remove the flakiness or make its source more apparent. +If the functionality is covered by other tests, perhaps the test can be removed. If not, perhaps it can be rewritten at a lower level which will remove the flakiness or make its source more apparent. Quarantine @@ -103,9 +102,9 @@ Research This is a limited list, please submit an issue or pull request to expand it! -* Gao, Zebao, Yalan Liang, Myra B. Cohen, Atif M. Memon, and Zhen Wang. "Making system user interactive tests repeatable: When and what should we control?." In *Software Engineering (ICSE), 2015 IEEE/ACM 37th IEEE International Conference on*, vol. 1, pp. 55-65. IEEE, 2015. `PDF `_ -* Palomba, Fabio, and Andy Zaidman. "Does refactoring of test smells induce fixing flaky tests?." In *Software Maintenance and Evolution (ICSME), 2017 IEEE International Conference on*, pp. 1-12. IEEE, 2017. `PDF in Google Drive `_ -* Bell, Jonathan, Owolabi Legunsen, Michael Hilton, Lamyaa Eloussi, Tifany Yung, and Darko Marinov. "DeFlaker: Automatically detecting flaky tests." In *Proceedings of the 2018 International Conference on Software Engineering*. 2018. `PDF `_ +* Gao, Zebao, Yalan Liang, Myra B. Cohen, Atif M. Memon, and Zhen Wang. "Making system user interactive tests repeatable: When and what should we control?." In *Software Engineering (ICSE), 2015 IEEE/ACM 37th IEEE International Conference on*, vol. 1, pp. 55-65. IEEE, 2015. `PDF `__ +* Palomba, Fabio, and Andy Zaidman. "Does refactoring of test smells induce fixing flaky tests?." In *Software Maintenance and Evolution (ICSME), 2017 IEEE International Conference on*, pp. 1-12. IEEE, 2017. `PDF in Google Drive `__ +* Bell, Jonathan, Owolabi Legunsen, Michael Hilton, Lamyaa Eloussi, Tifany Yung, and Darko Marinov. "DeFlaker: Automatically detecting flaky tests." In *Proceedings of the 2018 International Conference on Software Engineering*. 2018. `PDF `__ Resources @@ -124,5 +123,5 @@ Resources * `Flaky Tests at Google and How We Mitigate Them `_ by John Micco, 2016 * `Where do Google's flaky tests come from? `_ by Jeff Listfield, 2017 - +