python: skip pytest_pycollect_makeitem work on certain names

When a Python object (module/class/instance) is collected, for each name
in `obj.__dict__` (and up its MRO) the pytest_pycollect_makeitem hook is
called for potentially creating a node for it.

These Python objects have a bunch of builtin attributes that are
extremely unlikely to be collected. But due to their pervasiveness,
dispatching the hook for them ends up being mildly expensive and also
pollutes PYTEST_DEBUG=1 output and such.

Let's just ignore these attributes.

On the pandas test suite commit 04e9e0afd476b1b8bed930e47bf60e,
collect only, irrelevant lines snipped, about 5% improvement:

Before:

```
         51195095 function calls (48844352 primitive calls) in 39.089 seconds

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
226602/54    0.145    0.000   38.940    0.721 manager.py:90(_hookexec)
    72227    0.285    0.000   20.146    0.000 python.py:424(_makeitem)
    72227    0.171    0.000   16.678    0.000 python.py:218(pytest_pycollect_makeitem)
```

After:

```
          48410921 function calls (46240870 primitive calls) in 36.950 seconds

    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 181429/54    0.113    0.000   36.777    0.681 manager.py:90(_hookexec)
     27054    0.130    0.000   17.755    0.001 python.py:465(_makeitem)
     27054    0.121    0.000   16.219    0.001 python.py:218(pytest_pycollect_makeitem)
```
This commit is contained in:
Ran Benita
2020-08-21 17:27:06 +03:00
parent 8730a7bb14
commit 98891a5947
3 changed files with 57 additions and 0 deletions

View File

@@ -885,6 +885,34 @@ class TestConftestCustomization:
result = testdir.runpytest_subprocess()
result.stdout.fnmatch_lines(["*1 passed*"])
def test_early_ignored_attributes(self, testdir: Testdir) -> None:
"""Builtin attributes should be ignored early on, even if
configuration would otherwise allow them.
This tests a performance optimization, not correctness, really,
although it tests PytestCollectionWarning is not raised, while
it would have been raised otherwise.
"""
testdir.makeini(
"""
[pytest]
python_classes=*
python_functions=*
"""
)
testdir.makepyfile(
"""
class TestEmpty:
pass
test_empty = TestEmpty()
def test_real():
pass
"""
)
items, rec = testdir.inline_genitems()
assert rec.ret == 0
assert len(items) == 1
def test_setup_only_available_in_subdir(testdir):
sub1 = testdir.mkpydir("sub1")