DOC: add a readme for benchmarks/pybench
This commit is contained in:
parent
9f28161837
commit
81cf0db047
|
@ -0,0 +1,49 @@
|
|||
# Continuous benchmarking of OpenBLAS performance
|
||||
|
||||
We run a set of benchmarks of subset of OpenBLAS functionality.
|
||||
|
||||
## Benchmark runner
|
||||
|
||||
[](https://codspeed.io/OpenMathLib/OpenBLAS/)
|
||||
|
||||
Click on [benchmarks](https://codspeed.io/OpenMathLib/OpenBLAS/benchmarks) to see the performance of a particular benchmark over time;
|
||||
Click on [branches](https://codspeed.io/OpenMathLib/OpenBLAS/branches/) and then on the last PR link to see the flamegraphs.
|
||||
|
||||
## What are the benchmarks
|
||||
|
||||
We run raw BLAS/LAPACK subroutines, via f2py-generated python wrappers. The wrappers themselves are equivalent to [those from SciPy](https://docs.scipy.org/doc/scipy/reference/linalg.lapack.html).
|
||||
In fact, the wrappers _are_ from SciPy, we take a small subset simply to avoid having to build the whole SciPy for each CI run.
|
||||
|
||||
|
||||
## Adding a new benchmark
|
||||
|
||||
`.github/workflows/codspeed-bench.yml` does all the orchestration on CI.
|
||||
|
||||
Benchmarks live in the `benchmark/pybench` directory. It is organized as follows:
|
||||
|
||||
- benchmarks themselves live in the `benchmarks` folder. Note that the LAPACK routines are imported from the `openblas_wrap` package.
|
||||
- the `openblas_wrap` package is a simple trampoline: it contains an f2py extension, `_flapack`, which talks to OpenBLAS, and exports the python names in its `__init__.py`.
|
||||
This way, the `openblas_wrap` package shields the benchmarks from the details of where a particular LAPACK function comes from. If wanted, you may for instance swap the `_flapack` extension to
|
||||
`scipy.linalg.blas` and `scipy.linalg.lapack`.
|
||||
|
||||
To change parameters of an existing benchmark, edit python files in the `benchmark/pybench/benchmarks` directory.
|
||||
|
||||
To add a benchmark for a new BLAS or LAPACK function, you need to:
|
||||
|
||||
- add an f2py wrapper for the bare LAPACK function. You can simply copy a wrapper from SciPy (look for `*.pyf.src` files in https://github.com/scipy/scipy/tree/main/scipy/linalg)
|
||||
- add an import to `benchmark/pybench/openblas_wrap/__init__.py`
|
||||
|
||||
|
||||
## Running benchmarks locally
|
||||
|
||||
This benchmarking layer is orchestrated from python, therefore you'll need to
|
||||
have all what it takes to build OpenBLAS from source, plus `python` and
|
||||
|
||||
```
|
||||
$ python -mpip install numpy meson ninja pytest pytest-benchmark
|
||||
```
|
||||
|
||||
The benchmark syntax is consistent with that of `pytest-benchmark` framework. The incantation to run the suite locally is `$ pytest benchmark/pybench/benchmarks/test_blas.py`.
|
||||
|
||||
An ASV compatible benchmark suite is planned but currently not implemented.
|
||||
|
Loading…
Reference in New Issue