diff --git a/docs/build_android.md b/docs/build_android.md
deleted file mode 100644
index b6bc37735..000000000
--- a/docs/build_android.md
+++ /dev/null
@@ -1,188 +0,0 @@
-In addition to the Android NDK, you will need a C compiler on the build host as this is currently
-required by the OpenBLAS build environment.
-
-
-### Building with android NDK using clang compiler
-Around version 11 Android NDKs stopped supporting gcc, so you would need to use clang to compile OpenBLAS. clang is supported from OpenBLAS 0.2.20 version onwards. See below sections on how to build with clang for ARMV7 and ARMV8 targets. The same basic principles as described below for ARMV8 should also apply to building an x86 or x86_64 version (substitute something like NEHALEM for the target instead of ARMV8 and replace all the aarch64 in the toolchain paths obviously)
-"Historic" notes:
-Since version 19 the default toolchain is provided as a standalone toolchain, so building one yourself following [building a standalone toolchain](http://developer.android.com/ndk/guides/standalone_toolchain.html) should no longer be necessary.
-If you want to use static linking with an old NDK version older than about r17, you need to choose an API level below 23 currently due to NDK bug 272 (https://github.com/android-ndk/ndk/issues/272 , the libc.a lacks a definition of stderr) that will probably be fixed in r17 of the NDK.
-
-#### Build ARMV7 with clang
-```
-# Set path to ndk-bundle
-export NDK_BUNDLE_DIR=/path/to/ndk-bundle
-
-# Set the PATH to contain paths to clang and arm-linux-androideabi-* utilities
-export PATH=${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin:${NDK_BUNDLE_DIR}/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
-
-# Set LDFLAGS so that the linker finds the appropriate libgcc
-export LDFLAGS="-L${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x"
-
-# Set the clang cross compile flags
-export CLANG_FLAGS="-target arm-linux-androideabi -marm -mfpu=vfp -mfloat-abi=softfp --sysroot ${NDK_BUNDLE_DIR}/platforms/android-23/arch-arm -gcc-toolchain ${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/"
-
-#OpenBLAS Compile
-make TARGET=ARMV7 ONLY_CBLAS=1 AR=ar CC="clang ${CLANG_FLAGS}" HOSTCC=gcc ARM_SOFTFP_ABI=1 -j4
-```
-On a Mac, it may also be necessary to give the complete path to the `ar` utility in the make command above, like so:
-```
-AR=${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc-ar
-```
-otherwise you may get a linker error complaining about a "malformed archive header name at 8" when the native OSX ar command was invoked instead. Note that with recent NDK versions, the AR tool may be named `llvm-ar` rather than what is assumed above.
-
-#### Build ARMV8 with clang
-```
-# Set path to ndk-bundle
-export NDK_BUNDLE_DIR=/path/to/ndk-bundle/
-
-# Export PATH to contain directories of clang and aarch64-linux-android-* utilities
-export PATH=${NDK_BUNDLE_DIR}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/:${NDK_BUNDLE_DIR}/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
-
-# Setup LDFLAGS so that loader can find libgcc and pass -lm for sqrt
-export LDFLAGS="-L${NDK_BUNDLE_DIR}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/aarch64-linux-android/4.9.x -lm"
-
-# Setup the clang cross compile options
-export CLANG_FLAGS="-target aarch64-linux-android --sysroot ${NDK_BUNDLE_DIR}/platforms/android-23/arch-arm64 -gcc-toolchain ${NDK_BUNDLE_DIR}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/"
-
-# Compile
-make TARGET=ARMV8 ONLY_CBLAS=1 AR=ar CC="clang ${CLANG_FLAGS}" HOSTCC=gcc -j4
-```
-Note: Using TARGET=CORTEXA57 in place of ARMV8 will pick up better optimized routines. Implementations for CORTEXA57 target is compatible with all other armv8 targets.
-
-Note: For NDK 23b, something as simple as
-```
-export PATH=/opt/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH
-make HOSTCC=gcc CC=/opt/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android31-clang ONLY_CBLAS=1 TARGET=ARMV8
-```
-appears to be sufficient on Linux. On OSX, setting AR to the ar provided in the "bin" path of the NDK (probably llvm-ar) is also necessary.
-
-#### Alternative script which was tested on OSX with NDK(21.3.6528147)
-This script will build openblas for 3 architecture (ARMV7,ARMV8,X86) and put them with `sudo make install` to `/opt/OpenBLAS/lib`.
-Of course you can also copy only the section that is of interest to you - also notice that the AR= line may need adapting to the
-name of the ar tool provided in your $TOOLCHAIN/bin - for example llvm-ar in some recent NDK versions.
-```
-export NDK=YOUR_PATH_TO_SDK/Android/sdk/ndk/21.3.6528147
-export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/darwin-x86_64
-
-make clean
-make \
- TARGET=ARMV7 \
- ONLY_CBLAS=1 \
- CC="$TOOLCHAIN"/bin/armv7a-linux-androideabi21-clang \
- AR="$TOOLCHAIN"/bin/arm-linux-androideabi-ar \
- HOSTCC=gcc \
- ARM_SOFTFP_ABI=1 \
- -j4
-sudo make install
-
-make clean
-make \
- TARGET=CORTEXA57 \
- ONLY_CBLAS=1 \
- CC=$TOOLCHAIN/bin/aarch64-linux-android21-clang \
- AR=$TOOLCHAIN/bin/aarch64-linux-android-ar \
- HOSTCC=gcc \
- -j4
-sudo make install
-
-make clean
-make \
- TARGET=ATOM \
- ONLY_CBLAS=1 \
- CC="$TOOLCHAIN"/bin/i686-linux-android21-clang \
- AR="$TOOLCHAIN"/bin/i686-linux-android-ar \
- HOSTCC=gcc \
- ARM_SOFTFP_ABI=1 \
- -j4
-sudo make install
-
-# This will build for x86_64
-make clean
-make \
- TARGET=ATOM BINARY=64\
- ONLY_CBLAS=1 \
- CC="$TOOLCHAIN"/bin/x86_64-linux-android21-clang \
- AR="$TOOLCHAIN"/bin/x86_64-linux-android-ar \
- HOSTCC=gcc \
- ARM_SOFTFP_ABI=1 \
- -j4
-sudo make install
-```
-Also you can find full list of target architectures in [TargetsList.txt](https://github.com/xianyi/OpenBLAS/blob/develop/TargetList.txt)
-
-***
-anything below this line should be irrelevant nowadays unless you need to perform software archeology
-***
-#### Building OpenBLAS with very old gcc-based versions of the NDK, without Fortran
-
-The prebuilt Android NDK toolchains do not include Fortran, hence parts like LAPACK cannot be built. You can still build OpenBLAS without it. For instructions on how to build OpenBLAS with Fortran, see the [next section](#building-openblas-with-fortran).
-
-To use easily the prebuilt toolchains, follow [building a standalone toolchain](http://developer.android.com/ndk/guides/standalone_toolchain.html) for your desired architecture.
-This would be `arm-linux-androideabi-gcc-4.9` for ARMV7 and `aarch64-linux-android-gcc-4.9` for ARMV8.
-
-You can build OpenBLAS (0.2.19 and earlier) with:
-```
-# Add the toolchain to your path
-export PATH=/path/to/standalone-toolchain/bin:$PATH
-
-# Build without Fortran for ARMV7
-make TARGET=ARMV7 HOSTCC=gcc CC=arm-linux-androideabi-gcc NOFORTRAN=1 libs
-# Build without Fortran for ARMV8
-make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc NOFORTRAN=1 libs
-```
-
-Since we are cross-compiling, we make the `libs` recipe, not `all`. Otherwise you will get errors when trying to link/run tests as versions up to and including 0.2.19 cannot build a shared library for Android.
-
-From 0.2.20 on, you should leave off the "libs" to get a full build, and you may want to use the softfp ABI instead of the deprecated hardfp one on ARMV7 so you would use
-```
-# Add the toolchain to your path
-export PATH=/path/to/standalone-toolchain/bin:$PATH
-
-# Build without Fortran for ARMV7
-make TARGET=ARMV7 ARM_SOFTFP_ABI=1 HOSTCC=gcc CC=arm-linux-androideabi-gcc NOFORTRAN=1
-# Build without Fortran for ARMV8
-make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc NOFORTRAN=1
-```
-
-If you get an error about stdio.h not being found, you need to specify your sysroot in the CFLAGS argument to `make` like
-```CFLAGS=--sysroot=$NDK/platforms/android-16/arch-arm```
-When you are done, install OpenBLAS into the desired directory. Be sure to also use all command line options
-here that you specified for building, otherwise errors may occur as it tries to install things you did not build:
-```
-make PREFIX=/path/to/install-dir TARGET=... install
-```
-
-#### Building OpenBLAS with Fortran
-
-Instructions on how to build the GNU toolchains with Fortran can be found [here](https://github.com/buffer51/android-gfortran). The [Releases section](https://github.com/buffer51/android-gfortran/releases) provides prebuilt versions, use the standalone one.
-
-You can build OpenBLAS with:
-```
-# Add the toolchain to your path
-export PATH=/path/to/standalone-toolchain-with-fortran/bin:$PATH
-
-# Build with Fortran for ARMV7
-make TARGET=ARMV7 HOSTCC=gcc CC=arm-linux-androideabi-gcc FC=arm-linux-androideabi-gfortran libs
-# Build with LAPACK for ARMV8
-make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc FC=aarch64-linux-android-gfortran libs
-```
-
-As mentioned above you can leave off the `libs` argument here when building 0.2.20 and later, and you may want to add ARM_SOFTFP_ABI=1 when building for ARMV7.
-
-#### Linking OpenBLAS (0.2.19 and earlier) for ARMV7
-
-If you are using `ndk-build`, you need to set the ABI to hard floating points in your Application.mk:
-```
-APP_ABI := armeabi-v7a-hard
-```
-
-This will set the appropriate flags for you. If you are not using `ndk-build`, you will want to add the following flags:
-```
-TARGET_CFLAGS += -mhard-float -D_NDK_MATH_NO_SOFTFP=1
-TARGET_LDFLAGS += -Wl,--no-warn-mismatch -lm_hard
-```
-
-From 0.2.20 on, it is also possible to build for the softfp ABI by specifying ARM_SOFTFP_ABI=1 during the build.
-In that case, also make sure that all your dependencies are compiled with -mfloat-abi=softfp as well, as mixing
-"hard" and "soft" floating point ABIs in a program will make it crash.
diff --git a/docs/build_ios.md b/docs/build_ios.md
deleted file mode 100644
index f0b054e13..000000000
--- a/docs/build_ios.md
+++ /dev/null
@@ -1,14 +0,0 @@
-# How to build OpenBLAS for iPhone/iOS
-
-As none of the current developers uses iOS, the following instructions are what was found to work in our Azure CI setup, but as far as we know this builds a fully working OpenBLAS for this platform.
-
-Go to the directory where you unpacked OpenBLAS,and enter the following commands:
-```
- CC=/Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
-
-CFLAGS= -O2 -Wno-macro-redefined -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.4.sdk -arch arm64 -miphoneos-version-min=10.0
-
-make TARGET=ARMV8 DYNAMIC_ARCH=1 NUM_THREADS=32 HOSTCC=clang NOFORTRAN=1
-```
-Adjust MIN_IOS_VERSION as necessary for your installation, e.g. change the version number
-to the minimum iOS version you want to target and execute this file to build the library.
\ No newline at end of file
diff --git a/docs/build_m1.md b/docs/build_m1.md
deleted file mode 100644
index 77e3b9e83..000000000
--- a/docs/build_m1.md
+++ /dev/null
@@ -1 +0,0 @@
-On newer versions of Xcode and on arm64, you might need to compile with a newer macOS target (11.0) than the default (10.8) with `MACOSX_DEPLOYMENT_TARGET=11.0`, or switch your command-line tools to use an older SDK (e.g., [13.1](https://developer.apple.com/download/all/?q=Xcode%2013)).
\ No newline at end of file
diff --git a/docs/build_system.md b/docs/build_system.md
index 06242082d..98808fdfa 100644
--- a/docs/build_system.md
+++ b/docs/build_system.md
@@ -1,6 +1,7 @@
-**This page is made by someone who is not the developer and should not be considered as an official documentation of the build system. For getting the full picture, it is best to read the Makefiles and understand them yourself.**
+!!! warning
+ This page is made by someone who is not the developer and should not be considered as an official documentation of the build system. For getting the full picture, it is best to read the Makefiles and understand them yourself.
-### Makefile dep graph
+## Makefile dep graph
```
Makefile
@@ -41,12 +42,12 @@ Makefile
|~~~~~ relapack/
```
-### Important Variables
+## Important Variables
Most of the tunable variables are found in [Makefile.rule](https://github.com/xianyi/OpenBLAS/blob/develop/Makefile.rule), along with their detailed descriptions.
Most of the variables are detected automatically in [Makefile.prebuild](https://github.com/xianyi/OpenBLAS/blob/develop/Makefile.prebuild), if they are not set in the environment.
-#### CPU related
+### CPU related
```
ARCH - Target architecture (eg. x86_64)
TARGET - Target CPU architecture, in case of DYNAMIC_ARCH=1 means library will not be usable on less capable CPUs
@@ -55,7 +56,7 @@ DYNAMIC_ARCH - For building library for multiple TARGETs (does not lose any opti
DYNAMIC_LIST - optional user-provided subset of the DYNAMIC_CORE list in Makefile.system
```
-#### Toolchain related
+### Toolchain related
```
CC - TARGET C compiler used for compilation (can be cross-toolchains)
FC - TARGET Fortran compiler used for compilation (can be cross-toolchains, set NOFORTRAN=1 if used cross-toolchain has no fortran compiler)
@@ -65,7 +66,7 @@ HOSTCC - compiler of build machine, needed to create proper config f
HOST_CFLAGS - flags for build machine compiler
```
-#### Library related
+### Library related
```
BINARY - 32/64 bit library
diff --git a/docs/build_win_arm.md b/docs/build_win_arm.md
deleted file mode 100644
index 01c2a84df..000000000
--- a/docs/build_win_arm.md
+++ /dev/null
@@ -1,77 +0,0 @@
-This page describes how to natively build OpenBLAS library for windows on arm64 targets.
-
-See [below](#xcomp) for how to cross-compile OpenBLAS for WoA on an x86_64 Windows host.
-
-_We hope that the procedure can be simplified for/by LLVM 17+ and its flang-new compiler for the FORTRAN parts (LAPACK) of the code but we have no means of verifying that as of January 2024._
-
-
-# Prerequisite
-
-Following tools needs to be installed
-
-## 1. Download and install clang for windows on arm
-
-Find the latest LLVM build for WoA from [LLVM release page](https://releases.llvm.org/)
-
-E.g: LLVM 12 build for WoA64 can be found [here](https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.0/LLVM-12.0.0-woa64.exe)
-
-Run the LLVM installer and ensure that LLVM is added to environment PATH.
-
-## 2. Download and install classic flang for windows on arm
-
-Classic flang is the only available FORTRAN compiler for windows on arm for now and a pre-release build can be found [here](https://github.com/kaadam/flang/releases/tag/v0.1)
-
-There is no installer for classic flang and the zip package can be extracted and the path needs to be added to environment PATH.
-
-E.g: on PowerShell
-
-`$env:Path += ";C:\flang_woa\bin"`
-
-# Build
-
-The following steps describe how to build the static library for OpenBLAS with and without LAPACK
-
-## 1. Build OpenBLAS static library with BLAS and LAPACK routines with Make
-
-Following command can be used to build OpenBLAS static library with BLAS and LAPACK routines
-
-`make CC="clang-cl" HOSTCC="clang-cl" AR="llvm-ar" BUILD_WITHOUT_LAPACK=0 NOFORTRAN=0 DYNAMIC_ARCH=0 TARGET=ARMV8 ARCH=arm64 BINARY=64 USE_OPENMP=0 PARALLEL=1 RANLIB="llvm-ranlib" MAKE=make F_COMPILER=FLANG FC=FLANG FFLAGS_NOOPT="-march=armv8-a -cpp" FFLAGS="-march=armv8-a -cpp" NEED_PIC=0 HOSTARCH=arm64 libs netlib`
-
-## 2. Build static library with BLAS routines using CMake
-
-Classic flang has compatibility issues with cmake hence only BLAS routines can be compiled with CMake
-
-`mkdir build`
-
-`cd build`
-
-`cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DBUILD_WITHOUT_LAPACK=1 -DNOFORTRAN=1 -DDYNAMIC_ARCH=0 -DTARGET=ARMV8 -DARCH=arm64 -DBINARY=64 -DUSE_OPENMP=0 -DCMAKE_SYSTEM_PROCESSOR=ARM64 -DCMAKE_CROSSCOMPILING=1 -DCMAKE_SYSTEM_NAME=Windows`
-
-`cmake --build . --config Release`
-
-# Known issue
-
-## `getarch.exe` execution error
-
-If you notice that platform-specific headers by `getarch.exe` are not generated correctly, It could be due to a known debug runtime DLL issue for arm64 platforms. Please check out [link](https://linaro.atlassian.net/wiki/spaces/WOAR/pages/28677636097/Debug+run-time+DLL+issue#Workaround) for the workaround.
-
-# Alternative: cross-compiling on Windows x86
-## Prerequisites:
-a working installation of LLVM and Ninja (the versions included in the latest Visual Studio 2022 will do, if you install its optional Llvm.Clang component)
-
-## Build
- 1. Load into the appropriate cross compiling developer shell (in this case powershell, for cmd.exe it would be vcvarsamd64_arm64.bat):
- 'C:\Program Files\Microsoft Visual Studio\2022\Professional\Common7\Tools\Launch-VsDevShell.ps1' -Arch arm64 -HostArch amd64
-
- 2. Change to the folder where you unpacked the OpenBLAS sources if you haven't already, and create a subfolder for building.
- Then change into that folder.
-
- 3. Invoke the following CMake command
-
- ```
- cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DTARGET=ARMV8 -DCMAKE_CROSSCOMPILING=ON -DCMAKE_SYSTEM_NAME="Windows" -DARCH=arm -DBINARY=64 -DCMAKE_SYSTEM_PROCESSOR=ARM64 -DCMAKE_C_COMPILER=clang-cl -DCMAKE_C_COMPILER_TARGET=arm64-pc-windows-msvc -DCMAKE_ASM_COMPILER_TARGET=arm64-pc-windows-msvc.
- ```
- (Note the use of clang-cl instead of clang, and adding in the _COMPILER_TARGET entries. Without C_COMPILER_TARGET, clang-cl will be trying to compile x64 machine code and not make it past the cmake generation step. Without ASM_COMPILER_TARGET the assembly files in openblas will fail to assemble because the assembler is trying to interpret them as x64 assembly.)
-
- 4. Invoke Ninja for the actual compilation
-
diff --git a/docs/ci.md b/docs/ci.md
index 3e2a18192..e36fb633b 100644
--- a/docs/ci.md
+++ b/docs/ci.md
@@ -1,3 +1,5 @@
+# CI jobs overview
+
| Arch|Target CPU|OS|Build system|XComp to|C Compiler|Fortran Compiler|threading|DYN_ARCH|INT64|Libraries| CI Provider| CPU count|
| ------------|---|---|-----------|-------------|----------|----------------|------|------------|----------|-----------|----------|-------|
| x86_64 |Intel 32bit|Windows|CMAKE/VS2015| -|mingw6.3| - | pthreads | - | - | static | Appveyor| |
diff --git a/docs/developers.md b/docs/developers.md
index 0e0404025..08443b7e4 100644
--- a/docs/developers.md
+++ b/docs/developers.md
@@ -1,3 +1,5 @@
+# Developer manual
+
## Source codes Layout
```
diff --git a/docs/distributing.md b/docs/distributing.md
index 1e6372a28..2f3828d08 100644
--- a/docs/distributing.md
+++ b/docs/distributing.md
@@ -1,11 +1,12 @@
# Guidance for redistributing OpenBLAS
-*We note that this document contains recommendations only - packagers and other
-redistributors are in charge of how OpenBLAS is built and distributed in their
-systems, and may have good reasons to deviate from the guidance given on this
-page. These recommendations are aimed at general packaging systems, with a user
-base that typically is large, open source (or freely available at least), and
-doesn't behave uniformly or that the packager is directly connected with.*
+!!! note
+ This document contains recommendations only - packagers and other
+ redistributors are in charge of how OpenBLAS is built and distributed in their
+ systems, and may have good reasons to deviate from the guidance given on this
+ page. These recommendations are aimed at general packaging systems, with a user
+ base that typically is large, open source (or freely available at least), and
+ doesn't behave uniformly or that the packager is directly connected with.*
OpenBLAS has a large number of build-time options which can be used to change
how it behaves at runtime, how artifacts or symbols are named, etc. Variation
diff --git a/docs/donate.md b/docs/donate.md
index ca7efe915..94d73bab5 100644
--- a/docs/donate.md
+++ b/docs/donate.md
@@ -1,14 +1,4 @@
-
-
-
+# Donations
Thank you for the support.
@@ -18,8 +8,6 @@ You can read OpenBLAS statement of receipts and disbursement and cash balance on
* [2013.8] [Testbed for OpenBLAS project](https://www.bountysource.com/fundraisers/443-testbed-for-openblas-project)
- * completed.
-
Here is [Backer list](https://github.com/xianyi/OpenBLAS/blob/develop/BACKERS.md).
## For Personal Donation
@@ -28,4 +16,4 @@ You can create a bounty for an issue on [bountysource.com](https://www.bountysou
## For Hardware Vendors
-We welcome the hardware donation, including the latest CPU and boards.
\ No newline at end of file
+We welcome the hardware donation, including the latest CPU and boards.
diff --git a/docs/download.md b/docs/download.md
deleted file mode 100644
index cb0f32526..000000000
--- a/docs/download.md
+++ /dev/null
@@ -1,23 +0,0 @@
-
-
-
-
-## Binary Packages
-
-We provide binary packages for the following platform.
-
-* Windows x86/x86_64
-* ARM
-
-You can download them from [file hosting on sourceforge.net](http://sourceforge.net/projects/openblas/files/).
-
-## Source
-Download the latest [stable version](https://github.com/xianyi/OpenBLAS/releases) from release page.
\ No newline at end of file
diff --git a/docs/ecosystem.md b/docs/ecosystem.md
index d667f6c40..ae2ae543b 100644
--- a/docs/ecosystem.md
+++ b/docs/ecosystem.md
@@ -1,3 +1,5 @@
+# Users of OpenBLAS
+
## Softwares
* Julia - a high-level, high-performance dynamic programming language for technical computing
diff --git a/docs/faq.md b/docs/faq.md
index 3c30de0f6..bbcbb36e4 100644
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -1,16 +1,6 @@
-
-
-
+# FAQ
-### General questions
+## General questions
+ **[What is BLAS? Why is it important?](#whatblas)**
+ **[What functions are there and how can I call them from my C code?](#whatsinblas)**
+ **[What is OpenBLAS? Why did you create this project?](#what)**
@@ -23,7 +13,7 @@
+ **[What support is there for recent PC hardware ? What about GPU ?](#recent_hardware)**
+ **[How about the level 3 BLAS performance on Intel Sandy Bridge?](#sandybridge_perf)**
-### OS and Compiler
+## OS and Compiler
+ **[How can I call an OpenBLAS function in Microsoft Visual Studio?](#MSVC)**
+ **[How can I use CBLAS and LAPACKE without C99 complex number support (e.g. in Visual Studio)?](#C99_complex_number)**
+ **[I get a SEGFAULT with multi-threading on Linux. What's wrong?](#Linux_SEGFAULT)**
@@ -34,7 +24,6 @@
+ **[How to solve undefined reference errors when statically linking against libopenblas.a](#static_link)**
+ **[Building OpenBLAS for Haswell or Dynamic Arch on RHEL-6, CentOS-6, Rocks-6.1](#binutils)**
+ **[Building OpenBLAS in QEMU/KVM/XEN](#qemu)**
-+ **[Building OpenBLAS for MIPS](#mips)**
+ **[Building OpenBLAS on POWER fails with the IBM XL compiler](#ppcxl)**
+ **[Replacing system BLAS in Ubuntu/Debian](#debianlts)**
+ **[I built OpenBLAS for use with some other software, but that software cannot find it](#findblas)**
@@ -43,7 +32,7 @@
+ **[Build fails with lots of errors about undefined ?GEMM_UNROLL_M](#newcpu)**
+ **[CMAKE/OSX: Build fails with 'argument list too long'](#cmakeosx)**
+ **[Likely problems with AVX2 support in Docker Desktop for OSX](#xhyve)**
-### Usage
+## Usage
+ **[Program is Terminated. Because you tried to allocate too many memory regions](#allocmorebuffers)**
+ **[How to choose TARGET manually at runtime when compiled with DYNAMIC_ARCH](#choose_target_dynamic)**
+ **[After updating the installed OpenBLAS, a program complains about "undefined symbol gotoblas"](#missgoto)**
@@ -57,13 +46,13 @@
+ **[Using OpenBLAS with OpenMP](#OpenMP)**
-### General questions
+## General questions
-#### What is BLAS? Why is it important?
+### What is BLAS? Why is it important?
[BLAS](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms) stands for Basic Linear Algebra Subprograms. BLAS provides standard interfaces for [linear algebra](https://en.wikipedia.org/wiki/Linear_algebra), including BLAS1 (vector-vector operations), BLAS2 (matrix-vector operations), and BLAS3 (matrix-matrix operations). In general, BLAS is the computational kernel ("the bottom of the food chain") in linear algebra or scientific applications. Thus, if BLAS implementation is highly optimized, the whole application can get substantial benefit.
-#### What functions are there and how can I call them from my C code?
+### What functions are there and how can I call them from my C code?
As BLAS is a standardized interface, you can refer to the documentation of its reference implementation at [netlib.org](http://netlib.org/blas/index.html#_blas_routines). Calls from C go through its CBLAS interface,
so your code will need to include the provided cblas.h in addition to linking with -lopenblas.
@@ -77,11 +66,11 @@ where M,N,K are the dimensions of your data - see https://petewarden.files.wordp
(This image is part of an article on GEMM in the context of deep learning that is well worth reading in full -
https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/)
-#### What is OpenBLAS? Why did you create this project?
+### What is OpenBLAS? Why did you create this project?
OpenBLAS is an open source BLAS library forked from the GotoBLAS2-1.13 BSD version. Since Mr. Kazushige Goto left TACC, GotoBLAS is no longer being maintained. Thus, we created this project to continue developing OpenBLAS/GotoBLAS.
-#### What's the difference between OpenBLAS and GotoBLAS?
+### What's the difference between OpenBLAS and GotoBLAS?
In OpenBLAS 0.2.0, we optimized level 3 BLAS on the Intel Sandy Bridge 64-bit OS. We obtained a performance comparable with that Intel MKL.
@@ -93,24 +82,24 @@ We also added some minor features, e.g. supporting "make install", compiling wit
You can find the full list of modifications in Changelog.txt.
-#### Where do parameters GEMM_P, GEMM_Q, GEMM_R come from?
+### Where do parameters GEMM_P, GEMM_Q, GEMM_R come from?
The detailed explanation is probably in the original publication authored by Kazushige Goto - Goto, Kazushige; van de Geijn, Robert A; Anatomy of high-performance matrix multiplication. ACM Transactions on Mathematical Software (TOMS). Volume 34 Issue 3, May 2008
While this article is paywalled and too old for preprints to be available on arxiv.org, more recent
publications like https://arxiv.org/pdf/1609.00076 contain at least a brief description of the algorithm.
In practice, the values are derived by experimentation to yield the block sizes that give the highest performance. A general rule of thumb for selecting a starting point seems to be that PxQ is about half the size of L2 cache.
-#### How can I report a bug?
+### How can I report a bug?
Please file an issue at this [issue page](https://github.com/xianyi/OpenBLAS/issues) or send mail to the [OpenBLAS mailing list](https://groups.google.com/forum/#!forum/openblas-users).
Please provide the following information: CPU, OS, compiler, and OpenBLAS compiling flags (Makefile.rule). In addition, please describe how to reproduce this bug.
-#### How to reference OpenBLAS.
+### How to reference OpenBLAS.
-You can reference our papers in [this page](Publications). Alternatively, you can cite the OpenBLAS homepage http://www.openblas.net.
+You can reference our papers in [this page](publications.md). Alternatively, you can cite the OpenBLAS homepage http://www.openblas.net.
-#### How can I use OpenBLAS in multi-threaded applications?
+### How can I use OpenBLAS in multi-threaded applications?
If your application is already multi-threaded, it will conflict with OpenBLAS multi-threading. Thus, you must set OpenBLAS to use single thread as following.
@@ -125,12 +114,12 @@ If the application is parallelized by OpenMP, please build OpenBLAS with USE_OPE
With the increased availability of fast multicore hardware it has unfortunately become clear that the thread management provided by OpenMP is not sufficient to prevent race conditions when OpenBLAS was built single-threaded by USE_THREAD=0 and there are concurrent calls from multiple threads to OpenBLAS functions. In this case,
it is vital to also specify USE_LOCKING=1 (introduced with OpenBLAS 0.3.7).
-#### Does OpenBLAS support sparse matrices and/or vectors ?
+### Does OpenBLAS support sparse matrices and/or vectors ?
OpenBLAS implements only the standard (dense) BLAS and LAPACK functions with a select few extensions popularized by Intel's MKL. Some
cases can probably be made to work using e.g. GEMV or AXPBY, in general using a dedicated package like SuiteSparse (which can make use of OpenBLAS or equivalent for standard operations) is recommended.
-#### What support is there for recent PC hardware ? What about GPU ?
+### What support is there for recent PC hardware ? What about GPU ?
As OpenBLAS is a volunteer project, it can take some time for the combination of a capable developer,
free time, and particular hardware to come along, even for relatively common processors. Starting from 0.3.1, support
@@ -138,7 +127,7 @@ is being added for AVX 512 (TARGET=SKYLAKEX), requiring a compiler that is capab
While AMD Zen processors should be autodetected by the build system, as of 0.3.2 they are still handled exactly
like Intel Haswell. There once was an effort to build an OpenCL implementation that one can still find at https://github.com/xianyi/clOpenBLAS , but work on this stopped in 2015.
-#### How about the level 3 BLAS performance on Intel Sandy Bridge?
+### How about the level 3 BLAS performance on Intel Sandy Bridge?
We obtained a performance comparable with Intel MKL that actually outperformed Intel MKL in some cases.
Here is the result of the DGEMM subroutine's performance on Intel Core i5-2500K Windows 7 SP1 64-bit:
@@ -146,19 +135,19 @@ Here is the result of the DGEMM subroutine's performance on Intel Core i5-2500K
-### OS and Compiler
+## OS and Compiler
-#### How can I call an OpenBLAS function in Microsoft Visual Studio?
+### How can I call an OpenBLAS function in Microsoft Visual Studio?
-Please read [this page](How-to-use-OpenBLAS-in-Microsoft-Visual-Studio).
+Please read [this page](install.md#visual-studio).
-#### How can I use CBLAS and LAPACKE without C99 complex number support (e.g. in Visual Studio)?
+### How can I use CBLAS and LAPACKE without C99 complex number support (e.g. in Visual Studio)?
Zaheer has fixed this bug. You can now use the structure instead of C99 complex numbers. Please read [this issue page](http://github.com/xianyi/OpenBLAS/issues/95) for details.
[This issue](https://github.com/xianyi/OpenBLAS/issues/305) is for using LAPACKE in Visual Studio.
-#### I get a SEGFAULT with multi-threading on Linux. What's wrong?
+### I get a SEGFAULT with multi-threading on Linux. What's wrong?
This may be related to a bug in the Linux kernel 2.6.32 (?). Try applying the patch segaults.patch to disable mbind using
@@ -166,7 +155,7 @@ This may be related to a bug in the Linux kernel 2.6.32 (?). Try applying the pa
and see if the crashes persist. Note that this patch will lead to many compiler warnings.
-#### When I make the library, there is no such instruction: `xgetbv' error. What's wrong?
+### When I make the library, there is no such instruction: `xgetbv' error. What's wrong?
Please use GCC 4.4 and later version. This version supports xgetbv instruction. If you use the library for Sandy Bridge with AVX instructions, you should use GCC 4.6 and later version.
@@ -174,18 +163,18 @@ On Mac OS X, please use Clang 3.1 and later version. For example, make CC=clang
For the compatibility with old compilers (GCC < 4.4), you can enable NO_AVX flag. For example, make NO_AVX=1
-#### My build fails due to the linker error "multiple definition of `dlamc3_'". What is the problem?
+### My build fails due to the linker error "multiple definition of `dlamc3_'". What is the problem?
This linker error occurs if GNU patch is missing or if our patch for LAPACK fails to apply.
Background: OpenBLAS implements optimized versions of some LAPACK functions, so we need to disable the reference versions. If this process fails we end with duplicated implementations of the same function.
-#### My build worked fine and passed all tests, but running `make lapack-test` ends with segfaults
+### My build worked fine and passed all tests, but running `make lapack-test` ends with segfaults
Some of the LAPACK tests, notably in xeigtstz, try to allocate around 10MB on the stack. You may need to use
`ulimit -s` to change the default limits on your system to allow this.
-#### How could I disable OpenBLAS threading affinity on runtime?
+### How could I disable OpenBLAS threading affinity on runtime?
You can define the OPENBLAS_MAIN_FREE or GOTOBLAS_MAIN_FREE environment variable to disable threading affinity on runtime. For example, before the running,
```
@@ -194,7 +183,7 @@ export OPENBLAS_MAIN_FREE=1
Alternatively, you can disable affinity feature with enabling NO_AFFINITY=1 in Makefile.rule.
-#### How to solve undefined reference errors when statically linking against libopenblas.a
+### How to solve undefined reference errors when statically linking against libopenblas.a
On Linux, if OpenBLAS was compiled with threading support (`USE_THREAD=1` by default), custom programs statically linked against `libopenblas.a` should also link to the pthread library e.g.:
@@ -231,7 +220,7 @@ blas_server.c:(.text+0x50f): undefined reference to `pthread_mutex_unlock'
The `-lpthread` is not required when linking dynamically against `libopenblas.so.0`.
-#### Building OpenBLAS for Haswell or Dynamic Arch on RHEL-6, CentOS-6, Rocks-6.1,Scientific Linux 6
+### Building OpenBLAS for Haswell or Dynamic Arch on RHEL-6, CentOS-6, Rocks-6.1,Scientific Linux 6
Minimum requirement to actually run AVX2-enabled software like OpenBLAS is kernel-2.6.32-358, shipped with EL6U4 in 2013
@@ -254,56 +243,20 @@ $ scl enable devtoolset-3 -- make DYNAMIC_ARCH=1
```
AVX-512 (SKYLAKEX) support requires devtoolset-8-gcc-gfortran (which exceeds formal requirement for AVX-512 because of packaging issues in earlier packages) which dependency-installs respective binutils and gcc or later and kernel 2.6.32-696 aka 6U9 or 3.10.0-327 aka 7U2 or later to run. In absence of abovementioned toolset OpenBLAS will fall back to AVX2 instructions in place of AVX512 sacrificing some performance on SKYLAKE-X platform.
-#### Building OpenBLAS in QEMU/KVM/XEN
+### Building OpenBLAS in QEMU/KVM/XEN
By default, QEMU reports the CPU as "QEMU Virtual CPU version 2.2.0", which shares CPUID with existing 32bit CPU even in 64bit virtual machine, and OpenBLAS recognizes it as PENTIUM2. Depending on the exact combination of CPU features the hypervisor choses to expose, this may not correspond to any CPU that exists, and OpenBLAS will error when trying to build. To fix this, pass `-cpu host` or `-cpu passthough` to QEMU, or another CPU model.
Similarly, the XEN hypervisor may not pass through all features of the host cpu while reporting the cpu type itself correctly, which can
lead to compiler error messages about an "ABI change" when compiling AVX512 code. Again changing the Xen configuration by running e.g.
"xen-cmdline --set-xen cpuid=avx512" should get around this (as would building OpenBLAS for an older cpu lacking that particular feature, e.g. TARGET=HASWELL)
-#### Building OpenBLAS for MIPS
-
-For mips targets you will need latest toolchains
-P5600 - MTI GNU/Linux Toolchain
-I6400, P6600 - IMG GNU/Linux Toolchain
-
-The download link is below
-(http://codescape-mips-sdk.imgtec.com/components/toolchain/2016.05-03/downloads.html)
-
-You can use following commandlines for builds
-
-
- IMG_TOOLCHAIN_DIR={full IMG GNU/Linux Toolchain path including "bin" directory -- for example, /opt/linux_toolchain/bin}
- IMG_GCC_PREFIX=mips-img-linux-gnu
- IMG_TOOLCHAIN=${IMG_TOOLCHAIN_DIR}/${IMG_GCC_PREFIX}
-
- I6400 Build (n32):
- make BINARY=32 BINARY32=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL -mabi=n32" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=I6400
-
- I6400 Build (n64):
- make BINARY=64 BINARY64=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=I6400
-
- P6600 Build (n32):
- make BINARY=32 BINARY32=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL -mabi=n32" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=P6600
-
- P6600 Build (n64):
- make BINARY=64 BINARY64=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS="$CFLAGS" LDFLAGS="$CFLAGS" TARGET=P6600
-
- MTI_TOOLCHAIN_DIR={full MTI GNU/Linux Toolchain path including "bin" directory -- for example, /opt/linux_toolchain/bin}
- MTI_GCC_PREFIX=mips-mti-linux-gnu
- MTI_TOOLCHAIN=${IMG_TOOLCHAIN_DIR}/${IMG_GCC_PREFIX}
-
- P5600 Build:
-
- make BINARY=32 BINARY32=1 CC=$MTI_TOOLCHAIN-gcc AR=$MTI_TOOLCHAIN-ar FC="$MTI_TOOLCHAIN-gfortran -EL" RANLIB=$MTI_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=P5600
-
-#### Building OpenBLAS on POWER fails with IBM XL
+### Building OpenBLAS on POWER fails with IBM XL
Trying to compile OpenBLAS with IBM XL ends with error messages about unknown register names
like "vs32". Working around these by using known alternate names for the vector registers only leads to another assembler error about unsupported constraints. This is a known deficiency in the IBM compiler at least up to and including 16.1.0 (and in the POWER version of clang, from which it is derived) - use gcc instead. (See issues #1078
and #1699 for related discussions)
-#### Replacing system BLAS/updating APT OpenBLAS in Mint/Ubuntu/Debian
+### Replacing system BLAS/updating APT OpenBLAS in Mint/Ubuntu/Debian
Debian and Ubuntu LTS versions provide OpenBLAS package which is not updated after initial release, and under circumstances one might want to use more recent version of OpenBLAS e.g. to get support for newer CPUs
@@ -333,41 +286,41 @@ $ sudo update-alternatives --remove libblas.so.3 /opt/OpenBLAS/lib/libopenblas.s
In recent versions of the distributions, the installation path for the libraries has been changed to include the name of the host architecture, like /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 or libblas.so.3.x86_64-linux-gnu. Use ```$ update-alternatives --display libblas.so.3```
to find out what layout your system has.
-#### I built OpenBLAS for use with some other software, but that software cannot find it
+### I built OpenBLAS for use with some other software, but that software cannot find it
Openblas installs as a single library named libopenblas.so, while some programs may be searching for a separate libblas.so and liblapack.so so you may need to create appropriate symbolic links (`ln -s libopenblas.so libblas.so;
ln -s libopenblas.so liblapack.so`) or copies. Also make sure that the installation location (usually /opt/OpenBLAS/lib or /usr/local/lib) is among the library search paths of your system.
-#### I included cblas.h in my program, but the compiler complains about a missing common.h or functions from it
+### I included cblas.h in my program, but the compiler complains about a missing common.h or functions from it
You probably tried to include a cblas.h that you simply copied from the OpenBLAS source, instead you need to run
`make install` after building OpenBLAS and then use the modified cblas.h that this step builds in the installation
path (usually either /usr/local/include, /opt/OpenBLAS/include or whatever you specified as PREFIX= on the `make install`)
-#### Compiling OpenBLAS with gcc's -fbounds-check actually triggers aborts in programs
+### Compiling OpenBLAS with gcc's -fbounds-check actually triggers aborts in programs
This is due to different interpretations of the (informal) standard for passing characters as arguments between C and FORTRAN functions. As the method for storing text differs in the two languages, when C calls Fortran the text length is passed as an "invisible" additional parameter.
Historically, this has not been required when the text is just a single character, so older code like the Reference-LAPACK bundled with OpenBLAS
does not do it. Recently gcc's checking has changed to require it, but there is no consensus yet if and how the existing LAPACK (and many other codebases) should adapt. (And for actual compilation, gcc has mostly backtracked and provided compatibility options - hence the default build settings in the OpenBLAS Makefiles add -fno-optimize-sibling-calls to the gfortran options to prevent miscompilation with "affected" versions. See ticket 2154 in the issue tracker for more details and links)
-#### Build fails with lots of errors about undefined ?GEMM_UNROLL_M
+### Build fails with lots of errors about undefined ?GEMM_UNROLL_M
Your cpu is apparently too new to be recognized by the build scripts, so they failed to assign appropriate parameters for the block algorithm.
Do a `make clean` and try again with TARGET set to one of the cpu models listed in `TargetList.txt` - for x86_64 this will usually be HASWELL.
-#### CMAKE/OSX: Build fails with 'argument list too long'
+### CMAKE/OSX: Build fails with 'argument list too long'
This is a limitation in the maximum length of a command on OSX, coupled with how CMAKE works. You should be able to work around this
by adding the option `-DCMAKE_Fortran_USE_RESPONSE_FILE_FOR_OBJECTS=1` to your CMAKE arguments.
-#### Likely problems with AVX2 support in Docker Desktop for OSX
+### Likely problems with AVX2 support in Docker Desktop for OSX
There have been a few reports of wrong calculation results and build-time test failures when building in a container environment managed by the OSX version of Docker Desktop, which uses the xhyve virtualizer underneath. Judging from these reports, AVX2 support in xhyve appears to be subtly broken but a corresponding ticket in the xhyve issue tracker has not drawn any reaction or comment since 2019. Therefore it is strongly recommended to build OpenBLAS with the NO_AVX2=1 option when inside a container under (or for later use with) the Docker Desktop environment on Intel-based Apple hardware.
-### Usage
+## Usage
-#### Program is Terminated. Because you tried to allocate too many memory regions
+### Program is Terminated. Because you tried to allocate too many memory regions
In OpenBLAS, we mange a pool of memory buffers and allocate the number of buffers as the following.
```
@@ -378,16 +331,16 @@ This error indicates that the program exceeded the number of buffers.
Please build OpenBLAS with larger `NUM_THREADS`. For example, `make NUM_THREADS=32` or `make NUM_THREADS=64`.
In `Makefile.system`, we will set `MAX_CPU_NUMBER=NUM_THREADS`.
-#### How to choose TARGET manually at runtime when compiled with DYNAMIC_ARCH
+### How to choose TARGET manually at runtime when compiled with DYNAMIC_ARCH
The environment variable which control the kernel selection is `OPENBLAS_CORETYPE` (see `driver/others/dynamic.c`)
e.g. `export OPENBLAS_CORETYPE=Haswell`. And the function `char* openblas_get_corename()` returns the used target.
-#### After updating the installed OpenBLAS, a program complains about "undefined symbol gotoblas"
+### After updating the installed OpenBLAS, a program complains about "undefined symbol gotoblas"
This symbol gets defined only when OpenBLAS is built with "make DYNAMIC_ARCH=1" (which is what distributors will choose to ensure support for more than just one CPU type).
-#### How can I find out at runtime what options the library was built with ?
+### How can I find out at runtime what options the library was built with ?
OpenBLAS has two utility functions that may come in here:
@@ -395,31 +348,31 @@ openblas_get_parallel() will return 0 for a single-threaded library, 1 if multit
openblas_get_config() will return a string containing settings such as USE64BITINT or DYNAMIC_ARCH that were active at build time, as well as the target cpu (or in case of a dynamic_arch build, the currently detected one).
-#### After making OpenBLAS, I find that the static library is multithreaded, but the dynamic one is not ?
+### After making OpenBLAS, I find that the static library is multithreaded, but the dynamic one is not ?
The shared OpenBLAS library you built is probably working fine as well, but your program may be picking up a different (probably single-threaded) version from one of the standard system paths like /usr/lib on startup.
Running `ldd /path/to/your/program` will tell you which library the linkage loader will actually use.
Specifying the "correct" library location with the `-L` flag (like `-L /opt/OpenBLAS/lib`) when linking your program only defines which library will be used to see if all symbols _can_ be resolved, you will need to add an rpath entry to the binary (using `-Wl,rpath=/opt/OpenBLAS/lib`) to make it request searching that location. Alternatively, remove the "wrong old" library (if you can), or set LD_LIBRARY_PATH to the desired location before running your program.
-#### I want to use OpenBLAS with CUDA in the HPL 2.3 benchmark code but it keeps looking for Intel MKL
+### I want to use OpenBLAS with CUDA in the HPL 2.3 benchmark code but it keeps looking for Intel MKL
You need to edit file src/cuda/cuda_dgemm.c in the NVIDIA version of HPL, change the "handle2" and "handle" dlopen calls to use libopenblas.so instead of libmkl_intel_lp64.so, and add an trailing underscore in the dlsym lines for dgemm_mkl and dtrsm_mkl (like `dgemm_mkl = (void(*)())dlsym(handle, “dgemm_”);`)
-#### Multithreaded OpenBLAS runs no faster or is even slower than singlethreaded on my ARMV7 board
+### Multithreaded OpenBLAS runs no faster or is even slower than singlethreaded on my ARMV7 board
The power saving mechanisms of your board may have shut down some cores, making them invisible to OpenBLAS in its startup phase. Try bringing them online before starting your calculation.
-#### Speed varies wildly between individual runs on a typical ARMV8 smartphone processor
+### Speed varies wildly between individual runs on a typical ARMV8 smartphone processor
Check the technical specifications, it could be that the SoC combines fast and slow cpus and threads can end up on either. In that case, binding the process to specific cores e.g. by setting `OMP_PLACES=cores` may help. (You may need to experiment with OpenMP options, it has been reported that using `OMP_NUM_THREADS=2 OMP_PLACES=cores` caused
a huge drop in performance on a 4+4 core chip while `OMP_NUM_THREADS=2 OMP_PLACES=cores(2)` worked as intended - as did OMP_PLACES=cores with 4 threads)
-#### I cannot get OpenBLAS to use more than a small subset of available cores on a big system
+### I cannot get OpenBLAS to use more than a small subset of available cores on a big system
Multithreading support in OpenBLAS requires the use of internal buffers for sharing partial results, the number and size of which is defined at compile time. Unless you specify NUM_THREADS in your make or cmake command, the build scripts try to autodetect the number of cores available in your build host to size the library to match. This unfortunately means that if you move the resulting binary from a small "front-end node" to a larger "compute node" later, it will still be limited to the hardware capabilities of the original system. The solution is to set NUM_THREADS to a number big enough to encompass the biggest systems you expect to run the binary on - at runtime, it will scale down the maximum number of threads it uses to match the number of cores physically available.
-#### Getting "ELF load command address/offset not properly aligned" when loading libopenblas.so
+### Getting "ELF load command address/offset not properly aligned" when loading libopenblas.so
If you get a message "error while loading shared libraries: libopenblas.so.0: ELF load command address/offset not properly aligned" when starting a program that is (dynamically) linked to OpenBLAS, this is very likely due to a bug in the GNU linker (ld) that is part of the
GNU binutils package. This error was specifically observed on older versions of Ubuntu Linux updated with the (at the time) most recent binutils version 2.38, but an internet search turned up sporadic reports involving various other libraries dating back several years. A bugfix was created by the binutils developers and should be available in later versions of binutils.(See issue 3708 for details)
diff --git a/docs/import_mingw.md b/docs/import_mingw.md
deleted file mode 100644
index a0038c1de..000000000
--- a/docs/import_mingw.md
+++ /dev/null
@@ -1,36 +0,0 @@
-Microsoft Windows has this thing called "import libraries". You don't need it in MinGW because the `ld` linker from GNU Binutils is smart, but you may still want it for whatever reason.
-
-## Make the `.def`
-
-Import libraries are compiled from a list of what symbols to use, `.def`. This should be already in your `exports` directory: `cd OPENBLAS_TOP_DIR/exports`.
-
-## Making a MinGW import library
-
-MinGW import libraries have the suffix `.a`, same as static libraries. (It's actually more common to do `.dll.a`...)
-
-You need to first prepend `libopenblas.def` with a line `LIBRARY libopenblas.dll`:
-
- cat <(echo "LIBRARY libopenblas.dll") libopenblas.def > libopenblas.def.1
- mv libopenblas.def.1 libopenblas.def
-
-Now it probably looks like:
-
- LIBRARY libopenblas.dll
- EXPORTS
- caxpy=caxpy_ @1
- caxpy_=caxpy_ @2
- ...
-
-Then, generate the import library: `dlltool -d libopenblas.def -l libopenblas.a`
-
-Again, there is basically **no point** in making an import library for use in MinGW. It actually slows down linking.
-
-## Making a MSVC import library
-
-Unlike MinGW, MSVC absolutely requires an import library. Now the C ABI of MSVC and MinGW are actually identical, so linking is actually okay. (Any incompatibility in the C ABI would be a bug.)
-
-The import libraries of MSVC have the suffix `.lib`. They are generated from a `.def` file using MSVC's `lib.exe`. See [the MSVC instructions](https://github.com/xianyi/OpenBLAS/wiki/How-to-use-OpenBLAS-in-Microsoft-Visual-Studio#generate-import-library-before-0210-version).
-
-## Notes
-* Always remember that MinGW is **not the same** as MSYS2 or Cygwin. MSYS2 and Cygwin are full POSIX environments with a lot of magic such as `fork()` and its own `malloc()`. MinGW, which builds on the normal Microsoft C Runtime, has none of that. Be clear about which one you are building for.
-
diff --git a/docs/index.md b/docs/index.md
index 06b6f108d..279cfe57d 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,23 +1,8 @@
-# OpenBLAS Wiki
+## Introduction
-
-
-
-
-[](https://travis-ci.org/xianyi/OpenBLAS)
-
-### Introduction
OpenBLAS is an optimized Basic Linear Algebra Subprograms (BLAS) library based on [GotoBLAS2](https://www.tacc.utexas.edu/research-development/tacc-software/gotoblas2) 1.13 BSD version.
-### License
+## License
OpenBLAS is licensed under the 3-clause BSD license. Full license text follows:
@@ -49,8 +34,8 @@ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-### Acknowledgements
+## Acknowledgements
This work is partially supported by
* Research and Development of Compiler System and Toolchain for Domestic CPU, National S&T Major Projects: Core Electronic Devices, High-end General Chips and Fundamental Software (No.2009ZX01036-001-002)
-* National High-tech R&D Program of China (Grant No.2012AA010903)
\ No newline at end of file
+* National High-tech R&D Program of China (Grant No.2012AA010903)
diff --git a/docs/install.md b/docs/install.md
index 84c8e6799..e3a19db5b 100644
--- a/docs/install.md
+++ b/docs/install.md
@@ -1,21 +1,88 @@
-## Quick Installation
-[Precompiled packages](https://github.com/xianyi/OpenBLAS/wiki/Precompiled-installation-packages) have recently become available for a number of platforms through their normal installation procedures, so for users of desktop devices at least, the instructions below are mostly relevant when you want to try the most recent development snapshot from git.
+# Install OpenBLAS
+
+!!! note
+ Lists of precompiled packages are not comprehensive, is not meant to validate nor endorse a particular third-party build over others, and may not always lead to the newest version
+
+
+## Quick install
+
+Precompiled packages have recently become available for a number of platforms through their normal installation procedures, so for users of desktop devices at least, the instructions below are mostly relevant when you want to try the most recent development snapshot from git. See your platform's relevant "Precompiled packages" section.
+
+The [Conda-Forge](https://github.com/conda-forge) project maintains packages for the conda package manager at .
+
+## Source
+Download the latest [stable version](https://github.com/xianyi/OpenBLAS/releases) from release page.
+
+## Platforms
### Linux
Just type `make` to compile the library.
-Notes
+Notes:
+
* OpenBLAS doesn't support g77. Please use gfortran or other Fortran compilers. e.g. `make FC=gfortran`.
* When building in an emulator (KVM,QEMU etc.) make sure that the combination of CPU features exposed to
the virtual environment matches that of an existing CPU to allow detection of the cpu model to succeed.
(With qemu, this can be done by passing `-cpu host` or a supported model name at invocation)
+
+#### Precompiled packages
+
+##### Debian/Ubuntu/Mint/Kali
+ OpenBLAS package is available in default repositories and can act as default BLAS in system
+
+Example installation commands:
+```bash
+$ sudo apt update
+$ apt search openblas
+$ sudo apt install libopenblas-dev
+$ sudo update-alternatives --config libblas.so.3
+```
+ Alternatively, if distributor's package proves unsatisfactory, you may try latest version of OpenBLAS, [Following guide in OpenBLAS FAQ](faq.md#debianlts)
+
+##### openSuSE/SLE
+ Recent OpenSUSE versions include OpenBLAS in default repositories and also permit OpenBLAS to act as replacement of system-wide BLAS.
+
+ Example installation commands:
+```bash
+$ sudo zypper ref
+$ zypper se openblas
+$ sudo zypper in openblas-devel
+$ sudo update-alternatives --config libblas.so.3
+```
+Should you be using older OpenSUSE or SLE that provides no OpenBLAS, you can attach optional or experimental openSUSE repository as a new package source to acquire recent build of OpenBLAS following [instructions on openSUSE software site](https://software.opensuse.org/package/openblas)
+
+##### Fedora/CentOS/RHEL
+Fedora provides OpenBLAS in default installation repositories.
+
+To install it try following:
+```bash
+$ dnf search openblas
+$ dnf install openblas-devel
+```
+For CentOS/RHEL/Scientific Linux packages are provided via [Fedora EPEL repository](https://fedoraproject.org/wiki/EPEL)
+
+After adding repository and repository keys installation is pretty straightforward:
+```bash
+$ yum search openblas
+$ yum install openblas-devel
+```
+No alternatives mechanism is provided for BLAS, and packages in system repositories are linked against NetLib BLAS or ATLAS BLAS libraries. You may wish to re-package RPMs to use OpenBLAS instead [as described here](https://fedoraproject.org/wiki/How_to_create_an_RPM_package)
+
+##### Mageia
+Mageia offers ATLAS and NetLIB LAPACK in base repositories.
+You can build your own OpenBLAS replacement, and once installed in /opt
+TODO: populate /usr/lib64 /usr/include accurately to replicate netlib with update-alternatives
+
+##### Arch/Manjaro/Antergos
+```bash
+$ sudo pacman -S openblas
+```
+
### Windows
-See [[How-to-use-OpenBLAS-in-Microsoft-Visual-Studio]].
-
-The precompiled binaries available with each release (in https://github.com/xianyi/OpenBLAS/releases) are
+The precompiled binaries available with each release (in ) are
created with MinGW (as described in Section 2 of the wiki page mentioned above) using an option list of
"NUM_THREADS=64 TARGET=GENERIC DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 CONSISTENT_FPCSR=1" - they should work on
any x86 or x86_64 computer. The zip archive contains the include files, static and dll libraries as well
@@ -32,21 +99,546 @@ documentation of whatever program you are using with OpenBLAS mentions 64bit int
addressing huge matrix sizes, you will need to build OpenBLAS from source (or open an issue ticket to make
the demand for such a precompiled build known).
+#### Precompiled packages
+
+*
+*
+
+#### Visual Studio
+
+As of OpenBLAS v0.2.15, we support MinGW and Visual Studio (using CMake to generate visual studio solution files – note that you will need at least version 3.11 of CMake for linking to work correctly) to build OpenBLAS on Windows.
+
+Note that you need a Fortran compiler if you plan to build and use the LAPACK functions included with OpenBLAS. The sections below describe using either `flang` as an add-on to clang/LLVM or `gfortran` as part of MinGW for this purpose. If you want to use the Intel Fortran compiler `ifort` for this, be sure to also use the Intel C compiler `icc` for building the C parts, as the ABI imposed by `ifort` is incompatible with `msvc`.
+
+##### 1. Native (MSVC) ABI
+
+A fully-optimized OpenBLAS that can be statically or dynamically linked to your application can currently be built for the 64-bit architecture with the LLVM compiler infrastructure. We're going to use Miniconda3 to grab all of the tools we need, since some of them are in an experimental status. Before you begin, you'll need to have Microsoft Visual Studio 2015 or newer installed.
+
+1. Install Miniconda3 for 64 bits using `winget install --id Anaconda.Miniconda3` or easily download from [conda.io](https://docs.conda.io/en/latest/miniconda.html).
+2. Open the "Anaconda Command Prompt," now available in the Start Menu, or at `%USERPROFILE%\miniconda3\shell\condabin\conda-hook.ps1`.
+3. In that command prompt window, use `cd` to change to the directory where you want to build OpenBLAS
+4. Now install all of the tools we need:
+
+ ```
+ conda update -n base conda
+ conda config --add channels conda-forge
+ conda install -y cmake flang clangdev perl libflang ninja
+ ```
+
+5. Still in the Anaconda Command Prompt window, activate the MSVC environment for 64 bits with `vcvarsall x64`. On Windows 11 with Visual Studio 2022, this would be done by invoking:
+
+ ```shell
+ "c:\Program Files\Microsoft Visual Studio\2022\Preview\vc\Auxiliary\Build\vcvars64.bat"
+ ```
+
+ With VS2019, the command should be the same – except for the year number, obviously. For other/older versions of MSVC,
+ the VS documentation or a quick search on the web should turn up the exact wording you need.
+
+ Confirm that the environment is active by typing `link` – this should return a long list of possible options for the `link` command. If it just
+ returns "command not found" or similar, review and retype the call to vcvars64.bat.
+ **NOTE:** if you are working from a Visual Studio Command prompt window instead (so that you do not have to do the vcvars call), you need to invoke
+ `conda activate` so that CONDA_PREFIX etc. get set up correctly before proceeding to step 6. Failing to do so will lead to link errors like
+ libflangmain.lib not getting found later in the build.
+
+6. Now configure the project with CMake. Starting in the project directory, execute the following:
+
+ ```
+ set "LIB=%CONDA_PREFIX%\Library\lib;%LIB%"
+ set "CPATH=%CONDA_PREFIX%\Library\include;%CPATH%"
+ mkdir build
+ cd build
+ cmake .. -G "Ninja" -DCMAKE_CXX_COMPILER=clang-cl -DCMAKE_C_COMPILER=clang-cl -DCMAKE_Fortran_COMPILER=flang -DCMAKE_MT=mt -DBUILD_WITHOUT_LAPACK=no -DNOFORTRAN=0 -DDYNAMIC_ARCH=ON -DCMAKE_BUILD_TYPE=Release
+ ```
+
+ You may want to add further options in the `cmake` command here – for instance, the default only produces a static .lib version of the library. If you would rather have a DLL, add -DBUILD_SHARED_LIBS=ON above. Note that this step only creates some command files and directories, the actual build happens next.
+
+
+7. Build the project:
+
+ ```
+ cmake --build . --config Release
+ ```
+ This step will create the OpenBLAS library in the "lib" directory, and various build-time tests in the `test`, `ctest` and `openblas_utest` directories. However it will not separate the header files you might need for building your own programs from those used internally. To put all relevant files in a more convenient arrangement, run the next step.
+
+8. Install all relevant files created by the build
+
+ ```
+ cmake --install . --prefix c:\opt -v
+ ```
+ This will copy all files that are needed for building and running your own programs with OpenBLAS to the given location, creating appropriate subdirectories for the individual kinds of files. In the case of "C:\opt" as given above, this would be C:\opt\include\openblas for the header files,
+ C:\opt\bin for the libopenblas.dll and C:\opt\lib for the static library. C:\opt\share holds various support files that enable other cmake-based build scripts to find OpenBLAS automatically.
+
+###### Visual studio 2017+ (C++2017 standard)
+
+In newer visual studio versions, Microsoft has changed [how it handles complex types](https://docs.microsoft.com/en-us/cpp/c-runtime-library/complex-math-support?view=msvc-170#types-used-in-complex-math). Even when using a precompiled version of OpenBLAS, you might need to define `LAPACK_COMPLEX_CUSTOM` in order to define complex types properly for MSVC. For example, some variant of the following might help:
+
+```
+#if defined(_MSC_VER)
+ #include
+ #define LAPACK_COMPLEX_CUSTOM
+ #define lapack_complex_float _Fcomplex
+ #define lapack_complex_double _Dcomplex
+#endif
+```
+
+For reference, see https://github.com/xianyi/OpenBLAS/issues/3661, https://github.com/Reference-LAPACK/lapack/issues/683, and https://stackoverflow.com/questions/47520244/using-openblas-lapacke-in-visual-studio.
+
+###### CMake and Visual Studio
+
+To build OpenBLAS for the 32-bit architecture, you'll need to use the builtin Visual Studio compilers.
+
+!!! note
+ This method may produce binaries which demonstrate significantly lower performance than those built with the other methods. (The Visual Studio compiler does not support the dialect of assembly used in the cpu-specific optimized files, so only the "generic" TARGET which is
+ written in pure C will get built. For the same reason it is not possible (and not necessary) to use -DDYNAMIC_ARCH=ON in a Visual Studio build) You may consider building for the 32-bit architecture using the GNU (MinGW) ABI.
+
+####### 1. Install CMake at Windows
+
+####### 2. Use CMake to generate Visual Studio solution files
+
+```
+# Do this from Powershell so cmake can find visual studio
+cmake -G "Visual Studio 14 Win64" -DCMAKE_BUILD_TYPE=Release .
+```
+
+###### Build the solution at Visual Studio
+
+Note that this step depends on perl, so you'll need to install perl for windows, and put perl on your path so VS can start perl (http://stackoverflow.com/questions/3051049/active-perl-installation-on-windows-operating-system).
+
+Step 2 will build the OpenBLAS solution, open it in VS, and build the projects. Note that the dependencies do not seem to be automatically configured: if you try to build libopenblas directly, it will fail with a message saying that some .obj files aren't found, but if you build the projects libopenblas depends on before building libopenblas, the build will succeed.
+
+###### Build OpenBLAS for Universal Windows Platform
+
+OpenBLAS can be built for use on the [Universal Windows Platform](https://en.wikipedia.org/wiki/Universal_Windows_Platform) using a two step process since commit [c66b842](https://github.com/xianyi/OpenBLAS/commit/c66b842d66c5516e52804bf5a0544d18b1da1b44).
+
+####### 1. Follow steps 1 and 2 above to build the Visual Studio solution files for Windows. This builds the helper executables which are required when building the OpenBLAS Visual Studio solution files for UWP in step 2.
+
+####### 2. Remove the generated CMakeCache.txt and CMakeFiles directory from the OpenBLAS source directory and re-run CMake with the following options:
+
+```
+# do this to build UWP compatible solution files
+cmake -G "Visual Studio 14 Win64" -DCMAKE_SYSTEM_NAME=WindowsStore -DCMAKE_SYSTEM_VERSION="10.0" -DCMAKE_SYSTEM_PROCESSOR=AMD64 -DVS_WINRT_COMPONENT=TRUE -DCMAKE_BUILD_TYPE=Release .
+```
+
+####### Build the solution with Visual Studio
+
+This will build the OpenBLAS binaries with the required settings for use with UWP.
+
+##### 2. GNU (MinGW) ABI
+
+The resulting library can be used in Visual Studio, but it can only be linked dynamically. This configuration has not been thoroughly tested and should be considered experimental.
+
+###### Incompatible x86 calling conventions
+
+Due to incompatibilities between the calling conventions of MinGW and Visual Studio you will need to make the following modifications ( **32-bit only** ):
+
+1. Use the newer GCC 4.7.0. The older GCC (<4.7.0) has an ABI incompatibility for returning aggregate structures larger than 8 bytes with MSVC.
+
+
+###### Build OpenBLAS on Windows OS
+1. Install the MinGW (GCC) compiler suite, either 32-bit (http://www.mingw.org/) or 64-bit (http://mingw-w64.sourceforge.net/). Be sure to install its gfortran package as well (unless you really want to build the BLAS part of OpenBLAS only) and check that gcc and gfortran are the same version – mixing compilers from different sources or release versions can lead to strange error messages in the linking stage. In addition, please install MSYS with MinGW.
+1. Build OpenBLAS in the MSYS shell. Usually, you can just type "make". OpenBLAS will detect the compiler and CPU automatically.
+1. After the build is complete, OpenBLAS will generate the static library "libopenblas.a" and the shared dll library "libopenblas.dll" in the folder. You can type "make PREFIX=/your/installation/path install" to install the library to a certain location.
+
+!!! note
+ We suggest using official MinGW or MinGW-w64 compilers. A user reported that s/he met `Unhandled exception` by other compiler suite. https://groups.google.com/forum/#!topic/openblas-users/me2S4LkE55w
+
+Note also that older versions of the alternative builds of mingw-w64 available through http://www.msys2.org may contain a defect that leads to a compilation failure accompanied by the error message
+```
+:0:4: error: expected identifier or '(' before numeric constant
+```
+If you encounter this, please upgrade your msys2 setup or see https://github.com/xianyi/OpenBLAS/issues/1503 for a workaround.
+
+###### Generate import library (before 0.2.10 version)
+
+1. First, you will need to have the `lib.exe` tool in the Visual Studio command prompt.
+1. Open the command prompt and type `cd OPENBLAS_TOP_DIR/exports`, where OPENBLAS_TOP_DIR is the main folder of your OpenBLAS installation.
+1. For a 32-bit library, type `lib /machine:i386 /def:libopenblas.def`. For 64-bit, type `lib /machine:X64 /def:libopenblas.def`.
+1. This will generate the import library "libopenblas.lib" and the export library "libopenblas.exp" in OPENBLAS_TOP_DIR/exports. Although these two files have the same name, they are totally different.
+
+###### Generate import library (0.2.10 and after version)
+1. OpenBLAS already generated the import library "libopenblas.dll.a" for "libopenblas.dll".
+
+###### generate windows native PDB files from gcc/gfortran build
+Tool to do so is available at https://github.com/rainers/cv2pdb
+
+###### Use OpenBLAS .dll library in Visual Studio
+1. Copy the import library (before 0.2.10: "OPENBLAS_TOP_DIR/exports/libopenblas.lib", 0.2.10 and after: "OPENBLAS_TOP_DIR/libopenblas.dll.a") and .dll library "libopenblas.dll" into the same folder(The folder of your project that is going to use the BLAS library. You may need to add the libopenblas.dll.a to the linker input list: properties->Linker->Input).
+1. Please follow the documentation about using third-party .dll libraries in MS Visual Studio 2008 or 2010. Make sure to link against a library for the correct architecture. For example, you may receive an error such as "The application was unable to start correctly (0xc000007b)" which typically indicates a mismatch between 32/64-bit libraries.
+
+!!! note
+ If you need CBLAS, you should include cblas.h in /your/installation/path/include in Visual Studio. Please read [this page](http://github.com/xianyi/OpenBLAS/issues/95).
+
+###### Limitations
+* Both static and dynamic linking are supported with MinGW. With Visual Studio, however, only dynamic linking is supported and so you should use the import library.
+* Debugging from Visual Studio does not work because MinGW and Visual Studio have incompatible formats for debug information (PDB vs. DWARF/STABS). You should either debug with GDB on the command-line or with a visual frontend, for instance [Eclipse](http://www.eclipse.org/cdt/) or [Qt Creator](http://qt.nokia.com/products/developer-tools/).
+
+
+#### Windows on Arm
+
+##### Prerequisites
+
+Following tools needs to be installed
+
+###### 1. Download and install clang for windows on arm
+
+Find the latest LLVM build for WoA from [LLVM release page](https://releases.llvm.org/)
+
+E.g: LLVM 12 build for WoA64 can be found [here](https://github.com/llvm/llvm-project/releases/download/llvmorg-12.0.0/LLVM-12.0.0-woa64.exe)
+
+Run the LLVM installer and ensure that LLVM is added to environment PATH.
+
+###### 2. Download and install classic flang for windows on arm
+
+Classic flang is the only available FORTRAN compiler for windows on arm for now and a pre-release build can be found [here](https://github.com/kaadam/flang/releases/tag/v0.1)
+
+There is no installer for classic flang and the zip package can be extracted and the path needs to be added to environment PATH.
+
+E.g: on PowerShell
+
+```
+$env:Path += ";C:\flang_woa\bin"
+```
+
+##### Build
+
+The following steps describe how to build the static library for OpenBLAS with and without LAPACK
+
+###### 1. Build OpenBLAS static library with BLAS and LAPACK routines with Make
+
+Following command can be used to build OpenBLAS static library with BLAS and LAPACK routines
+
+```bash
+$ make CC="clang-cl" HOSTCC="clang-cl" AR="llvm-ar" BUILD_WITHOUT_LAPACK=0 NOFORTRAN=0 DYNAMIC_ARCH=0 TARGET=ARMV8 ARCH=arm64 BINARY=64 USE_OPENMP=0 PARALLEL=1 RANLIB="llvm-ranlib" MAKE=make F_COMPILER=FLANG FC=FLANG FFLAGS_NOOPT="-march=armv8-a -cpp" FFLAGS="-march=armv8-a -cpp" NEED_PIC=0 HOSTARCH=arm64 libs netlib
+```
+
+###### 2. Build static library with BLAS routines using CMake
+
+Classic flang has compatibility issues with cmake hence only BLAS routines can be compiled with CMake
+
+```bash
+$ mkdir build
+$ cd build
+$ cmake .. -G Ninja -DCMAKE_C_COMPILER=clang -DBUILD_WITHOUT_LAPACK=1 -DNOFORTRAN=1 -DDYNAMIC_ARCH=0 -DTARGET=ARMV8 -DARCH=arm64 -DBINARY=64 -DUSE_OPENMP=0 -DCMAKE_SYSTEM_PROCESSOR=ARM64 -DCMAKE_CROSSCOMPILING=1 -DCMAKE_SYSTEM_NAME=Windows
+$ cmake --build . --config Release
+```
+
+###### `getarch.exe` execution error
+
+If you notice that platform-specific headers by `getarch.exe` are not generated correctly, It could be due to a known debug runtime DLL issue for arm64 platforms. Please check out [link](https://linaro.atlassian.net/wiki/spaces/WOAR/pages/28677636097/Debug+run-time+DLL+issue#Workaround) for the workaround.
+
+#### MinGW import library
+
+Microsoft Windows has this thing called "import libraries". You don't need it in MinGW because the `ld` linker from GNU Binutils is smart, but you may still want it for whatever reason.
+
+##### Make the `.def`
+
+Import libraries are compiled from a list of what symbols to use, `.def`. This should be already in your `exports` directory: `cd OPENBLAS_TOP_DIR/exports`.
+
+##### Making a MinGW import library
+
+MinGW import libraries have the suffix `.a`, same as static libraries. (It's actually more common to do `.dll.a`...)
+
+You need to first prepend `libopenblas.def` with a line `LIBRARY libopenblas.dll`:
+
+ cat <(echo "LIBRARY libopenblas.dll") libopenblas.def > libopenblas.def.1
+ mv libopenblas.def.1 libopenblas.def
+
+Now it probably looks like:
+
+ LIBRARY libopenblas.dll
+ EXPORTS
+ caxpy=caxpy_ @1
+ caxpy_=caxpy_ @2
+ ...
+
+Then, generate the import library: `dlltool -d libopenblas.def -l libopenblas.a`
+
+Again, there is basically **no point** in making an import library for use in MinGW. It actually slows down linking.
+
+##### Making a MSVC import library
+
+Unlike MinGW, MSVC absolutely requires an import library. Now the C ABI of MSVC and MinGW are actually identical, so linking is actually okay. (Any incompatibility in the C ABI would be a bug.)
+
+The import libraries of MSVC have the suffix `.lib`. They are generated from a `.def` file using MSVC's `lib.exe`. See [the MSVC instructions](use_visual_studio.md#generate-import-library-before-0210-version).
+
+##### Notes
+
+* Always remember that MinGW is **not the same** as MSYS2 or Cygwin. MSYS2 and Cygwin are full POSIX environments with a lot of magic such as `fork()` and its own `malloc()`. MinGW, which builds on the normal Microsoft C Runtime, has none of that. Be clear about which one you are building for.
+
### Mac OSX
If your CPU is Sandy Bridge, please use Clang version 3.1 and above. The Clang 3.0 will generate the wrong AVX binary code of OpenBLAS.
+#### Precompiled packages
+
+
+
+`brew install openblas`
+
+or using the conda package manager from
+
+(which also has packages for the new M1 cpu)
+
+ `conda install openblas`
+
#### Build on Apple M1
+On newer versions of Xcode and on arm64, you might need to compile with a newer macOS target (11.0) than the default (10.8) with `MACOSX_DEPLOYMENT_TARGET=11.0`, or switch your command-line tools to use an older SDK (e.g., [13.1](https://developer.apple.com/download/all/?q=Xcode%2013)).
+
* without Fortran compiler （cannot build LAPACK)
-```
- $ make CC=cc NOFORTRAN=1
-```
+ ```bash
+ $ make CC=cc NOFORTRAN=1
+ ```
* with Fortran compiler (you could `brew install gfortran`) https://github.com/xianyi/OpenBLAS/issues/3032
+ ```bash
+ $ export MACOSX_DEPLOYMENT_TARGET=11.0
+ $ make CC=cc FC=gfortran
+ ```
+
+### Android
+
+#### Build
+
+##### Prerequisites
+
+In addition to the Android NDK, you will need both Perl and a C compiler on the build host as these are currently
+required by the OpenBLAS build environment.
+
+
+##### Building with android NDK using clang compiler
+Around version 11 Android NDKs stopped supporting gcc, so you would need to use clang to compile OpenBLAS. clang is supported from OpenBLAS 0.2.20 version onwards. See below sections on how to build with clang for ARMV7 and ARMV8 targets. The same basic principles as described below for ARMV8 should also apply to building an x86 or x86_64 version (substitute something like NEHALEM for the target instead of ARMV8 and replace all the aarch64 in the toolchain paths obviously)
+"Historic" notes:
+Since version 19 the default toolchain is provided as a standalone toolchain, so building one yourself following [building a standalone toolchain](http://developer.android.com/ndk/guides/standalone_toolchain.html) should no longer be necessary.
+If you want to use static linking with an old NDK version older than about r17, you need to choose an API level below 23 currently due to NDK bug 272 (https://github.com/android-ndk/ndk/issues/272 , the libc.a lacks a definition of stderr) that will probably be fixed in r17 of the NDK.
+
+##### Build ARMV7 with clang
```
- $ export MACOSX_DEPLOYMENT_TARGET=11.0
- $ make CC=cc FC=gfortran
+## Set path to ndk-bundle
+export NDK_BUNDLE_DIR=/path/to/ndk-bundle
+
+## Set the PATH to contain paths to clang and arm-linux-androideabi-* utilities
+export PATH=${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin:${NDK_BUNDLE_DIR}/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
+
+## Set LDFLAGS so that the linker finds the appropriate libgcc
+export LDFLAGS="-L${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x"
+
+## Set the clang cross compile flags
+export CLANG_FLAGS="-target arm-linux-androideabi -marm -mfpu=vfp -mfloat-abi=softfp --sysroot ${NDK_BUNDLE_DIR}/platforms/android-23/arch-arm -gcc-toolchain ${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/"
+
+#OpenBLAS Compile
+make TARGET=ARMV7 ONLY_CBLAS=1 AR=ar CC="clang ${CLANG_FLAGS}" HOSTCC=gcc ARM_SOFTFP_ABI=1 -j4
```
+On a Mac, it may also be necessary to give the complete path to the `ar` utility in the make command above, like so:
+```
+AR=${NDK_BUNDLE_DIR}/toolchains/arm-linux-androideabi-4.9/prebuilt/darwin-x86_64/bin/arm-linux-androideabi-gcc-ar
+```
+otherwise you may get a linker error complaining about a "malformed archive header name at 8" when the native OSX ar command was invoked instead.
+
+##### Build ARMV8 with clang
+```
+## Set path to ndk-bundle
+export NDK_BUNDLE_DIR=/path/to/ndk-bundle/
+
+## Export PATH to contain directories of clang and aarch64-linux-android-* utilities
+export PATH=${NDK_BUNDLE_DIR}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/:${NDK_BUNDLE_DIR}/toolchains/llvm/prebuilt/linux-x86_64/bin:$PATH
+
+## Setup LDFLAGS so that loader can find libgcc and pass -lm for sqrt
+export LDFLAGS="-L${NDK_BUNDLE_DIR}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/lib/gcc/aarch64-linux-android/4.9.x -lm"
+
+## Setup the clang cross compile options
+export CLANG_FLAGS="-target aarch64-linux-android --sysroot ${NDK_BUNDLE_DIR}/platforms/android-23/arch-arm64 -gcc-toolchain ${NDK_BUNDLE_DIR}/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/"
+
+## Compile
+make TARGET=ARMV8 ONLY_CBLAS=1 AR=ar CC="clang ${CLANG_FLAGS}" HOSTCC=gcc -j4
+```
+Note: Using TARGET=CORTEXA57 in place of ARMV8 will pick up better optimized routines. Implementations for CORTEXA57 target is compatible with all other armv8 targets.
+
+Note: For NDK 23b, something as simple as
+```
+export PATH=/opt/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH
+make HOSTCC=gcc CC=/opt/android-ndk-r23b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android31-clang ONLY_CBLAS=1 TARGET=ARMV8
+```
+appears to be sufficient on Linux.
+
+##### Alternative script which was tested on OSX with NDK(21.3.6528147)
+This script will build openblas for 3 architecture (ARMV7,ARMV8,X86) and put them with `sudo make install` to `/opt/OpenBLAS/lib`
+```
+export NDK=YOUR_PATH_TO_SDK/Android/sdk/ndk/21.3.6528147
+export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/darwin-x86_64
+
+make clean
+make \
+ TARGET=ARMV7 \
+ ONLY_CBLAS=1 \
+ CC="$TOOLCHAIN"/bin/armv7a-linux-androideabi21-clang \
+ AR="$TOOLCHAIN"/bin/arm-linux-androideabi-ar \
+ HOSTCC=gcc \
+ ARM_SOFTFP_ABI=1 \
+ -j4
+sudo make install
+
+make clean
+make \
+ TARGET=CORTEXA57 \
+ ONLY_CBLAS=1 \
+ CC=$TOOLCHAIN/bin/aarch64-linux-android21-clang \
+ AR=$TOOLCHAIN/bin/aarch64-linux-android-ar \
+ HOSTCC=gcc \
+ -j4
+sudo make install
+
+make clean
+make \
+ TARGET=ATOM \
+ ONLY_CBLAS=1 \
+ CC="$TOOLCHAIN"/bin/i686-linux-android21-clang \
+ AR="$TOOLCHAIN"/bin/i686-linux-android-ar \
+ HOSTCC=gcc \
+ ARM_SOFTFP_ABI=1 \
+ -j4
+sudo make install
+
+## This will build for x86_64
+make clean
+make \
+ TARGET=ATOM BINARY=64\
+ ONLY_CBLAS=1 \
+ CC="$TOOLCHAIN"/bin/x86_64-linux-android21-clang \
+ AR="$TOOLCHAIN"/bin/x86_64-linux-android-ar \
+ HOSTCC=gcc \
+ ARM_SOFTFP_ABI=1 \
+ -j4
+sudo make install
+```
+Also you can find full list of target architectures in [TargetsList.txt](https://github.com/xianyi/OpenBLAS/blob/develop/TargetList.txt)
+
+***
+anything below this line should be irrelevant nowadays unless you need to perform software archeology
+***
+##### Building OpenBLAS with very old gcc-based versions of the NDK, without Fortran
+
+The prebuilt Android NDK toolchains do not include Fortran, hence parts like LAPACK cannot be built. You can still build OpenBLAS without it. For instructions on how to build OpenBLAS with Fortran, see the [next section](#building-openblas-with-fortran).
+
+To use easily the prebuilt toolchains, follow [building a standalone toolchain](http://developer.android.com/ndk/guides/standalone_toolchain.html) for your desired architecture.
+This would be `arm-linux-androideabi-gcc-4.9` for ARMV7 and `aarch64-linux-android-gcc-4.9` for ARMV8.
+
+You can build OpenBLAS (0.2.19 and earlier) with:
+```
+## Add the toolchain to your path
+export PATH=/path/to/standalone-toolchain/bin:$PATH
+
+## Build without Fortran for ARMV7
+make TARGET=ARMV7 HOSTCC=gcc CC=arm-linux-androideabi-gcc NOFORTRAN=1 libs
+## Build without Fortran for ARMV8
+make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc NOFORTRAN=1 libs
+```
+
+Since we are cross-compiling, we make the `libs` recipe, not `all`. Otherwise you will get errors when trying to link/run tests as versions up to and including 0.2.19 cannot build a shared library for Android.
+
+From 0.2.20 on, you should leave off the "libs" to get a full build, and you may want to use the softfp ABI instead of the deprecated hardfp one on ARMV7 so you would use
+```
+## Add the toolchain to your path
+export PATH=/path/to/standalone-toolchain/bin:$PATH
+
+## Build without Fortran for ARMV7
+make TARGET=ARMV7 ARM_SOFTFP_ABI=1 HOSTCC=gcc CC=arm-linux-androideabi-gcc NOFORTRAN=1
+## Build without Fortran for ARMV8
+make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc NOFORTRAN=1
+```
+
+If you get an error about stdio.h not being found, you need to specify your sysroot in the CFLAGS argument to `make` like
+```CFLAGS=--sysroot=$NDK/platforms/android-16/arch-arm```
+When you are done, install OpenBLAS into the desired directory. Be sure to also use all command line options
+here that you specified for building, otherwise errors may occur as it tries to install things you did not build:
+```
+make PREFIX=/path/to/install-dir TARGET=... install
+```
+
+##### Building OpenBLAS with Fortran
+
+Instructions on how to build the GNU toolchains with Fortran can be found [here](https://github.com/buffer51/android-gfortran). The [Releases section](https://github.com/buffer51/android-gfortran/releases) provides prebuilt versions, use the standalone one.
+
+You can build OpenBLAS with:
+```
+## Add the toolchain to your path
+export PATH=/path/to/standalone-toolchain-with-fortran/bin:$PATH
+
+## Build with Fortran for ARMV7
+make TARGET=ARMV7 HOSTCC=gcc CC=arm-linux-androideabi-gcc FC=arm-linux-androideabi-gfortran libs
+## Build with LAPACK for ARMV8
+make TARGET=ARMV8 BINARY=64 HOSTCC=gcc CC=aarch64-linux-android-gcc FC=aarch64-linux-android-gfortran libs
+```
+
+As mentioned above you can leave off the `libs` argument here when building 0.2.20 and later, and you may want to add ARM_SOFTFP_ABI=1 when building for ARMV7.
+
+##### Linking OpenBLAS (0.2.19 and earlier) for ARMV7
+
+If you are using `ndk-build`, you need to set the ABI to hard floating points in your Application.mk:
+```
+APP_ABI := armeabi-v7a-hard
+```
+
+This will set the appropriate flags for you. If you are not using `ndk-build`, you will want to add the following flags:
+```
+TARGET_CFLAGS += -mhard-float -D_NDK_MATH_NO_SOFTFP=1
+TARGET_LDFLAGS += -Wl,--no-warn-mismatch -lm_hard
+```
+
+From 0.2.20 on, it is also possible to build for the softfp ABI by specifying ARM_SOFTFP_ABI=1 during the build.
+In that case, also make sure that all your dependencies are compiled with -mfloat-abi=softfp as well, as mixing
+"hard" and "soft" floating point ABIs in a program will make it crash.
+
+### iPhone/iOS
+
+As none of the current developers uses iOS, the following instructions are what was found to work in our Azure CI setup, but as far as we know this builds a fully working OpenBLAS for this platform.
+
+Go to the directory where you unpacked OpenBLAS,and enter the following commands:
+```
+ CC=/Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
+
+CFLAGS= -O2 -Wno-macro-redefined -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.4.sdk -arch arm64 -miphoneos-version-min=10.0
+
+make TARGET=ARMV8 DYNAMIC_ARCH=1 NUM_THREADS=32 HOSTCC=clang NOFORTRAN=1
+```
+Adjust MIN_IOS_VERSION as necessary for your installation, e.g. change the version number
+to the minimum iOS version you want to target and execute this file to build the library.
+
+### MIPS
+
+For mips targets you will need latest toolchains
+P5600 - MTI GNU/Linux Toolchain
+I6400, P6600 - IMG GNU/Linux Toolchain
+
+The download link is below
+(http://codescape-mips-sdk.imgtec.com/components/toolchain/2016.05-03/downloads.html)
+
+You can use following commandlines for builds
+
+
+ IMG_TOOLCHAIN_DIR={full IMG GNU/Linux Toolchain path including "bin" directory -- for example, /opt/linux_toolchain/bin}
+ IMG_GCC_PREFIX=mips-img-linux-gnu
+ IMG_TOOLCHAIN=${IMG_TOOLCHAIN_DIR}/${IMG_GCC_PREFIX}
+
+ I6400 Build (n32):
+ make BINARY=32 BINARY32=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL -mabi=n32" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=I6400
+
+ I6400 Build (n64):
+ make BINARY=64 BINARY64=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=I6400
+
+ P6600 Build (n32):
+ make BINARY=32 BINARY32=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL -mabi=n32" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=P6600
+
+ P6600 Build (n64):
+ make BINARY=64 BINARY64=1 CC=$IMG_TOOLCHAIN-gcc AR=$IMG_TOOLCHAIN-ar FC="$IMG_TOOLCHAIN-gfortran -EL" RANLIB=$IMG_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS="$CFLAGS" LDFLAGS="$CFLAGS" TARGET=P6600
+
+ MTI_TOOLCHAIN_DIR={full MTI GNU/Linux Toolchain path including "bin" directory -- for example, /opt/linux_toolchain/bin}
+ MTI_GCC_PREFIX=mips-mti-linux-gnu
+ MTI_TOOLCHAIN=${IMG_TOOLCHAIN_DIR}/${IMG_GCC_PREFIX}
+
+ P5600 Build:
+
+ make BINARY=32 BINARY32=1 CC=$MTI_TOOLCHAIN-gcc AR=$MTI_TOOLCHAIN-ar FC="$MTI_TOOLCHAIN-gfortran -EL" RANLIB=$MTI_TOOLCHAIN-ranlib HOSTCC=gcc CFLAGS="-EL" FFLAGS=$CFLAGS LDFLAGS=$CFLAGS TARGET=P5600
+
### FreeBSD
You will need to install the following tools from the FreeBSD ports tree:
@@ -64,8 +656,46 @@ Note that you need to build with GNU make and manually specify the compiler, oth
[1]: [Removal of Fortran from the FreeBSD base system](http://www.bsdunix.ch/serendipity/index.php?/archives/345-Removal-of-Fortran-from-the-FreeBSD-base-system.html)
-### Android
-See [this page](https://github.com/xianyi/OpenBLAS/wiki/How-to-build-OpenBLAS-for-Android)
-### MIPS
-See [this page](https://github.com/xianyi/OpenBLAS/wiki/Faq#mips)
\ No newline at end of file
+```
+pkg install openblas
+```
+
+see
+
+### Cortex-M
+
+Cortex-M is a widely used microcontroller that is present in a variety of industrial and consumer electronics.
+A common variant of the Cortex-M is the STM32F4xx series. Here, we will give instructions for building for
+the STM32F4xx.
+
+First, install the embedded arm gcc compiler from the arm website. Then, create the following toolchain file and build as follows.
+
+```cmake
+# cmake .. -G Ninja -DCMAKE_C_COMPILER=arm-none-eabi-gcc -DCMAKE_TOOLCHAIN_FILE:PATH="toolchain.cmake" -DNOFORTRAN=1 -DTARGET=ARMV5 -DEMBEDDED=1
+
+set(CMAKE_SYSTEM_NAME Generic)
+set(CMAKE_SYSTEM_PROCESSOR arm)
+
+set(CMAKE_C_COMPILER "arm-none-eabi-gcc.exe")
+set(CMAKE_CXX_COMPILER "arm-none-eabi-g++.exe")
+
+set(CMAKE_EXE_LINKER_FLAGS "--specs=nosys.specs" CACHE INTERNAL "")
+
+set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
+set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
+set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+```
+
+In your embedded application, the following functions need to be provided for OpenBLAS to work correctly:
+
+```C
+void free(void* ptr);
+void* malloc(size_t size);
+```
+
+!!! note
+ If you are developing for an embedded platform, it is your responsibility to make sure that the device has sufficient memory for malloc calls. [Libmemory][2] provides one implementation of malloc for embedded platforms.
+
+[2]: https://github.com/embeddedartistry/libmemory
diff --git a/docs/logo.svg b/docs/logo.svg
new file mode 100644
index 000000000..e5a5df296
--- /dev/null
+++ b/docs/logo.svg
@@ -0,0 +1,450 @@
+
+
diff --git a/docs/mailing_list.md b/docs/mailing_list.md
index 8de323e63..c3637a875 100644
--- a/docs/mailing_list.md
+++ b/docs/mailing_list.md
@@ -1,15 +1,3 @@
-
-
-
-
OpenBLAS users: https://groups.google.com/forum/#!forum/openblas-users
-OpenBLAS developers: https://groups.google.com/forum/#!forum/openblas-dev
\ No newline at end of file
+OpenBLAS developers: https://groups.google.com/forum/#!forum/openblas-dev
diff --git a/docs/packages.md b/docs/packages.md
deleted file mode 100644
index d7796db14..000000000
--- a/docs/packages.md
+++ /dev/null
@@ -1,84 +0,0 @@
-# Precompiled installation packages
-_note this list is not comprehensive, is not meant to validate nor endorse a particular third-party build over others and may not always lead to the newest version_
-
-## Multiple operating systems
-Binaries designed to be compatible with [Julia](https://github.com/JuliaLang/julia) are regularly provided in
-https://github.com/staticfloat/OpenBLASBuilder/releases . Note that their 64bit builds with INTERFACE64=1 have
-`_64` appended to the function symbol names, so your code needs to refer to e.g. gemm_64_ rather than gemm_
-
-the [Conda-Forge](https://github.com/conda-forge) project maintains packages for the conda package manager at
-https://github.com/conda-forge/openblas-feedstock
-## FreeBSD
- > pkg install openblas
-
- see https://www.freebsd.org/ports/index.html
-## Linux
-### Debian/Ubuntu/Mint/Kali
- OpenBLAS package is available in default repositories and can act as default BLAS in system
-
-Example installation commands:
-```
- $ sudo apt update
- $ apt search openblas
- $ sudo apt install libopenblas-dev
- $ sudo update-alternatives --config libblas.so.3
-```
- Alternatively, if distributor's package proves unsatisfactory, you may try latest version of OpenBLAS, [Following guide in OpenBLAS FAQ](https://github.com/xianyi/OpenBLAS/wiki/faq#debianlts)
-
-### openSuSE/SLE:
- Recent OpenSUSE versions include OpenBLAS in default repositories and also permit OpenBLAS to act as replacement of system-wide BLAS.
-
- Example installation commands:
-```
- $ sudo zypper ref
- $ zypper se openblas
- $ sudo zypper in openblas-devel
- $ sudo update-alternatives --config libblas.so.3
-```
-Should you be using older OpenSUSE or SLE that provides no OpenBLAS, you can attach optional or experimental openSUSE repository as a new package source to acquire recent build of OpenBLAS following [instructions on openSUSE software site](https://software.opensuse.org/package/openblas)
-
-### Fedora/CentOS/RHEL
-Fedora provides OpenBLAS in default installation repositories.
-
-To install it try following:
-```
- $ dnf search openblas
- $ dnf install openblas-devel
-```
-For CentOS/RHEL/Scientific Linux packages are provided via [Fedora EPEL repository](https://fedoraproject.org/wiki/EPEL)
-
-After adding repository and repository keys installation is pretty straightforward:
-```
- $ yum search openblas
- $ yum install openblas-devel
-```
-No alternatives mechanism is provided for BLAS, and packages in system repositories are linked against NetLib BLAS or ATLAS BLAS libraries. You may wish to re-package RPMs to use OpenBLAS instead [as described here](https://fedoraproject.org/wiki/How_to_create_an_RPM_package)
-
-### Mageia
-Mageia offers ATLAS and NetLIB LAPACK in base repositories.
-You can build your own OpenBLAS replacement, and once installed in /opt
-TODO: populate /usr/lib64 /usr/include accurately to replicate netlib with update-alternatives
-
-### Arch/Manjaro/Antergos
-```
- $ sudo pacman -S openblas
-```
-
-## Solaris (via pkgsrc)
-
-
-## OSX
-https://www.macports.org/ports.php?by=name&substr=openblas
-
-`brew install openblas`
-
-or using the conda package manager from
-https://github.com/conda-forge/miniforge#download
-(which also has packages for the new M1 cpu)
-
- `conda install openblas`
-
-## Windows
-http://sourceforge.net/projects/openblas/files
-
-https://www.nuget.org/packages?q=openblas
diff --git a/docs/publications.md b/docs/publications.md
index 4161fd444..37889eec0 100644
--- a/docs/publications.md
+++ b/docs/publications.md
@@ -1,19 +1,7 @@
-
-
-
-
### 2013
* Wang Qian, Zhang Xianyi, Zhang Yunquan, Qing Yi, **AUGEM: Automatically Generate High Performance Dense Linear Algebra Kernels on x86 CPUs**, In the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'13), Denver CO, November 2013. [[pdf](http://xianyi.github.io/paper/augem_SC13.pdf)]
### 2012
-* Zhang Xianyi, Wang Qian, Zhang Yunquan, **Model-driven Level 3 BLAS Performance Optimization on Loongson 3A Processor**, 2012 IEEE 18th International Conference on Parallel and Distributed Systems (ICPADS), 17-19 Dec. 2012.
\ No newline at end of file
+* Zhang Xianyi, Wang Qian, Zhang Yunquan, **Model-driven Level 3 BLAS Performance Optimization on Loongson 3A Processor**, 2012 IEEE 18th International Conference on Parallel and Distributed Systems (ICPADS), 17-19 Dec. 2012.
diff --git a/docs/todo.md b/docs/todo.md
index 006f2cfa9..080a250a8 100644
--- a/docs/todo.md
+++ b/docs/todo.md
@@ -1,14 +1,16 @@
+# TODO
+
Werner found a lot of bugs about lapack testing. In 0.2.9 version, we used a work-around,e.g. fallback to the old kernel, replacing the optimized kernel with reference kernel. We must fix them in future release.
-* ~~Nehalem dgemm kernel. Fallback to core2 kernel.~~
-* ~~Sandy bridge sgemm kernel. Fallback to Nehalem kernels.~~
-* ~~Sandy bridge cgemm kernel. Fallback to Nehalem kernels.~~
+* Nehalem dgemm kernel. Fallback to core2 kernel.
+* Sandy bridge sgemm kernel. Fallback to Nehalem kernels.
+* Sandy bridge cgemm kernel. Fallback to Nehalem kernels.
* sbmv, zsbmv, smp bug (interface/sbmv.c zsbmv.c). Now, it is only sequential.
-* zhbmv, smp bug.
-* ~~scal, zscal x86/x86_64 kernels. Fallback to C implementation.~~
-* ~~potri, zpotri. Fallback to LAPACK reference implementation.~~
-* ~~lauu2, zlauu2. Fallback to LAPACK reference implementation.~~
-* ~~trtri, ztrtri. Fallback to LAPACK reference implementation.~~
-* ~~lauum, zlauum. Fallback to LAPACK reference implementation.~~
-* ~~trti2, ztrti2. Fallback to LAPACK reference implementation.~~
-* Disable SMP in ger.c and zger.c
\ No newline at end of file
+* zhbmv, smp bug.
+* scal, zscal x86/x86_64 kernels. Fallback to C implementation.
+* potri, zpotri. Fallback to LAPACK reference implementation.
+* lauu2, zlauu2. Fallback to LAPACK reference implementation.
+* trtri, ztrtri. Fallback to LAPACK reference implementation.
+* lauum, zlauum. Fallback to LAPACK reference implementation.
+* trti2, ztrti2. Fallback to LAPACK reference implementation.
+* Disable SMP in ger.c and zger.c
diff --git a/docs/use_cortex.md b/docs/use_cortex.md
deleted file mode 100644
index a90b1324f..000000000
--- a/docs/use_cortex.md
+++ /dev/null
@@ -1,37 +0,0 @@
-Cortex-M is a widely used microcontroller that is present in a variety of industrial and consumer electronics.
-A common variant of the Cortex-M is the STM32F4xx series. Here, we will give instructions for building for
-the STM32F4xx.
-
-First, install the embedded arm gcc compiler from the arm website. Then, create the following toolchain file and build as follows.
-
-```cmake
-# cmake .. -G Ninja -DCMAKE_C_COMPILER=arm-none-eabi-gcc -DCMAKE_TOOLCHAIN_FILE:PATH="toolchain.cmake" -DNOFORTRAN=1 -DTARGET=ARMV5 -DEMBEDDED=1
-
-set(CMAKE_SYSTEM_NAME Generic)
-set(CMAKE_SYSTEM_PROCESSOR arm)
-
-set(CMAKE_C_COMPILER "arm-none-eabi-gcc.exe")
-set(CMAKE_CXX_COMPILER "arm-none-eabi-g++.exe")
-
-set(CMAKE_EXE_LINKER_FLAGS "--specs=nosys.specs" CACHE INTERNAL "")
-
-set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
-set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
-set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
-set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
-```
-
-In your embedded application, the following functions need to be provided for OpenBLAS to work correctly:
-
-```C
-void free(void* ptr);
-void* malloc(size_t size);
-```
-
-Note: if you are developing for an embedded platform, it is your responsibility to make sure that the device
-has sufficient memory for malloc calls. [Libmemory][1] provides one implementation of malloc for embedded
-platforms.
-
-
-[1]: https://github.com/embeddedartistry/libmemory
-
diff --git a/docs/use_visual_studio.md b/docs/use_visual_studio.md
deleted file mode 100644
index fe031383f..000000000
--- a/docs/use_visual_studio.md
+++ /dev/null
@@ -1,162 +0,0 @@
-As of OpenBLAS v0.2.15, we support MinGW and Visual Studio (using CMake to generate visual studio solution files – note that you will need at least version 3.11 of CMake for linking to work correctly) to build OpenBLAS on Windows.
-
-Note that you need a Fortran compiler if you plan to build and use the LAPACK functions included with OpenBLAS. The sections below describe using either `flang` as an add-on to clang/LLVM or `gfortran` as part of MinGW for this purpose. If you want to use the Intel Fortran compiler `ifort` for this, be sure to also use the Intel C compiler `icc` for building the C parts, as the ABI imposed by `ifort` is incompatible with `msvc`.
-
-## 1. Native (MSVC) ABI
-
-A fully-optimized OpenBLAS that can be statically or dynamically linked to your application can currently be built for the 64-bit architecture with the LLVM compiler infrastructure. We're going to use Miniconda3 to grab all of the tools we need, since some of them are in an experimental status. Before you begin, you'll need to have Microsoft Visual Studio 2015 or newer installed.
-
-1. Install Miniconda3 for 64 bits using `winget install --id Anaconda.Miniconda3` or easily download from [conda.io](https://docs.conda.io/en/latest/miniconda.html).
-2. Open the "Anaconda Command Prompt," now available in the Start Menu, or at `%USERPROFILE%\miniconda3\shell\condabin\conda-hook.ps1`.
-3. In that command prompt window, use `cd` to change to the directory where you want to build OpenBLAS
-4. Now install all of the tools we need:
-
- ```
- conda update -n base conda
- conda config --add channels conda-forge
- conda install -y cmake flang clangdev perl libflang ninja
- ```
-
-5. Still in the Anaconda Command Prompt window, activate the MSVC environment for 64 bits with `vcvarsall x64`. On Windows 11 with Visual Studio 2022, this would be done by invoking:
-
- ```shell
- "c:\Program Files\Microsoft Visual Studio\2022\Preview\vc\Auxiliary\Build\vcvars64.bat"
- ```
-
- With VS2019, the command should be the same – except for the year number, obviously. For other/older versions of MSVC,
- the VS documentation or a quick search on the web should turn up the exact wording you need.
-
- Confirm that the environment is active by typing `link` – this should return a long list of possible options for the `link` command. If it just
- returns "command not found" or similar, review and retype the call to vcvars64.bat.
- **NOTE:** if you are working from a Visual Studio Command prompt window instead (so that you do not have to do the vcvars call), you need to invoke
- `conda activate` so that CONDA_PREFIX etc. get set up correctly before proceeding to step 6. Failing to do so will lead to link errors like
- libflangmain.lib not getting found later in the build.
-
-6. Now configure the project with CMake. Starting in the project directory, execute the following:
-
- ```
- set "LIB=%CONDA_PREFIX%\Library\lib;%LIB%"
- set "CPATH=%CONDA_PREFIX%\Library\include;%CPATH%"
- mkdir build
- cd build
- cmake .. -G "Ninja" -DCMAKE_CXX_COMPILER=clang-cl -DCMAKE_C_COMPILER=clang-cl -DCMAKE_Fortran_COMPILER=flang -DCMAKE_MT=mt -DBUILD_WITHOUT_LAPACK=no -DNOFORTRAN=0 -DDYNAMIC_ARCH=ON -DCMAKE_BUILD_TYPE=Release
- ```
-
- You may want to add further options in the `cmake` command here – for instance, the default only produces a static .lib version of the library. If you would rather have a DLL, add -DBUILD_SHARED_LIBS=ON above. Note that this step only creates some command files and directories, the actual build happens next.
-
-
-7. Build the project:
-
- ```
- cmake --build . --config Release
- ```
- This step will create the OpenBLAS library in the "lib" directory, and various build-time tests in the `test`, `ctest` and `openblas_utest` directories. However it will not separate the header files you might need for building your own programs from those used internally. To put all relevant files in a more convenient arrangement, run the next step.
-
-8. Install all relevant files created by the build
-
- ```
- cmake --install . --prefix c:\opt -v
- ```
- This will copy all files that are needed for building and running your own programs with OpenBLAS to the given location, creating appropriate subdirectories for the individual kinds of files. In the case of "C:\opt" as given above, this would be C:\opt\include\openblas for the header files,
- C:\opt\bin for the libopenblas.dll and C:\opt\lib for the static library. C:\opt\share holds various support files that enable other cmake-based build scripts to find OpenBLAS automatically.
-
-### Visual studio 2017+ (C++2017 standard)
-
-In newer visual studio versions, Microsoft has changed [how it handles complex types](https://docs.microsoft.com/en-us/cpp/c-runtime-library/complex-math-support?view=msvc-170#types-used-in-complex-math). Even when using a precompiled version of OpenBLAS, you might need to define `LAPACK_COMPLEX_CUSTOM` in order to define complex types properly for MSVC. For example, some variant of the following might help:
-
-```
-#if defined(_MSC_VER)
- #include
- #define LAPACK_COMPLEX_CUSTOM
- #define lapack_complex_float _Fcomplex
- #define lapack_complex_double _Dcomplex
-#endif
-```
-
-For reference, see https://github.com/xianyi/OpenBLAS/issues/3661, https://github.com/Reference-LAPACK/lapack/issues/683, and https://stackoverflow.com/questions/47520244/using-openblas-lapacke-in-visual-studio.
-
-### CMake and Visual Studio
-
-To build OpenBLAS for the 32-bit architecture, you'll need to use the builtin Visual Studio compilers.
-
-**[Notice]** This method may produce binaries which demonstrate significantly lower performance than those built with the other methods. (The Visual Studio compiler does not support the dialect of assembly used in the cpu-specific optimized files, so only the "generic" TARGET which is
-written in pure C will get built. For the same reason it is not possible (and not necessary) to use -DDYNAMIC_ARCH=ON in a Visual Studio build) You may consider building for the 32-bit architecture using the GNU (MinGW) ABI.
-
-#### 1. Install CMake at Windows
-
-#### 2. Use CMake to generate Visual Studio solution files
-
-```
-# Do this from Powershell so cmake can find visual studio
-cmake -G "Visual Studio 14 Win64" -DCMAKE_BUILD_TYPE=Release .
-```
-
-### Build the solution at Visual Studio
-
-Note that this step depends on perl, so you'll need to install perl for windows, and put perl on your path so VS can start perl (http://stackoverflow.com/questions/3051049/active-perl-installation-on-windows-operating-system).
-
-Step 2 will build the OpenBLAS solution, open it in VS, and build the projects. Note that the dependencies do not seem to be automatically configured: if you try to build libopenblas directly, it will fail with a message saying that some .obj files aren't found, but if you build the projects libopenblas depends on before building libopenblas, the build will succeed.
-
-### Build OpenBLAS for Universal Windows Platform
-
-OpenBLAS can be built for use on the [Universal Windows Platform](https://en.wikipedia.org/wiki/Universal_Windows_Platform) using a two step process since commit [c66b842](https://github.com/xianyi/OpenBLAS/commit/c66b842d66c5516e52804bf5a0544d18b1da1b44).
-
-#### 1. Follow steps 1 and 2 above to build the Visual Studio solution files for Windows. This builds the helper executables which are required when building the OpenBLAS Visual Studio solution files for UWP in step 2.
-
-#### 2. Remove the generated CMakeCache.txt and CMakeFiles directory from the OpenBLAS source directory and re-run CMake with the following options:
-
-```
-# do this to build UWP compatible solution files
-cmake -G "Visual Studio 14 Win64" -DCMAKE_SYSTEM_NAME=WindowsStore -DCMAKE_SYSTEM_VERSION="10.0" -DCMAKE_SYSTEM_PROCESSOR=AMD64 -DVS_WINRT_COMPONENT=TRUE -DCMAKE_BUILD_TYPE=Release .
-```
-
-#### Build the solution with Visual Studio
-
-This will build the OpenBLAS binaries with the required settings for use with UWP.
-
-## 2. GNU (MinGW) ABI
-
-The resulting library can be used in Visual Studio, but it can only be linked dynamically. This configuration has not been thoroughly tested and should be considered experimental.
-
-### Incompatible x86 calling conventions
-
-Due to incompatibilities between the calling conventions of MinGW and Visual Studio you will need to make the following modifications ( **32-bit only** ):
-
-1. Use the newer GCC 4.7.0. The older GCC (<4.7.0) has an ABI incompatibility for returning aggregate structures larger than 8 bytes with MSVC.
-
-
-### Build OpenBLAS on Windows OS
-1. Install the MinGW (GCC) compiler suite, either 32-bit (http://www.mingw.org/) or 64-bit (http://mingw-w64.sourceforge.net/). Be sure to install its gfortran package as well (unless you really want to build the BLAS part of OpenBLAS only) and check that gcc and gfortran are the same version – mixing compilers from different sources or release versions can lead to strange error messages in the linking stage. In addition, please install MSYS with MinGW.
-1. Build OpenBLAS in the MSYS shell. Usually, you can just type "make". OpenBLAS will detect the compiler and CPU automatically.
-1. After the build is complete, OpenBLAS will generate the static library "libopenblas.a" and the shared dll library "libopenblas.dll" in the folder. You can type "make PREFIX=/your/installation/path install" to install the library to a certain location.
-
-**[Notice]** We suggest using official MinGW or MinGW-w64 compilers. A user reported that s/he met `Unhandled exception` by other compiler suite. https://groups.google.com/forum/#!topic/openblas-users/me2S4LkE55w
-
-Note also that older versions of the alternative builds of mingw-w64 available through http://www.msys2.org may contain a defect that leads to a compilation failure accompanied by the error message
-```
-:0:4: error: expected identifier or '(' before numeric constant
-```
-If you encounter this, please upgrade your msys2 setup or see https://github.com/xianyi/OpenBLAS/issues/1503 for a workaround.
-
-### Generate import library (before 0.2.10 version)
-
-1. First, you will need to have the `lib.exe` tool in the Visual Studio command prompt.
-1. Open the command prompt and type `cd OPENBLAS_TOP_DIR/exports`, where OPENBLAS_TOP_DIR is the main folder of your OpenBLAS installation.
-1. For a 32-bit library, type `lib /machine:i386 /def:libopenblas.def`. For 64-bit, type `lib /machine:X64 /def:libopenblas.def`.
-1. This will generate the import library "libopenblas.lib" and the export library "libopenblas.exp" in OPENBLAS_TOP_DIR/exports. Although these two files have the same name, they are totally different.
-
-### Generate import library (0.2.10 and after version)
-1. OpenBLAS already generated the import library "libopenblas.dll.a" for "libopenblas.dll".
-
-### generate windows native PDB files from gcc/gfortran build
-Tool to do so is available at https://github.com/rainers/cv2pdb
-
-### Use OpenBLAS .dll library in Visual Studio
-1. Copy the import library (before 0.2.10: "OPENBLAS_TOP_DIR/exports/libopenblas.lib", 0.2.10 and after: "OPENBLAS_TOP_DIR/libopenblas.dll.a") and .dll library "libopenblas.dll" into the same folder(The folder of your project that is going to use the BLAS library. You may need to add the libopenblas.dll.a to the linker input list: properties->Linker->Input).
-1. Please follow the documentation about using third-party .dll libraries in MS Visual Studio 2008 or 2010. Make sure to link against a library for the correct architecture. For example, you may receive an error such as "The application was unable to start correctly (0xc000007b)" which typically indicates a mismatch between 32/64-bit libraries.
-
-**[Notice]** If you need CBLAS, you should include cblas.h in /your/installation/path/include in Visual Studio. Please read [this page](http://github.com/xianyi/OpenBLAS/issues/95).
-
-### Limitations
-* Both static and dynamic linking are supported with MinGW. With Visual Studio, however, only dynamic linking is supported and so you should use the import library.
-* Debugging from Visual Studio does not work because MinGW and Visual Studio have incompatible formats for debug information (PDB vs. DWARF/STABS). You should either debug with GDB on the command-line or with a visual frontend, for instance [Eclipse](http://www.eclipse.org/cdt/) or [Qt Creator](http://qt.nokia.com/products/developer-tools/).
diff --git a/docs/user_manual.md b/docs/user_manual.md
index 2efb0f16d..b5cd632f1 100644
--- a/docs/user_manual.md
+++ b/docs/user_manual.md
@@ -1,11 +1,3 @@
-##### Table of Contents
-[Compile the library](#compile-the-library)
-[Link the library](#link-the-library)
-[Code examples](#code-examples)
-[Troubleshooting](#troubleshooting)
-[BLAS reference manual](#blas-reference-manual)
-
-
## Compile the library
### Normal compile
* type `make` to detect the CPU automatically.
@@ -47,7 +39,7 @@ Example:
The default directory is /opt/OpenBLAS. Note that any flags passed to `make` during build should also be passed to `make install` to circumvent any install errors, i.e. some headers not being copied over correctly.
-For more information, please read [Installation Guide](Installation-Guide).
+For more information, please read [Installation Guide](install.md).
## Link the library
@@ -173,7 +165,7 @@ gcc -o time_dgemm time_dgemm.c /your/path/libopenblas.a
## Troubleshooting
-* Please read [Faq](https://github.com/xianyi/OpenBLAS/wiki/Faq) at first.
+* Please read [Faq](faq.md) at first.
* Please use gcc version 4.6 and above to compile Sandy Bridge AVX kernels on Linux/MingW/BSD.
* Please use Clang version 3.1 and above to compile the library on Sandy Bridge microarchitecture. The Clang 3.0 will generate the wrong AVX binary code.
* The number of CPUs/Cores should less than or equal to 256. On Linux x86_64(amd64), there is experimental support for up to 1024 CPUs/Cores and 128 numa nodes if you build the library with BIGNUMA=1.
@@ -184,4 +176,4 @@ gcc -o time_dgemm time_dgemm.c /your/path/libopenblas.a
If you want to understand every BLAS function and definition, please read [Intel MKL reference manual](https://software.intel.com/en-us/intel-mkl/documentation) or [netlib.org](http://netlib.org/blas/)
-Here are [OpenBLAS extension functions](OpenBLAS-Extensions)
\ No newline at end of file
+Here are [OpenBLAS extension functions](extensions.md)
diff --git a/mkdocs.yml b/mkdocs.yml
index fa486d208..8c20365ba 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -1,3 +1,29 @@
site_name: OpenBLAS
theme:
name: material
+ logo: logo.svg
+ favicon: logo.svg
+ palette:
+ primary: grey
+markdown_extensions:
+ - admonition
+ - pymdownx.details
+ - pymdownx.superfences
+ - toc:
+ toc_depth: 4
+nav:
+ - index.md
+ - install.md
+ - user_manual.md
+ - extensions.md
+ - developers.md
+ - build_system.md
+ - distributing.md
+ - ci.md
+ - machine_list.md
+ - todo.md
+ - ecosystem.md
+ - publications.md
+ - mailing_list.md
+ - donate.md
+ - faq.md