Commit Graph

778 Commits

Author SHA1 Message Date
Chen, Guobing
5d86becdae Add all SBGEMM kernels for IA AVX512-BF16 based platforms
Added all SBGEMM kernels including NN/NT/TN/TT for both ColMajor and
RowMajor, based on AVX512-BF16 ISA set on IA.

Signed-off-by: Chen, Guobing <guobing.chen@intel.com>
2021-08-05 11:11:29 +08:00
Wangyang Guo
fa777f5517 Small Matrix: skylakex: add DGEMM_SMALL_M_PERMIT and tune for TN kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
8592c21af4 Small Matrix: skylakex: dgemm nn: fix typo in idx load 2021-08-02 07:06:54 +00:00
Wangyang Guo
3e79f6d89a Small Matrix: skylakex: add dgemm tn kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
323d7da4f7 Small Matrix: skylakex: add dgemm tt kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
f57fc932ac Small Matrix: skylakex: add dgemm nt kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
91ec21202b Small Matrix: skylakex: add dgemm nn kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
72e070539c Small Matrix: skylakex: add sgemm tt kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
02c6e764f2 Small Matrix: skylakex: add SGEMM_SMALL_M_PERMIT and tune for TN kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
642c393879 Small Matrix: skylakex: add sgemm tn kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
ae3f5c737c Small Matrix: skylakex: sgemm nt: optimize for M < 12 2021-08-02 07:06:54 +00:00
Wangyang Guo
0d72d75bf9 Small Matrix: skylakex: add sgemm nt kernel 2021-08-02 07:06:54 +00:00
Wangyang Guo
ca7682e3a3 Small Matrix: skylakex: sgemm nn: fix n6 conflicts with n4 2021-08-02 07:06:54 +00:00
Wangyang Guo
9967e61abb Small Matrix: skylakex: sgemm nn: fix error when beta not zero 2021-08-02 07:06:54 +00:00
Wangyang Guo
a87736346f Small Matrix: skylakex: sgemm nn: add n6 to improve performance 2021-08-02 07:06:54 +00:00
Wangyang Guo
4c9d9940fd Small Matrix: skylakex: sgemm nn: reduce store 4 N at a time 2021-08-02 07:06:54 +00:00
Wangyang Guo
13b32f69b7 Small Matrix: skylakex: sgemm nn: reduce store 4 M at a time 2021-08-02 07:06:54 +00:00
Wangyang Guo
3d8c6d9607 Small Matrix: skylakex: sgemm nn: clean up unused code 2021-08-02 07:06:54 +00:00
Wangyang Guo
49b61a3f30 Small Matrix: skylakex: sgemm_nn: optimize for M <= 8 2021-08-02 07:06:54 +00:00
Wangyang Guo
f88470323b Optimize M < 16 using AVX512 mask 2021-08-02 07:06:54 +00:00
Wangyang Guo
9186456a12 small matrix: SkylakeX: add SGEMM NN kernel 2021-08-02 07:06:54 +00:00
Martin Kroeker
5b4b385ecf Temporarily disable the SkylakeX sgemv_t microkernel due to LAPACK testsuite failures 2021-07-14 20:50:14 +02:00
Ma, Yu
706a08d4a0 Optimized sgemv_t for small N based on AVX512 2021-06-08 15:08:28 -04:00
Martin Kroeker
5f677e782e Merge pull request #3196 from guowangy/skylakex-gemm-batch-k
GEMM: skylake: improve the performance when m is small
2021-05-22 19:25:28 +02:00
Martin Kroeker
02087a62e7 Merge pull request #3205 from intelmy/sgemv_n_opt
optimize on sgemv_n for small n
2021-05-17 17:49:01 +02:00
Martin Kroeker
8b90e5f202 Drop redundant inclusion of complex.h 2021-05-14 15:06:44 +02:00
Martin Kroeker
c0ca63ea46 Fix missing conditionals for non-SKX kernels 2021-05-05 14:55:36 +02:00
pnp
3d4ccd2a13 fix for build error 2021-04-30 12:25:33 -04:00
pnp
c59652f0ce optimize on sgemv_n for small n 2021-04-30 12:14:58 -04:00
Wangyang Guo
aa7b3dc3db GEMM: skylake: improve the performance when m is small 2021-04-28 13:56:06 +00:00
Martin Kroeker
3d511f0e66 replace spurious avx512 requirement with fma check 2021-04-26 21:55:30 +02:00
Martin Kroeker
2dfb24730d Use "old" compute(24) function with clang due to register limitations 2021-04-06 19:58:32 +02:00
Martin Kroeker
7b8f580941 Merge pull request #3156 from martin-frbg/omatcopy_d
Move x86_64 DOMATCOPY_RT back to the C implementation
2021-03-19 15:22:48 +01:00
Martin Kroeker
0f5e86a0d9 Remove premature entry for DOMATCOPY_RT 2021-03-18 21:53:50 +01:00
Martin Kroeker
7b294a99fd Move common.h back to the top of the file so that SKYLAKEX (from config.h) is defined in time 2021-03-18 21:28:19 +01:00
Martin Kroeker
0934568d9c Move includes under the ifdef for compilers w/o intrinsics support 2021-03-12 12:42:05 +01:00
Martin Kroeker
a9f6f7ad39 Remove spurious AVX512 requirement and add AVX2/FMA3 guard 2021-03-06 14:35:49 +01:00
Martin Kroeker
292d1af1a0 Update omatcopy_rt.c 2021-02-24 09:34:14 +01:00
Martin Kroeker
325b398e3c Update omatcopy_rt.c 2021-02-24 09:13:12 +01:00
Martin Kroeker
6f5667b4d4 Enable optimized S/D OMATCOPY_RT 2021-02-24 09:03:41 +01:00
Martin Kroeker
cceeee7806 Add optimized omatcopy_rt 2021-02-24 09:00:54 +01:00
Martin Kroeker
47691c031f Use Haswell optimizations for Zen as well 2021-02-11 09:26:15 +01:00
Martin Kroeker
ce7ddd8921 Use Haswell optimizations for Zen as well 2021-02-11 09:25:36 +01:00
Martin Kroeker
950c047b49 Use Haswell optimizations for Zen as well 2021-02-11 09:24:51 +01:00
Martin Kroeker
46509953a9 Use Haswell optimizations for Zen as well 2021-02-11 09:24:16 +01:00
Martin Kroeker
db348dcff2 Enable optimized srot/drot kernels from Haswell 2021-02-11 09:23:05 +01:00
Martin Kroeker
69a5558203 Merge pull request #3059 from Guobing-Chen/BF16_gemm
Initial code for Cooperlake BF16 GEMM kernel
2021-01-23 19:08:05 +01:00
Alex Henrie
202fc9e8ed Fix uninitialized argument value in dasum_k 2021-01-14 19:40:31 -07:00
Chen, Guobing
b0beb0b1ca Initial code for Cooperlake BF16 GEMM kernel 2021-01-11 02:15:21 +08:00
Martin Kroeker
114eb159a4 Disable FMA intrinsics in the srot kernel when the compiler is PGI/NVIDIA 2020-12-19 22:15:58 +01:00