001 / vek
SIMD vector kernels
Dot product, L2 and cosine, hand-written across SSE2, AVX2, AVX-512 and NEON. One binary picks the widest path the CPU has, with the scalar path always compiled underneath it.
Measured, not marketed
- 11.0 ns
- dot product, n=128i5-1135G7, AVX-512; make bench on your CPU
- 46.3
- GFLOP/s at n=1024single core, same machine
- 5
- dispatch pathsSSE2, AVX2, AVX-512, NEON, scalar
- 0
- dependenciesone header, one source tree
01One binary, every machine
The library is compiled once. At startup it asks the CPU what it can do and points the function table at the widest path available — AVX-512 with VNNI and VPOPCNTDQ, AVX2 with FMA, NEON, or SSE2 as the x86 baseline. Nothing is chosen by build flag, so the same artifact runs everywhere and a distribution does not need a per-machine build.
Initialization is a lazy atomic, so no call-site has to remember vek_init(): the first call from any thread takes the acquire side, everyone else pays a load.
02Correctness sits under the speed
The scalar implementation is always compiled and is the reference the SIMD paths are tested against, on every backend, with a tolerance rather than an exact match because floating-point reassociation is real. f32 dot products use Neumaier compensated summation, which stops a long stream of products from cancelling into noise.
Nothing in the hot path allocates. Kernels read the caller's buffers and write a result; the memory behaviour is the caller's choice, not a hidden allocation.
03What is covered
Three similarity operations — dot, squared L2 and cosine — over four storage formats: f32, f16 and bf16 for floating point, int8/uint8 for quantized vectors, and 1-bit packed vectors using popcount. The 1-bit set adds Hamming distance alongside dot and cosine.
The API is extern "C" and the header is the documentation. Bindings ship for the languages that need them; anything that can call C can call this.
—What it does not do
- These are the kernels, not a vector store: no index, no storage, no persistence. The caller owns the data and the search structure.
- The benchmark table is one machine (Intel i5-1135G7). make bench on the hardware that matters is the only number worth quoting.
- ARM64's baseline is scalar plus NEON; there is no AVX-512 equivalent to dispatch to, so the widest path there is narrower.