llama.cpp's WebGPU backend adds quantised matrix-vector support

Release b11406 extends WebGPU's MMVQ path to six more quantisation formats, including MXFP4.

llama.cpp has tagged a new build, b11406. It extends the project's WebGPU backend to run matrix-vector multiplication, or MMVQ, against six more quantisation formats: Q1_0, Q5_0, Q5_1, Q3_K, Q5_K, Q6_K and MXFP4.

The change landed in PR #29483. It also adds a K_QUANTS_HANDLING macro to the Q1_0 path, bringing it in line with the other K-quant formats already supported.

MMVQ multiplies a weight matrix stored in a compressed, quantised format against an activation vector during inference. It does this without first decompressing the whole matrix to full precision. Before this release, WebGPU users running llama.cpp in the browser or on WebGPU-capable devices were limited to a narrower set of quantisation formats for that path. The addition of MXFP4, a 4-bit floating-point format gaining traction for low-memory inference, matters most. It lets WebGPU backends serve models quantised to that format without falling back to a slower, unoptimised multiplication path.

The release otherwise follows llama.cpp's usual cadence. Prebuilt binaries cover macOS on Apple Silicon and Intel, iOS, and Linux across x64, arm64 and s390x with CUDA 12.8 and 13.4, ROCm 10.0, Vulkan, SYCL and OpenVINO variants. They also cover Android, including Snapdragon NPU and GPU builds, Windows on x64 and arm64 with CUDA, Vulkan, OpenCL and ROCm variants, and openEuler on both x86 and aarch64.

Written from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.