llama.cpp patches Hexagon NPU attention bug in b11696

A single fix for Qualcomm's HMX flash-attention sinks ships in the latest build

Llama.cpp has pushed release b11696, a pre-release tagged on GitHub. It fixes a flash-attention bug on Qualcomm's Hexagon NPU. The change comes from pull request #30581, titled 'hexagon: correct HMX FA sinks'. It vectorises the sink handling inside the HMX kernels that the project uses for flash attention on Hexagon hardware. Max Krasnyansky of Qualcomm is listed as a co-author of the fix.

The release otherwise follows llama.cpp's usual pattern of shipping binaries across a wide matrix of platforms. These span macOS on Apple Silicon and Intel, and Linux builds for x64, arm64 and s390x. The Linux builds include CPU, Vulkan, CUDA, ROCm, OpenVINO and SYCL variants. Windows builds cover a similar spread, alongside openEuler images for both x86 and aarch64. Two build targets, a Linux arm64 image and an Android arm64 image, are built specifically for Snapdragon devices. These support CPU, Adreno GPU and Hexagon NPU, which is where the fixed kernel applies.

Flash attention lets transformer models compute attention without materialising the full attention matrix in memory. 'Sinks' refer to the attention-sink mechanism, which keeps early tokens resident to stabilise long-context generation. A bug in how HMX kernels handled sinks on Hexagon would have produced incorrect attention outputs. That fault sat specifically on Qualcomm's NPU path, rather than on the CPU, Vulkan or CUDA backends.

Written from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.