llama.cpp tunes CUDA kernel for Gated DeltaNet state updates
Release b11494 defaults GDN state updates to four columns per warp, a CUDA scheduling change in PR #30087.
The facts
- Release b11494 of llama.cpp merges PR #30087, 'ggml-cuda: assign four GDN state columns per warp'
- The commit history shows iteration: starting at two columns per warp, then four at S_v=128, before settling on a default of cols_per_warp=4
- A final commit addresses review nits from the GDN change
- The release ships binaries across macOS, Linux, Windows, Android and openEuler, with CUDA 12 and CUDA 13 builds for both x64 and arm64
llama.cpp's latest pre-release, b11494, changes how its CUDA backend handles Gated DeltaNet (GDN) state updates. Each warp is now assigned four state columns by default, instead of two. The change arrives via pull request #30087, titled 'ggml-cuda: assign four GDN state columns per warp.'
The commit log shows the project arrived at that number through testing. An initial version assigned two columns per warp. A follow-up moved to four at a specific state-value size, S_v=128. A final commit set cols_per_warp=4 as the default, before the team addressed review feedback.
A warp is the unit of 32 threads that a CUDA GPU schedules and executes together. How many columns of a state matrix each warp handles changes how much parallel work each group of threads does per step. Gated DeltaNet is a recurrent-style attention variant used in some newer model architectures. Its state update is a sequential bottleneck, and one that benefits from careful occupancy tuning on the GPU. The source text gives no benchmark numbers for the speedup, only the commit sequence that led to the new default.
The release otherwise follows llama.cpp's usual build matrix. CPU, Vulkan, CUDA 12.8, CUDA 13.4 and ROCm 10.0 builds cover Linux and Windows on x64 and arm64. There are also macOS, iOS, Android and openEuler packages, plus a Snapdragon-specific build covering CPU, Adreno GPU and Hexagon NPU.
Why it matters
Anyone self-hosting a GDN-based model, such as Qwen's recent linear-attention variants, on Nvidia GPUs via llama.cpp gets this scheduling change for free in the next pull. No serving configuration needs to change. But the practical gain is unquantified here, and worth benchmarking before assuming it matters for a given workload.
Primary source
llama.cpp releases — github.comWritten from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.