llama.cpp b11690 trims memory copies in speculative decoding cache
A single-line fix, but it is the kind that adds up at scale
The facts
- llama.cpp pre-release b11690 was published on 16 October 2026, built from commit 13b8048
- The release's sole listed change is PR #30061: "common: read ngram cache parts by reference instead of copying"
- Build artefacts cover macOS, Linux (including CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL, s390x and Snapdragon variants), Android, Windows and openEuler targets
- The commit carries a verified GitHub signature under GPG key ID B5690EEEBB952194
llama.cpp has shipped pre-release b11690, a one-commit update centred on how the project's n-gram cache is read during inference. The change is tracked as pull request #30061. It switches the common code path to read n-gram cache parts by reference rather than copying them. That cuts an avoidable memory operation out of a routine that supports speculative decoding and lookup-based generation.
The n-gram cache in llama.cpp stores short sequences of tokens seen during generation. That lets the engine guess ahead and verify guesses in bulk, a technique that speeds up text generation without changing its output. Copying cache parts on every read wastes CPU cycles and memory bandwidth. Reading by reference avoids that duplication. The source text gives no benchmark figures for the speedup, so the practical gain is not quantified here.
As with every tagged build, b11690 ships prebuilt binaries across a wide matrix of platforms. These include macOS on Apple Silicon and Intel, iOS, and several flavours of Linux. The Linux builds cover Ubuntu on x64, arm64 and s390x, with CPU, Vulkan, CUDA 12 and 13, ROCm, OpenVINO and SYCL backends. The matrix also covers Android, Windows across CPU, OpenCL, CUDA and Vulkan, and openEuler on x86 and aarch64. The commit was built and signed through GitHub's verified-signature process, tying the release to GitHub's own infrastructure.
Why it matters
The fix itself is narrow, but it is the kind of micro-optimisation that compounds for anyone running high-throughput inference with speculative decoding. Teams self-hosting llama.cpp at scale should keep pulling these incremental builds rather than pinning to an older tag. The gains arrive one small PR at a time, not in a single headline release.
Primary source
llama.cpp releases — github.comWritten from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.