llama.cpp patches server bug that corrupted busy slot's context
A slot-pinning bug could silently swap context mid-generation
The facts
- Release b11552 of llama.cpp fixes server bug tracked as pull request #30295.
- A request pinning a busy id_slot could trigger a prompt-cache update on that slot before being deferred.
- If the RAM cache held a better match, it loaded into the slot while another request was still generating, continuing on the wrong context.
- The fix makes the server return a busy slot as is and have the new request wait, rather than updating the cache early.
Llama.cpp's latest build, tagged b11552, fixes a bug in the project's server. It let a busy inference slot be silently overwritten mid-generation.
The server assigns each in-flight request a slot that holds its prompt cache. A new request asking for a specific slot ID that was still busy could trigger a prompt-cache update on that slot anyway. This happened if the RAM cache held a better-matching prompt. The new data loaded in while the original request kept generating, according to the release notes.
The practical effect: an in-progress generation could continue on a context swapped out from under it mid-stream. This produced output tied to the wrong prompt. The fix, merged as pull request #30295, makes the server leave a busy slot untouched. It queues the new request to wait instead of touching the cache early.
Llama.cpp ships binaries for this fix across its usual matrix of platforms. These include Linux, Windows and macOS builds with CUDA, ROCm, Vulkan, SYCL and OpenVINO backends, plus Android and openEuler targets. No other changes are listed in this release.
Why it matters
Anyone running llama.cpp's server for multi-request serving should update. The old behaviour could silently corrupt in-flight generations without erroring. It is a correctness bug, not a performance one, so no benchmark changes accompany it.
Primary source
llama.cpp releases — github.comWritten from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.