vLLM 0.31.0 adds fast-restart weight caching and new DeepSeek optimisations
The release adds a preload daemon that keeps quantized weights resident across restarts, plus deep DeepSeek-V4.1-Flash tuning.
The facts
- 717 commits from 307 contributors (96 new) went into vLLM v0.31.0
- New `vllm preload` CLI launches a weight-cache daemon keeping post-quantized weights resident in GPU memory across engine restarts, now with data parallelism, MTP draft models, a /health endpoint and a readiness wait
- Experimental `vllm snapshot create/restore` uses CRIU to restore a fully initialized TP1 engine
- FlashMLA mega attention with NVFP4 compressed KV cache is now the SM100 default for DeepSeek-V4.1-Flash
- New scheduling flag --max-num-active-seqs caps RUNNING admission independently of max_num_seqs
- Security fixes: per-request mm_processor_kwargs and media_io_kwargs now rejected unless --trust-request-mm-kwargs is set; LoRA path added to block hash to prevent cache key collisions
- Breaking changes include removal of tokenizer_mode='slow', renaming of --enable-mamba-fine-grained-prefix-cache, removal of quantization='fp8' in favor of fp8_per_tensor shorthand, and removal of the AllSpark INT8 W8A16 backend
vLLM's v0.31.0 release shipped on 5 October. It carries 717 commits from 307 contributors. The release centres on cutting the cost of restarting a serving engine. A new `vllm preload` CLI launches a weight-cache daemon. It keeps post-quantized weights resident in GPU memory across restarts. The daemon now works with data parallelism, MTP (multi-token prediction) draft models, a health-check endpoint and a readiness wait. There is also an experimental companion feature. `vllm snapshot create/restore` uses CRIU, a Linux checkpoint/restore tool. It brings back a fully initialized tensor-parallel-1 engine from a saved state, rather than re-initialising from scratch.
The release also deepens vLLM's tuning for DeepSeek-V4.1-Flash, which the project calls out as a highlight. FlashMLA attention paired with an NVFP4 compressed KV cache is now the default attention path on Nvidia's SM100, or Blackwell-class, GPUs. It arrives alongside DeepGEMM sparse MQA logits for the indexer, and a 'Mega-Gate' that fuses the gate GEMM with expert selection. Fused decoder boundary operations now combine the tensor-parallel all-reduce, mHC input preparation and MoE finalize steps into single kernels. Model Runner V2 gains draft-model speculative decoding and custom logits processors. Large-scale serving gets a MoonEP balanced expert-parallel all2all backend, plus prefill context parallelism combined with data parallelism. A sharding-aware NCCL weight-transfer backend rounds out the changes, aimed at reinforcement learning workloads.
Three security fixes are worth flagging for anyone running multi-tenant inference. Per-request `mm_processor_kwargs` and `media_io_kwargs` are now rejected by default. An operator must explicitly set `--trust-request-mm-kwargs` to allow them. Prefix-cache keys are now tagged by source. A LoRA adapter name and a cache_salt value can no longer collide. The LoRA path itself is folded into the block hash. This closes a path where mismatched cache entries could leak between requests.
The release also carries several breaking changes operators should check before upgrading. `tokenizer_mode="slow"` is removed. `--enable-mamba-fine-grained-prefix-cache` is renamed to `--enable-mamba-shared-prefix-checkpoint`. The `quantization="fp8"` option is replaced by an `fp8_per_tensor` shorthand. Quark's silent online quantization and the AllSpark INT8 W8A16 backend are both removed. `--enforce-eager` now also disables JIT kernel warmup. And XPU graphs are enabled by default, with the `VLLM_XPU_ENABLE_XPU_GRAPH` flag gone entirely.
Why it matters
For teams self-hosting large MoE models like DeepSeek-V4.1-Flash on Blackwell GPUs, the fused kernels and NVFP4 KV cache defaults mean meaningful throughput gains without code changes. But the removed flags and renamed options mean config files need a pass before upgrading. Engines serving large MoE models can take minutes to reload weights after any restart. That idle time is now something operators can design around, rather than just absorb.
Primary source
vLLM releases — github.comWritten from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.