Inference and deploymentSpeculative decoding1 story. An explainer is being written.Speculative decoding in the news3d ago·llama.cpp releasesllama.cpp v0.7.0 adds MoE GPU cache, new default server portA new expert cache and port change reshape how MoE models get servedRelated topicsInference and servingQuantizationOn-device AIAI hardware and kernels