OpenBMB ships quantized MiniCPM-V 4.7, its first MoE vision model
A 35-billion-parameter model that runs at 13+ tokens a second on an iPhone
The facts
- This AWQ (W4A16) quantized release packages OpenBMB's MiniCPM-V 4.7 35B-A3B, a mixture-of-experts model that is the first MoE variant in the MiniCPM-V series
- OpenBMB says the model decodes at 13+ tokens per second on an iPhone 17 Pro Max via a technique it calls Lookahead Expert Scheduling, combined with fine-grained layer-wise quantization and DSpark speculative sampling
- The company claims the 35B-A3B variant scores higher overall in Instruct mode than Qwen3.6 35B-A3B while using less inference time
- The model supports iOS, Android and HarmonyOS, with day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD and Arm, and ships in GGUF, BNB, AWQ, GPTQ and new FP8 quantized variants
OpenBMB has released a quantized AWQ version of MiniCPM-V 4.7 35B-A3B. It is a 35-billion-parameter vision-language model that activates only 3 billion parameters per token, using a mixture-of-experts design.
It is the first MoE model in the MiniCPM-V family. OpenBMB says it is also the first open-source MoE model to run smoothly on-device without sacrificing capability. On an iPhone 17 Pro Max, the company reports a decoding speed of more than 13 tokens per second.
The on-device result rests on three techniques stacked together. Lookahead Expert Scheduling predicts which experts the next layer will need and preloads them, overlapping that loading with computation on experts already fetched. Fine-grained layer-wise quantization shrinks the memory and bandwidth footprint of expert weights, and DSpark speculative sampling adds further decoding speedup on top. OpenBMB frames this as solving a core MoE deployment problem: devices cannot hold every expert in memory. Naive approaches either stall on repeated loading or restrict routing in ways that cut capability.
On benchmarks, OpenBMB says the 35B-A3B variant in Instruct mode beats Qwen3.6 35B-A3B on overall score while taking less inference time. The company describes this as advancing the performance-efficiency Pareto frontier for multimodal LLMs. The model also uses an intra-ViT early compression technique from LLaVA-UHD v4 that trims visual tokens inside the vision encoder. It supports mixed 4x/16x visual token compression, so developers can trade detail for speed.
It ships with day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD and Arm. It works with SGLang, vLLM, llama.cpp and Ollama, plus fine-tuning via SWIFT and LLaMA-Factory. Quantized formats include GGUF, BNB, AWQ, GPTQ and a newly added FP8 variant. Alongside these ships a fine-tuning method OpenBMB calls Self-Replay Fine-Tuning, meant to extend capabilities cheaply without forgetting existing ones.
Why it matters
A 35B MoE vision model claiming real-time decoding on a phone, if the figures hold outside OpenBMB's own tests, lowers the bar for running serious multimodal models without a server. Anyone building offline or edge vision apps now has a concrete on-device MoE reference to benchmark against, rather than just dense small models.
Written from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.