Meta releases Muse Spark Open, an 816B-parameter MoE model

Open weights, four modalities, a million-token window — but 1.5TB to serve

Meta has published Muse Spark Open, an open-weight mixture-of-experts model. It reads text, images, audio and video and answers in text. The model carries about 816 billion total parameters. It activates only around 42 billion per token, routing each token through 8 of 256 experts per layer. Hugging Face, which hosts the release, calls it Meta's most capable US open-weights model to date. That follows Muse Glimmer, a smaller dense model from the same family released in August.

Meta says it trained the model specifically for agentic coding. That means reading a codebase, writing and debugging across multiple files, and calling tools over long sessions. Those sessions can include browsing the web with images. The context window reaches up to 1,048,576 tokens and spans text, image, video and audio tokens together. Hugging Face notes that a full 1M-token context demands roughly 266GB of KV cache, even with grouped-query attention keeping memory down. Reasoning cannot be switched off entirely. The lowest of four effort settings, mapped to the integers 16, 64, 256 and 512, is the floor for how much the model will think before answering.

The two released checkpoints trade memory for speed in different ways. The BF16 version runs to about 1.5 terabytes in safetensors format. It needs four nodes of 8 Nvidia H200 GPUs under tensor parallelism. The NVFP4 version compresses weights to 4-bit precision, which Hugging Face says comes with negligible accuracy loss, and fits on a single Nvidia B200 node. Separate encoders handle vision and audio: a 1.8B-parameter perception encoder processes 14x14 image patches, and a convolutional tokenizer emits one token every 80 milliseconds, or 12.5 tokens per second. Meta provides inference recipes for transformers, llama.cpp, vLLM and SGLang.

All 256 experts must stay resident in memory regardless of how many run per token. Deployment still needs the footprint of an 816B-parameter model, even though compute cost tracks closer to 42B. That split is the real story for anyone deciding whether to self-host. The NVFP4 path puts a frontier-scale, four-modality model within reach of a single high-end GPU node. But that only holds if the operator already has a B200 to put it on.

Written from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.