IBM packs chat, decisions and speech into one Granite endpoint
Granite Switch 4.1 adds adapters, not GPUs, to cover three task types
The facts
- Granite-Switch-4.1-8B and -30B bundle a new Decision aLoRA, a Guardian aLoRA for safety decisions, and Granite-Speech-5.0-Turbo-CTC for transcription, all in one endpoint
- IBM says the Guardian aLoRA outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Clef-Flash on safety-related decision tasks on average
- The combined model uses roughly the same VRAM as the base model alone because adapters share its KV cache (key-value cache, the memory used to avoid recomputing attention)
- IBM says Granite-Switch-4.1-30B completed a 5-step RAG workflow faster than Jev-27B in a head-to-head test, and a custom 3B Granite Switch model made 60-100 decisions per second playing Doom versus Jev's reported 10 decisions per second
IBM has released Granite Switch 4.1, a pair of models that handle natural-language chat, decision-making and speech-to-text from a single hosted endpoint. The 8B and 30B versions combine a base Granite model with three add-ons. These are a general-purpose Decision aLoRA, a Guardian aLoRA tuned for safety-related decisions, and Granite-Speech-5.0-Turbo-CTC. IBM describes the speech model as the fastest automatic speech recognition model on the OpenASR Leaderboard. IBM says the Guardian adapter beats Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Clef-Flash on safety decision tasks, on average.
The trick is memory. All three capabilities, plus the base model, run in roughly the same VRAM footprint as the base model alone. That is because the Switch adapters are built as Activated LoRAs, which reuse the KV cache from a single generation across adapter calls. IBM pairs that with recent contributions to vLLM, the open-source inference engine, meant to speed up low-rank adapter inferencing. A co-located ASR encoder rounds out the stack. The result, IBM says, is fast switching between a chat request, a decision request and a spoken request, with no need to reload a separate model for each.
IBM is also pitching composability over a single bundled product. Granite-Switch-4.1-8B and -30B ship with a default set of adapters. But IBM's Granite Libraries let anyone mix specialized classification adapters with the generalist decision adapter at close to no extra cost, since they share the base model's KV cache. In one test, IBM says Granite-Switch-4.1-30B, running four specialized aLoRAs atop a frozen base model, finished a real-world five-step retrieval-augmented-generation workflow faster than Jev-27B. That model pairs a frozen Qwen base with one generic decision LoRA. A separate 3B-parameter Granite Switch model trained to play Doom reportedly made 60 to 100 decisions per second, against a figure of 10 decisions per second IBM attributes to Jev. It also narrated the game and answered questions in real time. IBM plans to release an Adapter SDK for training new adapters in its Mellea framework in the coming weeks.
Why it matters
For teams running separate models for chat, moderation and transcription, IBM is betting adapter composition can replace that stack with one GPU footprint. The comparisons against Jev and other decision models are IBM's own, run on IBM's chosen tasks. They are worth independent verification before any migration.
Primary source
Hugging Face — huggingface.coWritten from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.