New method quantizes Llama 3.2 11B Vision to 2.7 bits for phones

Researchers compress an 11B-parameter vision-language model to 3.7GB for Arm CPUs without needing the original training setup.

Three researchers have published a quantization method that shrinks Meta's Llama 3.2 11B Vision Instruct model to 3.7 gigabytes. That is small enough to run inference on the Arm CPUs found in ordinary phones. The paper, titled 'Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs,' describes a pipeline that compresses the vision-language model to an average of 2.7 bits per parameter. Activations stay at 8 bits.

The method's distinguishing feature is that it does not need access to the model's original training setup. Most deployers of open-weight models never see the data or code that produced them. So the pipeline works around that gap by using the model itself to generate training data for the quantization process. The authors say this preserves strong performance on a set of standard visual question answering tasks. But the paper does not name the benchmarks or give specific accuracy figures in the abstract.

The format is built specifically for efficient execution on Arm CPUs, the architecture that dominates mobile silicon. It is not built for GPUs or specialised accelerators. At under 4 gigabytes, the quantized model would fit within the RAM and storage budgets of current flagship phones. That size class previously required either a smaller base model or a less aggressive compression scheme.

Written from the primary source above by The Neural Net’s automated desk (claude-sonnet-5, theneuralnet/pipeline@1) without human review. How this works. Corrections: check the source first — if we got it wrong, the source link is the record.