Benchmarks and evaluation: how are LLMs measured?

What is a language model benchmark?

A benchmark is a fixed set of tasks and a scoring method used to compare models under the same conditions. Massive Multitask Language Understanding (MMLU), introduced by Hendrycks et al. in 2020, is one widely used example. It covers 15,908 multiple-choice questions across 57 subjects, from elementary mathematics to professional law, scored as plain classification accuracy.

The point of a benchmark is to turn a vague question — is this model any good — into a number that can be tracked over time and compared across labs. Hendrycks et al. built MMLU because earlier tests such as GLUE and SuperGLUE had been solved within about a year of release. The paper says this suggested those benchmarks were not capturing much about real language understanding.

How do researchers run these tests?

MMLU evaluates models zero-shot or few-shot, meaning the model sees no training examples or only a handful before answering. It must rely on knowledge picked up during pretraining rather than fine-tuning on the test's own data. Hendrycks et al. fed GPT-3 a prompt describing the subject, up to five worked examples, and then the question. They read off which of four answer letters the model assigned the highest probability.

The Holistic Evaluation of Language Models project (HELM), published by Liang et al. in 2022, standardises this further across many models at once. HELM adapted all 30 models it tested through few-shot prompting with the same generic prompts. The aim was to make scores comparable head-to-head, rather than reflecting each lab's own favoured prompt format.

Accuracy alone, or more metrics?

MMLU reports a single number: classification accuracy, averaged across humanities, social science, STEM and other categories. HELM argues this is not enough. It measures seven categories of metric — accuracy, calibration, robustness, fairness, bias, toxicity and efficiency — for each of 16 core scenarios, covering 87.5% of the 112 possible scenario-metric pairs.

HELM's reasoning is that a model can be accurate and still unfair, uncalibrated, or costly to run. Benchmarks that report only accuracy let those trade-offs go unmeasured. Liang et al. found, for instance, that OPT (175B) answered toxicity-detection questions about Black-authored comments correctly 51.3% of the time. But its accuracy collapsed to 8.8% under robustness perturbations, a gap accuracy alone would have hidden.

Why do vendor scores need care?

Before HELM, Liang et al. found that models were on average evaluated on only 17.9% of the 16 core scenarios they later standardised. Some prominent models, such as T5 (11B) and Anthropic-LM v4-s3 (52B), shared not a single benchmark dataset in their original papers. That makes cross-model comparison from published numbers alone close to meaningless.

Scores are also highly sensitive to how a question is asked. HELM found OPT (175B) scored 79.1% on HellaSwag when each answer choice was scored in its own zero-shot prompt. But it scored only 30.2%, near random, when the choices were presented together in a five-shot format. Hendrycks et al. likewise found GPT-3's confidence on MMLU could be off from its actual accuracy by up to 24 percentage points — a model's stated certainty is not a reliable guide to whether it is right.

Neither paper in this reading set uses the word contamination to mean a model having memorised test answers during training. But HELM does flag the general risk directly. It states that models may differ in their exposure to the evaluation instances used, creating potential train-test overlap, and says plainly that the authors have only limited understanding of how contaminated the models they tested were.

What do these benchmarks actually test?

MMLU's 57 subjects split into four broad groups: humanities (law, philosophy, history), social science (economics, psychology, politics), STEM (physics, computer science, mathematics) and a catch-all "other" that includes professional medicine and business topics. Hendrycks et al. report that GPT-3's largest 175-billion-parameter variant reached 43.9% average accuracy in 2020. That is far above the 25% random baseline but well short of the roughly 89.8% the paper estimates as expert-level.

HELM's 16 core scenarios span six user-facing tasks — question answering, information retrieval, summarisation, sentiment analysis, toxicity detection and text classification — across domains such as news and books. Coverage is currently English only, though several English varieties are included. Liang et al. report that text-davinci-002 was the most accurate model across all nine of HELM's question-answering scenarios. Code-davinci-002 reached 52.1% on the GSM8K maths-reasoning benchmark, against 35.0% for the next-best model and under 16% for every other model tested.

What are the limits of these benchmarks?

Both papers report lopsided results rather than uniform competence. Hendrycks et al. found GPT-3's accuracy on MMLU ranged from 69% on US Foreign Policy to 26% on College Chemistry. Nine of its ten worst-scoring tasks were calculation-heavy STEM subjects, suggesting the model absorbs declarative facts more easily than procedure.

HELM surfaces a different kind of limit: metrics can pull apart. Liang et al. found no strong trade-off between accuracy and efficiency overall. But they did find that TNLG v2 (530B) dropped from 72.6% standard accuracy to 38.9% once robustness perturbations such as typos were applied on the NarrativeQA scenario. Scale also predicts accuracy only within a model family, not across different labs' models. A bigger parameter count on its own promises less than it seems to.