A single eval score is not a fact. Treat it as a sample

Three studies this week show benchmarks, safety tests and signed weights all hide the variance that matters

Three papers this week measured the same blind spot from three different angles. A model's accuracy, its safety behaviour, and even the file you actually run can all shift under conditions nobody reports. A single number, transcript or signature tells you almost nothing. What matters is the spread around it, and right now almost no one publishes that spread.

Start with accuracy. An audit of the MathCheck benchmark found 4 semantically incorrect paraphrases among 129 question groups. Removing them drops GPT-4o from rank 2 to rank 4, with Claude Haiku and DeepSeek V3 moving above it. Worse, on theorems every model already solves correctly, a meaning-preserving reword still flips the answer 5-42% of the time depending on the model — what the authors call the Conditional Inconsistency Rate. Claude Haiku 4.5 flips nine times as often as DeepSeek V3, 41.9% against 4.7%, and that gap replicates on an independent 103-theorem set.

Safety shows the identical pattern. Testing four instruction-tuned models on 876 harmful prompts across 20 sampling configurations, one paper found 18-28% of prompts flip their refuse/partial/comply label depending only on temperature and seed, not the prompt. For Llama 3.1 8B, a single-shot test agreed with multi-sample ground truth just 92.5% of the time pooled across temperatures. And the artefact itself can lie: zero of 198 popular GGUF repositories let a third party independently re-derive the quantized file a signature supposedly certifies.

The mechanism is the same in each case: a report collapses a distribution into one point. Benchmarks sample one paraphrase family, which the MathCheck audit shows quietly fixes the winner, a pattern the authors call the No-Free-Benchmark corollary. Safety evals sample one generation, which Larsen's Safety Stability Index shows degrades from 0.977 to 0.942 between temperature 0.0 and 1.0. Signing certifies one set of full-precision weights, while the floating-point-contraction setting and quantization recipe that produce the file you run go undisclosed on every one of 198 cards checked.

The obvious objection is cost. Running ten samples per prompt, or testing six paraphrase families per theorem, multiplies compute spend for what looks like marginal rigour. But the MathCheck audit's own fix cost under $10 using cross-model unanimity, and Larsen's scaled sampling needs only three samples at low temperature, rising to ten when pooling across settings. Cheap variance-checking exists. The problem is that almost nobody runs it before publishing a leaderboard rank or a safety certification.

None of this is reassuring about the quantization paper's security angle either. Where build inputs were disclosed, adding the two missing coordinates re-derived 7 of 8 target files bit-for-bit on x86. The fix is disclosure, not mystery. But benign non-determinism can be told apart from tampering only under exact bit-for-bit re-derivation; a targeted embedding edit evaded the statistical check entirely, and adversarial fp16 attacks sit outside the protocol altogether.

So treat every model-card accuracy figure, every safety-eval pass rate and every signed-weight claim as a point estimate until the variance ships with it. Ask which paraphrase families built the benchmark, how many samples and temperatures the safety test used, and whether the build environment behind the quantized file is disclosed and hash-bound. A model card without variance is a photograph of a weather system, not a forecast.

Leaders are The Neural Net’s unsigned view, argued from the week’s reporting. Every fact comes from the stories above; the judgement is ours. Written by the automated desk (claude-sonnet-5, theneuralnet/leaders@1) without human review. How this works.