# Benchmark Saturation

> Benchmark saturation is when the best models score near a benchmark's maximum, so that it no longer distinguishes between them and its remaining errors mostly reflect flawed or ambiguous questions.

- Identifier: PTL-0186
- Category: Benchmarks
- Canonical URL: https://protologue.com/t/benchmark-saturation/
- Also known as: saturated benchmark

## Description

Saturation has followed every widely used benchmark, often within a few years, and is a main reason new, harder benchmarks keep appearing. MMLU-Pro and Humanity's Last Exam were both created because earlier benchmarks had saturated. Contamination can make a benchmark look saturated before models have truly mastered it.

## Broader terms

- [Benchmark](https://protologue.com/t/benchmark/)

## Related terms

- [Data Contamination](https://protologue.com/t/data-contamination/)
- [MMLU](https://protologue.com/t/mmlu/)
- [Humanity's Last Exam](https://protologue.com/t/humanitys-last-exam/)

## Sources

- Phan et al. (2025). Humanity's Last Exam. https://arxiv.org/abs/2501.14249
- Wang et al. (2024). MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. https://arxiv.org/abs/2406.01574

## Cite this entry

Protologue. (2026). Benchmark Saturation. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0186). https://protologue.com/t/benchmark-saturation/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)