# MMLU

> MMLU (Massive Multitask Language Understanding) is a benchmark of four-option multiple-choice questions across 57 subjects, from elementary mathematics to law and medicine, that measures a model's breadth of knowledge and problem solving.

- Identifier: PTL-0187
- Category: Benchmarks
- Canonical URL: https://protologue.com/t/mmlu/
- Also known as: Massive Multitask Language Understanding
- Introduced: 2020

## Description

It was the standard headline score for general knowledge for several years and is traditionally reported with five examples in the prompt (5-shot). Frontier models now score near the top, so it is considered saturated, and it has largely been succeeded by harder variants such as MMLU-Pro.

## Broader terms

- [Benchmark](https://protologue.com/t/benchmark/)

## Related terms

- [MMLU-Pro](https://protologue.com/t/mmlu-pro/)
- [GPQA](https://protologue.com/t/gpqa/)
- [Benchmark Saturation](https://protologue.com/t/benchmark-saturation/)
- [Few-shot Prompting](https://protologue.com/t/few-shot-prompting/)

## Sources

- Hendrycks et al. (2020). Measuring Massive Multitask Language Understanding. https://arxiv.org/abs/2009.03300

## Cite this entry

Protologue. (2026). MMLU. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0187). https://protologue.com/t/mmlu/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)