protologue

MMLU

Also called Massive Multitask Language Understanding.

MMLU (Massive Multitask Language Understanding) is a benchmark of four-option multiple-choice questions across 57 subjects, from elementary mathematics to law and medicine, that measures a model's breadth of knowledge and problem solving.

Description

It was the standard headline score for general knowledge for several years and is traditionally reported with five examples in the prompt (5-shot). Frontier models now score near the top, so it is considered saturated, and it has largely been succeeded by harder variants such as MMLU-Pro.

Sources

  1. Hendrycks et al. (2020). Measuring Massive Multitask Language Understanding.

Cite this entry

Protologue. (2026). MMLU. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0187). https://protologue.com/t/mmlu/

BibTeX
@misc{protologue_mmlu,
  title = {MMLU},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0187},
  url = {https://protologue.com/t/mmlu/}
}

Markdown JSON