# MMLU-Pro

> MMLU-Pro is a harder, reasoning-focused successor to MMLU that offers ten answer options instead of four and replaces trivial or noisy questions with more challenging ones.

- Identifier: PTL-0188
- Category: Benchmarks
- Canonical URL: https://protologue.com/t/mmlu-pro/
- Introduced: 2024

## Description

More options lower the score obtainable by guessing, and the harder questions make chain-of-thought prompting matter more than it does on MMLU. Its creators also reported that scores vary less with the wording of the prompt than on the original benchmark.

## Broader terms

- [Benchmark](https://protologue.com/t/benchmark/)

## Related terms

- [MMLU](https://protologue.com/t/mmlu/)
- [GPQA](https://protologue.com/t/gpqa/)
- [Chain-of-Thought Prompting](https://protologue.com/t/chain-of-thought/)
- [Prompt Sensitivity](https://protologue.com/t/prompt-sensitivity/)

## Sources

- Wang et al. (2024). MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark. https://arxiv.org/abs/2406.01574

## Cite this entry

Protologue. (2026). MMLU-Pro. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0188). https://protologue.com/t/mmlu-pro/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)