# Humanity's Last Exam

> Humanity's Last Exam (HLE) is a benchmark of 2,500 expert-written questions across dozens of academic subjects, some with images, designed to remain hard for frontier models after earlier academic benchmarks saturated.

- Identifier: PTL-0190
- Category: Benchmarks
- Canonical URL: https://protologue.com/t/humanitys-last-exam/
- Also known as: HLE
- Introduced: 2025

## Description

Questions were contributed by subject experts and kept only if leading models failed them at the time, and answers are short and automatically checkable. Its creators also measure calibration, how well a model's stated confidence matches its accuracy, because models often answer the questions they get wrong with high confidence.

## Broader terms

- [Benchmark](https://protologue.com/t/benchmark/)

## Related terms

- [Benchmark Saturation](https://protologue.com/t/benchmark-saturation/)
- [GPQA](https://protologue.com/t/gpqa/)
- [Confidence Calibration](https://protologue.com/t/confidence-calibration/)

## Sources

- Phan et al. (2025). Humanity's Last Exam. https://arxiv.org/abs/2501.14249

## Cite this entry

Protologue. (2026). Humanity's Last Exam. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0190). https://protologue.com/t/humanitys-last-exam/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)