protologue

Humanity's Last Exam

Also called HLE.

Humanity's Last Exam (HLE) is a benchmark of 2,500 expert-written questions across dozens of academic subjects, some with images, designed to remain hard for frontier models after earlier academic benchmarks saturated.

Description

Questions were contributed by subject experts and kept only if leading models failed them at the time, and answers are short and automatically checkable. Its creators also measure calibration, how well a model's stated confidence matches its accuracy, because models often answer the questions they get wrong with high confidence.

Sources

  1. Phan et al. (2025). Humanity's Last Exam.

Cite this entry

Protologue. (2026). Humanity's Last Exam. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0190). https://protologue.com/t/humanitys-last-exam/

BibTeX
@misc{protologue_humanitys_last_exam,
  title = {Humanity's Last Exam},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0190},
  url = {https://protologue.com/t/humanitys-last-exam/}
}

Markdown JSON