protologue

GPQA

Also called GPQA Diamond, Graduate-Level Google-Proof Q&A.

GPQA (Graduate-Level Google-Proof Q&A) is a benchmark of difficult multiple-choice questions in biology, physics, and chemistry, written by domain experts so that skilled non-experts cannot answer them reliably even with unrestricted web access.

Description

Its high-quality "Diamond" subset is the version most often reported in model announcements. Because the questions resist search, it measures expert-level reasoning more than retrieval. It is also used to study scalable oversight, the question of how people can supervise models that know more than they do.

Sources

  1. Rein et al. (2023). GPQA: A Graduate-Level Google-Proof Q&A Benchmark.

Cite this entry

Protologue. (2026). GPQA. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0189). https://protologue.com/t/gpqa/

BibTeX
@misc{protologue_gpqa,
  title = {GPQA},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0189},
  url = {https://protologue.com/t/gpqa/}
}

Markdown JSON