protologue

GSM8K

Also called Grade School Math 8K.

GSM8K is a benchmark of about 8,500 grade-school math word problems that each take several steps of basic arithmetic to solve, used to measure multi-step mathematical reasoning.

Description

It was the main testbed for chain-of-thought, self-consistency, and many other reasoning techniques, and was introduced together with the idea of training verifiers to check solutions. Frontier models now solve nearly all of it, so it is saturated, but variants that add irrelevant information or change numbers still reveal brittleness.

Sources

  1. Cobbe et al. (2021). Training Verifiers to Solve Math Word Problems.

Cite this entry

Protologue. (2026). GSM8K. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0197). https://protologue.com/t/gsm8k/

BibTeX
@misc{protologue_gsm8k,
  title = {GSM8K},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0197},
  url = {https://protologue.com/t/gsm8k/}
}

Markdown JSON