protologue

Irrelevant Context Distraction

Also called distractibility, GSM-IC.

Irrelevant context distraction is the drop in a model's accuracy when a prompt contains information that is unrelated to the question, such as an extra sentence in a math word problem, which the model wrongly incorporates into its reasoning.

Description

A benchmark built by adding irrelevant sentences to grade-school math problems showed large accuracy drops across prompting methods. Instructing the model to ignore irrelevant information, using self-consistency, and filtering context first, as in System 2 Attention, all reduce the effect. It matters for RAG, where retrieved passages often include off-topic text.

Sources

  1. Shi et al. (2023). Large Language Models Can Be Easily Distracted by Irrelevant Context.

Cite this entry

Protologue. (2026). Irrelevant Context Distraction. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0173). https://protologue.com/t/irrelevant-context-distraction/

BibTeX
@misc{protologue_irrelevant_context_distraction,
  title = {Irrelevant Context Distraction},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0173},
  url = {https://protologue.com/t/irrelevant-context-distraction/}
}

Markdown JSON