protologue

Data Contamination

Also called benchmark contamination, test set contamination, training data leakage.

Data contamination is the presence of benchmark or test examples in a model's training data, which inflates its measured performance because it may have memorized the answers.

Description

Since models are trained on large web crawls, public benchmarks often leak into training sets. Contamination can be detected, for example, by prompting a model to complete the beginning of test instances and checking for verbatim matches. For application evaluations, the safeguard is a private, regularly refreshed test set.

Sources

  1. Golchin and Surdeanu (2023). Time Travel in LLMs: Tracing Data Contamination in Large Language Models.

Cite this entry

Protologue. (2026). Data Contamination. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0178). https://protologue.com/t/data-contamination/

BibTeX
@misc{protologue_data_contamination,
  title = {Data Contamination},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0178},
  url = {https://protologue.com/t/data-contamination/}
}

Markdown JSON