protologue

LiveCodeBench

LiveCodeBench is a coding benchmark that continuously collects new problems from programming contests, such as LeetCode, AtCoder, and Codeforces, so that models can be evaluated only on problems published after their training data was collected.

Description

Dating each problem lets evaluators detect and avoid data contamination. Besides code generation, it covers self-repair, predicting the output of code, and predicting test outputs.

Sources

  1. Jain et al. (2024). LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code.

Cite this entry

Protologue. (2026). LiveCodeBench. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0201). https://protologue.com/t/livecodebench/

BibTeX
@misc{protologue_livecodebench,
  title = {LiveCodeBench},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0201},
  url = {https://protologue.com/t/livecodebench/}
}

Markdown JSON