protologue

Text Degeneration

Also called neural text degeneration, repetition, repetition loops.

Text degeneration is the tendency of language models, especially under greedy or beam-search decoding, to produce bland, repetitive, or looping text, such as the same phrase repeated over and over.

Description

Maximizing likelihood at each step favors safe, high-probability continuations that reinforce themselves once repetition starts. Sampling methods such as top-p and top-k, repetition or frequency penalties, and modern training largely mitigate it, but loops still appear in long generations and agent traces.

Sources

  1. Holtzman et al. (2019). The Curious Case of Neural Text Degeneration.

Cite this entry

Protologue. (2026). Text Degeneration. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0175). https://protologue.com/t/text-degeneration/

BibTeX
@misc{protologue_text_degeneration,
  title = {Text Degeneration},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0175},
  url = {https://protologue.com/t/text-degeneration/}
}

Markdown JSON