protologue

Top-k Sampling

Also called top-k.

Top-k sampling is a decoding method that, at each step, restricts sampling to the k most probable next tokens and renormalizes their probabilities.

Description

It removes the long tail of unlikely tokens that can derail generation, while keeping some variety. Because k is fixed, it can be too restrictive when many tokens are plausible and too permissive when only one is, which motivated top-p (nucleus) sampling. Many APIs expose top-k alongside temperature and top-p.

Sources

  1. Fan et al. (2018). Hierarchical Neural Story Generation.
  2. Holtzman et al. (2019). The Curious Case of Neural Text Degeneration.

Cite this entry

Protologue. (2026). Top-k Sampling. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0108). https://protologue.com/t/top-k-sampling/

BibTeX
@misc{protologue_top_k_sampling,
  title = {Top-k Sampling},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0108},
  url = {https://protologue.com/t/top-k-sampling/}
}

Markdown JSON