# Top-k Sampling vs Top-p Sampling

> Top-k sampling always restricts each next token to the k most probable candidates. Top-p (nucleus) sampling instead keeps the smallest set of candidates whose probabilities add up to p, so the number of candidates grows when many tokens are plausible and shrinks when one is clearly best.

- Canonical URL: https://protologue.com/compare/top-k-vs-top-p-sampling/

## Top-k Sampling

Top-k sampling is a decoding method that, at each step, restricts sampling to the k most probable next tokens and renormalizes their probabilities.
Entry: https://protologue.com/t/top-k-sampling/

## Top-p Sampling

Top-p sampling, also called nucleus sampling, draws each next token only from the smallest set of candidates whose cumulative probability exceeds a threshold p.
Entry: https://protologue.com/t/top-p-sampling/

## Differences

| | Top-k Sampling | Top-p Sampling |
|---|---|---|
| Cutoff | A fixed number of tokens | A probability mass |
| Candidate set size | Constant | Varies with the model's confidence |
| Weakness | Too narrow or too wide depending on the context | Can still include unlikely tokens when the distribution is flat |

## Which to use

- Top-k is a simple guard against very unlikely tokens.
- Top-p adapts to the model's confidence; either is usually combined with temperature.

## Sources

- Fan et al. (2018). Hierarchical Neural Story Generation. https://arxiv.org/abs/1805.04833
- Holtzman et al. (2019). The Curious Case of Neural Text Degeneration. https://arxiv.org/abs/1904.09751
- Holtzman et al. (2019). The Curious Case of Neural Text Degeneration. https://arxiv.org/abs/1904.09751

## Cite this page

Protologue. (2026). Top-k Sampling vs Top-p Sampling. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0). https://protologue.com/compare/top-k-vs-top-p-sampling/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)