# Speculative Decoding

> Speculative decoding is an inference technique in which a smaller, faster draft model proposes several next tokens and the large target model checks them in a single parallel pass, accepting or correcting them so that generation is faster while the output distribution stays the same as sampling from the target model alone.

- Identifier: PTL-0217
- Category: Foundations
- Canonical URL: https://protologue.com/t/speculative-decoding/
- Also known as: speculative sampling, assisted generation
- Introduced: 2022

## Description

Because the target model verifies several drafted tokens per forward pass instead of generating one token per pass, decoding needs fewer sequential calls to the large model. A sampling rule that accepts or rejects each drafted token is what preserves the target model's distribution. The speedup depends on how often the draft model's guesses are accepted. A closely related method was published shortly afterwards under the name speculative sampling.

## Related terms

- [KV Cache](https://protologue.com/t/kv-cache/)
- [Greedy Decoding](https://protologue.com/t/greedy-decoding/)
- [Knowledge Distillation](https://protologue.com/t/knowledge-distillation/)

## Sources

- Leviathan et al. (2022). Fast Inference from Transformers via Speculative Decoding. https://arxiv.org/abs/2211.17192
- Chen et al. (2023). Accelerating Large Language Model Decoding with Speculative Sampling. https://arxiv.org/abs/2302.01318

## Cite this entry

Protologue. (2026). Speculative Decoding. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0217). https://protologue.com/t/speculative-decoding/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)