# Prompt Caching vs KV Cache

> The KV cache stores attention keys and values for the tokens already processed within a single request, so each new token does not reprocess the whole sequence. Prompt caching keeps that processed state for a shared prompt prefix across requests, so later calls that begin with exactly the same content are cheaper and faster.

- Canonical URL: https://protologue.com/compare/prompt-caching-vs-kv-cache/

## Prompt Caching

Prompt caching stores the model's processed state for a reused prompt prefix, such as a long system prompt or document, so later requests sharing that prefix are cheaper and faster.
Entry: https://protologue.com/t/prompt-caching/

## KV Cache

The KV cache stores the attention keys and values a Transformer has already computed for earlier tokens, so that generating each new token requires computing only that token's keys and values instead of reprocessing the whole sequence.
Entry: https://protologue.com/t/kv-cache/

## Differences

| | Prompt Caching | KV Cache |
|---|---|---|
| Scope | Across requests | Within one request |
| Who controls it | The developer, by structuring prompts and marking breakpoints | The model server, automatically |
| Requirement | An identical prompt prefix | None |
| Benefit | Lower price and latency for repeated context | Fast token-by-token generation |

## Which to use

- Use prompt caching by putting stable content such as system prompts and documents first.
- The KV cache is always in use; it matters when sizing memory for long contexts and many concurrent requests.

## Sources

- Anthropic (2024). Prompt caching. https://platform.claude.com/docs/en/build-with-claude/prompt-caching
- Kwon et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. https://arxiv.org/abs/2309.06180
- Anthropic (2024). Prompt caching. https://platform.claude.com/docs/en/build-with-claude/prompt-caching

## Cite this page

Protologue. (2026). Prompt Caching vs KV Cache. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0). https://protologue.com/compare/prompt-caching-vs-kv-cache/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)