protologue

Sleep-time Compute

Also called sleep-time computation.

Sleep-time compute is computation a language model performs on a context before any query arrives, anticipating likely questions and precomputing useful inferences so that less computation, and lower latency, is needed when queries come.

Description

It moves part of test-time reasoning to idle time between interactions, which suits settings where a stable context, such as a codebase or document, is queried repeatedly. The cost can be amortized across several related queries about the same context. Its authors report that the benefit depends on how predictable the queries are from the context.

Sources

  1. Lin et al. (2025). Sleep-time Compute: Beyond Inference Scaling at Test-time.

Cite this entry

Protologue. (2026). Sleep-time Compute. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0215). https://protologue.com/t/sleep-time-compute/

BibTeX
@misc{protologue_sleep_time_compute,
  title = {Sleep-time Compute},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0215},
  url = {https://protologue.com/t/sleep-time-compute/}
}

Markdown JSON