# RULER

> RULER is a synthetic long-context benchmark that extends needle-in-a-haystack retrieval with harder tasks, such as multiple needles, multi-hop tracing, and aggregation, at configurable context lengths, to measure how much of its context window a model can actually use.

- Identifier: PTL-0211
- Category: Benchmarks
- Canonical URL: https://protologue.com/t/ruler/
- Introduced: 2024

## Description

Its creators found that many models performed well on simple needle retrieval but degraded well before their advertised context length on the harder tasks. It is a standard reference for a model's effective, as opposed to claimed, context size.

## Broader terms

- [Benchmark](https://protologue.com/t/benchmark/)

## Related terms

- [Needle in a Haystack](https://protologue.com/t/needle-in-a-haystack/)
- [Context Window](https://protologue.com/t/context-window/)
- [Context Rot](https://protologue.com/t/context-rot/)
- [Lost in the Middle](https://protologue.com/t/lost-in-the-middle/)

## Sources

- Hsieh et al. (2024). RULER: What's the Real Context Size of Your Long-Context Language Models?. https://arxiv.org/abs/2404.06654

## Cite this entry

Protologue. (2026). RULER. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0211). https://protologue.com/t/ruler/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)