# τ-bench

> τ-bench is a benchmark in which an agent with API tools and a written domain policy, for example for airline or retail customer service, converses with a simulated user and is judged by whether the final database state is correct.

- Identifier: PTL-0204
- Category: Benchmarks
- Canonical URL: https://protologue.com/t/tau-bench/
- Also known as: tau-bench, TAU-bench, pass^k
- Introduced: 2024

## Description

It tests tool use, policy following, and conversation together, as in real deployments. It introduced pass^k, the probability that an agent succeeds on all of k repeated trials of the same task, which exposes how inconsistent agents can be even when they often succeed once.

## Broader terms

- [Benchmark](https://protologue.com/t/benchmark/)

## Related terms

- [Function Calling](https://protologue.com/t/function-calling/)
- [AI Agent](https://protologue.com/t/ai-agent/)
- [pass@k](https://protologue.com/t/pass-at-k/)

## Sources

- Yao et al. (2024). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. https://arxiv.org/abs/2406.12045

## Cite this entry

Protologue. (2026). τ-bench. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0204). https://protologue.com/t/tau-bench/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)