protologue

Terminal-Bench

Terminal-Bench is a benchmark of hard, realistic tasks that an AI agent must complete by running commands in a containerized Linux terminal, graded by automated tests of the final state of the environment.

Description

Grading checks outcomes, not the commands used, so agents can solve tasks any way that works. It measures the command-line skills that coding and DevOps agents rely on, such as building software, configuring systems, and debugging.

Sources

  1. Merrill et al. (2026). Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces.

Cite this entry

Protologue. (2026). Terminal-Bench. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0203). https://protologue.com/t/terminal-bench/

BibTeX
@misc{protologue_terminal_bench,
  title = {Terminal-Bench},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0203},
  url = {https://protologue.com/t/terminal-bench/}
}

Markdown JSON