protologue

MT-Bench

MT-Bench is a benchmark of 80 two-turn questions across categories such as writing, reasoning, math, and coding, in which a strong LLM acts as the judge and scores each response.

Description

It was introduced with the systematic study of LLM-as-a-judge, which found that strong model judges agreed with human preferences about as well as people agreed with each other, and also documented judge biases. Its small size and reliance on a model judge limit how finely it can separate today's models.

Sources

  1. Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.

Cite this entry

Protologue. (2026). MT-Bench. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0209). https://protologue.com/t/mt-bench/

BibTeX
@misc{protologue_mt_bench,
  title = {MT-Bench},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0209},
  url = {https://protologue.com/t/mt-bench/}
}

Markdown JSON