protologue

Chatbot Arena

Also called LMArena, Arena, LMSYS Chatbot Arena.

Chatbot Arena is a public platform where people chat with two anonymous models side by side and vote for the better response, with the votes aggregated into a leaderboard of model ratings.

Description

Crowdsourced preferences capture open-ended quality that fixed test sets miss, and new questions from real users resist contamination. Ratings can also reflect style, such as length and formatting, which is why style-controlled rankings are offered. The platform was later renamed, first LMArena and then Arena.

Sources

  1. Chiang et al. (2024). Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference.

Cite this entry

Protologue. (2026). Chatbot Arena. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0210). https://protologue.com/t/chatbot-arena/

BibTeX
@misc{protologue_chatbot_arena,
  title = {Chatbot Arena},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0210},
  url = {https://protologue.com/t/chatbot-arena/}
}

Markdown JSON