# Evals

> Evals are systematic tests of a model or LLM application, consisting of a set of representative inputs and a grading method, used to measure performance against defined success criteria and to compare prompts, models, and changes.

- Identifier: PTL-0179
- Category: Failure Modes & Evaluation
- Canonical URL: https://protologue.com/t/evals/
- Also known as: evaluations, LLM evaluation, eval suite, test set

## Description

Grading can be exact-match or code-based, done by an LLM judge with a rubric, or done by people. Code-based grading is fastest and most reliable where it applies. A good eval set mirrors real usage, including edge cases, and turns prompt engineering from guesswork into measurement. Without one, changes that fix one case can silently break others.

## Narrower terms

- [pass@k](https://protologue.com/t/pass-at-k/)
- [RAG Evaluation](https://protologue.com/t/rag-evaluation/)

## Related terms

- [LLM-as-a-Judge](https://protologue.com/t/llm-as-a-judge/)
- [Prompt Optimization](https://protologue.com/t/prompt-optimization/)
- [Data Contamination](https://protologue.com/t/data-contamination/)

## Sources

- Anthropic (2024). Define success criteria and build evaluations. https://platform.claude.com/docs/en/test-and-evaluate/develop-tests
- Liang et al. (2022). Holistic Evaluation of Language Models. https://arxiv.org/abs/2211.09110

## Cite this entry

Protologue. (2026). Evals. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0179). https://protologue.com/t/evals/

License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)