protologue

Model Cascade

Also called LLM cascade, FrugalGPT, cascading.

A model cascade sends each query to a cheaper model first and escalates it to progressively more capable and expensive models only when a scoring function judges the cheaper model's answer unreliable.

Description

Because many queries are easy, most are answered cheaply, and only hard ones pay for the strongest model. FrugalGPT reported matching the best single model's performance at a large reduction in cost. The design depends on a reliable way to judge when an answer is good enough. Routing, by contrast, chooses a model before any answer is generated.

Sources

  1. Chen et al. (2023). FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance.

Cite this entry

Protologue. (2026). Model Cascade. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0155). https://protologue.com/t/model-cascade/

BibTeX
@misc{protologue_model_cascade,
  title = {Model Cascade},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0155},
  url = {https://protologue.com/t/model-cascade/}
}

Markdown JSON