Low-Rank Adaptation
Also called LoRA.
Low-rank adaptation (LoRA) fine-tunes a language model by training small low-rank matrices added to its weight layers while freezing the original weights, drastically reducing the number of trainable parameters.
Description
LoRA is a common alternative when prompting alone cannot reach the required behavior, and adapters can be swapped per task on one base model.
Sources
- Hu et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models.
Cite this entry
Protologue. (2026). Low-Rank Adaptation. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.0.0, PTL-0078). https://protologue.com/t/low-rank-adaptation/
BibTeX
@misc{protologue_low_rank_adaptation,
title = {Low-Rank Adaptation},
author = {{Protologue}},
year = {2026},
howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.0.0},
note = {Entry PTL-0078},
url = {https://protologue.com/t/low-rank-adaptation/}
}