protologue

Fine-Tuning

Also called model fine-tuning, transfer learning.

Fine-tuning is the further training of a pretrained model on a smaller, task- or domain-specific dataset so that its weights adapt to that task or domain.

Description

Fine-tuning complements prompting. Prompting changes only the input, while fine-tuning changes the model, which can teach a consistent format, style, or specialized skill that is hard to obtain from instructions alone, at the cost of training data, compute, and a separate model to maintain. Parameter-efficient methods such as LoRA update only a small number of added weights, and instruction tuning is the fine-tuning step that turns a base model into an assistant.

Sources

  1. Howard and Ruder (2018). Universal Language Model Fine-tuning for Text Classification.
  2. Hu et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models.

Cite this entry

Protologue. (2026). Fine-Tuning. In Protologue: A Taxonomy of Prompting and LLM Techniques (v1.4.0, PTL-0105). https://protologue.com/t/fine-tuning/

BibTeX
@misc{protologue_fine_tuning,
  title = {Fine-Tuning},
  author = {{Protologue}},
  year = {2026},
  howpublished = {Protologue: A Taxonomy of Prompting and LLM Techniques, v1.4.0},
  note = {Entry PTL-0105},
  url = {https://protologue.com/t/fine-tuning/}
}

Markdown JSON