Glossary
What is model fine-tuning?
In short
Model fine-tuning is the adjustment of an already trained model on your own examples, so that it performs a specific task more accurately or writes in the required style.
Language models · fine-tuning
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine you can already ride a bike and decide to learn mountain biking. You do not need to learn from scratch: a few rides on forest trails are enough. It is the same with a neural network: it already knows a lot, and it gets a little training on the right examples to become a master at one thing.
In plain words
A ready model knows the language and a lot about the world, but it does not know how your company usually answers. Fine-tuning solves this: the model is shown a set of “question and correct answer” pairs, and its internal settings are adjusted a little. There is no need to train from scratch, so a small set and fairly short computing are enough. For example, a delivery service collects good answers from its operators, and after training the model answers in the same tone and format. New facts are hard to add this way: it is easier to pass them to the model together with the question.
For business
Fine-tuning is worth it when the model must keep a format, a tone or a narrow classification stable: sorting requests by category, filling product cards, writing in the brand style. The price is made up of preparing labeled examples, renting graphics cards and repeat cycles on updates. Labeling often takes the most time: bad examples give confidently wrong answers. For knowledge that changes often, such as prices and internal rules, this path does not pay; document search (RAG) is used there. The risks: the model may lose some of its earlier skills, and customer data from the training set may surface in answers. Compare the fine-tuned model and the ordinary one on the same set of questions; a pilot is a good place to do it.
For a specialist
Fine-tuning continues the training of a pretrained model on a smaller target dataset. In the ULMFiT method, Howard and Ruder (2018) showed for text classification that with 100 labeled examples a fine-tuned language model performs on par with training from scratch on 100 times more data. For LLMs the main modes are supervised fine-tuning on instructions (SFT), preference alignment (RLHF, DPO) and parameter-efficient methods (PEFT). The LoRA method (Hu et al., 2021) freezes the original weights and trains low-rank add-on matrices; compared with full fine-tuning of GPT-3 175B, this cut the number of trainable parameters about 10,000 times and the GPU memory need about 3 times. QLoRA adds 4-bit quantization of the base model. The risks are catastrophic forgetting, overfitting on a small set and memorization of training data. Quality is compared on a held-out set and on general benchmarks before and after.
Example
A delivery service fine-tuned a model on several thousand labeled requests. Now the model sorts each email into one of twelve categories by itself and sends it to the right department.
Related terms
Questions and answers
How many examples does fine-tuning need?
Sources
Where companies use it
Not sure whether your company needs this?
AI solutions map