MMOOZZGG.com

Glossary

What is model fine-tuning?

In short

Model fine-tuning is the adjustment of an already trained model on your own examples, so that it performs a specific task more accurately or writes in the required style.

Language models · fine-tuning

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine you can already ride a bike and decide to learn mountain biking. You do not need to learn from scratch: a few rides on forest trails are enough. It is the same with a neural network: it already knows a lot, and it gets a little training on the right examples to become a master at one thing.

In plain words

A ready model knows the language and a lot about the world, but it does not know how your company usually answers. Fine-tuning solves this: the model is shown a set of “question and correct answer” pairs, and its internal settings are adjusted a little. There is no need to train from scratch, so a small set and fairly short computing are enough. For example, a delivery service collects good answers from its operators, and after training the model answers in the same tone and format. New facts are hard to add this way: it is easier to pass them to the model together with the question.

For business

Fine-tuning is worth it when the model must keep a format, a tone or a narrow classification stable: sorting requests by category, filling product cards, writing in the brand style. The price is made up of preparing labeled examples, renting graphics cards and repeat cycles on updates. Labeling often takes the most time: bad examples give confidently wrong answers. For knowledge that changes often, such as prices and internal rules, this path does not pay; document search (RAG) is used there. The risks: the model may lose some of its earlier skills, and customer data from the training set may surface in answers. Compare the fine-tuned model and the ordinary one on the same set of questions; a pilot is a good place to do it.

For a specialist

Fine-tuning continues the training of a pretrained model on a smaller target dataset. In the ULMFiT method, Howard and Ruder (2018) showed for text classification that with 100 labeled examples a fine-tuned language model performs on par with training from scratch on 100 times more data. For LLMs the main modes are supervised fine-tuning on instructions (SFT), preference alignment (RLHF, DPO) and parameter-efficient methods (PEFT). The LoRA method (Hu et al., 2021) freezes the original weights and trains low-rank add-on matrices; compared with full fine-tuning of GPT-3 175B, this cut the number of trainable parameters about 10,000 times and the GPU memory need about 3 times. QLoRA adds 4-bit quantization of the base model. The risks are catastrophic forgetting, overfitting on a small set and memorization of training data. Quality is compared on a held-out set and on general benchmarks before and after.

Example

A delivery service fine-tuned a model on several thousand labeled requests. Now the model sorts each email into one of twelve categories by itself and sends it to the right department.

Related terms

Questions and answers

How does fine-tuning differ from RAG? Fine-tuning changes the model’s weights and teaches it a skill or a style. RAG changes nothing in the model: at the moment of the question it gives the model the needed documents, so it suits knowledge that changes often.
How many examples does fine-tuning need? It depends on the task and the method. For narrow classification a few hundred good examples are sometimes enough; changing the style of complex texts needs more.

It is safer to start with a small set and check the quality gain on held-out questions.

Sources

Where companies use it

Not sure whether your company needs this?

AI solutions map
Contact us