MMOOZZGG.com

Glossary

What is parameter-efficient fine-tuning (PEFT)

In short

Parameter-efficient fine-tuning (PEFT) is a family of methods that adapt a large model to a task by training only a small share of its parameters.

Language models · parameter-efficient fine-tuning

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine a big toy robot that can already walk and talk. To make it water the flowers, you do not need to take it apart. It is enough to screw on a small watering can and change a couple of parts. The robot stays the same, but it has learned a new job. Smart programs learn new things in the same way, by changing very little.

In plain words

A large model has billions of settings, and changing all of them takes a long time and a lot of money. These methods keep the main part of the model as it is and train only a small piece: added layers, special prompts made of numbers, or small matrices next to the main ones. Such a piece takes little space, trains on an ordinary server and is easy to swap for another. One model can have many add-ons: for support, for lawyers, for reports.

For business

For a company, these methods mean that tuning its own model is no longer a big project. One base model serves several departments, and each one has a small adapter that is plugged in on request. The costs are a graphics card or rented capacity for the training time, preparing examples and testing. The risks are like those of any fine-tuning: bad data spoils the behavior, the license of the base model applies to the result, and when a new model version comes out, the adapters have to be trained again. Before you decide whether to keep a model in-house or use the cloud, estimate how many such tunings you will really need.

For a specialist

PEFT (parameter-efficient fine-tuning) covers methods that freeze the weights of a pretrained model and train a small number of new or selected parameters. The main groups are adapters, which are small layers inside transformer blocks; trainable soft prompts at the input, as in prompt tuning and prefix tuning; low-rank weight updates, which include LoRA and QLoRA; and selective training of part of the weights. Houlsby and co-authors (2019) showed that adapters, which add about 3.6% of parameters per task, trail full fine-tuning on GLUE by only 0.4 points. Lester, Al-Rfou and Constant (2021) found that prompt tuning catches up with full fine-tuning once the model has billions of parameters. Implementations are collected in the PEFT library from Hugging Face, which works with Transformers, Diffusers and Accelerate. The methods are combined with quantization and distillation.

Example

A law firm keeps one open model on its own server and several adapters for it: for contracts, for claims and for internal rules. Switching between them is fast, and there is no need to store copies of the large model.

Related terms

Questions and answers

How is PEFT different from LoRA? PEFT is the general name of a family of methods, and LoRA is one of them. PEFT is also the name of a Hugging Face library that collects implementations of LoRA, adapters and trainable prompts.
Is PEFT worse than full fine-tuning? On many tasks the difference is small: in the paper on adapters, the gap on GLUE was 0.4 points. In return, it needs far less computing and storage.

Full fine-tuning makes sense when the task is very different from what the model learned on.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us