MMOOZZGG.com

Glossary

What are model weights?

In short

Model weights are the numbers inside a neural network that set the strength of the links between its elements and store everything the model learned in training.

Language models · weights

MMOOZZGG.com editorsChecked: 1 min read

For a child

Imagine you are learning to throw a ball into a basket. After each miss, you change the force of the throw and the angle of your hand a little. After a week, your hand knows how to throw by itself. A neural network has habits like that too, only they are written as numbers. Each number tells how much one small part of the network should listen to another.

In plain words

A neural network is made of many simple elements linked to each other. Each link has a number that shows how strongly the signal of one element affects the next. These numbers are called weights. At the start of training they are chosen almost at random, so the answers make no sense. Then the model makes mistakes thousands of times, and after each mistake the program corrects the links a little. When training is over, the result is saved to disk. A downloaded open model takes tens of gigabytes because it holds billions of such coefficients.

For business

The weights are the model itself as a product. Whoever has the file can run the neural network on their own server, without the internet and without sending data to the vendor. So developers either keep the weights closed and sell access through an API, or publish them under a license. Before you run an open model in your company, check three things: whether the license allows commercial use, whether you have enough video memory for the file size, and where the file was downloaded from. A copy from an unofficial source may be altered. Fine-tuning on your examples changes the coefficients or adds an adapter to them, and that file becomes a company asset: you store and protect it like source code. The choice between your own server and the cloud is covered in the map of AI solutions.

For a specialist

Weights are the trainable coefficients of a neural network: the elements of the linear layer matrices that the input vector is multiplied by. Together with biases and normalization coefficients, they make up the model’s parameters. In a transformer, most of them sit in the attention projections and the feed-forward blocks, and the vocabulary embedding matrix is stored separately. Their values are found by gradient descent: backpropagation computes the gradient of the loss function for each weight, and an optimizer such as AdamW moves it by a small step. During inference they are frozen. In PyTorch they are kept in a state dictionary (state_dict) that maps each layer to its parameter tensors. For publishing, the safetensors format is used: Hugging Face describes it as a safe alternative to pickle with fast zero-copy loading. They are stored in FP32, BF16, or FP16 and are often quantized to lower precision for running, while LoRA adapters train small additions without touching the original matrices.

Example

A bank wants to run a model inside its own closed network. It downloads the weights file of an open model from the developer’s official page, checks the checksum, and deploys it on its own server: customer texts go nowhere.

Related terms

Questions and answers

What are weights in a neural network? They are the numbers the model found during training. They decide how the input text or image turns into an answer, and they hold all of the model’s knowledge.
How are weights different from model parameters? Often the two words mean the same. Strictly speaking, parameters include all trainable numbers, and weights are their main part: the coefficients of the links between the network’s elements.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us