Glossary
What are logits
In short
Logits are raw numeric scores that a model gives to each possible answer, for example to each token of its vocabulary, before they are turned into probabilities.
Language models · logits
MMOOZZGG.com editorsChecked: 1 min read
For a child
Imagine a drawing contest where each judge gives points any way they like: one gives from one to a hundred, another may even give minus five. To see who has a better chance of the prize, the points are later turned into clear shares. A neural network also gives such raw points to each possible answer first, and only then turns them into chances.
In plain words
Before it chooses the next word, the model scores each of the tens of thousands of words it knows and gives it a number. The numbers can be anything: large, small or negative. The higher the number, the more the model leans to that word. On their own they are not percentages, so the next step turns them into probabilities that sum to one. For example, after the words “The invoice is paid”, the word “fully” can have a high score, and the word “banana” a strongly negative one.
For business
A business needs these numbers indirectly, but they give a useful confidence signal. Some APIs return the log probabilities of tokens (logprobs) on request, and a model on your own server gives the scores themselves. With them you can tell an answer the model is sure of from a guess: for example, send a request to a person when the confidence of a classification is below a threshold. Format rules work through the same mechanism: forbidden tokens get a minimal score, so the model cannot output an invalid category. Remember that model confidence is not correctness. A model can be confidently wrong, so tune the threshold on a labeled sample. Where AI makes mistakes and when a person is needed is covered in risk assessment.
For a specialist
Logits are the output of the last linear layer of a model before normalization: a vector z the size of the vocabulary, which the softmax function turns into a distribution p_i = exp(z_i) / Σ exp(z_j). The name comes from statistics, where the logit is the logarithm of the odds ratio. Logits are defined up to a shift: adding a constant to all components does not change the probabilities. In PyTorch, the CrossEntropyLoss function expects unnormalized logits for each class as input. Generation processors work on logits too: division by temperature, masking for top-k and top-p, a penalty for repeats, and constrained decoding by a grammar or a schema. Hinton, Vinyals and Dean (2015) call the inputs of the final softmax logits in knowledge distillation, and train a small model on the distributions of a large one, softened by a high temperature. Absolute logit values of different models cannot be compared directly; only the differences inside one distribution are meaningful.
Example
A system sorts incoming emails by department. If, after the logits are turned into probabilities, no department gets more than eighty percent, the email goes to manual review and not into a random queue.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map