MMOOZZGG.com

Glossary

What is an AI hallucination?

In short

An AI hallucination is an answer from a neural network that sounds confident but contains invented or wrong facts.

Language models · hallucination

MMOOZZGG.com editorsChecked: 1 min read

For a child

Sometimes you don’t know the answer, but you really want to answer, so you make up something that sounds true. A neural network sometimes does the same: it speaks with confidence, but it invents things. Adults call this a hallucination.

In plain words

A neural network does not look for the truth in a database. It picks the most likely next words. Usually this gives a correct answer, but sometimes the result is smooth text with a number, a link or a name that does not exist. The model is not lying on purpose: it simply cannot tell knowledge from a plausible guess. So check the important facts in an AI answer.

For business

For a company, an AI hallucination is the risk of a wrong price, a wrong contract term or a wrong legal answer given to a customer. You lower the risk like this: the AI answers only from your documents through RAG, shows the source, says “I don’t know” to questions outside the knowledge base and hands them to a person. Measure the error rate on a set of real questions before launch and after every change.

For a specialist

Hallucination is generation that is meaningless or unfaithful to the source context (Ji et al., 2023, survey); for language models, answers that contradict facts about the world count too. Researchers separate intrinsic hallucinations (they contradict the source context) and extrinsic ones (they cannot be checked against it). Causes: maximizing the likelihood of the next token with no check, gaps and noise in training data, a shift in the distribution of requests, and decoding strategies. Methods to reduce them: grounding through RAG with required citations, limiting the answer to the context, self-checking and self-consistency, external checking models, confidence calibration and refusal to answer. Metrics: faithfulness and groundedness on labeled sets.

Example

An online shop’s AI assistant has no access to the price list and names a price that does not exist. After RAG is connected, it takes the price from the catalog and gives a link to the product.

Related terms

Questions and answers

Can you remove hallucinations completely? Not completely, but you can cut their share a lot: answers only from checked documents, a link to the source, and a hand-off to a person when the answer is not in the knowledge base.
Why does a neural network invent links and numbers? It picks a plausible continuation of the text and does not check the fact. If the needed fact is missing from the context, the model may compose a similar one.

Sources

Not sure whether your company needs this?

AI solutions map
Contact us