Glossary
What is an AI hallucination?
In short
An AI hallucination is an answer from a neural network that sounds confident but contains invented or wrong facts.
Language models · hallucination
MMOOZZGG.com editorsChecked: 1 min read
For a child
Sometimes you don’t know the answer, but you really want to answer, so you make up something that sounds true. A neural network sometimes does the same: it speaks with confidence, but it invents things. Adults call this a hallucination.
In plain words
A neural network does not look for the truth in a database. It picks the most likely next words. Usually this gives a correct answer, but sometimes the result is smooth text with a number, a link or a name that does not exist. The model is not lying on purpose: it simply cannot tell knowledge from a plausible guess. So check the important facts in an AI answer.
For business
For a company, an AI hallucination is the risk of a wrong price, a wrong contract term or a wrong legal answer given to a customer. You lower the risk like this: the AI answers only from your documents through RAG, shows the source, says “I don’t know” to questions outside the knowledge base and hands them to a person. Measure the error rate on a set of real questions before launch and after every change.
For a specialist
Hallucination is generation that is meaningless or unfaithful to the source context (Ji et al., 2023, survey); for language models, answers that contradict facts about the world count too. Researchers separate intrinsic hallucinations (they contradict the source context) and extrinsic ones (they cannot be checked against it). Causes: maximizing the likelihood of the next token with no check, gaps and noise in training data, a shift in the distribution of requests, and decoding strategies. Methods to reduce them: grounding through RAG with required citations, limiting the answer to the context, self-checking and self-consistency, external checking models, confidence calibration and refusal to answer. Metrics: faithfulness and groundedness on labeled sets.
Example
An online shop’s AI assistant has no access to the price list and names a price that does not exist. After RAG is connected, it takes the price from the catalog and gives a link to the product.
Related terms
Questions and answers
Sources
Not sure whether your company needs this?
AI solutions map