Hallucinations
When a model invents confident wrong answers—and practical ways to reduce the risk.
What you'll learn
- Define hallucination as fluent text not supported by facts or sources.
- List common triggers: rare facts, pressure to answer, missing context.
- Choose mitigations: RAG, citations, refusal, human review, evals.
In plain English
A hallucination is when a language model states something false as if it were true—made-up citations, wrong dates, imaginary product features. The scary part is the confidence: the same next-token engine that writes helpful prose also writes convincing nonsense.
Hallucination is not a random bug; it is what happens when a system optimized for plausible language meets questions it cannot reliably answer from weights alone.
How it works
The model picks high-probability continuations. If training data had many confident-sounding answers, the model learns that tone—even when content is uncertain. Prompts that demand an answer (“You must respond”) increase guess rate.
Mitigations change the setup: provide sources (RAG), require quotes, allow “I don't know,” use tools for math and lookups, and evaluate on your domain.
- Ground with retrieved documents and ask for citations.
- Separate factual tasks from creative ones in UX and prompts.
- Verify externally before high-stakes actions.
- Log prompts and outputs for regression testing.
Going deeper
Calibration—knowing when the model is unsure—is an active research area. Today, verbal confidence (“I'm certain…”) is not a reliable signal.
Fine-tuning for helpfulness can accidentally increase willingness to answer rather than refuse; product design must reward appropriate abstention.
Common misconceptions
- Hallucinations mean the model is broken.
- They are a predictable outcome of probabilistic text generation without guaranteed grounding.
- Newer models never hallucinate.
- Improvements reduce rates on many benchmarks but do not remove the need for verification.
- Asking the model to “only tell the truth” fixes it.
- Instructions help tone and refusal but cannot create facts that were never retrieved or computed.
Key facts
- Hallucinations are confident outputs not tied to verifiable truth.
- Next-token training rewards fluent structure, not factual guarantees.
- RAG, tools, and citations reduce but do not eliminate errors.
- Forcing answers increases fabricated detail.
- Domain evaluation is the practical measure of risk for your app.
Sources used
These free resources informed this page. ANN writes original explainers; we do not copy course text behind paywalls.
- Google Machine Learning Crash Course — General ML evaluation mindset for generative systems.
- Hugging Face LLM Course — How LLMs behave as text predictors in applications.
- Anthropic Prompt Engineering Interactive Tutorial — Hands-on patterns for clearer instructions and examples.
Also explore AI companies, Live Feed, and Weekly Brief.