The Dragon Called Hallucination
Why models invent confident answers, and how retrieval, citations, and checks tame the beast.
The village kept a dragon in the archive. It was charming, fast, and eager to finish every story. If the last page of a legend was missing, the dragon would invent an ending before anyone noticed. The ending sounded ancient, confident, and beautifully written. It was also sometimes false.
AI hallucination is the same kind of danger. A language model can produce fluent text that looks like knowledge even when it is not grounded in reliable evidence. The model is not checking truth the way a database checks a record. It is generating likely language from patterns, context, and probability.
Hallucinations happen for several reasons. The question may ask for information outside the model's knowledge. The prompt may contain false assumptions. The model may be pushed to answer when it should say it is uncertain. In some cases, the source material is missing, conflicting, or too vague for a dependable response.
The remedy is not to stop using AI. The remedy is to decide where accuracy matters and design around it. Retrieval can provide current documents. Citations can show which passages support an answer. Structured outputs can limit free-form invention. Verification steps can compare claims against trusted systems before users rely on them.
Product design matters too. A customer support assistant should distinguish between policy it has retrieved and general suggestions. A coding assistant should run tests when possible. A research assistant should show sources and uncertainty. A creative brainstorming tool can tolerate more imagination than a compliance workflow.
Teams should also evaluate hallucination risk with real examples. Collect user questions, expected answers, missing-source cases, and adversarial prompts. Then measure how often the system gives unsupported claims, refuses appropriately, or asks for clarification. Without evaluation, teams only know how the demo feels.
The dragon became useful after the librarian gave it rules: read the scroll first, name the scroll, admit when the scroll is missing, and never pretend a guess is a citation. AI systems need similar discipline. Confidence is not the same as evidence.
