Hallucination is the term for an LLM producing plausible-sounding but factually wrong output. The model is not lying; it has no concept of truth, only of likely text. When asked a question outside its training data or that requires precise recall, it generates the most statistically plausible answer, which can be wrong, misleading, or completely invented. Common hallucinations: fake citations, made-up API methods, wrong historical dates, fabricated statistics, invented people.
Hallucinations happen because language models are trained to predict the next token, not to verify facts. When the training signal is ambiguous (the model has seen many similar but not identical claims), it averages or interpolates, producing output that resembles its training data without matching any specific source. Models are also better at generating fluent text than at admitting uncertainty, so they will confidently produce wrong answers rather than say "I do not know".
Mitigations: ground the model with retrieved context, use chain-of-thought to expose flawed reasoning, ask the model to cite sources, run multiple samples and check consistency, use reasoning-trained models that are better at expressing uncertainty.
Hallucination is the single biggest reliability problem with LLMs. For consumer chat, occasional hallucinations are tolerable. For agents acting on data (writing code, sending emails, filing reports), a hallucinated fact can become a costly mistake. Designing systems that fail safely under hallucination is core to production AI.
The Schema Drift article covers how AI agents propagating bad data is structurally similar to a hallucination, and how to defend against both.