What is an AI hallucination?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
An AI hallucination is generated content that is incorrect or unsupported by the relevant evidence, even though it may sound plausible. It can take the form of an invented fact, a fabricated reference, a distorted summary or a conclusion that the source does not justify. The term describes an output failure, not a claim that the computer has a human perceptual experience.
Suppose you ask an assistant for a summary of a report. It correctly describes the general subject but adds a result that never appeared in the document. That addition is a problem even if the result sounds reasonable. The task was to represent the report, not to fill gaps with a likely-sounding story.
A language model is a system trained to recognize patterns in language and generate responses. Producing fluent text is not the same as verifying each claim against an authoritative source. The surrounding application may provide search or documents, but it must actually obtain and use the relevant evidence.
There are useful distinctions within the term. An answer can contradict the supplied source. It can make a claim the source does not support. Or it can accurately report a source that is itself mistaken. That last case may be a source error rather than a hallucination by the model. These failures need different remedies: checking that a summary matches a document does not establish that the document is correct.
Fiction is different again. An invented character in a requested story is not a factual error. The intended task determines the standard. The same sentence may be acceptable creative writing and unacceptable as an account of a real event.
Retrieval-augmented generation, or RAG, supplies a model with information found in external material before it answers. Retrieval means finding and selecting that material. This can make evidence available, but the wrong source can be retrieved and the right source can be misread. A citation can also be genuine while failing to support the sentence beside it.
For a consequential factual claim, follow the source and check what it actually says. Asking the same assistant “Are you sure?” may prompt another fluent explanation without adding independent evidence. A useful system should be able to say when the information is insufficient.
The Consensus discussion in class emphasized the ability to inspect the evidence behind an answer. A healthcare-research example is a summary that cites a real study but describes a patient population or outcome the study did not examine. A genuine reference does not rescue an unsupported claim. Open the paper and check the relevant passage, including its limitations. In a clinical-record summary, apply the same discipline: an undocumented diagnosis must not appear merely because it makes the narrative sound more complete.
Source: Ji and colleagues — Survey of Hallucination in Natural Language Generation.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy