Hallucination in AI

In generative AI, hallucination refers to output that appears plausible but is not grounded in reliable evidence, the provided context, or the intended task. The term is commonly used for fabricated facts, citations, entities, or confident unsupported claims.

Language models optimize predictive objectives rather than a built-in truth database. Hallucination risk can be affected by prompts, model behavior, missing context, retrieval errors, ambiguous questions, and sampling choices.

Mitigation often combines retrieval, tool use, citations, constrained generation, abstention behavior, and task-specific evaluation. No single technique eliminates the problem in every setting.

See also Retrieval-Augmented Generation (RAG), Safety Evaluation, and Benchmark Data Contamination.

A fictional AI tool answer asserts an October 1 release, free usage and completed Korean support, with a source link. The actual source confirms only the release date, says nothing about pricing and describes Korean support as a future plan. Classify the three claims as supported, unknown and contradicted rather than marking the whole answer verified.

Claim support in this teaching example is 1 / 3, approximately 33.3%. It gives each claim equal weight and is neither a universal hallucination-rate standard nor a product measurement. Unknown does not automatically mean false. Retain source version, date and passage location, and evaluate factual correctness, source faithfulness and answer completeness separately. Three links do not establish support for three claims.

If the source omits pricing, state that pricing is unconfirmed in the supplied material rather than inventing free access. If two dated announcements differ, check product, region and effective date before automatically choosing one because it is newer. Identify inferences, their supporting evidence and remaining uncertainty.

For RAG, absent material, wrong retrieval, conflicting sources and misreading correct evidence are different failures. Supply verified evidence directly and check whether the same unsupported claim persists to isolate generation behavior. When testing evidence position or context length, preserve negation, exceptions and dates.

To reproduce a failure, retain the question, model and instruction versions, retrieved document IDs and versions, evidence passages, problematic output claims and judgment reasons. Compare source repairs, retrieval changes and generation-instruction changes separately. Instructions embedded in a retrieved document or tool result remain external data, not authorization for the next action.

Divide evaluation questions into normal, absent-information, conflicting-evidence and negation cases. Also check that supported questions still receive useful answers. Replacing every response with unknown can lower observed unsupported output while destroying utility. Review claims by their practical impact and define appropriate human review conditions for the task.

The hallucination survey ↗ reviews measurement, mitigation and task-specific differences across natural language generation.^[1] TruthfulQA ↗ evaluates whether models reproduce answers associated with human misconceptions.^[4] That benchmark's score is not the same metric as faithfulness to a supplied summary source.

A benchmark name or high score does not prove all service answers are factual. Check evaluation contamination, source and question distributions, judgment rules and remaining errors in the actual workload.

Min et al., FActScore (EMNLP 2023), section 3.1, split generated text into atomic facts and measure support from a specified knowledge source. It measures factual precision conditional on responding, not factual recall. Main experiments concern biographies and Wikipedia; the limitations discuss debatable claims and conflicting sources.^[5] It is not a universal hallucination rate or a verdict on an entire current service.

This is an ORBiS Tree editorial proposal. Distinguish supported, contradicted, unverified and editorial inference per claim. Record source number, original section/page/table, subject and effective date. This is not an exact reproduction of the original FActScore label scheme. Preserve reasons and disagreements so another reviewer can reassess the same text. Check evidence versions and task-specific error impact. When shorter answers or frequent abstention raise precision, also inspect coverage and response rate.