Digital Lexicon Hub logo

Digital Lexicon Hub

THE TECHNICAL LANGUAGE OF DIGITAL B2B INFRASTRUCTURE

CONCEPT
INTELLIGENT AUTOMATION

AI Hallucination

Output from an AI model that is presented confidently but is false, unsupported by its sources, or inconsistent with the context it was given.


BUSINESS RELEVANCE

Hallucinated facts in contracts, support answers, reports or code can cause legal, financial and reputational damage, so output verification must be part of any process that relies on generative AI.


IMPLEMENTATION EXAMPLE

A legal operations team requires its contract-review assistant to quote the exact clause behind each finding, drops any finding without a matching quote, and has a lawyer review every flag on contracts above €50,000.


LIMITATIONS

Mitigations reduce hallucinations but do not remove them, and retrieval can still return wrong or outdated documents. A confident tone is not a reliable signal of accuracy, and some errors can only be spotted with domain expertise.


TECHNICAL EXPLANATION

NIST's Generative AI Profile names the risk confabulation: systems generate and confidently present erroneous or false content, which may also contradict the prompt or earlier statements. It arises because models predict likely text rather than check facts. Controls include grounding answers with Retrieval-Augmented Generation (RAG), requiring quotes or citations, allowing the model to say it does not know, AI Guardrails and Human-in-the-Loop (HITL) review.


Secondary Topics

AI Governance

Sources

NIST — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1) — https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf; Claude Platform Docs — Reduce hallucinations — https://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations

Related terms

Application Programming Interface (API)

CONCEPT
SYSTEMS ARCHITECTURE

A defined software interface that lets applications or components request data or functionality from each other through documented operations.

Harmonized Standard

CONCEPT
AI GOVERNANCE

A European standard that, once cited in the Official Journal, gives a presumption of conformity with the EU law it supports, including the AI Act.

Microservices

CONCEPT
SYSTEMS ARCHITECTURE

Applications split into independently deployable services that own their data and communicate through APIs or events instead of shared code and databases.

Indexability

CONCEPT
SEARCH OPTIMIZATION

The technical state determining whether a search engine can add a crawled URL to its index, based on status codes, directives and canonicals.

General-Purpose AI Model (GPAI)

CONCEPT
AI GOVERNANCE

General-purpose AI models, such as large language models, carry specific EU AI Act duties for providers, with stricter rules for models with systemic risk.