Explainable AI (XAI)

Technical explanation

Explainability may describe how a model works globally or why it produced a particular result locally. Approaches include interpretable models, feature attribution, examples, counterfactuals, uncertainty information, and process documentation. Explanations must be designed for the audience and decision being supported.

Business relevance

Useful explanations support validation, human oversight, incident investigation, contestability, adoption, and regulatory or assurance requirements. They can reveal when a system relies on inappropriate signals.

Implementation example

A fraud system provides investigators with influential transaction factors, comparable cases, confidence information, and a route to inspect the underlying evidence before action is taken.

Limitations and common misconceptions

An explanation can be plausible without being faithful to the model. Simplification can hide interactions, and technical explanations may not answer a person’s practical question. Explainability does not guarantee fairness, accuracy, or accountability.

Topics

Sources