Explainable AI (XAI)
Methods and system properties that help relevant people understand the basis, behavior, or limitations of an AI system and its outputs.
Useful explanations support validation, human oversight, incident investigation, contestability, adoption, and regulatory or assurance requirements. They can reveal when a system relies on inappropriate signals.
A fraud system provides investigators with influential transaction factors, comparable cases, confidence information, and a route to inspect the underlying evidence before action is taken.
An explanation can be plausible without being faithful to the model. Simplification can hide interactions, and technical explanations may not answer a person's practical question. Explainability does not guarantee fairness, accuracy, or accountability.
Explainability may describe how a model works globally or why it produced a particular result locally. Approaches include interpretable models, feature attribution, examples, counterfactuals, uncertainty information, and process documentation. Explanations must be designed for the audience and decision being supported.
Data Engineering
NISTIR 8312 — Four Principles of Explainable AI — https://doi.org/10.6028/NIST.IR.8312; NIST — AI Risk Management Framework — https://www.nist.gov/itl/ai-risk-management-framework