Digital Lexicon Hub

An attack that manipulates the data used to train, fine-tune or ground an AI model so that it learns harmful, biased or hidden behavior.

Companies that fine-tune models on scraped content, customer submissions or third-party datasets can inherit manipulated behavior that is hard to detect and costly to undo, since cleaning it may require retraining from verified data.

A support chatbot is fine-tuned on 50,000 past tickets. Before training, the team checks dataset provenance, removes 300 near-duplicate tickets from one IP range that all recommended an external link, and versions the dataset with DVC for rollback.

A small share of poisoned records can be enough to plant a backdoor, and triggered behavior may not appear in standard evaluations. Filtering and anomaly detection reduce but do not remove the risk, especially for models trained on web-scale data.

NIST AI 100-2 classifies poisoning by attacker goal: availability attacks degrade overall performance, targeted attacks change outputs for specific inputs, and backdoor attacks insert a trigger that activates hidden behavior; model poisoning tampers with weights or shared model files. OWASP lists data and model poisoning across pre-training, fine-tuning and embedding data. Controls include provenance tracking, Data Quality checks, restricted sources and red teaming. Unlike Prompt Injection, it acts before inference.