Model Drift
Decline in a deployed machine learning model's predictive performance as real-world data or the relationships behind it change after training.
Drifting models keep producing confident outputs, so lead scores, forecasts or fraud flags can degrade for months before anyone notices, quietly distorting budget, pricing and risk decisions built on them.
A lead-scoring model trained on 2024 data is checked weekly; the population stability index for company size rises from 0.04 to 0.27 after a new paid channel launches, so the team retrains on the last 90 days and recalibrates score thresholds.
Input drift does not always reduce accuracy, and accuracy can fall without visible input drift. Ground-truth labels often arrive late, so drift statistics act as proxies, and fixed alert thresholds raise false alarms on small samples.
Two main forms are distinguished: data drift (covariate shift), where the distribution of input features changes, and concept drift, where the relationship between inputs and the target changes suddenly, gradually or seasonally. Monitoring compares production data with a reference window using tests such as Kolmogorov-Smirnov, population stability index or Wasserstein distance, and tracks accuracy once labels arrive. Findings should feed AI Risk Management and update the Model Card.
Data Engineering, Intelligent Automation
IBM — What is model drift? — https://www.ibm.com/think/topics/model-drift; Evidently AI — Model monitoring for ML in production — https://www.evidentlyai.com/ml-in-production/model-monitoring