Digital Lexicon Hub

The process of discovering patterns, relationships or anomalies in datasets using statistical, computational and machine-learning methods.

Data mining turns historical records into patterns that support decisions such as which customers are likely to churn, which products sell together and which transactions look fraudulent.

A grocery chain runs association rule mining on 12 months of basket data, finds that buyers of fresh pasta often buy a specific sauce, and tests a shelf placement change in 20 stores against 20 control stores.

Patterns found in data are correlations and may not hold over time or imply causation. Mining many variables raises the risk of false discoveries, and biased or poor-quality data produces misleading results and fairness concerns.

Data mining is often run as a process such as CRISP-DM: understand the business problem, understand and prepare the data, model, evaluate and deploy. Common techniques include classification, regression, clustering, association rule learning and anomaly detection, using algorithms such as decision trees, k-means and Apriori. Results are validated on held-out data before being used.