Extract, Transform, Load (ETL)
A data-integration pattern in which data is retrieved from sources, transformed into the required structure, and then loaded into a target system.
ETL supports controlled, repeatable preparation of data for reporting, migration, compliance, and operational use, especially when the target should receive only curated data.
A nightly process extracts CRM and billing records, standardizes account identifiers, validates currencies, calculates revenue fields, and loads approved tables into a warehouse.
Pre-load transformation can increase latency and infrastructure complexity. Rigid pipelines may be difficult to adapt when source schemas or analytical needs change, and transformation errors can silently propagate without testing.
Extraction collects source data; transformation validates, cleans, joins, standardizes, and applies business rules; loading writes the prepared result to a warehouse or other destination. ETL often uses a separate processing layer before the target receives data.
Systems Architecture
IBM — ETL — https://www.ibm.com/think/topics/etl; Apache Airflow Documentation — Pythonic Dags with the TaskFlow API — https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html
