ETL

ETL solves the problem of preparing data before it enters an analytics system. It is a classic data integration pattern: extract data from sources, transform it into the desired shape, then load it into a warehouse or target database.

Motivation

Teams use ETL when raw source data should be cleaned, validated, joined, filtered, or standardized before it becomes available to downstream users.

Where it fits

ETL belongs to the Product track, inside Data Applications. It is closely related to ELT, data pipelines, datasets, warehouses, and analytics systems.

Mental model

Think of ETL as a preparation line: data is taken from source systems, processed in the middle, and only then placed into the final destination.

Subconcepts

Extract

Extract is the step that reads data from source systems such as APIs, databases, files, or event streams.

Transform

Transform is the step that cleans, joins, reshapes, validates, or enriches data before storage.

Load

Load is the step that writes the transformed data into the target system.

ETL vs ELT

ETL transforms before loading. ELT loads first and transforms inside the target system. ETL is often useful when transformation must happen before storage; ELT is often useful when the target warehouse has powerful compute and can preserve raw data.

Common mistakes

• Treating ETL and ELT as identical.
• Hiding business logic in pipeline code that is difficult to test or audit.
• Forgetting that data quality and lineage matter as much as moving the data.