Mermaid code
flowchart LR
subgraph SRC[Sources]
S1[(App database)]
S2[Event logs]
S3[Partner CSV files]
end
S1 --> E[Extract<br/>nightly 02:00]
S2 --> E
S3 --> E
E --> ST[(Staging tables)]
ST --> V{Schema and<br/>null checks pass?}
V -->|no| Q[Quarantine table<br/>+ alert]
V -->|yes| T[Transform<br/>dedupe, join, aggregate]
T --> W[(Data warehouse)]
W --> B[BI dashboards]
W --> M[ML feature store] Paste this into any tool that renders Mermaid, or into the editor to change the theme and export PNG or SVG. Syntax reference: Flowchart guide.
What this diagram shows
Three sources inside the Sources subgraph feed one Extract step scheduled at 02:00. Everything lands in Staging tables first, then the Schema and null checks pass? diamond either sends the batch to Transform or diverts it to a Quarantine table with an alert. Clean data reaches the Data warehouse, which has two consumers: BI dashboards and an ML feature store. The cylinder shape marks every place data is stored at rest.
How to adapt it
- Rename the three sources to your real systems and add or remove nodes inside the
Sourcessubgraph as needed. - Change
Extract<br/>nightly 02:00toStream ingestand replaceStaging tableswith aMessage queuenode for a streaming pipeline. - Split
Transforminto separateClean,Join, andAggregatenodes if each is a separate job with its own owner. - Add a
Data catalognode fed from the warehouse if you publish table documentation. - For ELT, move
Transformafter the warehouse node so the diagram shows transformation happening inside the warehouse.
FAQ
Why validate in staging instead of at extraction?
Staging keeps a copy of the raw batch, so a failed check can be inspected and the batch replayed after a fix. Validating during extraction loses the raw rows. The quarantine path only works because staging exists.
What is the difference between ETL and ELT in this diagram?
Only the position of the Transform node. ETL transforms before loading into the warehouse, as drawn. ELT loads raw data first and transforms inside the warehouse with SQL, so the node moves to the right of Data warehouse.