ETL and data pipelines · Intermediate · 6 h

Build your first pipeline

Extract, validate, transform and load monthly files into partitioned Parquet, with logging and a rerun that does no harm.

Why employers care: Every data engineering interview asks you to describe a pipeline you built. Analysts who can automate their own data feeds stand out.

In the roadmaps for: Data Analyst, Data Scientist, Data Engineer, AI/ML Engineer

Lessons, in order

Next shelf: Incremental and idempotent loads