Apache Airflow and orchestration · Intermediate · 5 h

Your first DAG

DAGs, tasks, dependencies, schedules, retries and passing small values between tasks.

Why employers care: Every production pipeline needs a scheduler that retries, alerts and records history. Airflow is the most common one.

In the roadmaps for: Data Engineer

Lessons, in order

Next shelf: Backfills, idempotency and alerting