Y

Library · tool

Apache Airflow and orchestration

Schedule, retry and monitor data pipelines as code, and run backfills safely. Taught with Airflow 3.

Orchestration appeared in 22% of the Data Engineer job descriptions we read, with Airflow the most named tool; Dagster and Prefect follow the same ideas.

Examples use Airflow 3 and the TaskFlow API, tested locally with airflow dags test.

  1. 1Your first DAGA monthly pipeline as an Airflow DAG4 min read

Apache Airflow and orchestration lessons

Loading lessons…