Mini Data Engineering Project: Monitor Apache Airflow with Airbyte, Snowflake, and Superset

Опубликовано: 05 Ноябрь 2024
на канале: Data with Marc

1,945

101

Mini Data Engineering Project: Monitor Apache Airflow with Airbyte, Snowflake, and Superset

Notion Page: https://robust-dinosaur-2ef.notion.si...

In this video, you will build a dashboard to monitor your Airflow instance using different metrics.
You will learn:
✅ Using the Airflow REST API to build metrics
✅ Extracting data from your Airflow instance using Airbyte
✅ Creating an Airflow connector with the Airbyte AI Builder
✅ Storing data of your Airflow instance into Snowflake with Airbyte
✅ Defining metrics to monitor with Superset
✅ Running Airflow in the cloud with Astronomer
and more!

Enjoy ❤️