Apache Airflow Workshop [ENG]
In this workshop we explore Apache Airflow as a modern and the most effective orchestration tool for Big Data Developers. A short overview of the Apache Airflow architecture + dive into a live coding session.
The practical part contain:
Building a pipeline with sensors and several operators,
Writing the results from our pipeline to the database,
Setting up notifications in Telegram for the successful pipeline results,
Live discussion: the distribution of the tasks by workers, why pools are so essential.