Airflow, Spark, EMR - Building a Batch Data Pipeline by Emma Tang
Robust and user friendly data pipelines are at the foundation of powerful analytics, machine learning, and is at the core of allowing companies scale with their data. In this talk, we will walk through how to get started building a batch processing data pipeline end to end using Airflow, Spark on EMR. Through real code and live examples we will explore one of the most popular OSS data pipeline stacks.