TabularConf 2021
Building efficient and reliable data pipelines can be a time-consuming task. There are many aspects to consider; infrastructure, code, data ingress/egress, reporting, metrics, etc. Data Fusion is Google's answer to this problem, offering a managed CDAP environment to create data processing pipelines using an intuitive visual interface, with the added benefit of tight integration with the rest of Google Cloud Platform services and APIs. We'll take a tour around Data Fusion, build a real processing pipeline, throw some data at it and discuss pros and cons of this approach to ETL.