Databricks decoded: deployments, case studies, and lessons learned (keynote by Aditya & Arman)
Ever wondered how Databricks transforms real-world data projects? In their keynote at the Data Engineering Meetup in Berlin, Data Engineer Aditya Kadiwal and Cloud Engineer Arman Yazdani delved into various projects showcasing both the successes and challenges encountered with Databricks deployments.
✦ Learn more about how we apply AI in real-world solutions: https://applydata.io/
✦ Stay connected and follow our latest insights on LinkedIn: / applydata-by-diconium
✦ Why not join our Berlin Meetup community? https://www.meetup.com/applydata-berlin/
If you found this helpful, don’t forget to like, comment, and subscribe for more content updates!
00:00 Introduction to Databricks decoded
00:33 E-vehicle data analytics
03:17 Solution & data storage overview
07:13 Why Databricks?
12:33 The medallion architecture
14:20 Medallion architecture with DLT
14:42 The ingest & materialized view
18:45 The data quality
22:10 Pipeline dashboard & monitoring
25:30 Outro
#databricks #dataengineering #cloudengineering