Delta Lake with Azure Databricks - Let's build a reliable Data Lake! | Mohit Batra
Want to learn more about GroupBy’s Sponsors? Check them out below!
SolarWinds: https://loom.ly/ljHr3eg
Redgate: https://loom.ly/Q65t2z0
Data Masterminds: https://loom.ly/JgisLIw
SSG: https://loom.ly/MotcEjE
While Data Lakes allow us to store massive amounts of data, there are many challenges that come along – maintaining data and multiple files, providing consistency to downstream applications, handling updates, data recovery in case of failures, slower transactional query performance, consolidation of streaming data and much more. On the other hand, Data Warehouses based on relational databases, handle many of these challenges, but they do not scale well and are expensive to maintain.
Are you interested to build a large-scale, reliable Data Lake?
This session will take you through Delta Lake – an open-source storage layer that brings ACID transactions and reliability to your data lakes. It can scale to handle TBs and PBs of data, handle updates, combine and store batch and streaming data, as well as data and metadata management. And this is what we will look into, via demos: