Realtime Streaming with Data Lakehouse - End to End Data Engineering Project
In this video you will learn to design, implement and maintain secure, scalable and cost effective lakehouse architectures leveraging Apache Spark, Apache Kafka, Apache Flink, Delta Lake, AWS, and open-source tools. Unlock data's full potential through advanced analytics and machine learning.
Part 1: • Building Data Lakehouse from Scratch - E...
FULL COURSE AVAILABLE: https://sh.datamasterylab.com/costsaver
Like this video?
Support us: / @codewithyu
Timestamps:
0:00 Setting up Kafka Broker in KRaft Mode
21:30 Setting up Minio
35:30 Producing data into Kafka
39:10 Acquiring Secret and Access Key for S3
59:00 Creating S3 Bucket Event Listener for Lakehouse
1:05:53 Data Preview and Results
1:07:42 Outro
Resources:
Youtube Source Code:
https://buymeacoffee.com/yusuf.ganiyu...
🌟 Please LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
👦🏻 My Linkedin: / yusuf-ganiyu-b90140107
🚀 X(Twitter): https://x.com/YusufOGaniyu
📝 Medium: / yusuf.ganiyu
Hashtags:
#dataengineering #bigdata #dataanalytics #realtimeanalytics #streaming, #datalakehouse, #datalake, #datawarehouse, #dataintegration, #datatransformation, #datagovernance, #datasecurity, #apachespark, #apachekafka, #apacheflink, #deltalake, #aws, #opensource, #dataingestion, #structureddata, #unstructureddata, #semi-structureddata, #dataanalysis, #advancedanalytics, #dataarchitecture, #costoptimization, #cloudcomputing, #awscloud