Mastering Joins In Apache Spark: Complete Deep Dive
This video is sponsored by SerpApi. Get started with SerpApi using 250 free credits: https://serpapi.com/?utm_source=youtu...
A deep dive into the four core physical joins in Apache Spark: Sort Merge Join, Broadcast Hash Join, Shuffle Hash Join, and Broadcast Nested Loop Join. For each join, we do into depth and cover three things: the exact conditions that make Spark choose it, a step-by-step visual walkthrough of how it works under the hood, and a hands-on lab proving it with real code, the Spark physical plan, and the Spark UI. If you work with joins, partitions, and shuffles in Spark, this will give you the intuition to reason about join strategies and debug performance issues.
Chapters:
0:00 Intro
6:42 Sort Merge Join
19:38 Lab: Sort Merge Join
32:27 Broadcast Hash Join
39:57 Lab: Broadcast Hash Join
47:26 Shuffle Hash Join
58:40 Lab: Shuffle Hash Join – Lab
1:04:05 Broadcast Nested Loop Join
1:13:42 Lab: Broadcast Nested Loop Join
1:20:52 Conclusion
Connect w/ me here:
LinkedIn: / afaque-ahmad-5a5847129
YouTube Channel: / @afaqueahmad7117
Playlists:
Databricks Masterclass (Zero-To-Expert): • Databricks Masterclass: Zero To Expert
Ultimate Guide To Spark Performance Tuning: • Ultimate Guide To Apache Spark Performance...
Interview Preparation: • Data Engineering Interview Preparation
How I Mastered System Design Interviews: • How I Mastered System Design Interviews
Databricks End-To-End Project | Zero-To-Expert | Streaming, AI, Lakeflow, Unity Catalog, AI/BI: • How I Mastered System Design Interviews
Apache Spark Was Hard Until I Learned These 30 Concepts: • Apache Spark Was Hard Until I Learned Thes...
Data Engineering Roadmap: • The $500k+ Data Engineering Roadmap: Exact...
How I Mastered Data Modeling: • How I Mastered Data Modeling Interviews
Cracking Interviews @ Apple, Uber, Atlassian, Databricks: • This Is How You Can Crack Data Engineering...
Github: https://github.com/afaqueahmad7117
Spark Performance Tuning Codes: https://github.com/afaqueahmad7117/sp...
#DataEngineering #ApacheSpark #Databricks #sqljoins #joins