Apache Spark Optimization - Optimizer Rules And How To Add Custom Rules.
This video explains about Apache Spark optimizations. How Spark built in optimizations like Rule based, cost based etc work in details. And how can you add your own custom optimization rules.
0:00 Introduction
2:00 Optimizer - Pattern Matching - Trees - Rules
7:26 Optimizer Execution - Analysis
8:24 Optimizer Execution - Logical Optimization
13:55 Optimizer Execution - Physical Planning
15:59 Optimizer Execution - Code Generation
18:03 Non Optimized Query Plan
22:22 Multiple Rules
25:57 Logical Plan to Physical Plan
33:38 Spark Optimizer - Cost Based Optimizer
Link for Ppts used the the video is - https://drive.google.com/file/d/1jNA3...