How spark works internally | what are the different components and how do they interact
This video talks about how spark components work together and what is a Job, Stage and task. We will walk through a spark program to see how stages and tasks are created.
0:00 Intro
0:36 A spark program walk through
1:25 creation of RDD
2:37 Execution
4:51 Spark Stages
6:55 Spark Tasks
9:00 Executor and task parallelism
12:10 spark component hierarchy