Run Spark Applications On Local Laptop Using Single Node Spark Cluster Having Hdfs, Yarn and Hive.
Before you push your Spark code to code repo like GitHub, its good practice to test it on a cluster on your laptop itself using spark-submit. It saves time and gives confidence to developer about the code running on cluster without any problem. This video explains how to go about it. It uses docker image I have created which has Spark + hdfs + Yarn + Hive setup done.