Video explains - What is Data Skewness? How Data Skewness leads to Data Spillage ? How to Identify Skewness in Spark? What is Salting Technique? How to implement Salting Technique?
Chapters
00:00 - Introduction
03:57 - What is Data Skewness?
04:25 - Types of Spillage in Spark
05:28 - How to identify Skewness?
08:32 - How to fix Skewness ?
09:03 - What is Salting Technique?
09:53 - How is Salting done ?
Medium Link for Salting - / pyspark-the-famous-salting-technique
Local PySpark Jupyter Lab setup - • 03 Setup Docker PySpark Jupyter Lab Enviro...
Python Basics - https://www.learnpython.org/
GitHub URL for code - https://github.com/subhamkharwal/pysp...
The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing.
New video in every 3 days ❤️
#spark #pyspark #python #dataengineering