13 Read Complex File Formats | Parquet | ORC | Performance benefit of Parquet |Recursive File Lookup
Video explains - How to read Parquet files ? How to read ORC files ? Difference between Parquet and ORC files? How Row and Columnar File format works? How to read data recursively?
Chapters
00:00 - Introduction
01:24 - Benefit of Columnar Format
02:09 - Parquet vs ORC vs AVRO
03:39 - Read Parquet Data Files
04:32 - Read ORC Data Files
06:12 - Parquet and ORC documentation
06:33 - Performance benefit for Parquet
08:22 - Bonus Tip - Recursive File Lookup
Spark Parquet Documentation - https://spark.apache.org/docs/latest/...
Local PySpark Jupyter Lab setup - • 03 Setup Docker PySpark Jupyter Lab Enviro...
Python Basics - https://www.learnpython.org/
GitHub URL for code - https://github.com/subhamkharwal/pysp...
The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing.
New video in every 3 days ❤️
#spark #pyspark #python #dataengineering