How to read large files in Apache spark || spark Performance tuning tips and tricks
In this video, have explained few concepts related to read large files in Spark
1. How to read large compressed csv files ?
2. while file format gives better performance?
3. What is the best file format for spark reads and writes?
4. How to utilize the full capacity of the spark resources?
5. Which compression is good ? why?
6. what is splitable and non-splitable compressions?
#azuredataengineer #sparkoptimization #sparkinterviewquestions #compression