In this video we see how to skip records when loading CSV files using PySpark.
You can find the Databricks Notebooks here:
https://github.com/apostolos1927/PySp...
Follow me on social media:
LinkedIn: www.linkedin.com/in/apostolos-athanasiou-9a0baa119
GitHub: https://github.com/apostolos1927/
00:00 - Intro
01:20 - Skip records when loading CSV files in PySpark