Working in Big Data? You should know about Spark - Spark Basics Simply Explained (Part 1)
You are working in Big Data and AI? You should know about Spark! If Data Engineer, Data Scientist, Stakeholder or Manager: You should watch the video below!
Are you a Data Engineer, Data Scientist, Stakeholder, Manager in Data and AI? Do you want to process Big Data lightning fast and efficient for your analytics use-cases?
Then you should know about Apache Spark! A state of the art framework for data processing and more. Simple, fast, scalable and reliable. On top all big cloud providers like Azure and AWS support it fully set up!
We will talk about:
What are the features of Spark?
How does it work lightning fast?
You want to master Data Engineering with PySpark? Subscribe here: https://www.youtube.com/@DataNikktheG...
Feel free to comment or challenge my explanations as always. Happy to learn also myself more by the community.
Link to slides: https://github.com/datanikkthegreek/S...
00:00 - Intro
01:30 - Apache Sparks Key Features
03:32 - Cloud providers for Apache Spark
05:07 - Key Components of Apache Spark on Python
09:15 - Why is Apache Spark lightning fast?
11:00 - Architecture on Apache Spark
16:57 - How Apache Spark optimizes using the Catalyst Optimizer and Lazy Evaluation
20:00 - Summary
#spark #pyspark #dataengineering #dataengineeringessentials