DATALEARN | DE - 101 | MODULE 7-3 GETTING STARTED WITH APACHE SPARK
Apache Spark is the most popular tool among data engineers, analysts, and machine learning engineers. Its primary purpose is data processing. With Spark, you can connect to any data source, read big data, and process it in-memory using distributed computing.
In this video:
📌 Download and run Apache Spark
📌 Learn how to run Spark on Windows
📌 Explore the Spark UI
📌 Learn about the main components of Spark
📌 Get started using PySpark
Run a Spark program using spark-submit
For this lab, we'll take the existing M&Ms code and run it locally using Spark Submit. Then, we'll run the same code in a Databricks notebook, where we can see how the code executes interactively, piece by piece.
=====
In Module 7, we'll explore the open source solution for data analytics and engineering – Apache Spark and its commercial version, Databricks, along with its analogs, Amazon Glue and Azure Synapse. You'll learn about industry use cases and popular use cases. I'll share my experience with Apache Spark at Amazon and Microsoft, teach you how to work with data using PySpark and Spark SQL, and share the best books and resources on Spark.
🔔 Subscribe to the "Datalearn" channel to stay up to date with the rest of the series, and give it a thumbs up!
📕 Enroll and take the Data Engineer course.
⚠️ THE COURSE IS FREE!
🔗 You can enroll on our portal https://datalearn.ru/
👍🏻 Enrolling in the course will not only give you the opportunity to watch videos but also access restricted materials, complete homework assignments, and receive a course completion certificate.
🔥 Find the latest analytics news on our Telegram channel: https://t.me/rockyourdata