HOW TO: Setup And Use Pyspark In Python (windows 10)
How to setup and use pyspark in Jupyter notebook?
1) pip install pyspark
2) pip install sparksql-magic
3) Download and install java: https://www.java.com/download/ie_manu...
4) Download and install latest Apache Spark: http://spark.apache.org/downloads.html
5) Paste extracted folder in C:\Spark\spark-3.1.2-bin-hadoop2.7
6) Set environment variable SPARK_HOME to point to this folder
7) Add SPARK_HOME\bin directory to PATH environment variable.
8) Move the Java folder from Program Files to just C:\
9) Set environment variable JAVA_HOME to point to C:\Java
10) Restart PC
11) Open jupyter-lab or jupyter-notebook
12) Start using pyspark!
P.S. maybe steps 6 and 7 can be neglected?