Apache Spark 2.x Installation in Ubuntu
In this Video I am explaining about How to install Spark 2.0 in Ubuntu. For more spark tips subscribe into my blog http://www.bigdataanalyst.in
Documentation:
download from http://d3kbcqa49mib13.cloudfront.net/...
#add these lines in spark-env.sh file
export SPARK_CLASSPATH="/home/hadoop/work/apache-hive-2.0.0-bin/lib/mysql-connector-java-5.1.38-bin.jar"
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export SPARK_LOG_DIR=/home/hadoop/work/spark-2.0.0-preview-bin-hadoop2.7/logs
export SPARK_WORKER_DIR=/home/hadoop/work/spark-2.0.0-preview-bin-hadoop2.7/logs/worker_dir
If scala is not installed follow this video and run these script to install scala
#Install Java
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get update
sudo apt-get install openjdk-8-jdk
sudo update-alternatives --config java
sudo update-alternatives --config javac
or
sudo apt-get install openjdk-8-jdk
sudo apt-get install openjdk-8-jre
#Install Scala
cd ~/work
wget http://downloads.lightbend.com/scala/...
sudo tar xvf scala-2.11.8.tgz
gedit ~/.bashrc
export SCALA_HOME=/home/hadoop/work/scala-2.11.8
export PATH=$SCALA_HOME/bin:$PATH
#Test scala is installed or not?
scala
More info: • How to install (Latest) Apache Spark ...