NOTE:
Please use a headset to get the optimal sound as Youtube has decreased the volume considerably!

This video is part of the Spark learning Series, where we will be learning Apache Spark step by step.

Prerequisites: JDK 8 should be installed and javac -version should point to your jdk 8 installation.
Set JAVA_HOME to your Java 8 path.
Please Set JAVA_HOME and HADOOP_HOME for the setup to work seamlessly.

Getting Started with the latest version of Apache Spark using Python and Scala in your local PC using Intellij , Windows, Mac , Linux Databricks and Apache Zeppelin.

This video on Spark installation will let you learn how to install and set up Apache Spark on Windows and MAC.

Please do provide your comments and subscribe to my channel!
[ Ссылка ]

Timestamps:
00:00 Introduction
01:04 Types of Spark Setups that can be done!
03:47 Downloading the Spark Binaries!!!
08:19 Setting HADOOP_HOME ...
10:17 Scala Spark In the command prompt...
14:48 PYSPARK In the command prompt...
17:12 Setting Up Intellij IDEA.
22:36 Scala Build Tool - SBT in Intellij
27:19 Adding Spark Dependencies in build.sbt and Reloading SBT.(Key Step).
28:19 Reimport Dependencies.
32:13 Setting Up The Databricks Community Edition.
47:34 Setting up Apache Zeppelin.

Spark Download: [ Ссылка ]
WinUtils : [ Ссылка ]
Databricks : [ Ссылка ]
Intellij Community Edition : [ Ссылка ]
Apache Zeppelin : [ Ссылка ]

Apache Spark is an open-source cluster-computing framework. Originally developed at the University of California, Berkeley's AMPLab, the Spark codebase was later donated to the Apache Software Foundation, which has maintained it since. Spark provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

We will see in detail on how to start from scratch with respect to learning Apache Spark.

Methods in which we can interact with Apache Spark:

Command Line REPL-(Scala and Python).
Intellij IDEA - (Covering Sbt Plugin).
Databricks Notebooks-(Scala and Python).
Zeppelin Notebooks.

First, you will see how to download the latest release of Spark.

Then you will set up a winutils executable file along with installing Spark.

You will also see how to setup environment variables as part of this installation and finally, you will understand how to run a small demo using scala in Spark.

Now, let's get started with installing Spark on windows and get some hands-on experience.

We will then see on how to get started with Intellij, Databricks and Zeppelin.

Coding in 4k.

#Spark #dataengineering #DataEngineer #freshers #beginners #ApacheSpark #4k #Learnin4K #CodingIn4K
#programmer #bigdataengineer #java #bigdataanalysis #onlinebigdatacourse #apache #apachesparktraining #computerscience #bigdatajobs #apachehadoop #hdfs #webdeveloper #dataanalyst #artificialintelligenceai #technology #coder #databricks #hadooptraining #apachekafka #itsecurity #dataprotection #debugging #github #codingtutorial
#bigdata #python #datascience #scala #hadoop #bigdataanalytics #machinelearning #aws #pyspark #bigdatatraining #coding #pythonprogramming #datascientist #data #growthmindset #dataanalytics #programming #bigdatacourse #onlinetraining #india #onlinebusiness #bigdatatechnologies #artificialintelligence #bigdatahadoop #zeppelin #winutils #sparkinwindows #Stream2Learn

свернуть