Канал: BigDataElearning

Spark-submit command and flags

Spark-submit command and flags

Spark Broadcast variable

Spark Broadcast variable

Spark Accumulators

Spark Accumulators

Spark User-defined functions

Spark User-defined functions

Which filesystem to use HDFS or Amazon S3

Which filesystem to use HDFS or Amazon S3

Most common filesystems used by apache Spark

Most common filesystems used by apache Spark

How to remove the duplicate column when joining the datasets?

How to remove the duplicate column when joining the datasets?

Joining Datasets: How to join 2 datasets

Joining Datasets: How to join 2 datasets

RDD vs Dataframe vs Dataset

RDD vs Dataframe vs Dataset

How to create a Dataset in Spark : 4 ways to create a spark dataset

How to create a Dataset in Spark : 4 ways to create a spark dataset

What is a Dataset: 3 specific features that Dataset provides

What is a Dataset: 3 specific features that Dataset provides

How to create a dataframe from a CSV file

How to create a dataframe from a CSV file

How to create a dataframe from a text file

How to create a dataframe from a text file

How to create a Spark Dataframe from Parquet file?

How to create a Spark Dataframe from Parquet file?

How to create a dataframe from ElasticSearch?

How to create a dataframe from ElasticSearch?

How to create a dataframe from relational database table using JDBC?

How to create a dataframe from relational database table using JDBC?

How to create a Dataframe from JSON file? How to write the dataframe contents into a JSON file?

How to create a Dataframe from JSON file? How to write the dataframe contents into a JSON file?

Dataframe operations

Dataframe operations

What is a Spark Dataframe?

What is a Spark Dataframe?

How to programmatically specify a schema?

How to programmatically specify a schema?

what is Spark SQL

what is Spark SQL

Determining the number of partitions

Determining the number of partitions

How to create partitions in RDD

How to create partitions in RDD

Why should we partition the data in spark?

Why should we partition the data in spark?

What is RDD partitioning

What is RDD partitioning

Actions on pair RDDs

Actions on pair RDDs

Transformations on multi pair RDDs

Transformations on multi pair RDDs

combineByKey pair RDD Transformation

combineByKey pair RDD Transformation

Transformations on one pair RDD

Transformations on one pair RDD

How to Create Pair RDDs: Convert regular RDD to pair RDD: Create pair RDD from in-memory collection

How to Create Pair RDDs: Convert regular RDD to pair RDD: Create pair RDD from in-memory collection

What are Pair RDDs

What are Pair RDDs

Persistence and storage levels

Persistence and storage levels

Get familiarized with Scala and Python shells

Get familiarized with Scala and Python shells

How to install Spark in Windows

How to install Spark in Windows

What is Apache Spark

What is Apache Spark

Class Structure and components Part 2

Class Structure and components Part 2

JVM Architecture

JVM Architecture

Class Structure and components - Part I

Class Structure and components - Part I

How to install Java : How to install IntelliJ : How to create Hello World program in Java

How to install Java : How to install IntelliJ : How to create Hello World program in Java

Why is Java so famous : Why is Java so popular among Enterprises, Developers, and Students?

Why is Java so famous : Why is Java so popular among Enterprises, Developers, and Students?

Delete, Drop, and Truncate Statements in Hive

Delete, Drop, and Truncate Statements in Hive

How To Alter Hive Tables : ALTER Statement

How To Alter Hive Tables : ALTER Statement

How to Query Hive Tables : SELECT Statement

How to Query Hive Tables : SELECT Statement

Different Ways to Insert, Update Data in Hive Table

Different Ways to Insert, Update Data in Hive Table

Different Types of Hive Table: Managed Table and External Table

Different Types of Hive Table: Managed Table and External Table

HIVE data types: Hive data model

HIVE data types: Hive data model

What is Apache Hive? : Understanding Hive

What is Apache Hive? : Understanding Hive

Apache Spark : Commonly used Transformations : Map, Filter, Flatmap Transformations

Apache Spark : Commonly used Transformations : Map, Filter, Flatmap Transformations

Apache Spark RDD operations : Transformations and Actions

Apache Spark RDD operations : Transformations and Actions

Apache Spark RDD Basics : What is RDD, How to create an RDD

Apache Spark RDD Basics : What is RDD, How to create an RDD

Apache Spark Architecture : Run Time Architecture of Spark Application

Apache Spark Architecture : Run Time Architecture of Spark Application

Apache Spark Components : Different Components in Spark Framework

Apache Spark Components : Different Components in Spark Framework

Pig Describe , Explain , Illustrate Operators : Some useful development tools in Pig

Pig Describe , Explain , Illustrate Operators : Some useful development tools in Pig

Filtering the Data : How to Filter the data in pig

Filtering the Data : How to Filter the data in pig

Loading and Storing Data : How to Load and Store Data in Apache Pig

Loading and Storing Data : How to Load and Store Data in Apache Pig

Apache Pig Data Types : How to define data types in apache Pig

Apache Pig Data Types : How to define data types in apache Pig

Installing and Running Apache Pig : How to Install pig

Installing and Running Apache Pig : How to Install pig