@backstreetbrogrammer

--------------------------------------------------------------------------------
Chapter 03 - Apache Spark Web UI - Jobs tab
--------------------------------------------------------------------------------
Spark Web UI can be viewed in browser using default port of 4040:

[ Ссылка ]

Spark UI is separated into following tabs:

- Jobs
The details provided in this tab are:

Scheduling mode: As we are running on a local machine, its Standalone mode. The standalone cluster mode currently only supports a simple FIFO scheduler across applications.

Number of Spark Jobs: The number of Spark jobs is equal to the number of actions in the application and each Spark job should have at least one Stage.

Number of Stages: Only Wide Transformation results in a separate Number of Stages.

Number of Tasks we could see in each stage is the number of partitions that Spark is going to work on and each task inside a stage is the same work that will be done by spark but on a different partition of data.

Description: Description links the complete details of the associated SparkJob like Spark Job Status, DAG Visualization, Completed Stages, etc.


Github: [ Ссылка ]

- Apache Spark for Java Developers Playlist: [ Ссылка ]
- Java Serialization Playlist: [ Ссылка ]
- Dynamic Programming Playlist: [ Ссылка ]_

#java #javadevelopers #javaprogramming #apachespark #spark

свернуть