Amazon EMR
Management Guide

The AWS Documentation website is getting a new look!
Try it now and let us know what you think. Switch to the new look >>

You can return to the original look by selecting English in the language selector above.

View Application History

Using Amazon EMR version 5.8.0 or later, you can view YARN application details using the Application history tab of a cluster's detail page in the console. Using Amazon EMR application history makes it easier for you to troubleshoot and analyze active jobs and job history. Instead of setting up and connecting to the master node to view open-source troubleshooting UIs or sift through log files, you can quickly view application metrics and access relevant log files.

Application history is enabled automatically for all clusters that run YARN applications. No special setup is required. Amazon EMR keeps historical information for up to seven days after an application has completed. Detailed application history is available only for Spark. There are additional minor limitations in Amazon EMR version 5.8.0. For more information, see Known Issues for version 5.8.0 in the Amazon EMR Release Guide.

Example: View Job Details for a Spark Application

The following sequence demonstrates a drill-down through a Spark application into job detail to evaluate stages, tasks, and executors using the Application history on a cluster details page (to view cluster details, from the Clusters list, select a cluster Name).

On the Application history tab, two rows are expanded to show the diagnostic summaries for two different Spark applications, and then an Application ID is selected to view further application detail:

On the Jobs tab of YARN application details, the Description of Job 0 is selected to see Job 0 details:

On the Job 0 details page, information about individual job stages is expanded, and then the Description for Stage 1 is selected to see Stage 1 details:

On the Stage 1 details page, key metrics for stage tasks and executors can be seen, and task and executor logs can be viewed using links: