Adding steps to a cluster using the console - Amazon EMR

Adding steps to a cluster using the console

Use the following procedures to add steps to a cluster using the AWS Management Console. For detailed information about how to submit steps for specific big data applications, see the Amazon EMR Release Guide.

To add steps during cluster creation

Using the AWS Management Console, you can add steps to a cluster when the cluster is created.

  1. In the Amazon EMR console, choose Create Cluster - Advanced Options.

  2. On the Step 1: Software and Steps page, for Steps (optional), select Run multiple steps in parallel to improve cluster utilization and save cost. The default value for the concurrency level is 10. You can choose between 2 and 256 steps that can run in parallel.

    Note

    Running multiple steps in parallel is only supported with Amazon EMR version 5.28.0 and later.

  3. For After last step completes, choose Cluster enters waiting state or Auto-terminate the cluster.

  4. Choose Step type, then Add step.

  5. Type appropriate values in the fields in the Add Step dialog. Options differ depending on the step type. If you have enabled Run multiple steps in parallel to improve cluster utilization and save cost, the only available option for Action on failure is Continue. Next, choose Add.

To add steps to a running cluster

Using the AWS Management Console, you can add steps to a long-running cluster - that is, a cluster with the auto-terminate option disabled.

  1. In the Amazon EMR console, on the Cluster List page, select the link for your cluster.

  2. On the Cluster Details page, choose the Steps tab.

  3. On the Steps tab, choose Add step.

  4. Type appropriate values in the fields in the Add Step dialog, and then choose Add. The options differ depending on the step type.

To modify the step concurrency level in a running cluster

Using the AWS Management Console, you can modify the step concurrency level in a running cluster.

Note

Running multiple steps in parallel is only supported with Amazon EMR version 5.28.0 and later.

  1. In the Amazon EMR console, on the Cluster List page, select the link for your cluster.

  2. On the Cluster Details page, choose the Steps tab.

  3. For Concurrency, choose Change. Select a new value for the step concurrency level and then save.