Menu
AWS CloudFormation
User Guide (API Version 2010-05-15)

AWS::EMR::Cluster

The AWS::EMR::Cluster resource creates an Amazon EMR cluster, which is a collection of EC2 instances on which you can run big data frameworks to process and analyze vast amounts of data. For more information, see Plan an Amazon EMR Cluster in the Amazon EMR Management Guide.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON

Copy
{ "Type" : "AWS::EMR::Cluster", "Properties" : { "AdditionalInfo" : JSON object, "Applications" : [ Applications, ... ], "AutoScalingRole" : String, "BootstrapActions" [ Bootstrap Actions, ... ], "Configurations" : [ Configurations, ... ], "Instances" : JobFlowInstancesConfig, "JobFlowRole" : String, "LogUri" : String, "Name" : String, "ReleaseLabel" : String, "ScaleDownBehavior" : String, "SecurityConfiguration" : String, "ServiceRole" : String, "Tags" : [ Resource Tag, ... ], "VisibleToAllUsers" : Boolean } }

YAML

Copy
Type: "AWS::EMR::Cluster" Properties: AdditionalInfo: JSON object Applications: - Applications AutoScalingRole: String BootstrapActions: - Bootstrap Actions Configurations: - Configurations Instances: JobFlowInstancesConfig JobFlowRole: String LogUri: String Name: String ReleaseLabel: String ScaleDownBehavior: String SecurityConfiguration: String ServiceRole: String Tags: - Resource Tag VisibleToAllUsers: Boolean

Properties

Note

For more information about the constraints and valid values of each property, see the Cluster data type in the Amazon EMR API Reference.

AdditionalInfo

Additional features that you want to select.

Required: No

Type: JSON object

Update requires: Replacement

Applications

The software applications to deploy on the cluster, and the arguments that Amazon EMR passes to those applications.

Required: No

Type: List of Amazon EMR Cluster Application

Update requires: Replacement

AutoScalingRole

An AWS Identity and Access Management (IAM) role for automatic scaling policies. The default role is EMR_AutoScaling_DefaultRole. The IAM role provides permissions that the automatic scaling feature requires to launch and terminate Amazon EC2 instances in an instance group.

Required: No

Type: String

Update requires: Replacement

BootstrapActions

A list of bootstrap actions that Amazon EMR runs before starting applications on the cluster.

Required: No

Type: List of Amazon EMR Cluster BootstrapActionConfig

Update requires: Replacement

Configurations

The software configuration of the Amazon EMR cluster.

Required: No

Type: List of Amazon EMR Cluster Configurations

Update requires: Replacement

Instances

Configures the EC2 instances that will run jobs in the Amazon EMR cluster.

Required: Yes

Type: Amazon EMR Cluster JobFlowInstancesConfig

Update requires: Replacement

JobFlowRole

Also called instance profile and EC2 role. Accepts an instance profile associated with the role that you want to use. All EC2 instances in the cluster assume this role.

Required: Yes

Type: String

Update requires: Replacement

LogUri

An S3 bucket location to which Amazon EMR writes logs files from a job flow. If you don't specify a value, Amazon EMR doesn't write any log files.

Required: No

Type: String

Update requires: Replacement

Name

A name for the Amazon EMR cluster.

Required: Yes

Type: String

Update requires: Replacement

ReleaseLabel

The Amazon EMR software release label. A release is a set of software applications and components that you can install and configure on an Amazon EMR cluster. For more information, see About Amazon EMR Releases in the Amazon EMR Release Guide.

Currently, AWS CloudFormation supports only Amazon EMR 4.0 and later software releases.

Required: Conditional. If you specify the Applications property, you must specify this property.

Type: String

Update requires: Replacement

ScaleDownBehavior

Indicates how individual EC2 instances terminate when an automatic scale-in activity occurs or an instance group is resized. For more information, see Cluster in the Amazon EMR API Reference.

Required: No

Type: String

Update requires: Replacement

SecurityConfiguration

The name of the security configuration applied to the cluster.

Required: No

Type: String

Update requires: Replacement

ServiceRole

The IAM role that Amazon EMR assumes to access AWS resources on your behalf. For more information, see Configure IAM Roles for Amazon EMR in the Amazon EMR Management Guide.

Required: Yes

Type: String

Update requires: Replacement

Tags

An arbitrary set of tags (key–value pairs) to help you identify the Amazon EMR cluster.

Required: No

Type: AWS CloudFormation Resource Tags

Update requires: No interruption

VisibleToAllUsers

Indicates whether the instances in the cluster are visible to all IAM users in the AWS account. If you specify true, all IAM users can view and (if they have permissions) manage the instances. If you specify false, only the IAM user that created the cluster can view and manage it. By default, AWS CloudFormation sets this property to false.

Required: No

Type: Boolean

Update requires: No interruption

Return Values

Ref

When the logical ID of this resource is provided to the Ref intrinsic function, Ref returns the cluster ID, such as j-1ABCD123AB1A.

For more information about using the Ref function, see Ref.

Fn::GetAtt

Fn::GetAtt returns a value for a specified attribute of this type. The following are the available attributes and sample return values.

MasterPublicDNS

The public DNS name of the master node (instance), such as ec2-12-123-123-123.us-west-2.compute.amazonaws.com.

For more information about using Fn::GetAtt, see Fn::GetAtt.

Examples

Create a Cluster With Two Core Nodes

The following example creates an Amazon EMR cluster with one master node and two core nodes. The specified IAM roles are the default roles provided by Amazon EMR. The example also assumes that the cluster is launched in an AWS Region with a default VPC and subnet. If you don't have these, use the Ec2SubnetId property to specify the VPC and subnet for the cluster. Otherwise, AWS CloudFormation can't launch the cluster and returns the following status message: ElasticMapReduce Cluster failed to stabilize.

JSON

Copy
"TestCluster": { "Type": "AWS::EMR::Cluster", "Properties": { "Instances": { "MasterInstanceGroup": { "InstanceCount": 1, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Master" }, "CoreInstanceGroup": { "InstanceCount": 2, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Core" }, "TerminationProtected" : true }, "Name": "TestCluster", "JobFlowRole": "EMR_EC2_DefaultRole", "ServiceRole": "EMR_DefaultRole", "ReleaseLabel": "emr-4.2.0", "Tags": [ { "Key": "IsTest", "Value": "True" } ] } }

YAML

Copy
TestCluster: Type: "AWS::EMR::Cluster" Properties: Instances: MasterInstanceGroup: InstanceCount: 1 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Master" CoreInstanceGroup: InstanceCount: 2 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Core" TerminationProtected: true Name: "TestCluster" JobFlowRole: "EMR_EC2_DefaultRole" ServiceRole: "EMR_DefaultRole" ReleaseLabel: "emr-4.2.0" Tags: - Key: "IsTest" Value: "True"

Create a Cluster with a Bootstrap Action

The following example creates an Amazon EMR cluster with a bootstrap action.

JSON

Copy
"TestCluster": { "Type": "AWS::EMR::Cluster", "Properties": { "BootstrapActions": [{ "Name": "SomeBootStrapAction", "ScriptBootstrapAction": { "Path": "/path/to/s3" } }], "Instances": { "MasterInstanceGroup": { "InstanceCount": 1, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Master" }, "CoreInstanceGroup": { "InstanceCount": 2, "InstanceType": "m3.xlarge", "Market": "ON_DEMAND", "Name": "Core" }, "TerminationProtected": true }, "Name": "TestCluster", "JobFlowRole": "EMR_EC2_DefaultRole", "ScaleDownBehavior": "TERMINATE_AT_TASK_COMPLETION", "ServiceRole": "EMR_DefaultRole", "ReleaseLabel": "emr-4.2.0", "Tags": [ { "Key": "IsTest", "Value": "True" } ] } }

YAML

Copy
TestCluster: Type: "AWS::EMR::Cluster" Properties: BootstrapActions: - Name: "SomeBootStrapAction" ScriptBootstrapAction: Path: "/path/to/s3" Instances: MasterInstanceGroup: InstanceCount: 1 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Master" CoreInstanceGroup: InstanceCount: 2 InstanceType: "m3.xlarge" Market: "ON_DEMAND" Name: "Core" TerminationProtected: true Name: "TestCluster" JobFlowRole: "EMR_EC2_DefaultRole" ScaleDownBehavior: "TERMINATE_AT_TASK_COMPLETION" ServiceRole: "EMR_DefaultRole" ReleaseLabel: "emr-4.2.0" Tags: - Key: "IsTest" Value: "True"