AWS::EMR::Cluster JobFlowInstancesConfig - AWS CloudFormation
SyntaxProperties

AWS::EMR::Cluster JobFlowInstancesConfig

JobFlowInstancesConfig is a property of the AWS::EMR::Cluster resource. JobFlowInstancesConfig defines the instance groups or instance fleets that comprise the cluster. JobFlowInstancesConfig must contain either InstanceFleetConfig or InstanceGroupConfig. They cannot be used together.

You can now define task instance groups or task instance fleets using the TaskInstanceGroups and TaskInstanceFleets subproperties. Using these subproperties reduces delays in provisioning task nodes compared to specifying task nodes with the InstanceFleetConfig and InstanceGroupConfig resources.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

Properties

AdditionalMasterSecurityGroups

A list of additional Amazon EC2 security group IDs for the master node.

Required: No

Type: Array of String

Update requires: No interruption

AdditionalSlaveSecurityGroups

A list of additional Amazon EC2 security group IDs for the core and task nodes.

Required: No

Type: Array of String

Update requires: No interruption

CoreInstanceFleet

Describes the EC2 instances and instance configurations for the core instance fleet when using clusters with the instance fleet configuration.

Required: No

Type: InstanceFleetConfig

Update requires: No interruption

CoreInstanceGroup

Describes the EC2 instances and instance configurations for core instance groups when using clusters with the uniform instance group configuration.

Required: No

Type: InstanceGroupConfig

Update requires: No interruption

Ec2KeyName

The name of the Amazon EC2 key pair that can be used to connect to the master node using SSH as the user called "hadoop."

Required: No

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Minimum: 0

Maximum: 256

Update requires: No interruption

Ec2SubnetId

Applies to clusters that use the uniform instance group configuration. To launch the cluster in Amazon Virtual Private Cloud (Amazon VPC), set this parameter to the identifier of the Amazon VPC subnet where you want the cluster to launch. If you do not specify this value and your account supports EC2-Classic, the cluster launches in EC2-Classic.

Required: No

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Minimum: 0

Maximum: 256

Update requires: No interruption

Ec2SubnetIds

Applies to clusters that use the instance fleet configuration. When multiple Amazon EC2 subnet IDs are specified, Amazon EMR evaluates them and launches instances in the optimal subnet.

Note

The instance fleet configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions.

Required: No

Type: Array of String

Update requires: No interruption

EmrManagedMasterSecurityGroup

The identifier of the Amazon EC2 security group for the master node. If you specify EmrManagedMasterSecurityGroup, you must also specify EmrManagedSlaveSecurityGroup.

Required: No

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Minimum: 0

Maximum: 256

Update requires: No interruption

EmrManagedSlaveSecurityGroup

The identifier of the Amazon EC2 security group for the core and task nodes. If you specify EmrManagedSlaveSecurityGroup, you must also specify EmrManagedMasterSecurityGroup.

Required: No

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Minimum: 0

Maximum: 256

Update requires: No interruption

HadoopVersion

Applies only to Amazon EMR release versions earlier than 4.0. The Hadoop version for the cluster. Valid inputs are "0.18" (no longer maintained), "0.20" (no longer maintained), "0.20.205" (no longer maintained), "1.0.3", "2.2.0", or "2.4.0". If you do not set this value, the default of 0.18 is used, unless the AmiVersion parameter is set in the RunJobFlow call, in which case the default version of Hadoop for that AMI version is used.

Required: No

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Minimum: 0

Maximum: 256

Update requires: No interruption

KeepJobFlowAliveWhenNoSteps

Specifies whether the cluster should remain available after completing all steps. Defaults to false. For more information about configuring cluster termination, see Control Cluster Termination in the EMR Management Guide.

Required: No

Type: Boolean

Update requires: No interruption

MasterInstanceFleet

Describes the EC2 instances and instance configurations for the master instance fleet when using clusters with the instance fleet configuration.

Required: No

Type: InstanceFleetConfig

Update requires: No interruption

MasterInstanceGroup

Describes the EC2 instances and instance configurations for the master instance group when using clusters with the uniform instance group configuration.

Required: No

Type: InstanceGroupConfig

Update requires: No interruption

Placement

The Availability Zone in which the cluster runs.

Required: No

Type: PlacementType

Update requires: No interruption

ServiceAccessSecurityGroup

The identifier of the Amazon EC2 security group for the Amazon EMR service to access clusters in VPC private subnets.

Required: No

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Minimum: 0

Maximum: 256

Update requires: No interruption

TaskInstanceFleets

Describes the EC2 instances and instance configurations for the task instance fleets when using clusters with the instance fleet configuration. These task instance fleets are added to the cluster as part of the cluster launch. Each task instance fleet must have a unique name specified so that CloudFormation can differentiate between the task instance fleets.

Note

You can currently specify only one task instance fleet for a cluster. After creating the cluster, you can only modify the mutable properties of InstanceFleetConfig, which are TargetOnDemandCapacity and TargetSpotCapacity. Modifying any other property results in cluster replacement.

Important

To allow a maximum of 30 Amazon EC2 instance types per fleet, include TaskInstanceFleets when you create your cluster. If you create your cluster without TaskInstanceFleets, Amazon EMR uses its default allocation strategy, which allows for a maximum of five Amazon EC2 instance types.

Required: No

Type: Array of InstanceFleetConfig

Update requires: No interruption

TaskInstanceGroups

Describes the EC2 instances and instance configurations for task instance groups when using clusters with the uniform instance group configuration. These task instance groups are added to the cluster as part of the cluster launch. Each task instance group must have a unique name specified so that CloudFormation can differentiate between the task instance groups.

Note

After creating the cluster, you can only modify the mutable properties of InstanceGroupConfig, which are AutoScalingPolicy and InstanceCount. Modifying any other property results in cluster replacement.

Required: No

Type: Array of InstanceGroupConfig

Update requires: No interruption

TerminationProtected

Specifies whether to lock the cluster to prevent the Amazon EC2 instances from being terminated by API call, user intervention, or in the event of a job-flow error.

Required: No

Type: Boolean

Update requires: No interruption

UnhealthyNodeReplacement

Indicates whether Amazon EMR should gracefully replace core nodes that have degraded within the cluster.

Required: No

Type: Boolean

Update requires: No interruption