AWS CloudFormation
User Guide (Version )

AWS::EMR::InstanceGroupConfig

Use InstanceGroupConfig to define instance groups for an EMR cluster. A cluster can not use both instance groups and instance fleets. For more information, see Create a Cluster with Instance Fleets or Uniform Instance Groups in the Amazon EMR Management Guide.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON

{ "Type" : "AWS::EMR::InstanceGroupConfig", "Properties" : { "AutoScalingPolicy" : AutoScalingPolicy, "BidPrice" : String, "Configurations" : [ Configuration, ... ], "EbsConfiguration" : EbsConfiguration, "InstanceCount" : Integer, "InstanceRole" : String, "InstanceType" : String, "JobFlowId" : String, "Market" : String, "Name" : String } }

YAML

Type: AWS::EMR::InstanceGroupConfig Properties: AutoScalingPolicy: AutoScalingPolicy BidPrice: String Configurations: - Configuration EbsConfiguration: EbsConfiguration InstanceCount: Integer InstanceRole: String InstanceType: String JobFlowId: String Market: String Name: String

Properties

AutoScalingPolicy

AutoScalingPolicy is a subproperty of InstanceGroupConfig. AutoScalingPolicy defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. For more information, see Using Automatic Scaling in Amazon EMR in the Amazon EMR Management Guide.

Required: No

Type: AutoScalingPolicy

Update requires: No interruption

BidPrice

The maximum Spot price your are willing to pay for EC2 instances.

If BidPrice is specified, Amazon EMR uses Spot Instances for the instance group. Specified in USD. Alternatively, a value of OnDemandPrice indicates that the maximum Spot price is set equal to the On-Demand price.

Required: No

Type: String

Minimum: 0

Maximum: 256

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Update requires: Replacement

Configurations

Note

Amazon EMR releases 4.x or later.

The list of configurations supplied for an EMR cluster instance group. You can specify a separate configuration for each instance group (master, core, and task).

Required: No

Type: List of Configuration

Update requires: Replacement

EbsConfiguration

EbsConfiguration determines the EBS volumes to attach to EMR cluster instances.

Required: No

Type: EbsConfiguration

Update requires: Replacement

InstanceCount

Target number of instances for the instance group.

Required: Yes

Type: Integer

Update requires: No interruption

InstanceRole

The role of the instance group in the cluster.

Required: Yes

Type: String

Allowed Values: CORE | MASTER | TASK

Update requires: Replacement

InstanceType

The EC2 instance type for all instances in the instance group.

Required: Yes

Type: String

Minimum: 1

Maximum: 256

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Update requires: Replacement

JobFlowId

The ID of an Amazon EMR cluster that you want to associate this instance group with.

Required: Yes

Type: String

Update requires: Replacement

Market

Market type of the EC2 instances used to create a cluster node.

Required: No

Type: String

Allowed Values: ON_DEMAND | SPOT

Update requires: Replacement

Name

Friendly name given to the instance group.

Required: No

Type: String

Minimum: 0

Maximum: 256

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Update requires: Replacement

Return Values

Ref

When you pass the logical ID of this resource to the intrinsic Ref function, Ref returns returns the ID of the instance group.

For more information about using the Ref function, see Ref.

Examples

Example instance group configuration specifications.

Add a task instance group

The following example adds a task instance group to a cluster named TestCluster.

JSON

"TestInstanceGroupConfig": { "Type": "AWS: : EMR: : InstanceGroupConfig", "Properties": { "InstanceCount": 2, "InstanceType": "m3.xlarge", "InstanceRole": "TASK", "Market": "ON_DEMAND", "Name": "cfnTask2", "JobFlowId": { "Ref": "cluster" } } }

YAML

TestInstanceGroupConfig: Properties: InstanceCount: 2 InstanceRole: TASK InstanceType: m3.xlarge JobFlowId: Ref: cluster Market: ON_DEMAND Name: cfnTask2 Type: "AWS::EMR::InstanceGroupConfig"

Specify an Automatic Scaling Policy

The following example defines an AutoScalingPolicy for an InstanceGroupConfig resource.

JSON

"MyInstanceGroupConfig": { "Type": "AWS::EMR::InstanceGroupConfig", "Properties": { "InstanceCount": 1, "InstanceType": { "Ref": "InstanceType" }, "InstanceRole": "TASK", "Market": "ON_DEMAND", "Name": "cfnTask", "JobFlowId": { "Ref": "MyCluster" }, "AutoScalingPolicy": { "Constraints": { "MinCapacity": { "Ref": "MinCapacity" }, "MaxCapacity": { "Ref": "MaxCapacity" } }, "Rules": [ { "Name": "Scale-out", "Description": "Scale-out policy", "Action": { "SimpleScalingPolicyConfiguration": { "AdjustmentType": "CHANGE_IN_CAPACITY", "ScalingAdjustment": 1, "CoolDown": 300 } }, "Trigger": { "CloudWatchAlarmDefinition": { "Dimensions": [ { "Key": "JobFlowId", "Value": "${emr.clusterId}" } ], "EvaluationPeriods": 1, "Namespace": "AWS/ElasticMapReduce", "Period": 300, "ComparisonOperator": "LESS_THAN", "Statistic": "AVERAGE", "Threshold": 15, "Unit": "PERCENT", "MetricName": "YARNMemoryAvailablePercentage" } } }, { "Name": "Scale-in", "Description": "Scale-in policy", "Action": { "SimpleScalingPolicyConfiguration": { "AdjustmentType": "CHANGE_IN_CAPACITY", "ScalingAdjustment": -1, "CoolDown": 300 } }, "Trigger": { "CloudWatchAlarmDefinition": { "Dimensions": [ { "Key": "JobFlowId", "Value": "${emr.clusterId}" } ], "EvaluationPeriods": 1, "Namespace": "AWS/ElasticMapReduce", "Period": 300, "ComparisonOperator": "GREATER_THAN", "Statistic": "AVERAGE", "Threshold": 75, "Unit": "PERCENT", "MetricName": "YARNMemoryAvailablePercentage" } } } ] } } }

YAML

MyInstanceGroupConfig: Type: 'AWS::EMR::InstanceGroupConfig' Properties: InstanceCount: 1 InstanceType: !Ref InstanceType InstanceRole: TASK Market: ON_DEMAND Name: cfnTask JobFlowId: !Ref MyCluster AutoScalingPolicy: Constraints: MinCapacity: !Ref MinCapacity MaxCapacity: !Ref MaxCapacity Rules: - Name: Scale-out Description: Scale-out policy Action: SimpleScalingPolicyConfiguration: AdjustmentType: CHANGE_IN_CAPACITY ScalingAdjustment: 1 CoolDown: 300 Trigger: CloudWatchAlarmDefinition: Dimensions: - Key: JobFlowId Value: '${emr.clusterId}' EvaluationPeriods: 1 Namespace: AWS/ElasticMapReduce Period: 300 ComparisonOperator: LESS_THAN Statistic: AVERAGE Threshold: 15 Unit: PERCENT MetricName: YARNMemoryAvailablePercentage - Name: Scale-in Description: Scale-in policy Action: SimpleScalingPolicyConfiguration: AdjustmentType: CHANGE_IN_CAPACITY ScalingAdjustment: -1 CoolDown: 300 Trigger: CloudWatchAlarmDefinition: Dimensions: - Key: JobFlowId Value: '${emr.clusterId}' EvaluationPeriods: 1 Namespace: AWS/ElasticMapReduce Period: 300 ComparisonOperator: GREATER_THAN Statistic: AVERAGE Threshold: 75 Unit: PERCENT MetricName: YARNMemoryAvailablePercentage