CfnCluster
- class aws_cdk.aws_emr.CfnCluster(scope, id, *, instances, job_flow_role, name, service_role, additional_info=None, applications=None, auto_scaling_role=None, auto_termination_policy=None, bootstrap_actions=None, configurations=None, custom_ami_id=None, ebs_root_volume_size=None, kerberos_attributes=None, log_encryption_kms_key_id=None, log_uri=None, managed_scaling_policy=None, os_release_label=None, release_label=None, scale_down_behavior=None, security_configuration=None, step_concurrency_level=None, steps=None, tags=None, visible_to_all_users=None)
Bases:
CfnResource
A CloudFormation
AWS::EMR::Cluster
.The
AWS::EMR::Cluster
resource specifies an Amazon EMR cluster. This cluster is a collection of Amazon EC2 instances that run open source big data frameworks and applications to process and analyze vast amounts of data. For more information, see the Amazon EMR Management Guide .Amazon EMR now supports launching task instance groups and task instance fleets as part of the
AWS::EMR::Cluster
resource. This can be done by using theJobFlowInstancesConfig
property type’sTaskInstanceGroups
andTaskInstanceFleets
subproperties. Using these subproperties reduces delays in provisioning task nodes compared to specifying task nodes with theAWS::EMR::InstanceGroupConfig
andAWS::EMR::InstanceFleetConfig
resources. Please refer to the examples at the bottom of this page to learn how to use these subproperties.- CloudformationResource:
AWS::EMR::Cluster
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr # additional_info: Any # configuration_property_: emr.CfnCluster.ConfigurationProperty cfn_cluster = emr.CfnCluster(self, "MyCfnCluster", instances=emr.CfnCluster.JobFlowInstancesConfigProperty( additional_master_security_groups=["additionalMasterSecurityGroups"], additional_slave_security_groups=["additionalSlaveSecurityGroups"], core_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 ), core_instance_group=emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" ), ec2_key_name="ec2KeyName", ec2_subnet_id="ec2SubnetId", ec2_subnet_ids=["ec2SubnetIds"], emr_managed_master_security_group="emrManagedMasterSecurityGroup", emr_managed_slave_security_group="emrManagedSlaveSecurityGroup", hadoop_version="hadoopVersion", keep_job_flow_alive_when_no_steps=False, master_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 ), master_instance_group=emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" ), placement=emr.CfnCluster.PlacementTypeProperty( availability_zone="availabilityZone" ), service_access_security_group="serviceAccessSecurityGroup", task_instance_fleets=[emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 )], task_instance_groups=[emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" )], termination_protected=False ), job_flow_role="jobFlowRole", name="name", service_role="serviceRole", # the properties below are optional additional_info=additional_info, applications=[emr.CfnCluster.ApplicationProperty( additional_info={ "additional_info_key": "additionalInfo" }, args=["args"], name="name", version="version" )], auto_scaling_role="autoScalingRole", auto_termination_policy=emr.CfnCluster.AutoTerminationPolicyProperty( idle_timeout=123 ), bootstrap_actions=[emr.CfnCluster.BootstrapActionConfigProperty( name="name", script_bootstrap_action=emr.CfnCluster.ScriptBootstrapActionConfigProperty( path="path", # the properties below are optional args=["args"] ) )], configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_root_volume_size=123, kerberos_attributes=emr.CfnCluster.KerberosAttributesProperty( kdc_admin_password="kdcAdminPassword", realm="realm", # the properties below are optional ad_domain_join_password="adDomainJoinPassword", ad_domain_join_user="adDomainJoinUser", cross_realm_trust_principal_password="crossRealmTrustPrincipalPassword" ), log_encryption_kms_key_id="logEncryptionKmsKeyId", log_uri="logUri", managed_scaling_policy=emr.CfnCluster.ManagedScalingPolicyProperty( compute_limits=emr.CfnCluster.ComputeLimitsProperty( maximum_capacity_units=123, minimum_capacity_units=123, unit_type="unitType", # the properties below are optional maximum_core_capacity_units=123, maximum_on_demand_capacity_units=123 ) ), os_release_label="osReleaseLabel", release_label="releaseLabel", scale_down_behavior="scaleDownBehavior", security_configuration="securityConfiguration", step_concurrency_level=123, steps=[emr.CfnCluster.StepConfigProperty( hadoop_jar_step=emr.CfnCluster.HadoopJarStepConfigProperty( jar="jar", # the properties below are optional args=["args"], main_class="mainClass", step_properties=[emr.CfnCluster.KeyValueProperty( key="key", value="value" )] ), name="name", # the properties below are optional action_on_failure="actionOnFailure" )], tags=[CfnTag( key="key", value="value" )], visible_to_all_users=False )
Create a new
AWS::EMR::Cluster
.- Parameters:
scope (
Construct
) –scope in which this resource is defined.
id (
str
) –scoped id of the resource.
instances (
Union
[JobFlowInstancesConfigProperty
,Dict
[str
,Any
],IResolvable
]) – A specification of the number and type of Amazon EC2 instances.job_flow_role (
str
) – Also called instance profile and Amazon EC2 role. An IAM role for an Amazon EMR cluster. The Amazon EC2 instances of the cluster assume this role. The default role isEMR_EC2_DefaultRole
. In order to use the default role, you must have already created it using the AWS CLI or console.name (
str
) – The name of the cluster.service_role (
str
) – The IAM role that Amazon EMR assumes in order to access AWS resources on your behalf.additional_info (
Optional
[Any
]) – A JSON string for selecting additional features.applications (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,ApplicationProperty
,Dict
[str
,Any
]]],None
]) – The applications to install on this cluster, for example, Spark, Flink, Oozie, Zeppelin, and so on.auto_scaling_role (
Optional
[str
]) – An IAM role for automatic scaling policies. The default role isEMR_AutoScaling_DefaultRole
. The IAM role provides permissions that the automatic scaling feature requires to launch and terminate Amazon EC2 instances in an instance group.auto_termination_policy (
Union
[IResolvable
,AutoTerminationPolicyProperty
,Dict
[str
,Any
],None
]) –AWS::EMR::Cluster.AutoTerminationPolicy
.bootstrap_actions (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,BootstrapActionConfigProperty
,Dict
[str
,Any
]]],None
]) – A list of bootstrap actions to run before Hadoop starts on the cluster nodes.configurations (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,ConfigurationProperty
,Dict
[str
,Any
]]],None
]) – Applies only to Amazon EMR releases 4.x and later. The list of configurations that are supplied to the Amazon EMR cluster.custom_ami_id (
Optional
[str
]) – Available only in Amazon EMR releases 5.7.0 and later. The ID of a custom Amazon EBS-backed Linux AMI if the cluster uses a custom AMI.ebs_root_volume_size (
Union
[int
,float
,None
]) – The size, in GiB, of the Amazon EBS root device volume of the Linux AMI that is used for each Amazon EC2 instance. Available in Amazon EMR releases 4.x and later.kerberos_attributes (
Union
[IResolvable
,KerberosAttributesProperty
,Dict
[str
,Any
],None
]) – Attributes for Kerberos configuration when Kerberos authentication is enabled using a security configuration. For more information see Use Kerberos Authentication in the Amazon EMR Management Guide .log_encryption_kms_key_id (
Optional
[str
]) – The AWS KMS key used for encrypting log files. This attribute is only available with Amazon EMR 5.30.0 and later, excluding Amazon EMR 6.0.0.log_uri (
Optional
[str
]) – The path to the Amazon S3 location where logs for this cluster are stored.managed_scaling_policy (
Union
[IResolvable
,ManagedScalingPolicyProperty
,Dict
[str
,Any
],None
]) – Creates or updates a managed scaling policy for an Amazon EMR cluster. The managed scaling policy defines the limits for resources, such as Amazon EC2 instances that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.os_release_label (
Optional
[str
]) –AWS::EMR::Cluster.OSReleaseLabel
.release_label (
Optional
[str
]) – The Amazon EMR release label, which determines the version of open-source application packages installed on the cluster. Release labels are in the formemr-x.x.x
, where x.x.x is an Amazon EMR release version such asemr-5.14.0
. For more information about Amazon EMR release versions and included application versions and features, see ` <https://docs.aws.amazon.com/emr/latest/ReleaseGuide/>`_ . The release label applies only to Amazon EMR releases version 4.0 and later. Earlier versions useAmiVersion
.scale_down_behavior (
Optional
[str
]) – The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an instance group is resized.TERMINATE_AT_INSTANCE_HOUR
indicates that Amazon EMR terminates nodes at the instance-hour boundary, regardless of when the request to terminate the instance was submitted. This option is only available with Amazon EMR 5.1.0 and later and is the default for clusters created using that version.TERMINATE_AT_TASK_COMPLETION
indicates that Amazon EMR adds nodes to a deny list and drains tasks from nodes before terminating the Amazon EC2 instances, regardless of the instance-hour boundary. With either behavior, Amazon EMR removes the least active nodes first and blocks instance termination if it could lead to HDFS corruption.TERMINATE_AT_TASK_COMPLETION
is available only in Amazon EMR releases 4.1.0 and later, and is the default for versions of Amazon EMR earlier than 5.1.0.security_configuration (
Optional
[str
]) – The name of the security configuration applied to the cluster.step_concurrency_level (
Union
[int
,float
,None
]) – Specifies the number of steps that can be executed concurrently. The default value is1
. The maximum value is256
.steps (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,StepConfigProperty
,Dict
[str
,Any
]]],None
]) – A list of steps to run.tags (
Optional
[Sequence
[Union
[CfnTag
,Dict
[str
,Any
]]]]) – A list of tags associated with a cluster.visible_to_all_users (
Union
[bool
,IResolvable
,None
]) – Indicates whether the cluster is visible to all IAM users of the AWS account associated with the cluster. If this value is set totrue
, all IAM users of that AWS account can view and manage the cluster if they have the proper policy permissions set. If this value isfalse
, only the IAM user that created the cluster can view and manage it. This value can be changed using the SetVisibleToAllUsers action. .. epigraph:: When you create clusters directly through the EMR console or API, this value is set totrue
by default. However, forAWS::EMR::Cluster
resources in CloudFormation, the default isfalse
.
Methods
- add_deletion_override(path)
Syntactic sugar for
addOverride(path, undefined)
.- Parameters:
path (
str
) – The path of the value to delete.- Return type:
None
- add_depends_on(target)
Indicates that this resource depends on another resource and cannot be provisioned unless the other resource has been successfully provisioned.
This can be used for resources across stacks (or nested stack) boundaries and the dependency will automatically be transferred to the relevant scope.
- Parameters:
target (
CfnResource
)- Return type:
None
- add_metadata(key, value)
Add a value to the CloudFormation Resource Metadata.
- Parameters:
key (
str
)value (
Any
)
- See:
- Return type:
None
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html
Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.
- add_override(path, value)
Adds an override to the synthesized CloudFormation resource.
To add a property override, either use
addPropertyOverride
or prefixpath
with “Properties.” (i.e.Properties.TopicName
).If the override is nested, separate each nested level using a dot (.) in the path parameter. If there is an array as part of the nesting, specify the index in the path.
To include a literal
.
in the property name, prefix with a\
. In most programming languages you will need to write this as"\\."
because the\
itself will need to be escaped.For example:
cfn_resource.add_override("Properties.GlobalSecondaryIndexes.0.Projection.NonKeyAttributes", ["myattribute"]) cfn_resource.add_override("Properties.GlobalSecondaryIndexes.1.ProjectionType", "INCLUDE")
would add the overrides Example:
"Properties": { "GlobalSecondaryIndexes": [ { "Projection": { "NonKeyAttributes": [ "myattribute" ] ... } ... }, { "ProjectionType": "INCLUDE" ... }, ] ... }
The
value
argument toaddOverride
will not be processed or translated in any way. Pass raw JSON values in here with the correct capitalization for CloudFormation. If you pass CDK classes or structs, they will be rendered with lowercased key names, and CloudFormation will reject the template.- Parameters:
path (
str
) –The path of the property, you can use dot notation to override values in complex types. Any intermdediate keys will be created as needed.
value (
Any
) –The value. Could be primitive or complex.
- Return type:
None
- add_property_deletion_override(property_path)
Adds an override that deletes the value of a property from the resource definition.
- Parameters:
property_path (
str
) – The path to the property.- Return type:
None
- add_property_override(property_path, value)
Adds an override to a resource property.
Syntactic sugar for
addOverride("Properties.<...>", value)
.- Parameters:
property_path (
str
) – The path of the property.value (
Any
) – The value.
- Return type:
None
- apply_removal_policy(policy=None, *, apply_to_update_replace_policy=None, default=None)
Sets the deletion policy of the resource based on the removal policy specified.
The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you’ve removed it from the CDK application or because you’ve made a change that requires the resource to be replaced.
The resource can be deleted (
RemovalPolicy.DESTROY
), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN
).- Parameters:
policy (
Optional
[RemovalPolicy
])apply_to_update_replace_policy (
Optional
[bool
]) – Apply the same deletion policy to the resource’s “UpdateReplacePolicy”. Default: truedefault (
Optional
[RemovalPolicy
]) – The default policy to apply in case the removal policy is not defined. Default: - Default value is resource specific. To determine the default value for a resoure, please consult that specific resource’s documentation.
- Return type:
None
- get_att(attribute_name)
Returns a token for an runtime attribute of this resource.
Ideally, use generated attribute accessors (e.g.
resource.arn
), but this can be used for future compatibility in case there is no generated attribute.- Parameters:
attribute_name (
str
) – The name of the attribute.- Return type:
- get_metadata(key)
Retrieve a value value from the CloudFormation Resource Metadata.
- Parameters:
key (
str
)- See:
- Return type:
Any
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html
Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.
- inspect(inspector)
Examines the CloudFormation resource and discloses attributes.
- Parameters:
inspector (
TreeInspector
) –tree inspector to collect and process attributes.
- Return type:
None
- override_logical_id(new_logical_id)
Overrides the auto-generated logical ID with a specific ID.
- Parameters:
new_logical_id (
str
) – The new logical ID to use for this stack element.- Return type:
None
- to_string()
Returns a string representation of this construct.
- Return type:
str
- Returns:
a string representation of this resource
Attributes
- CFN_RESOURCE_TYPE_NAME = 'AWS::EMR::Cluster'
- additional_info
A JSON string for selecting additional features.
- applications
The applications to install on this cluster, for example, Spark, Flink, Oozie, Zeppelin, and so on.
- attr_master_public_dns
The public DNS name of the master node (instance), such as
ec2-12-123-123-123.us-west-2.compute.amazonaws.com
.- CloudformationAttribute:
MasterPublicDNS
- auto_scaling_role
An IAM role for automatic scaling policies.
The default role is
EMR_AutoScaling_DefaultRole
. The IAM role provides permissions that the automatic scaling feature requires to launch and terminate Amazon EC2 instances in an instance group.
- auto_termination_policy
AWS::EMR::Cluster.AutoTerminationPolicy
.
- bootstrap_actions
A list of bootstrap actions to run before Hadoop starts on the cluster nodes.
- cfn_options
Options for this resource, such as condition, update policy etc.
- cfn_resource_type
AWS resource type.
- configurations
Applies only to Amazon EMR releases 4.x and later. The list of configurations that are supplied to the Amazon EMR cluster.
- creation_stack
return:
the stack trace of the point where this Resource was created from, sourced from the +metadata+ entry typed +aws:cdk:logicalId+, and with the bottom-most node +internal+ entries filtered.
- custom_ami_id
Available only in Amazon EMR releases 5.7.0 and later. The ID of a custom Amazon EBS-backed Linux AMI if the cluster uses a custom AMI.
- ebs_root_volume_size
The size, in GiB, of the Amazon EBS root device volume of the Linux AMI that is used for each Amazon EC2 instance.
Available in Amazon EMR releases 4.x and later.
- instances
A specification of the number and type of Amazon EC2 instances.
- job_flow_role
Also called instance profile and Amazon EC2 role.
An IAM role for an Amazon EMR cluster. The Amazon EC2 instances of the cluster assume this role. The default role is
EMR_EC2_DefaultRole
. In order to use the default role, you must have already created it using the AWS CLI or console.
- kerberos_attributes
Attributes for Kerberos configuration when Kerberos authentication is enabled using a security configuration.
For more information see Use Kerberos Authentication in the Amazon EMR Management Guide .
- log_encryption_kms_key_id
The AWS KMS key used for encrypting log files.
This attribute is only available with Amazon EMR 5.30.0 and later, excluding Amazon EMR 6.0.0.
- log_uri
The path to the Amazon S3 location where logs for this cluster are stored.
- logical_id
The logical ID for this CloudFormation stack element.
The logical ID of the element is calculated from the path of the resource node in the construct tree.
To override this value, use
overrideLogicalId(newLogicalId)
.- Returns:
the logical ID as a stringified token. This value will only get resolved during synthesis.
- managed_scaling_policy
Creates or updates a managed scaling policy for an Amazon EMR cluster.
The managed scaling policy defines the limits for resources, such as Amazon EC2 instances that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
- name
The name of the cluster.
- node
The construct tree node associated with this construct.
- os_release_label
AWS::EMR::Cluster.OSReleaseLabel
.
- ref
Return a string that will be resolved to a CloudFormation
{ Ref }
for this element.If, by any chance, the intrinsic reference of a resource is not a string, you could coerce it to an IResolvable through
Lazy.any({ produce: resource.ref })
.
- release_label
The Amazon EMR release label, which determines the version of open-source application packages installed on the cluster.
Release labels are in the form
emr-x.x.x
, where x.x.x is an Amazon EMR release version such asemr-5.14.0
. For more information about Amazon EMR release versions and included application versions and features, see ` <https://docs.aws.amazon.com/emr/latest/ReleaseGuide/>`_ . The release label applies only to Amazon EMR releases version 4.0 and later. Earlier versions useAmiVersion
.
- scale_down_behavior
The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an instance group is resized.
TERMINATE_AT_INSTANCE_HOUR
indicates that Amazon EMR terminates nodes at the instance-hour boundary, regardless of when the request to terminate the instance was submitted. This option is only available with Amazon EMR 5.1.0 and later and is the default for clusters created using that version.TERMINATE_AT_TASK_COMPLETION
indicates that Amazon EMR adds nodes to a deny list and drains tasks from nodes before terminating the Amazon EC2 instances, regardless of the instance-hour boundary. With either behavior, Amazon EMR removes the least active nodes first and blocks instance termination if it could lead to HDFS corruption.TERMINATE_AT_TASK_COMPLETION
is available only in Amazon EMR releases 4.1.0 and later, and is the default for versions of Amazon EMR earlier than 5.1.0.
- security_configuration
The name of the security configuration applied to the cluster.
- service_role
The IAM role that Amazon EMR assumes in order to access AWS resources on your behalf.
- stack
The stack in which this element is defined.
CfnElements must be defined within a stack scope (directly or indirectly).
- step_concurrency_level
Specifies the number of steps that can be executed concurrently.
The default value is
1
. The maximum value is256
.
- steps
A list of steps to run.
- tags
A list of tags associated with a cluster.
- visible_to_all_users
Indicates whether the cluster is visible to all IAM users of the AWS account associated with the cluster.
If this value is set to
true
, all IAM users of that AWS account can view and manage the cluster if they have the proper policy permissions set. If this value isfalse
, only the IAM user that created the cluster can view and manage it. This value can be changed using the SetVisibleToAllUsers action. .. epigraph:When you create clusters directly through the EMR console or API, this value is set to ``true`` by default. However, for ``AWS::EMR::Cluster`` resources in CloudFormation, the default is ``false`` .
Static Methods
- classmethod is_cfn_element(x)
Returns
true
if a construct is a stack element (i.e. part of the synthesized cloudformation template).Uses duck-typing instead of
instanceof
to allow stack elements from different versions of this library to be included in the same stack.- Parameters:
x (
Any
)- Return type:
bool
- Returns:
The construct as a stack element or undefined if it is not a stack element.
- classmethod is_cfn_resource(construct)
Check whether the given construct is a CfnResource.
- Parameters:
construct (
IConstruct
)- Return type:
bool
- classmethod is_construct(x)
Return whether the given object is a Construct.
- Parameters:
x (
Any
)- Return type:
bool
ApplicationProperty
- class CfnCluster.ApplicationProperty(*, additional_info=None, args=None, name=None, version=None)
Bases:
object
Application
is a property ofAWS::EMR::Cluster
.The
Application
property type defines the open-source big data applications for EMR to install and configure when a cluster is created.With Amazon EMR release version 4.0 and later, the only accepted parameter is the application
Name
. To pass arguments to these applications, you use configuration classifications specified using JSON objects in aConfiguration
property. For more information, see Configuring Applications .With earlier Amazon EMR releases, the application is any AWS or third-party software that you can add to the cluster. You can specify the version of the application and arguments to pass to it. Amazon EMR accepts and forwards the argument list to the corresponding installation script as a bootstrap action argument.
- Parameters:
additional_info (
Union
[IResolvable
,Mapping
[str
,str
],None
]) – This option is for advanced users only. This is meta information about clusters and applications that are used for testing and troubleshooting.args (
Optional
[Sequence
[str
]]) – Arguments for Amazon EMR to pass to the application.name (
Optional
[str
]) – The name of the application.version (
Optional
[str
]) – The version of the application.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr application_property = emr.CfnCluster.ApplicationProperty( additional_info={ "additional_info_key": "additionalInfo" }, args=["args"], name="name", version="version" )
Attributes
- additional_info
This option is for advanced users only.
This is meta information about clusters and applications that are used for testing and troubleshooting.
- args
Arguments for Amazon EMR to pass to the application.
- name
The name of the application.
- version
The version of the application.
AutoScalingPolicyProperty
- class CfnCluster.AutoScalingPolicyProperty(*, constraints, rules)
Bases:
object
AutoScalingPolicy
is a subproperty ofInstanceGroupConfig
.AutoScalingPolicy
defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. For more information, see Using Automatic Scaling in Amazon EMR in the Amazon EMR Management Guide .- Parameters:
constraints (
Union
[IResolvable
,ScalingConstraintsProperty
,Dict
[str
,Any
]]) – The upper and lower Amazon EC2 instance limits for an automatic scaling policy. Automatic scaling activity will not cause an instance group to grow above or below these limits.rules (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,ScalingRuleProperty
,Dict
[str
,Any
]]]]) – The scale-in and scale-out rules that comprise the automatic scaling policy.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr auto_scaling_policy_property = emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] )
Attributes
- constraints
The upper and lower Amazon EC2 instance limits for an automatic scaling policy.
Automatic scaling activity will not cause an instance group to grow above or below these limits.
- rules
The scale-in and scale-out rules that comprise the automatic scaling policy.
AutoTerminationPolicyProperty
- class CfnCluster.AutoTerminationPolicyProperty(*, idle_timeout=None)
Bases:
object
- Parameters:
idle_timeout (
Union
[int
,float
,None
]) –CfnCluster.AutoTerminationPolicyProperty.IdleTimeout
.- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr auto_termination_policy_property = emr.CfnCluster.AutoTerminationPolicyProperty( idle_timeout=123 )
Attributes
- idle_timeout
CfnCluster.AutoTerminationPolicyProperty.IdleTimeout
.
BootstrapActionConfigProperty
- class CfnCluster.BootstrapActionConfigProperty(*, name, script_bootstrap_action)
Bases:
object
BootstrapActionConfig
is a property ofAWS::EMR::Cluster
that can be used to run bootstrap actions on EMR clusters.You can use a bootstrap action to install software and configure EC2 instances for all cluster nodes before EMR installs and configures open-source big data applications on cluster instances. For more information, see Create Bootstrap Actions to Install Additional Software in the Amazon EMR Management Guide .
- Parameters:
name (
str
) – The name of the bootstrap action.script_bootstrap_action (
Union
[IResolvable
,ScriptBootstrapActionConfigProperty
,Dict
[str
,Any
]]) – The script run by the bootstrap action.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr bootstrap_action_config_property = emr.CfnCluster.BootstrapActionConfigProperty( name="name", script_bootstrap_action=emr.CfnCluster.ScriptBootstrapActionConfigProperty( path="path", # the properties below are optional args=["args"] ) )
Attributes
- name
The name of the bootstrap action.
- script_bootstrap_action
The script run by the bootstrap action.
CloudWatchAlarmDefinitionProperty
- class CfnCluster.CloudWatchAlarmDefinitionProperty(*, comparison_operator, metric_name, period, threshold, dimensions=None, evaluation_periods=None, namespace=None, statistic=None, unit=None)
Bases:
object
CloudWatchAlarmDefinition
is a subproperty of theScalingTrigger
property, which determines when to trigger an automatic scaling activity.Scaling activity begins when you satisfy the defined alarm conditions.
- Parameters:
comparison_operator (
str
) – Determines how the metric specified byMetricName
is compared to the value specified byThreshold
.metric_name (
str
) – The name of the CloudWatch metric that is watched to determine an alarm condition.period (
Union
[int
,float
]) – The period, in seconds, over which the statistic is applied. CloudWatch metrics for Amazon EMR are emitted every five minutes (300 seconds), so if you specify a CloudWatch metric, specify300
.threshold (
Union
[int
,float
]) – The value against which the specified statistic is compared.dimensions (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,MetricDimensionProperty
,Dict
[str
,Any
]]],None
]) – A CloudWatch metric dimension.evaluation_periods (
Union
[int
,float
,None
]) – The number of periods, in five-minute increments, during which the alarm condition must exist before the alarm triggers automatic scaling activity. The default value is1
.namespace (
Optional
[str
]) – The namespace for the CloudWatch metric. The default isAWS/ElasticMapReduce
.statistic (
Optional
[str
]) – The statistic to apply to the metric associated with the alarm. The default isAVERAGE
.unit (
Optional
[str
]) – The unit of measure associated with the CloudWatch metric being watched. The value specified forUnit
must correspond to the units specified in the CloudWatch metric.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr cloud_watch_alarm_definition_property = emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" )
Attributes
- comparison_operator
Determines how the metric specified by
MetricName
is compared to the value specified byThreshold
.
- dimensions
A CloudWatch metric dimension.
- evaluation_periods
The number of periods, in five-minute increments, during which the alarm condition must exist before the alarm triggers automatic scaling activity.
The default value is
1
.
- metric_name
The name of the CloudWatch metric that is watched to determine an alarm condition.
- namespace
The namespace for the CloudWatch metric.
The default is
AWS/ElasticMapReduce
.
- period
The period, in seconds, over which the statistic is applied.
CloudWatch metrics for Amazon EMR are emitted every five minutes (300 seconds), so if you specify a CloudWatch metric, specify
300
.
- statistic
The statistic to apply to the metric associated with the alarm.
The default is
AVERAGE
.
- threshold
The value against which the specified statistic is compared.
- unit
The unit of measure associated with the CloudWatch metric being watched.
The value specified for
Unit
must correspond to the units specified in the CloudWatch metric.
ComputeLimitsProperty
- class CfnCluster.ComputeLimitsProperty(*, maximum_capacity_units, minimum_capacity_units, unit_type, maximum_core_capacity_units=None, maximum_on_demand_capacity_units=None)
Bases:
object
The Amazon EC2 unit limits for a managed scaling policy.
The managed scaling activity of a cluster can not be above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
- Parameters:
maximum_capacity_units (
Union
[int
,float
]) – The upper boundary of Amazon EC2 units. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.minimum_capacity_units (
Union
[int
,float
]) – The lower boundary of Amazon EC2 units. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.unit_type (
str
) – The unit type used for specifying a managed scaling policy.maximum_core_capacity_units (
Union
[int
,float
,None
]) – The upper boundary of Amazon EC2 units for core node type in a cluster. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The core units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between core and task nodes.maximum_on_demand_capacity_units (
Union
[int
,float
,None
]) – The upper boundary of On-Demand Amazon EC2 units. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The On-Demand units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between On-Demand and Spot Instances.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr compute_limits_property = emr.CfnCluster.ComputeLimitsProperty( maximum_capacity_units=123, minimum_capacity_units=123, unit_type="unitType", # the properties below are optional maximum_core_capacity_units=123, maximum_on_demand_capacity_units=123 )
Attributes
- maximum_capacity_units
The upper boundary of Amazon EC2 units.
It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
- maximum_core_capacity_units
The upper boundary of Amazon EC2 units for core node type in a cluster.
It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The core units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between core and task nodes.
- maximum_on_demand_capacity_units
The upper boundary of On-Demand Amazon EC2 units.
It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The On-Demand units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between On-Demand and Spot Instances.
- minimum_capacity_units
The lower boundary of Amazon EC2 units.
It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
- unit_type
The unit type used for specifying a managed scaling policy.
ConfigurationProperty
- class CfnCluster.ConfigurationProperty(*, classification=None, configuration_properties=None, configurations=None)
Bases:
object
Used only with Amazon EMR release 4.0 and later.
Configuration
is a subproperty ofInstanceFleetConfig
orInstanceGroupConfig
.Configuration
specifies optional configurations for customizing open-source big data applications and environment parameters. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file. For more information, see Configuring Applications in the Amazon EMR Release Guide .Attributes
- classification
The classification within a configuration.
- configuration_properties
A list of additional configurations to apply within a configuration object.
- configurations
A list of additional configurations to apply within a configuration object.
EbsBlockDeviceConfigProperty
- class CfnCluster.EbsBlockDeviceConfigProperty(*, volume_specification, volumes_per_instance=None)
Bases:
object
EbsBlockDeviceConfig
is a subproperty of theEbsConfiguration
property type.EbsBlockDeviceConfig
defines the number and type of EBS volumes to associate with all EC2 instances in an EMR cluster.- Parameters:
volume_specification (
Union
[IResolvable
,VolumeSpecificationProperty
,Dict
[str
,Any
]]) – EBS volume specifications such as volume type, IOPS, size (GiB) and throughput (MiB/s) that are requested for the EBS volume attached to an Amazon EC2 instance in the cluster.volumes_per_instance (
Union
[int
,float
,None
]) – Number of EBS volumes with a specific volume configuration that are associated with every instance in the instance group.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr ebs_block_device_config_property = emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )
Attributes
- volume_specification
EBS volume specifications such as volume type, IOPS, size (GiB) and throughput (MiB/s) that are requested for the EBS volume attached to an Amazon EC2 instance in the cluster.
- volumes_per_instance
Number of EBS volumes with a specific volume configuration that are associated with every instance in the instance group.
EbsConfigurationProperty
- class CfnCluster.EbsConfigurationProperty(*, ebs_block_device_configs=None, ebs_optimized=None)
Bases:
object
EbsConfiguration
is a subproperty ofInstanceFleetConfig
orInstanceGroupConfig
.EbsConfiguration
determines the EBS volumes to attach to EMR cluster instances.- Parameters:
ebs_block_device_configs (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,EbsBlockDeviceConfigProperty
,Dict
[str
,Any
]]],None
]) – An array of Amazon EBS volume specifications attached to a cluster instance.ebs_optimized (
Union
[bool
,IResolvable
,None
]) – Indicates whether an Amazon EBS volume is EBS-optimized.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr ebs_configuration_property = emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False )
Attributes
- ebs_block_device_configs
An array of Amazon EBS volume specifications attached to a cluster instance.
- ebs_optimized
Indicates whether an Amazon EBS volume is EBS-optimized.
HadoopJarStepConfigProperty
- class CfnCluster.HadoopJarStepConfigProperty(*, jar, args=None, main_class=None, step_properties=None)
Bases:
object
The
HadoopJarStepConfig
property type specifies a job flow step consisting of a JAR file whose main function will be executed.The main function submits a job for the cluster to execute as a step on the master node, and then waits for the job to finish or fail before executing subsequent steps.
- Parameters:
jar (
str
) – A path to a JAR file run during the step.args (
Optional
[Sequence
[str
]]) – A list of command line arguments passed to the JAR file’s main function when executed.main_class (
Optional
[str
]) – The name of the main class in the specified Java file. If not specified, the JAR file should specify a Main-Class in its manifest file.step_properties (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,KeyValueProperty
,Dict
[str
,Any
]]],None
]) – A list of Java properties that are set when the step runs. You can use these properties to pass key-value pairs to your main function.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr hadoop_jar_step_config_property = emr.CfnCluster.HadoopJarStepConfigProperty( jar="jar", # the properties below are optional args=["args"], main_class="mainClass", step_properties=[emr.CfnCluster.KeyValueProperty( key="key", value="value" )] )
Attributes
- args
A list of command line arguments passed to the JAR file’s main function when executed.
- jar
A path to a JAR file run during the step.
- main_class
The name of the main class in the specified Java file.
If not specified, the JAR file should specify a Main-Class in its manifest file.
- step_properties
A list of Java properties that are set when the step runs.
You can use these properties to pass key-value pairs to your main function.
InstanceFleetConfigProperty
- class CfnCluster.InstanceFleetConfigProperty(*, instance_type_configs=None, launch_specifications=None, name=None, target_on_demand_capacity=None, target_spot_capacity=None)
Bases:
object
Use
InstanceFleetConfig
to define instance fleets for an EMR cluster.A cluster can not use both instance fleets and instance groups. For more information, see Configure Instance Fleets in the Amazon EMR Management Guide . .. epigraph:
The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
- Parameters:
instance_type_configs (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,InstanceTypeConfigProperty
,Dict
[str
,Any
]]],None
]) – The instance type configurations that define the Amazon EC2 instances in the instance fleet.launch_specifications (
Union
[IResolvable
,InstanceFleetProvisioningSpecificationsProperty
,Dict
[str
,Any
],None
]) – The launch specification for the instance fleet.name (
Optional
[str
]) – The friendly name of the instance fleet.target_on_demand_capacity (
Union
[int
,float
,None
]) – The target capacity of On-Demand units for the instance fleet, which determines how many On-Demand instances to provision. When the instance fleet launches, Amazon EMR tries to provision On-Demand instances as specified byInstanceTypeConfig
. Each instance configuration has a specifiedWeightedCapacity
. When an On-Demand instance is provisioned, theWeightedCapacity
units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with aWeightedCapacity
of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:: If not specified or set to 0, only Spot instances are provisioned for the instance fleet usingTargetSpotCapacity
. At least one ofTargetSpotCapacity
andTargetOnDemandCapacity
should be greater than 0. For a master instance fleet, only one ofTargetSpotCapacity
andTargetOnDemandCapacity
can be specified, and its value must be 1.target_spot_capacity (
Union
[int
,float
,None
]) – The target capacity of Spot units for the instance fleet, which determines how many Spot instances to provision. When the instance fleet launches, Amazon EMR tries to provision Spot instances as specified byInstanceTypeConfig
. Each instance configuration has a specifiedWeightedCapacity
. When a Spot instance is provisioned, theWeightedCapacity
units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with aWeightedCapacity
of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:: If not specified or set to 0, only On-Demand instances are provisioned for the instance fleet. At least one ofTargetSpotCapacity
andTargetOnDemandCapacity
should be greater than 0. For a master instance fleet, only one ofTargetSpotCapacity
andTargetOnDemandCapacity
can be specified, and its value must be 1.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr # configuration_property_: emr.CfnCluster.ConfigurationProperty instance_fleet_config_property = emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 )
Attributes
- instance_type_configs
The instance type configurations that define the Amazon EC2 instances in the instance fleet.
- launch_specifications
The launch specification for the instance fleet.
- name
The friendly name of the instance fleet.
- target_on_demand_capacity
The target capacity of On-Demand units for the instance fleet, which determines how many On-Demand instances to provision.
When the instance fleet launches, Amazon EMR tries to provision On-Demand instances as specified by
InstanceTypeConfig
. Each instance configuration has a specifiedWeightedCapacity
. When an On-Demand instance is provisioned, theWeightedCapacity
units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with aWeightedCapacity
of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:If not specified or set to 0, only Spot instances are provisioned for the instance fleet using ``TargetSpotCapacity`` . At least one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` should be greater than 0. For a master instance fleet, only one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` can be specified, and its value must be 1.
- target_spot_capacity
The target capacity of Spot units for the instance fleet, which determines how many Spot instances to provision.
When the instance fleet launches, Amazon EMR tries to provision Spot instances as specified by
InstanceTypeConfig
. Each instance configuration has a specifiedWeightedCapacity
. When a Spot instance is provisioned, theWeightedCapacity
units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with aWeightedCapacity
of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:If not specified or set to 0, only On-Demand instances are provisioned for the instance fleet. At least one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` should be greater than 0. For a master instance fleet, only one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` can be specified, and its value must be 1.
InstanceFleetProvisioningSpecificationsProperty
- class CfnCluster.InstanceFleetProvisioningSpecificationsProperty(*, on_demand_specification=None, spot_specification=None)
Bases:
object
InstanceFleetProvisioningSpecification
is a subproperty ofInstanceFleetConfig
.InstanceFleetProvisioningSpecification
defines the launch specification for Spot instances in an instance fleet, which determines the defined duration and provisioning timeout behavior for Spot instances. .. epigraph:The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
- Parameters:
on_demand_specification (
Union
[IResolvable
,OnDemandProvisioningSpecificationProperty
,Dict
[str
,Any
],None
]) – The launch specification for On-Demand Instances in the instance fleet, which determines the allocation strategy. .. epigraph:: The instance fleet configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions. On-Demand Instances allocation strategy is available in Amazon EMR releases 5.12.1 and later.spot_specification (
Union
[IResolvable
,SpotProvisioningSpecificationProperty
,Dict
[str
,Any
],None
]) – The launch specification for Spot instances in the fleet, which determines the defined duration, provisioning timeout behavior, and allocation strategy.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr instance_fleet_provisioning_specifications_property = emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) )
Attributes
- on_demand_specification
The launch specification for On-Demand Instances in the instance fleet, which determines the allocation strategy.
The instance fleet configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions. On-Demand Instances allocation strategy is available in Amazon EMR releases 5.12.1 and later.
- spot_specification
The launch specification for Spot instances in the fleet, which determines the defined duration, provisioning timeout behavior, and allocation strategy.
InstanceGroupConfigProperty
- class CfnCluster.InstanceGroupConfigProperty(*, instance_count, instance_type, auto_scaling_policy=None, bid_price=None, configurations=None, custom_ami_id=None, ebs_configuration=None, market=None, name=None)
Bases:
object
Use
InstanceGroupConfig
to define instance groups for an EMR cluster.A cluster can not use both instance groups and instance fleets. For more information, see Create a Cluster with Instance Fleets or Uniform Instance Groups in the Amazon EMR Management Guide .
- Parameters:
instance_count (
Union
[int
,float
]) – Target number of instances for the instance group.instance_type (
str
) – The Amazon EC2 instance type for all instances in the instance group.auto_scaling_policy (
Union
[IResolvable
,AutoScalingPolicyProperty
,Dict
[str
,Any
],None
]) –AutoScalingPolicy
is a subproperty of the InstanceGroupConfig property type that specifies the constraints and rules of an automatic scaling policy in Amazon EMR . The automatic scaling policy defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. Only core and task instance groups can use automatic scaling policies. For more information, see Using Automatic Scaling in Amazon EMR .bid_price (
Optional
[str
]) – If specified, indicates that the instance group uses Spot Instances. This is the maximum price you are willing to pay for Spot Instances. SpecifyOnDemandPrice
to set the amount equal to the On-Demand price, or specify an amount in USD.configurations (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,ConfigurationProperty
,Dict
[str
,Any
]]],None
]) –Amazon EMR releases 4.x or later. The list of configurations supplied for an Amazon EMR cluster instance group. You can specify a separate configuration for each instance group (master, core, and task).
custom_ami_id (
Optional
[str
]) – The custom AMI ID to use for the provisioned instance group.ebs_configuration (
Union
[IResolvable
,EbsConfigurationProperty
,Dict
[str
,Any
],None
]) – EBS configurations that will be attached to each Amazon EC2 instance in the instance group.market (
Optional
[str
]) – Market type of the Amazon EC2 instances used to create a cluster node.name (
Optional
[str
]) – Friendly name given to the instance group.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr # configuration_property_: emr.CfnCluster.ConfigurationProperty instance_group_config_property = emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" )
Attributes
- auto_scaling_policy
AutoScalingPolicy
is a subproperty of the InstanceGroupConfig property type that specifies the constraints and rules of an automatic scaling policy in Amazon EMR . The automatic scaling policy defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. Only core and task instance groups can use automatic scaling policies. For more information, see Using Automatic Scaling in Amazon EMR .
- bid_price
If specified, indicates that the instance group uses Spot Instances.
This is the maximum price you are willing to pay for Spot Instances. Specify
OnDemandPrice
to set the amount equal to the On-Demand price, or specify an amount in USD.
- configurations
Amazon EMR releases 4.x or later.
The list of configurations supplied for an Amazon EMR cluster instance group. You can specify a separate configuration for each instance group (master, core, and task).
- custom_ami_id
The custom AMI ID to use for the provisioned instance group.
- ebs_configuration
EBS configurations that will be attached to each Amazon EC2 instance in the instance group.
- instance_count
Target number of instances for the instance group.
- instance_type
The Amazon EC2 instance type for all instances in the instance group.
- market
Market type of the Amazon EC2 instances used to create a cluster node.
- name
Friendly name given to the instance group.
InstanceTypeConfigProperty
- class CfnCluster.InstanceTypeConfigProperty(*, instance_type, bid_price=None, bid_price_as_percentage_of_on_demand_price=None, configurations=None, custom_ami_id=None, ebs_configuration=None, weighted_capacity=None)
Bases:
object
The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
InstanceTypeConfig
is a sub-property ofInstanceFleetConfig
.InstanceTypeConfig
determines the EC2 instances that Amazon EMR attempts to provision to fulfill On-Demand and Spot target capacities.Attributes
- bid_price
The bid price for each Amazon EC2 Spot Instance type as defined by
InstanceType
.Expressed in USD. If neither
BidPrice
norBidPriceAsPercentageOfOnDemandPrice
is provided,BidPriceAsPercentageOfOnDemandPrice
defaults to 100%.
- bid_price_as_percentage_of_on_demand_price
The bid price, as a percentage of On-Demand price, for each Amazon EC2 Spot Instance as defined by
InstanceType
.Expressed as a number (for example, 20 specifies 20%). If neither
BidPrice
norBidPriceAsPercentageOfOnDemandPrice
is provided,BidPriceAsPercentageOfOnDemandPrice
defaults to 100%.
- configurations
A configuration classification that applies when provisioning cluster instances, which can include configurations for applications and software that run on the cluster.
- custom_ami_id
The custom AMI ID to use for the instance type.
- ebs_configuration
The configuration of Amazon Elastic Block Store (Amazon EBS) attached to each instance as defined by
InstanceType
.
- instance_type
An Amazon EC2 instance type, such as
m3.xlarge
.
- weighted_capacity
The number of units that a provisioned instance of this type provides toward fulfilling the target capacities defined in
InstanceFleetConfig
.This value is 1 for a master instance fleet, and must be 1 or greater for core and task instance fleets. Defaults to 1 if not specified.
JobFlowInstancesConfigProperty
- class CfnCluster.JobFlowInstancesConfigProperty(*, additional_master_security_groups=None, additional_slave_security_groups=None, core_instance_fleet=None, core_instance_group=None, ec2_key_name=None, ec2_subnet_id=None, ec2_subnet_ids=None, emr_managed_master_security_group=None, emr_managed_slave_security_group=None, hadoop_version=None, keep_job_flow_alive_when_no_steps=None, master_instance_fleet=None, master_instance_group=None, placement=None, service_access_security_group=None, task_instance_fleets=None, task_instance_groups=None, termination_protected=None)
Bases:
object
JobFlowInstancesConfig
is a property of theAWS::EMR::Cluster
resource.JobFlowInstancesConfig
defines the instance groups or instance fleets that comprise the cluster.JobFlowInstancesConfig
must contain eitherInstanceFleetConfig
orInstanceGroupConfig
. They cannot be used together.You can now define task instance groups or task instance fleets using the
TaskInstanceGroups
andTaskInstanceFleets
subproperties. Using these subproperties reduces delays in provisioning task nodes compared to specifying task nodes with theInstanceFleetConfig
andInstanceGroupConfig
resources.- Parameters:
additional_master_security_groups (
Optional
[Sequence
[str
]]) – A list of additional Amazon EC2 security group IDs for the master node.additional_slave_security_groups (
Optional
[Sequence
[str
]]) – A list of additional Amazon EC2 security group IDs for the core and task nodes.core_instance_fleet (
Union
[IResolvable
,InstanceFleetConfigProperty
,Dict
[str
,Any
],None
]) – Describes the EC2 instances and instance configurations for the core instance fleet when using clusters with the instance fleet configuration.core_instance_group (
Union
[IResolvable
,InstanceGroupConfigProperty
,Dict
[str
,Any
],None
]) – Describes the EC2 instances and instance configurations for core instance groups when using clusters with the uniform instance group configuration.ec2_key_name (
Optional
[str
]) – The name of the Amazon EC2 key pair that can be used to connect to the master node using SSH as the user called “hadoop.”.ec2_subnet_id (
Optional
[str
]) – Applies to clusters that use the uniform instance group configuration. To launch the cluster in Amazon Virtual Private Cloud (Amazon VPC), set this parameter to the identifier of the Amazon VPC subnet where you want the cluster to launch. If you do not specify this value and your account supports EC2-Classic, the cluster launches in EC2-Classic.ec2_subnet_ids (
Optional
[Sequence
[str
]]) – Applies to clusters that use the instance fleet configuration. When multiple Amazon EC2 subnet IDs are specified, Amazon EMR evaluates them and launches instances in the optimal subnet. .. epigraph:: The instance fleet configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions.emr_managed_master_security_group (
Optional
[str
]) – The identifier of the Amazon EC2 security group for the master node. If you specifyEmrManagedMasterSecurityGroup
, you must also specifyEmrManagedSlaveSecurityGroup
.emr_managed_slave_security_group (
Optional
[str
]) – The identifier of the Amazon EC2 security group for the core and task nodes. If you specifyEmrManagedSlaveSecurityGroup
, you must also specifyEmrManagedMasterSecurityGroup
.hadoop_version (
Optional
[str
]) – Applies only to Amazon EMR release versions earlier than 4.0. The Hadoop version for the cluster. Valid inputs are “0.18” (no longer maintained), “0.20” (no longer maintained), “0.20.205” (no longer maintained), “1.0.3”, “2.2.0”, or “2.4.0”. If you do not set this value, the default of 0.18 is used, unless theAmiVersion
parameter is set in the RunJobFlow call, in which case the default version of Hadoop for that AMI version is used.keep_job_flow_alive_when_no_steps (
Union
[bool
,IResolvable
,None
]) – Specifies whether the cluster should remain available after completing all steps. Defaults totrue
. For more information about configuring cluster termination, see Control Cluster Termination in the EMR Management Guide .master_instance_fleet (
Union
[IResolvable
,InstanceFleetConfigProperty
,Dict
[str
,Any
],None
]) – Describes the EC2 instances and instance configurations for the master instance fleet when using clusters with the instance fleet configuration.master_instance_group (
Union
[IResolvable
,InstanceGroupConfigProperty
,Dict
[str
,Any
],None
]) – Describes the EC2 instances and instance configurations for the master instance group when using clusters with the uniform instance group configuration.placement (
Union
[IResolvable
,PlacementTypeProperty
,Dict
[str
,Any
],None
]) – The Availability Zone in which the cluster runs.service_access_security_group (
Optional
[str
]) – The identifier of the Amazon EC2 security group for the Amazon EMR service to access clusters in VPC private subnets.task_instance_fleets (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,InstanceFleetConfigProperty
,Dict
[str
,Any
]]],None
]) – Describes the EC2 instances and instance configurations for the task instance fleets when using clusters with the instance fleet configuration. These task instance fleets are added to the cluster as part of the cluster launch. Each task instance fleet must have a unique name specified so that CloudFormation can differentiate between the task instance fleets. .. epigraph:: You can currently specify only one task instance fleet for a cluster. After creating the cluster, you can only modify the mutable properties ofInstanceFleetConfig
, which areTargetOnDemandCapacity
andTargetSpotCapacity
. Modifying any other property results in cluster replacement. > To allow a maximum of 30 Amazon EC2 instance types per fleet, includeTaskInstanceFleets
when you create your cluster. If you create your cluster withoutTaskInstanceFleets
, Amazon EMR uses its default allocation strategy, which allows for a maximum of five Amazon EC2 instance types.task_instance_groups (
Union
[IResolvable
,Sequence
[Union
[IResolvable
,InstanceGroupConfigProperty
,Dict
[str
,Any
]]],None
]) – Describes the EC2 instances and instance configurations for task instance groups when using clusters with the uniform instance group configuration. These task instance groups are added to the cluster as part of the cluster launch. Each task instance group must have a unique name specified so that CloudFormation can differentiate between the task instance groups. .. epigraph:: After creating the cluster, you can only modify the mutable properties ofInstanceGroupConfig
, which areAutoScalingPolicy
andInstanceCount
. Modifying any other property results in cluster replacement.termination_protected (
Union
[bool
,IResolvable
,None
]) – Specifies whether to lock the cluster to prevent the Amazon EC2 instances from being terminated by API call, user intervention, or in the event of a job-flow error.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr # configuration_property_: emr.CfnCluster.ConfigurationProperty job_flow_instances_config_property = emr.CfnCluster.JobFlowInstancesConfigProperty( additional_master_security_groups=["additionalMasterSecurityGroups"], additional_slave_security_groups=["additionalSlaveSecurityGroups"], core_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 ), core_instance_group=emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" ), ec2_key_name="ec2KeyName", ec2_subnet_id="ec2SubnetId", ec2_subnet_ids=["ec2SubnetIds"], emr_managed_master_security_group="emrManagedMasterSecurityGroup", emr_managed_slave_security_group="emrManagedSlaveSecurityGroup", hadoop_version="hadoopVersion", keep_job_flow_alive_when_no_steps=False, master_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 ), master_instance_group=emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" ), placement=emr.CfnCluster.PlacementTypeProperty( availability_zone="availabilityZone" ), service_access_security_group="serviceAccessSecurityGroup", task_instance_fleets=[emr.CfnCluster.InstanceFleetConfigProperty( instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty( instance_type="instanceType", # the properties below are optional bid_price="bidPrice", bid_price_as_percentage_of_on_demand_price=123, configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), weighted_capacity=123 )], launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty( on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" ), spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 ) ), name="name", target_on_demand_capacity=123, target_spot_capacity=123 )], task_instance_groups=[emr.CfnCluster.InstanceGroupConfigProperty( instance_count=123, instance_type="instanceType", # the properties below are optional auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty( constraints=emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 ), rules=[emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )] ), bid_price="bidPrice", configurations=[emr.CfnCluster.ConfigurationProperty( classification="classification", configuration_properties={ "configuration_properties_key": "configurationProperties" }, configurations=[configuration_property_] )], custom_ami_id="customAmiId", ebs_configuration=emr.CfnCluster.EbsConfigurationProperty( ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty( volume_specification=emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 ), # the properties below are optional volumes_per_instance=123 )], ebs_optimized=False ), market="market", name="name" )], termination_protected=False )
Attributes
- additional_master_security_groups
A list of additional Amazon EC2 security group IDs for the master node.
- additional_slave_security_groups
A list of additional Amazon EC2 security group IDs for the core and task nodes.
- core_instance_fleet
Describes the EC2 instances and instance configurations for the core instance fleet when using clusters with the instance fleet configuration.
- core_instance_group
Describes the EC2 instances and instance configurations for core instance groups when using clusters with the uniform instance group configuration.
- ec2_key_name
The name of the Amazon EC2 key pair that can be used to connect to the master node using SSH as the user called “hadoop.”.
- ec2_subnet_id
Applies to clusters that use the uniform instance group configuration.
To launch the cluster in Amazon Virtual Private Cloud (Amazon VPC), set this parameter to the identifier of the Amazon VPC subnet where you want the cluster to launch. If you do not specify this value and your account supports EC2-Classic, the cluster launches in EC2-Classic.
- ec2_subnet_ids
Applies to clusters that use the instance fleet configuration.
When multiple Amazon EC2 subnet IDs are specified, Amazon EMR evaluates them and launches instances in the optimal subnet. .. epigraph:
The instance fleet configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions.
- emr_managed_master_security_group
The identifier of the Amazon EC2 security group for the master node.
If you specify
EmrManagedMasterSecurityGroup
, you must also specifyEmrManagedSlaveSecurityGroup
.
- emr_managed_slave_security_group
The identifier of the Amazon EC2 security group for the core and task nodes.
If you specify
EmrManagedSlaveSecurityGroup
, you must also specifyEmrManagedMasterSecurityGroup
.
- hadoop_version
Applies only to Amazon EMR release versions earlier than 4.0. The Hadoop version for the cluster. Valid inputs are “0.18” (no longer maintained), “0.20” (no longer maintained), “0.20.205” (no longer maintained), “1.0.3”, “2.2.0”, or “2.4.0”. If you do not set this value, the default of 0.18 is used, unless the
AmiVersion
parameter is set in the RunJobFlow call, in which case the default version of Hadoop for that AMI version is used.
- keep_job_flow_alive_when_no_steps
Specifies whether the cluster should remain available after completing all steps.
Defaults to
true
. For more information about configuring cluster termination, see Control Cluster Termination in the EMR Management Guide .
- master_instance_fleet
Describes the EC2 instances and instance configurations for the master instance fleet when using clusters with the instance fleet configuration.
- master_instance_group
Describes the EC2 instances and instance configurations for the master instance group when using clusters with the uniform instance group configuration.
- placement
The Availability Zone in which the cluster runs.
- service_access_security_group
The identifier of the Amazon EC2 security group for the Amazon EMR service to access clusters in VPC private subnets.
- task_instance_fleets
Describes the EC2 instances and instance configurations for the task instance fleets when using clusters with the instance fleet configuration.
These task instance fleets are added to the cluster as part of the cluster launch. Each task instance fleet must have a unique name specified so that CloudFormation can differentiate between the task instance fleets. .. epigraph:
You can currently specify only one task instance fleet for a cluster. After creating the cluster, you can only modify the mutable properties of ``InstanceFleetConfig`` , which are ``TargetOnDemandCapacity`` and ``TargetSpotCapacity`` . Modifying any other property results in cluster replacement. > To allow a maximum of 30 Amazon EC2 instance types per fleet, include ``TaskInstanceFleets`` when you create your cluster. If you create your cluster without ``TaskInstanceFleets`` , Amazon EMR uses its default allocation strategy, which allows for a maximum of five Amazon EC2 instance types.
- task_instance_groups
Describes the EC2 instances and instance configurations for task instance groups when using clusters with the uniform instance group configuration.
These task instance groups are added to the cluster as part of the cluster launch. Each task instance group must have a unique name specified so that CloudFormation can differentiate between the task instance groups. .. epigraph:
After creating the cluster, you can only modify the mutable properties of ``InstanceGroupConfig`` , which are ``AutoScalingPolicy`` and ``InstanceCount`` . Modifying any other property results in cluster replacement.
- termination_protected
Specifies whether to lock the cluster to prevent the Amazon EC2 instances from being terminated by API call, user intervention, or in the event of a job-flow error.
KerberosAttributesProperty
- class CfnCluster.KerberosAttributesProperty(*, kdc_admin_password, realm, ad_domain_join_password=None, ad_domain_join_user=None, cross_realm_trust_principal_password=None)
Bases:
object
KerberosAttributes
is a property of theAWS::EMR::Cluster
resource.KerberosAttributes
define the cluster-specific Kerberos configuration when Kerberos authentication is enabled using a security configuration. The cluster-specific configuration must be compatible with the security configuration. For more information see Use Kerberos Authentication in the EMR Management Guide .- Parameters:
kdc_admin_password (
str
) – The password used within the cluster for the kadmin service on the cluster-dedicated KDC, which maintains Kerberos principals, password policies, and keytabs for the cluster.realm (
str
) – The name of the Kerberos realm to which all nodes in a cluster belong. For example,EC2.INTERNAL
.ad_domain_join_password (
Optional
[str
]) – The Active Directory password forADDomainJoinUser
.ad_domain_join_user (
Optional
[str
]) – Required only when establishing a cross-realm trust with an Active Directory domain. A user with sufficient privileges to join resources to the domain.cross_realm_trust_principal_password (
Optional
[str
]) – Required only when establishing a cross-realm trust with a KDC in a different realm. The cross-realm principal password, which must be identical across realms.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr kerberos_attributes_property = emr.CfnCluster.KerberosAttributesProperty( kdc_admin_password="kdcAdminPassword", realm="realm", # the properties below are optional ad_domain_join_password="adDomainJoinPassword", ad_domain_join_user="adDomainJoinUser", cross_realm_trust_principal_password="crossRealmTrustPrincipalPassword" )
Attributes
- ad_domain_join_password
The Active Directory password for
ADDomainJoinUser
.
- ad_domain_join_user
Required only when establishing a cross-realm trust with an Active Directory domain.
A user with sufficient privileges to join resources to the domain.
- cross_realm_trust_principal_password
Required only when establishing a cross-realm trust with a KDC in a different realm.
The cross-realm principal password, which must be identical across realms.
- kdc_admin_password
The password used within the cluster for the kadmin service on the cluster-dedicated KDC, which maintains Kerberos principals, password policies, and keytabs for the cluster.
- realm
The name of the Kerberos realm to which all nodes in a cluster belong.
For example,
EC2.INTERNAL
.
KeyValueProperty
- class CfnCluster.KeyValueProperty(*, key=None, value=None)
Bases:
object
KeyValue
is a subproperty of theHadoopJarStepConfig
property type.KeyValue
is used to pass parameters to a step.- Parameters:
key (
Optional
[str
]) – The unique identifier of a key-value pair.value (
Optional
[str
]) – The value part of the identified key.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr key_value_property = emr.CfnCluster.KeyValueProperty( key="key", value="value" )
Attributes
- key
The unique identifier of a key-value pair.
- value
The value part of the identified key.
ManagedScalingPolicyProperty
- class CfnCluster.ManagedScalingPolicyProperty(*, compute_limits=None)
Bases:
object
Managed scaling policy for an Amazon EMR cluster.
The policy specifies the limits for resources that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
- Parameters:
compute_limits (
Union
[IResolvable
,ComputeLimitsProperty
,Dict
[str
,Any
],None
]) – The Amazon EC2 unit limits for a managed scaling policy. The managed scaling activity of a cluster is not allowed to go above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr managed_scaling_policy_property = emr.CfnCluster.ManagedScalingPolicyProperty( compute_limits=emr.CfnCluster.ComputeLimitsProperty( maximum_capacity_units=123, minimum_capacity_units=123, unit_type="unitType", # the properties below are optional maximum_core_capacity_units=123, maximum_on_demand_capacity_units=123 ) )
Attributes
- compute_limits
The Amazon EC2 unit limits for a managed scaling policy.
The managed scaling activity of a cluster is not allowed to go above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.
MetricDimensionProperty
- class CfnCluster.MetricDimensionProperty(*, key, value)
Bases:
object
MetricDimension
is a subproperty of theCloudWatchAlarmDefinition
property type.MetricDimension
specifies a CloudWatch dimension, which is specified with aKey
Value
pair. The key is known as aName
in CloudWatch. By default, Amazon EMR uses one dimension whoseKey
isJobFlowID
andValue
is a variable representing the cluster ID, which is${emr.clusterId}
. This enables the automatic scaling rule for EMR to bootstrap when the cluster ID becomes available during cluster creation.- Parameters:
key (
str
) – The dimension name.value (
str
) – The dimension value.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr metric_dimension_property = emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )
Attributes
- key
The dimension name.
OnDemandProvisioningSpecificationProperty
- class CfnCluster.OnDemandProvisioningSpecificationProperty(*, allocation_strategy)
Bases:
object
The launch specification for On-Demand Instances in the instance fleet, which determines the allocation strategy.
The instance fleet configuration is available only in Amazon EMR releases 4.8.0 and later, excluding 5.0.x versions. On-Demand Instances allocation strategy is available in Amazon EMR releases 5.12.1 and later.
- Parameters:
allocation_strategy (
str
) – Specifies the strategy to use in launching On-Demand instance fleets. Currently, the only option islowest-price
(the default), which launches the lowest price first.- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr on_demand_provisioning_specification_property = emr.CfnCluster.OnDemandProvisioningSpecificationProperty( allocation_strategy="allocationStrategy" )
Attributes
- allocation_strategy
Specifies the strategy to use in launching On-Demand instance fleets.
Currently, the only option is
lowest-price
(the default), which launches the lowest price first.
PlacementTypeProperty
- class CfnCluster.PlacementTypeProperty(*, availability_zone)
Bases:
object
PlacementType
is a property of theAWS::EMR::Cluster
resource.PlacementType
determines the Amazon EC2 Availability Zone configuration of the cluster (job flow).- Parameters:
availability_zone (
str
) – The Amazon EC2 Availability Zone for the cluster.AvailabilityZone
is used for uniform instance groups, whileAvailabilityZones
(plural) is used for instance fleets.- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr placement_type_property = emr.CfnCluster.PlacementTypeProperty( availability_zone="availabilityZone" )
Attributes
- availability_zone
The Amazon EC2 Availability Zone for the cluster.
AvailabilityZone
is used for uniform instance groups, whileAvailabilityZones
(plural) is used for instance fleets.
ScalingActionProperty
- class CfnCluster.ScalingActionProperty(*, simple_scaling_policy_configuration, market=None)
Bases:
object
ScalingAction
is a subproperty of theScalingRule
property type.ScalingAction
determines the type of adjustment the automatic scaling activity makes when triggered, and the periodicity of the adjustment.- Parameters:
simple_scaling_policy_configuration (
Union
[IResolvable
,SimpleScalingPolicyConfigurationProperty
,Dict
[str
,Any
]]) – The type of adjustment the automatic scaling activity makes when triggered, and the periodicity of the adjustment.market (
Optional
[str
]) – Not available for instance groups. Instance groups use the market type specified for the group.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr scaling_action_property = emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" )
Attributes
- market
Not available for instance groups.
Instance groups use the market type specified for the group.
- simple_scaling_policy_configuration
The type of adjustment the automatic scaling activity makes when triggered, and the periodicity of the adjustment.
ScalingConstraintsProperty
- class CfnCluster.ScalingConstraintsProperty(*, max_capacity, min_capacity)
Bases:
object
ScalingConstraints
is a subproperty of theAutoScalingPolicy
property type.ScalingConstraints
defines the upper and lower EC2 instance limits for an automatic scaling policy. Automatic scaling activities triggered by automatic scaling rules will not cause an instance group to grow above or shrink below these limits.- Parameters:
max_capacity (
Union
[int
,float
]) – The upper boundary of Amazon EC2 instances in an instance group beyond which scaling activities are not allowed to grow. Scale-out activities will not add instances beyond this boundary.min_capacity (
Union
[int
,float
]) – The lower boundary of Amazon EC2 instances in an instance group below which scaling activities are not allowed to shrink. Scale-in activities will not terminate instances below this boundary.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr scaling_constraints_property = emr.CfnCluster.ScalingConstraintsProperty( max_capacity=123, min_capacity=123 )
Attributes
- max_capacity
The upper boundary of Amazon EC2 instances in an instance group beyond which scaling activities are not allowed to grow.
Scale-out activities will not add instances beyond this boundary.
- min_capacity
The lower boundary of Amazon EC2 instances in an instance group below which scaling activities are not allowed to shrink.
Scale-in activities will not terminate instances below this boundary.
ScalingRuleProperty
- class CfnCluster.ScalingRuleProperty(*, action, name, trigger, description=None)
Bases:
object
ScalingRule
is a subproperty of theAutoScalingPolicy
property type.ScalingRule
defines the scale-in or scale-out rules for scaling activity, including the CloudWatch metric alarm that triggers activity, how EC2 instances are added or removed, and the periodicity of adjustments. The automatic scaling policy for an instance group can comprise one or more automatic scaling rules.- Parameters:
action (
Union
[IResolvable
,ScalingActionProperty
,Dict
[str
,Any
]]) – The conditions that trigger an automatic scaling activity.name (
str
) – The name used to identify an automatic scaling rule. Rule names must be unique within a scaling policy.trigger (
Union
[IResolvable
,ScalingTriggerProperty
,Dict
[str
,Any
]]) – The CloudWatch alarm definition that determines when automatic scaling activity is triggered.description (
Optional
[str
]) – A friendly, more verbose description of the automatic scaling rule.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr scaling_rule_property = emr.CfnCluster.ScalingRuleProperty( action=emr.CfnCluster.ScalingActionProperty( simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 ), # the properties below are optional market="market" ), name="name", trigger=emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) ), # the properties below are optional description="description" )
Attributes
- action
The conditions that trigger an automatic scaling activity.
- description
A friendly, more verbose description of the automatic scaling rule.
- name
The name used to identify an automatic scaling rule.
Rule names must be unique within a scaling policy.
- trigger
The CloudWatch alarm definition that determines when automatic scaling activity is triggered.
ScalingTriggerProperty
- class CfnCluster.ScalingTriggerProperty(*, cloud_watch_alarm_definition)
Bases:
object
ScalingTrigger
is a subproperty of theScalingRule
property type.ScalingTrigger
determines the conditions that trigger an automatic scaling activity.- Parameters:
cloud_watch_alarm_definition (
Union
[IResolvable
,CloudWatchAlarmDefinitionProperty
,Dict
[str
,Any
]]) – The definition of a CloudWatch metric alarm. When the defined alarm conditions are met along with other trigger parameters, scaling activity begins.- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr scaling_trigger_property = emr.CfnCluster.ScalingTriggerProperty( cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty( comparison_operator="comparisonOperator", metric_name="metricName", period=123, threshold=123, # the properties below are optional dimensions=[emr.CfnCluster.MetricDimensionProperty( key="key", value="value" )], evaluation_periods=123, namespace="namespace", statistic="statistic", unit="unit" ) )
Attributes
- cloud_watch_alarm_definition
The definition of a CloudWatch metric alarm.
When the defined alarm conditions are met along with other trigger parameters, scaling activity begins.
ScriptBootstrapActionConfigProperty
- class CfnCluster.ScriptBootstrapActionConfigProperty(*, path, args=None)
Bases:
object
ScriptBootstrapActionConfig
is a subproperty of theBootstrapActionConfig
property type.ScriptBootstrapActionConfig
specifies the arguments and location of the bootstrap script for EMR to run on all cluster nodes before it installs open-source big data applications on them.- Parameters:
path (
str
) – Location in Amazon S3 of the script to run during a bootstrap action.args (
Optional
[Sequence
[str
]]) – A list of command line arguments to pass to the bootstrap action script.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr script_bootstrap_action_config_property = emr.CfnCluster.ScriptBootstrapActionConfigProperty( path="path", # the properties below are optional args=["args"] )
Attributes
- args
A list of command line arguments to pass to the bootstrap action script.
- path
Location in Amazon S3 of the script to run during a bootstrap action.
SimpleScalingPolicyConfigurationProperty
- class CfnCluster.SimpleScalingPolicyConfigurationProperty(*, scaling_adjustment, adjustment_type=None, cool_down=None)
Bases:
object
SimpleScalingPolicyConfiguration
is a subproperty of theScalingAction
property type.SimpleScalingPolicyConfiguration
determines how an automatic scaling action adds or removes instances, the cooldown period, and the number of EC2 instances that are added each time the CloudWatch metric alarm condition is satisfied.- Parameters:
scaling_adjustment (
Union
[int
,float
]) – The amount by which to scale in or scale out, based on the specifiedAdjustmentType
. A positive value adds to the instance group’s Amazon EC2 instance count while a negative number removes instances. IfAdjustmentType
is set toEXACT_CAPACITY
, the number should only be a positive integer. IfAdjustmentType
is set toPERCENT_CHANGE_IN_CAPACITY
, the value should express the percentage as an integer. For example, -20 indicates a decrease in 20% increments of cluster capacity.adjustment_type (
Optional
[str
]) – The way in which Amazon EC2 instances are added (ifScalingAdjustment
is a positive number) or terminated (ifScalingAdjustment
is a negative number) each time the scaling activity is triggered.CHANGE_IN_CAPACITY
is the default.CHANGE_IN_CAPACITY
indicates that the Amazon EC2 instance count increments or decrements byScalingAdjustment
, which should be expressed as an integer.PERCENT_CHANGE_IN_CAPACITY
indicates the instance count increments or decrements by the percentage specified byScalingAdjustment
, which should be expressed as an integer. For example, 20 indicates an increase in 20% increments of cluster capacity.EXACT_CAPACITY
indicates the scaling activity results in an instance group with the number of Amazon EC2 instances specified byScalingAdjustment
, which should be expressed as a positive integer.cool_down (
Union
[int
,float
,None
]) – The amount of time, in seconds, after a scaling activity completes before any further trigger-related scaling activities can start. The default value is 0.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr simple_scaling_policy_configuration_property = emr.CfnCluster.SimpleScalingPolicyConfigurationProperty( scaling_adjustment=123, # the properties below are optional adjustment_type="adjustmentType", cool_down=123 )
Attributes
- adjustment_type
The way in which Amazon EC2 instances are added (if
ScalingAdjustment
is a positive number) or terminated (ifScalingAdjustment
is a negative number) each time the scaling activity is triggered.CHANGE_IN_CAPACITY
is the default.CHANGE_IN_CAPACITY
indicates that the Amazon EC2 instance count increments or decrements byScalingAdjustment
, which should be expressed as an integer.PERCENT_CHANGE_IN_CAPACITY
indicates the instance count increments or decrements by the percentage specified byScalingAdjustment
, which should be expressed as an integer. For example, 20 indicates an increase in 20% increments of cluster capacity.EXACT_CAPACITY
indicates the scaling activity results in an instance group with the number of Amazon EC2 instances specified byScalingAdjustment
, which should be expressed as a positive integer.
- cool_down
The amount of time, in seconds, after a scaling activity completes before any further trigger-related scaling activities can start.
The default value is 0.
- scaling_adjustment
The amount by which to scale in or scale out, based on the specified
AdjustmentType
.A positive value adds to the instance group’s Amazon EC2 instance count while a negative number removes instances. If
AdjustmentType
is set toEXACT_CAPACITY
, the number should only be a positive integer. IfAdjustmentType
is set toPERCENT_CHANGE_IN_CAPACITY
, the value should express the percentage as an integer. For example, -20 indicates a decrease in 20% increments of cluster capacity.
SpotProvisioningSpecificationProperty
- class CfnCluster.SpotProvisioningSpecificationProperty(*, timeout_action, timeout_duration_minutes, allocation_strategy=None, block_duration_minutes=None)
Bases:
object
SpotProvisioningSpecification
is a subproperty of theInstanceFleetProvisioningSpecifications
property type.SpotProvisioningSpecification
determines the launch specification for Spot instances in the instance fleet, which includes the defined duration and provisioning timeout behavior. .. epigraph:The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
- Parameters:
timeout_action (
str
) – The action to take whenTargetSpotCapacity
has not been fulfilled when theTimeoutDurationMinutes
has expired; that is, when all Spot Instances could not be provisioned within the Spot provisioning timeout. Valid values areTERMINATE_CLUSTER
andSWITCH_TO_ON_DEMAND
. SWITCH_TO_ON_DEMAND specifies that if no Spot Instances are available, On-Demand Instances should be provisioned to fulfill any remaining Spot capacity.timeout_duration_minutes (
Union
[int
,float
]) – The Spot provisioning timeout period in minutes. If Spot Instances are not provisioned within this time period, theTimeOutAction
is taken. Minimum value is 5 and maximum value is 1440. The timeout applies only during initial provisioning, when the cluster is first created.allocation_strategy (
Optional
[str
]) – Specifies one of the following strategies to launch Spot Instance fleets:price-capacity-optimized
,capacity-optimized
,lowest-price
, ordiversified
. For more information on the provisioning strategies, see Allocation strategies for Spot Instances in the Amazon EC2 User Guide for Linux Instances . .. epigraph:: When you launch a Spot Instance fleet with the old console, it automatically launches with thecapacity-optimized
strategy. You can’t change the allocation strategy from the old console.block_duration_minutes (
Union
[int
,float
,None
]) – The defined duration for Spot Instances (also known as Spot blocks) in minutes. When specified, the Spot Instance does not terminate before the defined duration expires, and defined duration pricing for Spot Instances applies. Valid values are 60, 120, 180, 240, 300, or 360. The duration period starts as soon as a Spot Instance receives its instance ID. At the end of the duration, Amazon EC2 marks the Spot Instance for termination and provides a Spot Instance termination notice, which gives the instance a two-minute warning before it terminates. .. epigraph:: Spot Instances with a defined duration (also known as Spot blocks) are no longer available to new customers from July 1, 2021. For customers who have previously used the feature, we will continue to support Spot Instances with a defined duration until December 31, 2022.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr spot_provisioning_specification_property = emr.CfnCluster.SpotProvisioningSpecificationProperty( timeout_action="timeoutAction", timeout_duration_minutes=123, # the properties below are optional allocation_strategy="allocationStrategy", block_duration_minutes=123 )
Attributes
- allocation_strategy
price-capacity-optimized
,capacity-optimized
,lowest-price
, ordiversified
.For more information on the provisioning strategies, see Allocation strategies for Spot Instances in the Amazon EC2 User Guide for Linux Instances . .. epigraph:
When you launch a Spot Instance fleet with the old console, it automatically launches with the ``capacity-optimized`` strategy. You can't change the allocation strategy from the old console.
- Link:
- Type:
Specifies one of the following strategies to launch Spot Instance fleets
- block_duration_minutes
The defined duration for Spot Instances (also known as Spot blocks) in minutes.
When specified, the Spot Instance does not terminate before the defined duration expires, and defined duration pricing for Spot Instances applies. Valid values are 60, 120, 180, 240, 300, or 360. The duration period starts as soon as a Spot Instance receives its instance ID. At the end of the duration, Amazon EC2 marks the Spot Instance for termination and provides a Spot Instance termination notice, which gives the instance a two-minute warning before it terminates. .. epigraph:
Spot Instances with a defined duration (also known as Spot blocks) are no longer available to new customers from July 1, 2021. For customers who have previously used the feature, we will continue to support Spot Instances with a defined duration until December 31, 2022.
- timeout_action
The action to take when
TargetSpotCapacity
has not been fulfilled when theTimeoutDurationMinutes
has expired;that is, when all Spot Instances could not be provisioned within the Spot provisioning timeout. Valid values are
TERMINATE_CLUSTER
andSWITCH_TO_ON_DEMAND
. SWITCH_TO_ON_DEMAND specifies that if no Spot Instances are available, On-Demand Instances should be provisioned to fulfill any remaining Spot capacity.
- timeout_duration_minutes
The Spot provisioning timeout period in minutes.
If Spot Instances are not provisioned within this time period, the
TimeOutAction
is taken. Minimum value is 5 and maximum value is 1440. The timeout applies only during initial provisioning, when the cluster is first created.
StepConfigProperty
- class CfnCluster.StepConfigProperty(*, hadoop_jar_step, name, action_on_failure=None)
Bases:
object
StepConfig
is a property of theAWS::EMR::Cluster
resource.The
StepConfig
property type specifies a cluster (job flow) step, which runs only on the master node. Steps are used to submit data processing jobs to the cluster.- Parameters:
hadoop_jar_step (
Union
[IResolvable
,HadoopJarStepConfigProperty
,Dict
[str
,Any
]]) – The JAR file used for the step.name (
str
) – The name of the step.action_on_failure (
Optional
[str
]) – The action to take when the cluster step fails. Possible values areCANCEL_AND_WAIT
andCONTINUE
.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr step_config_property = emr.CfnCluster.StepConfigProperty( hadoop_jar_step=emr.CfnCluster.HadoopJarStepConfigProperty( jar="jar", # the properties below are optional args=["args"], main_class="mainClass", step_properties=[emr.CfnCluster.KeyValueProperty( key="key", value="value" )] ), name="name", # the properties below are optional action_on_failure="actionOnFailure" )
Attributes
- action_on_failure
The action to take when the cluster step fails.
Possible values are
CANCEL_AND_WAIT
andCONTINUE
.
- hadoop_jar_step
The JAR file used for the step.
VolumeSpecificationProperty
- class CfnCluster.VolumeSpecificationProperty(*, size_in_gb, volume_type, iops=None)
Bases:
object
VolumeSpecification
is a subproperty of theEbsBlockDeviceConfig
property type.VolumeSecification
determines the volume type, IOPS, and size (GiB) for EBS volumes attached to EC2 instances.- Parameters:
size_in_gb (
Union
[int
,float
]) – The volume size, in gibibytes (GiB). This can be a number from 1 - 1024. If the volume type is EBS-optimized, the minimum value is 10.volume_type (
str
) – The volume type. Volume types supported are gp3, gp2, io1, st1, sc1, and standard.iops (
Union
[int
,float
,None
]) – The number of I/O operations per second (IOPS) that the volume supports.
- Link:
- ExampleMetadata:
fixture=_generated
Example:
# The code below shows an example of how to instantiate this type. # The values are placeholders you should change. import aws_cdk.aws_emr as emr volume_specification_property = emr.CfnCluster.VolumeSpecificationProperty( size_in_gb=123, volume_type="volumeType", # the properties below are optional iops=123 )
Attributes
- iops
The number of I/O operations per second (IOPS) that the volume supports.
- size_in_gb
The volume size, in gibibytes (GiB).
This can be a number from 1 - 1024. If the volume type is EBS-optimized, the minimum value is 10.
- volume_type
The volume type.
Volume types supported are gp3, gp2, io1, st1, sc1, and standard.