CfnCluster

class aws_cdk.aws_emr.CfnCluster(scope, id, *, instances, job_flow_role, name, service_role, additional_info=None, applications=None, auto_scaling_role=None, auto_termination_policy=None, bootstrap_actions=None, configurations=None, custom_ami_id=None, ebs_root_volume_size=None, kerberos_attributes=None, log_encryption_kms_key_id=None, log_uri=None, managed_scaling_policy=None, release_label=None, scale_down_behavior=None, security_configuration=None, step_concurrency_level=None, steps=None, tags=None, visible_to_all_users=None)

Bases: aws_cdk.core.CfnResource

A CloudFormation AWS::EMR::Cluster.

The AWS::EMR::Cluster resource specifies an Amazon EMR cluster. This cluster is a collection of Amazon EC2 instances that run open source big data frameworks and applications to process and analyze vast amounts of data. For more information, see the Amazon EMR Management Guide .

Amazon EMR now supports launching task instance groups and task instance fleets as part of the AWS::EMR::Cluster resource. This can be done by using the JobFlowInstancesConfig property type’s TaskInstanceGroups and TaskInstanceFleets subproperties. Using these subproperties reduces delays in provisioning task nodes compared to specifying task nodes with the AWS::EMR::InstanceGroupConfig and AWS::EMR::InstanceFleetConfig resources. Please refer to the examples at the bottom of this page to learn how to use these subproperties.

CloudformationResource

AWS::EMR::Cluster

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

# additional_info: Any
# configuration_property_: emr.CfnCluster.ConfigurationProperty

cfn_cluster = emr.CfnCluster(self, "MyCfnCluster",
    instances=emr.CfnCluster.JobFlowInstancesConfigProperty(
        additional_master_security_groups=["additionalMasterSecurityGroups"],
        additional_slave_security_groups=["additionalSlaveSecurityGroups"],
        core_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty(
            instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
                instance_type="instanceType",

                # the properties below are optional
                bid_price="bidPrice",
                bid_price_as_percentage_of_on_demand_price=123,
                configurations=[emr.CfnCluster.ConfigurationProperty(
                    classification="classification",
                    configuration_properties={
                        "configuration_properties_key": "configurationProperties"
                    },
                    configurations=[configuration_property_]
                )],
                custom_ami_id="customAmiId",
                ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                    ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                        volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                            size_in_gb=123,
                            volume_type="volumeType",

                            # the properties below are optional
                            iops=123
                        ),

                        # the properties below are optional
                        volumes_per_instance=123
                    )],
                    ebs_optimized=False
                ),
                weighted_capacity=123
            )],
            launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
                on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
                    allocation_strategy="allocationStrategy"
                ),
                spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
                    timeout_action="timeoutAction",
                    timeout_duration_minutes=123,

                    # the properties below are optional
                    allocation_strategy="allocationStrategy",
                    block_duration_minutes=123
                )
            ),
            name="name",
            target_on_demand_capacity=123,
            target_spot_capacity=123
        ),
        core_instance_group=emr.CfnCluster.InstanceGroupConfigProperty(
            instance_count=123,
            instance_type="instanceType",

            # the properties below are optional
            auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
                constraints=emr.CfnCluster.ScalingConstraintsProperty(
                    max_capacity=123,
                    min_capacity=123
                ),
                rules=[emr.CfnCluster.ScalingRuleProperty(
                    action=emr.CfnCluster.ScalingActionProperty(
                        simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                            scaling_adjustment=123,

                            # the properties below are optional
                            adjustment_type="adjustmentType",
                            cool_down=123
                        ),

                        # the properties below are optional
                        market="market"
                    ),
                    name="name",
                    trigger=emr.CfnCluster.ScalingTriggerProperty(
                        cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                            comparison_operator="comparisonOperator",
                            metric_name="metricName",
                            period=123,
                            threshold=123,

                            # the properties below are optional
                            dimensions=[emr.CfnCluster.MetricDimensionProperty(
                                key="key",
                                value="value"
                            )],
                            evaluation_periods=123,
                            namespace="namespace",
                            statistic="statistic",
                            unit="unit"
                        )
                    ),

                    # the properties below are optional
                    description="description"
                )]
            ),
            bid_price="bidPrice",
            configurations=[emr.CfnCluster.ConfigurationProperty(
                classification="classification",
                configuration_properties={
                    "configuration_properties_key": "configurationProperties"
                },
                configurations=[configuration_property_]
            )],
            custom_ami_id="customAmiId",
            ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                        size_in_gb=123,
                        volume_type="volumeType",

                        # the properties below are optional
                        iops=123
                    ),

                    # the properties below are optional
                    volumes_per_instance=123
                )],
                ebs_optimized=False
            ),
            market="market",
            name="name"
        ),
        ec2_key_name="ec2KeyName",
        ec2_subnet_id="ec2SubnetId",
        ec2_subnet_ids=["ec2SubnetIds"],
        emr_managed_master_security_group="emrManagedMasterSecurityGroup",
        emr_managed_slave_security_group="emrManagedSlaveSecurityGroup",
        hadoop_version="hadoopVersion",
        keep_job_flow_alive_when_no_steps=False,
        master_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty(
            instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
                instance_type="instanceType",

                # the properties below are optional
                bid_price="bidPrice",
                bid_price_as_percentage_of_on_demand_price=123,
                configurations=[emr.CfnCluster.ConfigurationProperty(
                    classification="classification",
                    configuration_properties={
                        "configuration_properties_key": "configurationProperties"
                    },
                    configurations=[configuration_property_]
                )],
                custom_ami_id="customAmiId",
                ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                    ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                        volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                            size_in_gb=123,
                            volume_type="volumeType",

                            # the properties below are optional
                            iops=123
                        ),

                        # the properties below are optional
                        volumes_per_instance=123
                    )],
                    ebs_optimized=False
                ),
                weighted_capacity=123
            )],
            launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
                on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
                    allocation_strategy="allocationStrategy"
                ),
                spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
                    timeout_action="timeoutAction",
                    timeout_duration_minutes=123,

                    # the properties below are optional
                    allocation_strategy="allocationStrategy",
                    block_duration_minutes=123
                )
            ),
            name="name",
            target_on_demand_capacity=123,
            target_spot_capacity=123
        ),
        master_instance_group=emr.CfnCluster.InstanceGroupConfigProperty(
            instance_count=123,
            instance_type="instanceType",

            # the properties below are optional
            auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
                constraints=emr.CfnCluster.ScalingConstraintsProperty(
                    max_capacity=123,
                    min_capacity=123
                ),
                rules=[emr.CfnCluster.ScalingRuleProperty(
                    action=emr.CfnCluster.ScalingActionProperty(
                        simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                            scaling_adjustment=123,

                            # the properties below are optional
                            adjustment_type="adjustmentType",
                            cool_down=123
                        ),

                        # the properties below are optional
                        market="market"
                    ),
                    name="name",
                    trigger=emr.CfnCluster.ScalingTriggerProperty(
                        cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                            comparison_operator="comparisonOperator",
                            metric_name="metricName",
                            period=123,
                            threshold=123,

                            # the properties below are optional
                            dimensions=[emr.CfnCluster.MetricDimensionProperty(
                                key="key",
                                value="value"
                            )],
                            evaluation_periods=123,
                            namespace="namespace",
                            statistic="statistic",
                            unit="unit"
                        )
                    ),

                    # the properties below are optional
                    description="description"
                )]
            ),
            bid_price="bidPrice",
            configurations=[emr.CfnCluster.ConfigurationProperty(
                classification="classification",
                configuration_properties={
                    "configuration_properties_key": "configurationProperties"
                },
                configurations=[configuration_property_]
            )],
            custom_ami_id="customAmiId",
            ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                        size_in_gb=123,
                        volume_type="volumeType",

                        # the properties below are optional
                        iops=123
                    ),

                    # the properties below are optional
                    volumes_per_instance=123
                )],
                ebs_optimized=False
            ),
            market="market",
            name="name"
        ),
        placement=emr.CfnCluster.PlacementTypeProperty(
            availability_zone="availabilityZone"
        ),
        service_access_security_group="serviceAccessSecurityGroup",
        task_instance_fleets=[emr.CfnCluster.InstanceFleetConfigProperty(
            instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
                instance_type="instanceType",

                # the properties below are optional
                bid_price="bidPrice",
                bid_price_as_percentage_of_on_demand_price=123,
                configurations=[emr.CfnCluster.ConfigurationProperty(
                    classification="classification",
                    configuration_properties={
                        "configuration_properties_key": "configurationProperties"
                    },
                    configurations=[configuration_property_]
                )],
                custom_ami_id="customAmiId",
                ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                    ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                        volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                            size_in_gb=123,
                            volume_type="volumeType",

                            # the properties below are optional
                            iops=123
                        ),

                        # the properties below are optional
                        volumes_per_instance=123
                    )],
                    ebs_optimized=False
                ),
                weighted_capacity=123
            )],
            launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
                on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
                    allocation_strategy="allocationStrategy"
                ),
                spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
                    timeout_action="timeoutAction",
                    timeout_duration_minutes=123,

                    # the properties below are optional
                    allocation_strategy="allocationStrategy",
                    block_duration_minutes=123
                )
            ),
            name="name",
            target_on_demand_capacity=123,
            target_spot_capacity=123
        )],
        task_instance_groups=[emr.CfnCluster.InstanceGroupConfigProperty(
            instance_count=123,
            instance_type="instanceType",

            # the properties below are optional
            auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
                constraints=emr.CfnCluster.ScalingConstraintsProperty(
                    max_capacity=123,
                    min_capacity=123
                ),
                rules=[emr.CfnCluster.ScalingRuleProperty(
                    action=emr.CfnCluster.ScalingActionProperty(
                        simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                            scaling_adjustment=123,

                            # the properties below are optional
                            adjustment_type="adjustmentType",
                            cool_down=123
                        ),

                        # the properties below are optional
                        market="market"
                    ),
                    name="name",
                    trigger=emr.CfnCluster.ScalingTriggerProperty(
                        cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                            comparison_operator="comparisonOperator",
                            metric_name="metricName",
                            period=123,
                            threshold=123,

                            # the properties below are optional
                            dimensions=[emr.CfnCluster.MetricDimensionProperty(
                                key="key",
                                value="value"
                            )],
                            evaluation_periods=123,
                            namespace="namespace",
                            statistic="statistic",
                            unit="unit"
                        )
                    ),

                    # the properties below are optional
                    description="description"
                )]
            ),
            bid_price="bidPrice",
            configurations=[emr.CfnCluster.ConfigurationProperty(
                classification="classification",
                configuration_properties={
                    "configuration_properties_key": "configurationProperties"
                },
                configurations=[configuration_property_]
            )],
            custom_ami_id="customAmiId",
            ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                        size_in_gb=123,
                        volume_type="volumeType",

                        # the properties below are optional
                        iops=123
                    ),

                    # the properties below are optional
                    volumes_per_instance=123
                )],
                ebs_optimized=False
            ),
            market="market",
            name="name"
        )],
        termination_protected=False
    ),
    job_flow_role="jobFlowRole",
    name="name",
    service_role="serviceRole",

    # the properties below are optional
    additional_info=additional_info,
    applications=[emr.CfnCluster.ApplicationProperty(
        additional_info={
            "additional_info_key": "additionalInfo"
        },
        args=["args"],
        name="name",
        version="version"
    )],
    auto_scaling_role="autoScalingRole",
    auto_termination_policy=emr.CfnCluster.AutoTerminationPolicyProperty(
        idle_timeout=123
    ),
    bootstrap_actions=[emr.CfnCluster.BootstrapActionConfigProperty(
        name="name",
        script_bootstrap_action=emr.CfnCluster.ScriptBootstrapActionConfigProperty(
            path="path",

            # the properties below are optional
            args=["args"]
        )
    )],
    configurations=[emr.CfnCluster.ConfigurationProperty(
        classification="classification",
        configuration_properties={
            "configuration_properties_key": "configurationProperties"
        },
        configurations=[configuration_property_]
    )],
    custom_ami_id="customAmiId",
    ebs_root_volume_size=123,
    kerberos_attributes=emr.CfnCluster.KerberosAttributesProperty(
        kdc_admin_password="kdcAdminPassword",
        realm="realm",

        # the properties below are optional
        ad_domain_join_password="adDomainJoinPassword",
        ad_domain_join_user="adDomainJoinUser",
        cross_realm_trust_principal_password="crossRealmTrustPrincipalPassword"
    ),
    log_encryption_kms_key_id="logEncryptionKmsKeyId",
    log_uri="logUri",
    managed_scaling_policy=emr.CfnCluster.ManagedScalingPolicyProperty(
        compute_limits=emr.CfnCluster.ComputeLimitsProperty(
            maximum_capacity_units=123,
            minimum_capacity_units=123,
            unit_type="unitType",

            # the properties below are optional
            maximum_core_capacity_units=123,
            maximum_on_demand_capacity_units=123
        )
    ),
    release_label="releaseLabel",
    scale_down_behavior="scaleDownBehavior",
    security_configuration="securityConfiguration",
    step_concurrency_level=123,
    steps=[emr.CfnCluster.StepConfigProperty(
        hadoop_jar_step=emr.CfnCluster.HadoopJarStepConfigProperty(
            jar="jar",

            # the properties below are optional
            args=["args"],
            main_class="mainClass",
            step_properties=[emr.CfnCluster.KeyValueProperty(
                key="key",
                value="value"
            )]
        ),
        name="name",

        # the properties below are optional
        action_on_failure="actionOnFailure"
    )],
    tags=[CfnTag(
        key="key",
        value="value"
    )],
    visible_to_all_users=False
)

Create a new AWS::EMR::Cluster.

Parameters
  • scope (Construct) –

    • scope in which this resource is defined.

  • id (str) –

    • scoped id of the resource.

  • instances (Union[JobFlowInstancesConfigProperty, Dict[str, Any], IResolvable]) – A specification of the number and type of Amazon EC2 instances.

  • job_flow_role (str) – Also called instance profile and EC2 role. An IAM role for an EMR cluster. The EC2 instances of the cluster assume this role. The default role is EMR_EC2_DefaultRole . In order to use the default role, you must have already created it using the CLI or console.

  • name (str) – The name of the cluster.

  • service_role (str) – The IAM role that Amazon EMR assumes in order to access AWS resources on your behalf.

  • additional_info (Optional[Any]) – A JSON string for selecting additional features.

  • applications (Union[IResolvable, Sequence[Union[IResolvable, ApplicationProperty, Dict[str, Any]]], None]) – The applications to install on this cluster, for example, Spark, Flink, Oozie, Zeppelin, and so on.

  • auto_scaling_role (Optional[str]) – An IAM role for automatic scaling policies. The default role is EMR_AutoScaling_DefaultRole . The IAM role provides permissions that the automatic scaling feature requires to launch and terminate EC2 instances in an instance group.

  • auto_termination_policy (Union[IResolvable, AutoTerminationPolicyProperty, Dict[str, Any], None]) – AWS::EMR::Cluster.AutoTerminationPolicy.

  • bootstrap_actions (Union[IResolvable, Sequence[Union[IResolvable, BootstrapActionConfigProperty, Dict[str, Any]]], None]) – A list of bootstrap actions to run before Hadoop starts on the cluster nodes.

  • configurations (Union[IResolvable, Sequence[Union[IResolvable, ConfigurationProperty, Dict[str, Any]]], None]) – Applies only to Amazon EMR releases 4.x and later. The list of Configurations supplied to the EMR cluster.

  • custom_ami_id (Optional[str]) – Available only in Amazon EMR version 5.7.0 and later. The ID of a custom Amazon EBS-backed Linux AMI if the cluster uses a custom AMI.

  • ebs_root_volume_size (Union[int, float, None]) – The size, in GiB, of the Amazon EBS root device volume of the Linux AMI that is used for each EC2 instance. Available in Amazon EMR version 4.x and later.

  • kerberos_attributes (Union[IResolvable, KerberosAttributesProperty, Dict[str, Any], None]) – Attributes for Kerberos configuration when Kerberos authentication is enabled using a security configuration. For more information see Use Kerberos Authentication in the Amazon EMR Management Guide .

  • log_encryption_kms_key_id (Optional[str]) – The AWS KMS key used for encrypting log files. This attribute is only available with EMR version 5.30.0 and later, excluding EMR 6.0.0.

  • log_uri (Optional[str]) – The path to the Amazon S3 location where logs for this cluster are stored.

  • managed_scaling_policy (Union[IResolvable, ManagedScalingPolicyProperty, Dict[str, Any], None]) – Creates or updates a managed scaling policy for an Amazon EMR cluster. The managed scaling policy defines the limits for resources, such as EC2 instances that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

  • release_label (Optional[str]) – The Amazon EMR release label, which determines the version of open-source application packages installed on the cluster. Release labels are in the form emr-x.x.x , where x.x.x is an Amazon EMR release version such as emr-5.14.0 . For more information about Amazon EMR release versions and included application versions and features, see ` <https://docs.aws.amazon.com/emr/latest/ReleaseGuide/>`_ . The release label applies only to Amazon EMR releases version 4.0 and later. Earlier versions use AmiVersion .

  • scale_down_behavior (Optional[str]) – The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an instance group is resized. TERMINATE_AT_INSTANCE_HOUR indicates that Amazon EMR terminates nodes at the instance-hour boundary, regardless of when the request to terminate the instance was submitted. This option is only available with Amazon EMR 5.1.0 and later and is the default for clusters created using that version. TERMINATE_AT_TASK_COMPLETION indicates that Amazon EMR adds nodes to a deny list and drains tasks from nodes before terminating the Amazon EC2 instances, regardless of the instance-hour boundary. With either behavior, Amazon EMR removes the least active nodes first and blocks instance termination if it could lead to HDFS corruption. TERMINATE_AT_TASK_COMPLETION is available only in Amazon EMR version 4.1.0 and later, and is the default for versions of Amazon EMR earlier than 5.1.0.

  • security_configuration (Optional[str]) – The name of the security configuration applied to the cluster.

  • step_concurrency_level (Union[int, float, None]) – Specifies the number of steps that can be executed concurrently. The default value is 1 . The maximum value is 256 .

  • steps (Union[IResolvable, Sequence[Union[IResolvable, StepConfigProperty, Dict[str, Any]]], None]) – A list of steps to run.

  • tags (Optional[Sequence[Union[CfnTag, Dict[str, Any]]]]) – A list of tags associated with a cluster.

  • visible_to_all_users (Union[bool, IResolvable, None]) – Indicates whether the cluster is visible to all IAM users of the AWS account associated with the cluster. If this value is set to true , all IAM users of that AWS account can view and manage the cluster if they have the proper policy permissions set. If this value is false , only the IAM user that created the cluster can view and manage it. This value can be changed using the SetVisibleToAllUsers action. .. epigraph:: When you create clusters directly through the EMR console or API, this value is set to true by default. However, for AWS::EMR::Cluster resources in CloudFormation, the default is false .

Methods

add_deletion_override(path)

Syntactic sugar for addOverride(path, undefined).

Parameters

path (str) – The path of the value to delete.

Return type

None

add_depends_on(target)

Indicates that this resource depends on another resource and cannot be provisioned unless the other resource has been successfully provisioned.

This can be used for resources across stacks (or nested stack) boundaries and the dependency will automatically be transferred to the relevant scope.

Parameters

target (CfnResource) –

Return type

None

add_metadata(key, value)

Add a value to the CloudFormation Resource Metadata.

Parameters
  • key (str) –

  • value (Any) –

See

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html

Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.

Return type

None

add_override(path, value)

Adds an override to the synthesized CloudFormation resource.

To add a property override, either use addPropertyOverride or prefix path with “Properties.” (i.e. Properties.TopicName).

If the override is nested, separate each nested level using a dot (.) in the path parameter. If there is an array as part of the nesting, specify the index in the path.

To include a literal . in the property name, prefix with a \. In most programming languages you will need to write this as "\\." because the \ itself will need to be escaped.

For example:

cfn_resource.add_override("Properties.GlobalSecondaryIndexes.0.Projection.NonKeyAttributes", ["myattribute"])
cfn_resource.add_override("Properties.GlobalSecondaryIndexes.1.ProjectionType", "INCLUDE")

would add the overrides Example:

"Properties": {
   "GlobalSecondaryIndexes": [
     {
       "Projection": {
         "NonKeyAttributes": [ "myattribute" ]
         ...
       }
       ...
     },
     {
       "ProjectionType": "INCLUDE"
       ...
     },
   ]
   ...
}

The value argument to addOverride will not be processed or translated in any way. Pass raw JSON values in here with the correct capitalization for CloudFormation. If you pass CDK classes or structs, they will be rendered with lowercased key names, and CloudFormation will reject the template.

Parameters
  • path (str) –

    • The path of the property, you can use dot notation to override values in complex types. Any intermdediate keys will be created as needed.

  • value (Any) –

    • The value. Could be primitive or complex.

Return type

None

add_property_deletion_override(property_path)

Adds an override that deletes the value of a property from the resource definition.

Parameters

property_path (str) – The path to the property.

Return type

None

add_property_override(property_path, value)

Adds an override to a resource property.

Syntactic sugar for addOverride("Properties.<...>", value).

Parameters
  • property_path (str) – The path of the property.

  • value (Any) – The value.

Return type

None

apply_removal_policy(policy=None, *, apply_to_update_replace_policy=None, default=None)

Sets the deletion policy of the resource based on the removal policy specified.

The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you’ve removed it from the CDK application or because you’ve made a change that requires the resource to be replaced.

The resource can be deleted (RemovalPolicy.DESTROY), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN).

Parameters
  • policy (Optional[RemovalPolicy]) –

  • apply_to_update_replace_policy (Optional[bool]) – Apply the same deletion policy to the resource’s “UpdateReplacePolicy”. Default: true

  • default (Optional[RemovalPolicy]) – The default policy to apply in case the removal policy is not defined. Default: - Default value is resource specific. To determine the default value for a resoure, please consult that specific resource’s documentation.

Return type

None

get_att(attribute_name)

Returns a token for an runtime attribute of this resource.

Ideally, use generated attribute accessors (e.g. resource.arn), but this can be used for future compatibility in case there is no generated attribute.

Parameters

attribute_name (str) – The name of the attribute.

Return type

Reference

get_metadata(key)

Retrieve a value value from the CloudFormation Resource Metadata.

Parameters

key (str) –

See

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/metadata-section-structure.html

Note that this is a different set of metadata from CDK node metadata; this metadata ends up in the stack template under the resource, whereas CDK node metadata ends up in the Cloud Assembly.

Return type

Any

inspect(inspector)

Examines the CloudFormation resource and discloses attributes.

Parameters

inspector (TreeInspector) –

  • tree inspector to collect and process attributes.

Return type

None

override_logical_id(new_logical_id)

Overrides the auto-generated logical ID with a specific ID.

Parameters

new_logical_id (str) – The new logical ID to use for this stack element.

Return type

None

to_string()

Returns a string representation of this construct.

Return type

str

Returns

a string representation of this resource

Attributes

CFN_RESOURCE_TYPE_NAME = 'AWS::EMR::Cluster'
additional_info

A JSON string for selecting additional features.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-additionalinfo

Return type

Any

applications

The applications to install on this cluster, for example, Spark, Flink, Oozie, Zeppelin, and so on.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-applications

Return type

Union[IResolvable, List[Union[IResolvable, ApplicationProperty]], None]

attr_master_public_dns

The public DNS name of the master node (instance), such as ec2-12-123-123-123.us-west-2.compute.amazonaws.com .

CloudformationAttribute

MasterPublicDNS

Return type

str

auto_scaling_role

An IAM role for automatic scaling policies.

The default role is EMR_AutoScaling_DefaultRole . The IAM role provides permissions that the automatic scaling feature requires to launch and terminate EC2 instances in an instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-autoscalingrole

Return type

Optional[str]

auto_termination_policy

AWS::EMR::Cluster.AutoTerminationPolicy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-autoterminationpolicy

Return type

Union[IResolvable, AutoTerminationPolicyProperty, None]

bootstrap_actions

A list of bootstrap actions to run before Hadoop starts on the cluster nodes.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-bootstrapactions

Return type

Union[IResolvable, List[Union[IResolvable, BootstrapActionConfigProperty]], None]

cfn_options

Options for this resource, such as condition, update policy etc.

Return type

ICfnResourceOptions

cfn_resource_type

AWS resource type.

Return type

str

configurations

Applies only to Amazon EMR releases 4.x and later. The list of Configurations supplied to the EMR cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-configurations

Return type

Union[IResolvable, List[Union[IResolvable, ConfigurationProperty]], None]

creation_stack

return:

the stack trace of the point where this Resource was created from, sourced from the +metadata+ entry typed +aws:cdk:logicalId+, and with the bottom-most node +internal+ entries filtered.

Return type

List[str]

custom_ami_id

Available only in Amazon EMR version 5.7.0 and later. The ID of a custom Amazon EBS-backed Linux AMI if the cluster uses a custom AMI.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-customamiid

Return type

Optional[str]

ebs_root_volume_size

The size, in GiB, of the Amazon EBS root device volume of the Linux AMI that is used for each EC2 instance.

Available in Amazon EMR version 4.x and later.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-ebsrootvolumesize

Return type

Union[int, float, None]

instances

A specification of the number and type of Amazon EC2 instances.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-instances

Return type

Union[JobFlowInstancesConfigProperty, IResolvable]

job_flow_role

Also called instance profile and EC2 role.

An IAM role for an EMR cluster. The EC2 instances of the cluster assume this role. The default role is EMR_EC2_DefaultRole . In order to use the default role, you must have already created it using the CLI or console.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-jobflowrole

Return type

str

kerberos_attributes

Attributes for Kerberos configuration when Kerberos authentication is enabled using a security configuration.

For more information see Use Kerberos Authentication in the Amazon EMR Management Guide .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-kerberosattributes

Return type

Union[IResolvable, KerberosAttributesProperty, None]

log_encryption_kms_key_id

The AWS KMS key used for encrypting log files.

This attribute is only available with EMR version 5.30.0 and later, excluding EMR 6.0.0.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-logencryptionkmskeyid

Return type

Optional[str]

log_uri

The path to the Amazon S3 location where logs for this cluster are stored.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-loguri

Return type

Optional[str]

logical_id

The logical ID for this CloudFormation stack element.

The logical ID of the element is calculated from the path of the resource node in the construct tree.

To override this value, use overrideLogicalId(newLogicalId).

Return type

str

Returns

the logical ID as a stringified token. This value will only get resolved during synthesis.

managed_scaling_policy

Creates or updates a managed scaling policy for an Amazon EMR cluster.

The managed scaling policy defines the limits for resources, such as EC2 instances that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-managedscalingpolicy

Return type

Union[IResolvable, ManagedScalingPolicyProperty, None]

name

The name of the cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-name

Return type

str

node

The construct tree node associated with this construct.

Return type

ConstructNode

ref

Return a string that will be resolved to a CloudFormation { Ref } for this element.

If, by any chance, the intrinsic reference of a resource is not a string, you could coerce it to an IResolvable through Lazy.any({ produce: resource.ref }).

Return type

str

release_label

The Amazon EMR release label, which determines the version of open-source application packages installed on the cluster.

Release labels are in the form emr-x.x.x , where x.x.x is an Amazon EMR release version such as emr-5.14.0 . For more information about Amazon EMR release versions and included application versions and features, see ` <https://docs.aws.amazon.com/emr/latest/ReleaseGuide/>`_ . The release label applies only to Amazon EMR releases version 4.0 and later. Earlier versions use AmiVersion .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-releaselabel

Return type

Optional[str]

scale_down_behavior

The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an instance group is resized.

TERMINATE_AT_INSTANCE_HOUR indicates that Amazon EMR terminates nodes at the instance-hour boundary, regardless of when the request to terminate the instance was submitted. This option is only available with Amazon EMR 5.1.0 and later and is the default for clusters created using that version. TERMINATE_AT_TASK_COMPLETION indicates that Amazon EMR adds nodes to a deny list and drains tasks from nodes before terminating the Amazon EC2 instances, regardless of the instance-hour boundary. With either behavior, Amazon EMR removes the least active nodes first and blocks instance termination if it could lead to HDFS corruption. TERMINATE_AT_TASK_COMPLETION is available only in Amazon EMR version 4.1.0 and later, and is the default for versions of Amazon EMR earlier than 5.1.0.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-scaledownbehavior

Return type

Optional[str]

security_configuration

The name of the security configuration applied to the cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-securityconfiguration

Return type

Optional[str]

service_role

The IAM role that Amazon EMR assumes in order to access AWS resources on your behalf.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-servicerole

Return type

str

stack

The stack in which this element is defined.

CfnElements must be defined within a stack scope (directly or indirectly).

Return type

Stack

step_concurrency_level

Specifies the number of steps that can be executed concurrently.

The default value is 1 . The maximum value is 256 .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-stepconcurrencylevel

Return type

Union[int, float, None]

steps

A list of steps to run.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-steps

Return type

Union[IResolvable, List[Union[IResolvable, StepConfigProperty]], None]

tags

A list of tags associated with a cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-tags

Return type

TagManager

visible_to_all_users

Indicates whether the cluster is visible to all IAM users of the AWS account associated with the cluster.

If this value is set to true , all IAM users of that AWS account can view and manage the cluster if they have the proper policy permissions set. If this value is false , only the IAM user that created the cluster can view and manage it. This value can be changed using the SetVisibleToAllUsers action. .. epigraph:

When you create clusters directly through the EMR console or API, this value is set to ``true`` by default. However, for ``AWS::EMR::Cluster`` resources in CloudFormation, the default is ``false`` .
Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticmapreduce-cluster.html#cfn-elasticmapreduce-cluster-visibletoallusers

Return type

Union[bool, IResolvable, None]

Static Methods

classmethod is_cfn_element(x)

Returns true if a construct is a stack element (i.e. part of the synthesized cloudformation template).

Uses duck-typing instead of instanceof to allow stack elements from different versions of this library to be included in the same stack.

Parameters

x (Any) –

Return type

bool

Returns

The construct as a stack element or undefined if it is not a stack element.

classmethod is_cfn_resource(construct)

Check whether the given construct is a CfnResource.

Parameters

construct (IConstruct) –

Return type

bool

classmethod is_construct(x)

Return whether the given object is a Construct.

Parameters

x (Any) –

Return type

bool

ApplicationProperty

class CfnCluster.ApplicationProperty(*, additional_info=None, args=None, name=None, version=None)

Bases: object

Application is a property of AWS::EMR::Cluster .

The Application property type defines the open-source big data applications for EMR to install and configure when a cluster is created.

With Amazon EMR release version 4.0 and later, the only accepted parameter is the application Name . To pass arguments to these applications, you use configuration classifications specified using JSON objects in a Configuration property. For more information, see Configuring Applications .

With earlier Amazon EMR releases, the application is any AWS or third-party software that you can add to the cluster. You can specify the version of the application and arguments to pass to it. Amazon EMR accepts and forwards the argument list to the corresponding installation script as a bootstrap action argument.

Parameters
  • additional_info (Union[IResolvable, Mapping[str, str], None]) – This option is for advanced users only. This is meta information about clusters and applications that are used for testing and troubleshooting.

  • args (Optional[Sequence[str]]) – Arguments for Amazon EMR to pass to the application.

  • name (Optional[str]) – The name of the application.

  • version (Optional[str]) – The version of the application.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-application.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

application_property = emr.CfnCluster.ApplicationProperty(
    additional_info={
        "additional_info_key": "additionalInfo"
    },
    args=["args"],
    name="name",
    version="version"
)

Attributes

additional_info

This option is for advanced users only.

This is meta information about clusters and applications that are used for testing and troubleshooting.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-application.html#cfn-elasticmapreduce-cluster-application-additionalinfo

Return type

Union[IResolvable, Mapping[str, str], None]

args

Arguments for Amazon EMR to pass to the application.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-application.html#cfn-elasticmapreduce-cluster-application-args

Return type

Optional[List[str]]

name

The name of the application.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-application.html#cfn-elasticmapreduce-cluster-application-name

Return type

Optional[str]

version

The version of the application.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-application.html#cfn-elasticmapreduce-cluster-application-version

Return type

Optional[str]

AutoScalingPolicyProperty

class CfnCluster.AutoScalingPolicyProperty(*, constraints, rules)

Bases: object

AutoScalingPolicy is a subproperty of InstanceGroupConfig .

AutoScalingPolicy defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. For more information, see Using Automatic Scaling in Amazon EMR in the Amazon EMR Management Guide .

Parameters
  • constraints (Union[IResolvable, ScalingConstraintsProperty, Dict[str, Any]]) – The upper and lower EC2 instance limits for an automatic scaling policy. Automatic scaling activity will not cause an instance group to grow above or below these limits.

  • rules (Union[IResolvable, Sequence[Union[IResolvable, ScalingRuleProperty, Dict[str, Any]]]]) – The scale-in and scale-out rules that comprise the automatic scaling policy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-autoscalingpolicy.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

auto_scaling_policy_property = emr.CfnCluster.AutoScalingPolicyProperty(
    constraints=emr.CfnCluster.ScalingConstraintsProperty(
        max_capacity=123,
        min_capacity=123
    ),
    rules=[emr.CfnCluster.ScalingRuleProperty(
        action=emr.CfnCluster.ScalingActionProperty(
            simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                scaling_adjustment=123,

                # the properties below are optional
                adjustment_type="adjustmentType",
                cool_down=123
            ),

            # the properties below are optional
            market="market"
        ),
        name="name",
        trigger=emr.CfnCluster.ScalingTriggerProperty(
            cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                comparison_operator="comparisonOperator",
                metric_name="metricName",
                period=123,
                threshold=123,

                # the properties below are optional
                dimensions=[emr.CfnCluster.MetricDimensionProperty(
                    key="key",
                    value="value"
                )],
                evaluation_periods=123,
                namespace="namespace",
                statistic="statistic",
                unit="unit"
            )
        ),

        # the properties below are optional
        description="description"
    )]
)

Attributes

constraints

The upper and lower EC2 instance limits for an automatic scaling policy.

Automatic scaling activity will not cause an instance group to grow above or below these limits.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-autoscalingpolicy.html#cfn-elasticmapreduce-cluster-autoscalingpolicy-constraints

Return type

Union[IResolvable, ScalingConstraintsProperty]

rules

The scale-in and scale-out rules that comprise the automatic scaling policy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-autoscalingpolicy.html#cfn-elasticmapreduce-cluster-autoscalingpolicy-rules

Return type

Union[IResolvable, List[Union[IResolvable, ScalingRuleProperty]]]

AutoTerminationPolicyProperty

class CfnCluster.AutoTerminationPolicyProperty(*, idle_timeout=None)

Bases: object

Parameters

idle_timeout (Union[int, float, None]) – CfnCluster.AutoTerminationPolicyProperty.IdleTimeout.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-autoterminationpolicy.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

auto_termination_policy_property = emr.CfnCluster.AutoTerminationPolicyProperty(
    idle_timeout=123
)

Attributes

idle_timeout

CfnCluster.AutoTerminationPolicyProperty.IdleTimeout.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-autoterminationpolicy.html#cfn-elasticmapreduce-cluster-autoterminationpolicy-idletimeout

Return type

Union[int, float, None]

BootstrapActionConfigProperty

class CfnCluster.BootstrapActionConfigProperty(*, name, script_bootstrap_action)

Bases: object

BootstrapActionConfig is a property of AWS::EMR::Cluster that can be used to run bootstrap actions on EMR clusters.

You can use a bootstrap action to install software and configure EC2 instances for all cluster nodes before EMR installs and configures open-source big data applications on cluster instances. For more information, see Create Bootstrap Actions to Install Additional Software in the Amazon EMR Management Guide .

Parameters
Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-bootstrapactionconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

bootstrap_action_config_property = emr.CfnCluster.BootstrapActionConfigProperty(
    name="name",
    script_bootstrap_action=emr.CfnCluster.ScriptBootstrapActionConfigProperty(
        path="path",

        # the properties below are optional
        args=["args"]
    )
)

Attributes

name

The name of the bootstrap action.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-bootstrapactionconfig.html#cfn-elasticmapreduce-cluster-bootstrapactionconfig-name

Return type

str

script_bootstrap_action

The script run by the bootstrap action.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-bootstrapactionconfig.html#cfn-elasticmapreduce-cluster-bootstrapactionconfig-scriptbootstrapaction

Return type

Union[IResolvable, ScriptBootstrapActionConfigProperty]

CloudWatchAlarmDefinitionProperty

class CfnCluster.CloudWatchAlarmDefinitionProperty(*, comparison_operator, metric_name, period, threshold, dimensions=None, evaluation_periods=None, namespace=None, statistic=None, unit=None)

Bases: object

CloudWatchAlarmDefinition is a subproperty of the ScalingTrigger property, which determines when to trigger an automatic scaling activity.

Scaling activity begins when you satisfy the defined alarm conditions.

Parameters
  • comparison_operator (str) – Determines how the metric specified by MetricName is compared to the value specified by Threshold .

  • metric_name (str) – The name of the CloudWatch metric that is watched to determine an alarm condition.

  • period (Union[int, float]) – The period, in seconds, over which the statistic is applied. EMR CloudWatch metrics are emitted every five minutes (300 seconds), so if an EMR CloudWatch metric is specified, specify 300 .

  • threshold (Union[int, float]) – The value against which the specified statistic is compared.

  • dimensions (Union[IResolvable, Sequence[Union[IResolvable, MetricDimensionProperty, Dict[str, Any]]], None]) – A CloudWatch metric dimension.

  • evaluation_periods (Union[int, float, None]) – The number of periods, in five-minute increments, during which the alarm condition must exist before the alarm triggers automatic scaling activity. The default value is 1 .

  • namespace (Optional[str]) – The namespace for the CloudWatch metric. The default is AWS/ElasticMapReduce .

  • statistic (Optional[str]) – The statistic to apply to the metric associated with the alarm. The default is AVERAGE .

  • unit (Optional[str]) – The unit of measure associated with the CloudWatch metric being watched. The value specified for Unit must correspond to the units specified in the CloudWatch metric.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

cloud_watch_alarm_definition_property = emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
    comparison_operator="comparisonOperator",
    metric_name="metricName",
    period=123,
    threshold=123,

    # the properties below are optional
    dimensions=[emr.CfnCluster.MetricDimensionProperty(
        key="key",
        value="value"
    )],
    evaluation_periods=123,
    namespace="namespace",
    statistic="statistic",
    unit="unit"
)

Attributes

comparison_operator

Determines how the metric specified by MetricName is compared to the value specified by Threshold .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-comparisonoperator

Return type

str

dimensions

A CloudWatch metric dimension.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-dimensions

Return type

Union[IResolvable, List[Union[IResolvable, MetricDimensionProperty]], None]

evaluation_periods

The number of periods, in five-minute increments, during which the alarm condition must exist before the alarm triggers automatic scaling activity.

The default value is 1 .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-evaluationperiods

Return type

Union[int, float, None]

metric_name

The name of the CloudWatch metric that is watched to determine an alarm condition.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-metricname

Return type

str

namespace

The namespace for the CloudWatch metric.

The default is AWS/ElasticMapReduce .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-namespace

Return type

Optional[str]

period

The period, in seconds, over which the statistic is applied.

EMR CloudWatch metrics are emitted every five minutes (300 seconds), so if an EMR CloudWatch metric is specified, specify 300 .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-period

Return type

Union[int, float]

statistic

The statistic to apply to the metric associated with the alarm.

The default is AVERAGE .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-statistic

Return type

Optional[str]

threshold

The value against which the specified statistic is compared.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-threshold

Return type

Union[int, float]

unit

The unit of measure associated with the CloudWatch metric being watched.

The value specified for Unit must correspond to the units specified in the CloudWatch metric.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-cloudwatchalarmdefinition.html#cfn-elasticmapreduce-cluster-cloudwatchalarmdefinition-unit

Return type

Optional[str]

ComputeLimitsProperty

class CfnCluster.ComputeLimitsProperty(*, maximum_capacity_units, minimum_capacity_units, unit_type, maximum_core_capacity_units=None, maximum_on_demand_capacity_units=None)

Bases: object

The EC2 unit limits for a managed scaling policy.

The managed scaling activity of a cluster can not be above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Parameters
  • maximum_capacity_units (Union[int, float]) – The upper boundary of EC2 units. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

  • minimum_capacity_units (Union[int, float]) – The lower boundary of EC2 units. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

  • unit_type (str) – The unit type used for specifying a managed scaling policy.

  • maximum_core_capacity_units (Union[int, float, None]) – The upper boundary of EC2 units for core node type in a cluster. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The core units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between core and task nodes.

  • maximum_on_demand_capacity_units (Union[int, float, None]) – The upper boundary of On-Demand EC2 units. It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The On-Demand units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between On-Demand and Spot Instances.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-computelimits.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

compute_limits_property = emr.CfnCluster.ComputeLimitsProperty(
    maximum_capacity_units=123,
    minimum_capacity_units=123,
    unit_type="unitType",

    # the properties below are optional
    maximum_core_capacity_units=123,
    maximum_on_demand_capacity_units=123
)

Attributes

maximum_capacity_units

The upper boundary of EC2 units.

It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-computelimits.html#cfn-elasticmapreduce-cluster-computelimits-maximumcapacityunits

Return type

Union[int, float]

maximum_core_capacity_units

The upper boundary of EC2 units for core node type in a cluster.

It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The core units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between core and task nodes.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-computelimits.html#cfn-elasticmapreduce-cluster-computelimits-maximumcorecapacityunits

Return type

Union[int, float, None]

maximum_on_demand_capacity_units

The upper boundary of On-Demand EC2 units.

It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. The On-Demand units are not allowed to scale beyond this boundary. The parameter is used to split capacity allocation between On-Demand and Spot Instances.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-computelimits.html#cfn-elasticmapreduce-cluster-computelimits-maximumondemandcapacityunits

Return type

Union[int, float, None]

minimum_capacity_units

The lower boundary of EC2 units.

It is measured through vCPU cores or instances for instance groups and measured through units for instance fleets. Managed scaling activities are not allowed beyond this boundary. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-computelimits.html#cfn-elasticmapreduce-cluster-computelimits-minimumcapacityunits

Return type

Union[int, float]

unit_type

The unit type used for specifying a managed scaling policy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-computelimits.html#cfn-elasticmapreduce-cluster-computelimits-unittype

Return type

str

ConfigurationProperty

class CfnCluster.ConfigurationProperty(*, classification=None, configuration_properties=None, configurations=None)

Bases: object

Used only with Amazon EMR release 4.0 and later.

Configuration is a subproperty of InstanceFleetConfig or InstanceGroupConfig . Configuration specifies optional configurations for customizing open-source big data applications and environment parameters. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file. For more information, see Configuring Applications in the Amazon EMR Release Guide .

param classification

The classification within a configuration.

param configuration_properties

A list of additional configurations to apply within a configuration object.

param configurations

A list of additional configurations to apply within a configuration object.

link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-configuration.html

exampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

# configuration_property_: emr.CfnCluster.ConfigurationProperty

configuration_property = emr.CfnCluster.ConfigurationProperty(
    classification="classification",
    configuration_properties={
        "configuration_properties_key": "configurationProperties"
    },
    configurations=[emr.CfnCluster.ConfigurationProperty(
        classification="classification",
        configuration_properties={
            "configuration_properties_key": "configurationProperties"
        },
        configurations=[configuration_property_]
    )]
)

Attributes

classification

The classification within a configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-configuration.html#cfn-elasticmapreduce-cluster-configuration-classification

Return type

Optional[str]

configuration_properties

A list of additional configurations to apply within a configuration object.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-configuration.html#cfn-elasticmapreduce-cluster-configuration-configurationproperties

Return type

Union[IResolvable, Mapping[str, str], None]

configurations

A list of additional configurations to apply within a configuration object.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-configuration.html#cfn-elasticmapreduce-cluster-configuration-configurations

Return type

Union[IResolvable, List[Union[IResolvable, ConfigurationProperty]], None]

EbsBlockDeviceConfigProperty

class CfnCluster.EbsBlockDeviceConfigProperty(*, volume_specification, volumes_per_instance=None)

Bases: object

EbsBlockDeviceConfig is a subproperty of the EbsConfiguration property type.

EbsBlockDeviceConfig defines the number and type of EBS volumes to associate with all EC2 instances in an EMR cluster.

Parameters
  • volume_specification (Union[IResolvable, VolumeSpecificationProperty, Dict[str, Any]]) – EBS volume specifications such as volume type, IOPS, and size (GiB) that will be requested for the EBS volume attached to an EC2 instance in the cluster.

  • volumes_per_instance (Union[int, float, None]) – Number of EBS volumes with a specific volume configuration that will be associated with every instance in the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ebsblockdeviceconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

ebs_block_device_config_property = emr.CfnCluster.EbsBlockDeviceConfigProperty(
    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
        size_in_gb=123,
        volume_type="volumeType",

        # the properties below are optional
        iops=123
    ),

    # the properties below are optional
    volumes_per_instance=123
)

Attributes

volume_specification

EBS volume specifications such as volume type, IOPS, and size (GiB) that will be requested for the EBS volume attached to an EC2 instance in the cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ebsblockdeviceconfig.html#cfn-elasticmapreduce-cluster-ebsblockdeviceconfig-volumespecification

Return type

Union[IResolvable, VolumeSpecificationProperty]

volumes_per_instance

Number of EBS volumes with a specific volume configuration that will be associated with every instance in the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ebsblockdeviceconfig.html#cfn-elasticmapreduce-cluster-ebsblockdeviceconfig-volumesperinstance

Return type

Union[int, float, None]

EbsConfigurationProperty

class CfnCluster.EbsConfigurationProperty(*, ebs_block_device_configs=None, ebs_optimized=None)

Bases: object

EbsConfiguration is a subproperty of InstanceFleetConfig or InstanceGroupConfig .

EbsConfiguration determines the EBS volumes to attach to EMR cluster instances.

Parameters
  • ebs_block_device_configs (Union[IResolvable, Sequence[Union[IResolvable, EbsBlockDeviceConfigProperty, Dict[str, Any]]], None]) – An array of Amazon EBS volume specifications attached to a cluster instance.

  • ebs_optimized (Union[bool, IResolvable, None]) – Indicates whether an Amazon EBS volume is EBS-optimized.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ebsconfiguration.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

ebs_configuration_property = emr.CfnCluster.EbsConfigurationProperty(
    ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
        volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
            size_in_gb=123,
            volume_type="volumeType",

            # the properties below are optional
            iops=123
        ),

        # the properties below are optional
        volumes_per_instance=123
    )],
    ebs_optimized=False
)

Attributes

ebs_block_device_configs

An array of Amazon EBS volume specifications attached to a cluster instance.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ebsconfiguration.html#cfn-elasticmapreduce-cluster-ebsconfiguration-ebsblockdeviceconfigs

Return type

Union[IResolvable, List[Union[IResolvable, EbsBlockDeviceConfigProperty]], None]

ebs_optimized

Indicates whether an Amazon EBS volume is EBS-optimized.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ebsconfiguration.html#cfn-elasticmapreduce-cluster-ebsconfiguration-ebsoptimized

Return type

Union[bool, IResolvable, None]

HadoopJarStepConfigProperty

class CfnCluster.HadoopJarStepConfigProperty(*, jar, args=None, main_class=None, step_properties=None)

Bases: object

The HadoopJarStepConfig property type specifies a job flow step consisting of a JAR file whose main function will be executed.

The main function submits a job for the cluster to execute as a step on the master node, and then waits for the job to finish or fail before executing subsequent steps.

Parameters
  • jar (str) – A path to a JAR file run during the step.

  • args (Optional[Sequence[str]]) – A list of command line arguments passed to the JAR file’s main function when executed.

  • main_class (Optional[str]) – The name of the main class in the specified Java file. If not specified, the JAR file should specify a Main-Class in its manifest file.

  • step_properties (Union[IResolvable, Sequence[Union[IResolvable, KeyValueProperty, Dict[str, Any]]], None]) – A list of Java properties that are set when the step runs. You can use these properties to pass key-value pairs to your main function.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-hadoopjarstepconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

hadoop_jar_step_config_property = emr.CfnCluster.HadoopJarStepConfigProperty(
    jar="jar",

    # the properties below are optional
    args=["args"],
    main_class="mainClass",
    step_properties=[emr.CfnCluster.KeyValueProperty(
        key="key",
        value="value"
    )]
)

Attributes

args

A list of command line arguments passed to the JAR file’s main function when executed.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-hadoopjarstepconfig.html#cfn-elasticmapreduce-cluster-hadoopjarstepconfig-args

Return type

Optional[List[str]]

jar

A path to a JAR file run during the step.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-hadoopjarstepconfig.html#cfn-elasticmapreduce-cluster-hadoopjarstepconfig-jar

Return type

str

main_class

The name of the main class in the specified Java file.

If not specified, the JAR file should specify a Main-Class in its manifest file.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-hadoopjarstepconfig.html#cfn-elasticmapreduce-cluster-hadoopjarstepconfig-mainclass

Return type

Optional[str]

step_properties

A list of Java properties that are set when the step runs.

You can use these properties to pass key-value pairs to your main function.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-hadoopjarstepconfig.html#cfn-elasticmapreduce-cluster-hadoopjarstepconfig-stepproperties

Return type

Union[IResolvable, List[Union[IResolvable, KeyValueProperty]], None]

InstanceFleetConfigProperty

class CfnCluster.InstanceFleetConfigProperty(*, instance_type_configs=None, launch_specifications=None, name=None, target_on_demand_capacity=None, target_spot_capacity=None)

Bases: object

Use InstanceFleetConfig to define instance fleets for an EMR cluster.

A cluster can not use both instance fleets and instance groups. For more information, see Configure Instance Fleets in the Amazon EMR Management Guide . .. epigraph:

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
Parameters
  • instance_type_configs (Union[IResolvable, Sequence[Union[IResolvable, InstanceTypeConfigProperty, Dict[str, Any]]], None]) – The instance type configurations that define the EC2 instances in the instance fleet.

  • launch_specifications (Union[IResolvable, InstanceFleetProvisioningSpecificationsProperty, Dict[str, Any], None]) – The launch specification for the instance fleet.

  • name (Optional[str]) – The friendly name of the instance fleet.

  • target_on_demand_capacity (Union[int, float, None]) – The target capacity of On-Demand units for the instance fleet, which determines how many On-Demand instances to provision. When the instance fleet launches, Amazon EMR tries to provision On-Demand instances as specified by InstanceTypeConfig . Each instance configuration has a specified WeightedCapacity . When an On-Demand instance is provisioned, the WeightedCapacity units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with a WeightedCapacity of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:: If not specified or set to 0, only Spot instances are provisioned for the instance fleet using TargetSpotCapacity . At least one of TargetSpotCapacity and TargetOnDemandCapacity should be greater than 0. For a master instance fleet, only one of TargetSpotCapacity and TargetOnDemandCapacity can be specified, and its value must be 1.

  • target_spot_capacity (Union[int, float, None]) – The target capacity of Spot units for the instance fleet, which determines how many Spot instances to provision. When the instance fleet launches, Amazon EMR tries to provision Spot instances as specified by InstanceTypeConfig . Each instance configuration has a specified WeightedCapacity . When a Spot instance is provisioned, the WeightedCapacity units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with a WeightedCapacity of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:: If not specified or set to 0, only On-Demand instances are provisioned for the instance fleet. At least one of TargetSpotCapacity and TargetOnDemandCapacity should be greater than 0. For a master instance fleet, only one of TargetSpotCapacity and TargetOnDemandCapacity can be specified, and its value must be 1.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

# configuration_property_: emr.CfnCluster.ConfigurationProperty

instance_fleet_config_property = emr.CfnCluster.InstanceFleetConfigProperty(
    instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
        instance_type="instanceType",

        # the properties below are optional
        bid_price="bidPrice",
        bid_price_as_percentage_of_on_demand_price=123,
        configurations=[emr.CfnCluster.ConfigurationProperty(
            classification="classification",
            configuration_properties={
                "configuration_properties_key": "configurationProperties"
            },
            configurations=[configuration_property_]
        )],
        custom_ami_id="customAmiId",
        ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
            ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                    size_in_gb=123,
                    volume_type="volumeType",

                    # the properties below are optional
                    iops=123
                ),

                # the properties below are optional
                volumes_per_instance=123
            )],
            ebs_optimized=False
        ),
        weighted_capacity=123
    )],
    launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
        on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
            allocation_strategy="allocationStrategy"
        ),
        spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
            timeout_action="timeoutAction",
            timeout_duration_minutes=123,

            # the properties below are optional
            allocation_strategy="allocationStrategy",
            block_duration_minutes=123
        )
    ),
    name="name",
    target_on_demand_capacity=123,
    target_spot_capacity=123
)

Attributes

instance_type_configs

The instance type configurations that define the EC2 instances in the instance fleet.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetconfig.html#cfn-elasticmapreduce-cluster-instancefleetconfig-instancetypeconfigs

Return type

Union[IResolvable, List[Union[IResolvable, InstanceTypeConfigProperty]], None]

launch_specifications

The launch specification for the instance fleet.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetconfig.html#cfn-elasticmapreduce-cluster-instancefleetconfig-launchspecifications

Return type

Union[IResolvable, InstanceFleetProvisioningSpecificationsProperty, None]

name

The friendly name of the instance fleet.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetconfig.html#cfn-elasticmapreduce-cluster-instancefleetconfig-name

Return type

Optional[str]

target_on_demand_capacity

The target capacity of On-Demand units for the instance fleet, which determines how many On-Demand instances to provision.

When the instance fleet launches, Amazon EMR tries to provision On-Demand instances as specified by InstanceTypeConfig . Each instance configuration has a specified WeightedCapacity . When an On-Demand instance is provisioned, the WeightedCapacity units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with a WeightedCapacity of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:

If not specified or set to 0, only Spot instances are provisioned for the instance fleet using ``TargetSpotCapacity`` . At least one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` should be greater than 0. For a master instance fleet, only one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` can be specified, and its value must be 1.
Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetconfig.html#cfn-elasticmapreduce-cluster-instancefleetconfig-targetondemandcapacity

Return type

Union[int, float, None]

target_spot_capacity

The target capacity of Spot units for the instance fleet, which determines how many Spot instances to provision.

When the instance fleet launches, Amazon EMR tries to provision Spot instances as specified by InstanceTypeConfig . Each instance configuration has a specified WeightedCapacity . When a Spot instance is provisioned, the WeightedCapacity units count toward the target capacity. Amazon EMR provisions instances until the target capacity is totally fulfilled, even if this results in an overage. For example, if there are 2 units remaining to fulfill capacity, and Amazon EMR can only provision an instance with a WeightedCapacity of 5 units, the instance is provisioned, and the target capacity is exceeded by 3 units. .. epigraph:

If not specified or set to 0, only On-Demand instances are provisioned for the instance fleet. At least one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` should be greater than 0. For a master instance fleet, only one of ``TargetSpotCapacity`` and ``TargetOnDemandCapacity`` can be specified, and its value must be 1.
Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetconfig.html#cfn-elasticmapreduce-cluster-instancefleetconfig-targetspotcapacity

Return type

Union[int, float, None]

InstanceFleetProvisioningSpecificationsProperty

class CfnCluster.InstanceFleetProvisioningSpecificationsProperty(*, on_demand_specification=None, spot_specification=None)

Bases: object

InstanceFleetProvisioningSpecification is a subproperty of InstanceFleetConfig .

InstanceFleetProvisioningSpecification defines the launch specification for Spot instances in an instance fleet, which determines the defined duration and provisioning timeout behavior for Spot instances. .. epigraph:

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
Parameters
  • on_demand_specification (Union[IResolvable, OnDemandProvisioningSpecificationProperty, Dict[str, Any], None]) – The launch specification for On-Demand Instances in the instance fleet, which determines the allocation strategy. .. epigraph:: The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions. On-Demand Instances allocation strategy is available in Amazon EMR version 5.12.1 and later.

  • spot_specification (Union[IResolvable, SpotProvisioningSpecificationProperty, Dict[str, Any], None]) – The launch specification for Spot Instances in the fleet, which determines the defined duration, provisioning timeout behavior, and allocation strategy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetprovisioningspecifications.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

instance_fleet_provisioning_specifications_property = emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
    on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
        allocation_strategy="allocationStrategy"
    ),
    spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
        timeout_action="timeoutAction",
        timeout_duration_minutes=123,

        # the properties below are optional
        allocation_strategy="allocationStrategy",
        block_duration_minutes=123
    )
)

Attributes

on_demand_specification

The launch specification for On-Demand Instances in the instance fleet, which determines the allocation strategy.

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions. On-Demand Instances allocation strategy is available in Amazon EMR version 5.12.1 and later.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetprovisioningspecifications.html#cfn-elasticmapreduce-cluster-instancefleetprovisioningspecifications-ondemandspecification

Return type

Union[IResolvable, OnDemandProvisioningSpecificationProperty, None]

spot_specification

The launch specification for Spot Instances in the fleet, which determines the defined duration, provisioning timeout behavior, and allocation strategy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancefleetprovisioningspecifications.html#cfn-elasticmapreduce-cluster-instancefleetprovisioningspecifications-spotspecification

Return type

Union[IResolvable, SpotProvisioningSpecificationProperty, None]

InstanceGroupConfigProperty

class CfnCluster.InstanceGroupConfigProperty(*, instance_count, instance_type, auto_scaling_policy=None, bid_price=None, configurations=None, custom_ami_id=None, ebs_configuration=None, market=None, name=None)

Bases: object

Use InstanceGroupConfig to define instance groups for an EMR cluster.

A cluster can not use both instance groups and instance fleets. For more information, see Create a Cluster with Instance Fleets or Uniform Instance Groups in the Amazon EMR Management Guide .

Parameters
  • instance_count (Union[int, float]) – Target number of instances for the instance group.

  • instance_type (str) – The EC2 instance type for all instances in the instance group.

  • auto_scaling_policy (Union[IResolvable, AutoScalingPolicyProperty, Dict[str, Any], None]) –

    AutoScalingPolicy is a subproperty of the InstanceGroupConfig property type that specifies the constraints and rules of an automatic scaling policy in Amazon EMR . The automatic scaling policy defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. Only core and task instance groups can use automatic scaling policies. For more information, see Using Automatic Scaling in Amazon EMR .

  • bid_price (Optional[str]) – If specified, indicates that the instance group uses Spot Instances. This is the maximum price you are willing to pay for Spot Instances. Specify OnDemandPrice to set the amount equal to the On-Demand price, or specify an amount in USD.

  • configurations (Union[IResolvable, Sequence[Union[IResolvable, ConfigurationProperty, Dict[str, Any]]], None]) –

    Amazon EMR releases 4.x or later. The list of configurations supplied for an EMR cluster instance group. You can specify a separate configuration for each instance group (master, core, and task).

  • custom_ami_id (Optional[str]) – The custom AMI ID to use for the provisioned instance group.

  • ebs_configuration (Union[IResolvable, EbsConfigurationProperty, Dict[str, Any], None]) – EBS configurations that will be attached to each EC2 instance in the instance group.

  • market (Optional[str]) – Market type of the EC2 instances used to create a cluster node.

  • name (Optional[str]) – Friendly name given to the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

# configuration_property_: emr.CfnCluster.ConfigurationProperty

instance_group_config_property = emr.CfnCluster.InstanceGroupConfigProperty(
    instance_count=123,
    instance_type="instanceType",

    # the properties below are optional
    auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
        constraints=emr.CfnCluster.ScalingConstraintsProperty(
            max_capacity=123,
            min_capacity=123
        ),
        rules=[emr.CfnCluster.ScalingRuleProperty(
            action=emr.CfnCluster.ScalingActionProperty(
                simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                    scaling_adjustment=123,

                    # the properties below are optional
                    adjustment_type="adjustmentType",
                    cool_down=123
                ),

                # the properties below are optional
                market="market"
            ),
            name="name",
            trigger=emr.CfnCluster.ScalingTriggerProperty(
                cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                    comparison_operator="comparisonOperator",
                    metric_name="metricName",
                    period=123,
                    threshold=123,

                    # the properties below are optional
                    dimensions=[emr.CfnCluster.MetricDimensionProperty(
                        key="key",
                        value="value"
                    )],
                    evaluation_periods=123,
                    namespace="namespace",
                    statistic="statistic",
                    unit="unit"
                )
            ),

            # the properties below are optional
            description="description"
        )]
    ),
    bid_price="bidPrice",
    configurations=[emr.CfnCluster.ConfigurationProperty(
        classification="classification",
        configuration_properties={
            "configuration_properties_key": "configurationProperties"
        },
        configurations=[configuration_property_]
    )],
    custom_ami_id="customAmiId",
    ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
        ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
            volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                size_in_gb=123,
                volume_type="volumeType",

                # the properties below are optional
                iops=123
            ),

            # the properties below are optional
            volumes_per_instance=123
        )],
        ebs_optimized=False
    ),
    market="market",
    name="name"
)

Attributes

auto_scaling_policy

//docs.aws.amazon.com//emr/latest/ManagementGuide/emr-automatic-scaling.html>`_ .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-autoscalingpolicy

Type

AutoScalingPolicy is a subproperty of the `InstanceGroupConfig <https

Type

//docs.aws.amazon.com//AWSCloudFormation/latest/UserGuide/aws-properties-emr-cluster-jobflowinstancesconfig-instancegroupconfig.html>`_ property type that specifies the constraints and rules of an automatic scaling policy in Amazon EMR . The automatic scaling policy defines how an instance group dynamically adds and terminates EC2 instances in response to the value of a CloudWatch metric. Only core and task instance groups can use automatic scaling policies. For more information, see `Using Automatic Scaling in Amazon EMR <https

Return type

Union[IResolvable, AutoScalingPolicyProperty, None]

bid_price

If specified, indicates that the instance group uses Spot Instances.

This is the maximum price you are willing to pay for Spot Instances. Specify OnDemandPrice to set the amount equal to the On-Demand price, or specify an amount in USD.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-bidprice

Return type

Optional[str]

configurations

Amazon EMR releases 4.x or later.

The list of configurations supplied for an EMR cluster instance group. You can specify a separate configuration for each instance group (master, core, and task).

link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-configurations

Return type

Union[IResolvable, List[Union[IResolvable, ConfigurationProperty]], None]

custom_ami_id

The custom AMI ID to use for the provisioned instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-customamiid

Return type

Optional[str]

ebs_configuration

EBS configurations that will be attached to each EC2 instance in the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-ebsconfiguration

Return type

Union[IResolvable, EbsConfigurationProperty, None]

instance_count

Target number of instances for the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-instancecount

Return type

Union[int, float]

instance_type

The EC2 instance type for all instances in the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-instancetype

Return type

str

market

Market type of the EC2 instances used to create a cluster node.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-market

Return type

Optional[str]

name

Friendly name given to the instance group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancegroupconfig.html#cfn-elasticmapreduce-cluster-instancegroupconfig-name

Return type

Optional[str]

InstanceTypeConfigProperty

class CfnCluster.InstanceTypeConfigProperty(*, instance_type, bid_price=None, bid_price_as_percentage_of_on_demand_price=None, configurations=None, custom_ami_id=None, ebs_configuration=None, weighted_capacity=None)

Bases: object

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.

InstanceTypeConfig is a sub-property of InstanceFleetConfig . InstanceTypeConfig determines the EC2 instances that Amazon EMR attempts to provision to fulfill On-Demand and Spot target capacities.

param instance_type

An EC2 instance type, such as m3.xlarge .

param bid_price

The bid price for each EC2 Spot Instance type as defined by InstanceType . Expressed in USD. If neither BidPrice nor BidPriceAsPercentageOfOnDemandPrice is provided, BidPriceAsPercentageOfOnDemandPrice defaults to 100%.

param bid_price_as_percentage_of_on_demand_price

The bid price, as a percentage of On-Demand price, for each EC2 Spot Instance as defined by InstanceType . Expressed as a number (for example, 20 specifies 20%). If neither BidPrice nor BidPriceAsPercentageOfOnDemandPrice is provided, BidPriceAsPercentageOfOnDemandPrice defaults to 100%.

param configurations

A configuration classification that applies when provisioning cluster instances, which can include configurations for applications and software that run on the cluster.

param custom_ami_id

The custom AMI ID to use for the instance type.

param ebs_configuration

The configuration of Amazon Elastic Block Store (Amazon EBS) attached to each instance as defined by InstanceType .

param weighted_capacity

The number of units that a provisioned instance of this type provides toward fulfilling the target capacities defined in InstanceFleetConfig . This value is 1 for a master instance fleet, and must be 1 or greater for core and task instance fleets. Defaults to 1 if not specified.

link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html

exampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

# configuration_property_: emr.CfnCluster.ConfigurationProperty

instance_type_config_property = emr.CfnCluster.InstanceTypeConfigProperty(
    instance_type="instanceType",

    # the properties below are optional
    bid_price="bidPrice",
    bid_price_as_percentage_of_on_demand_price=123,
    configurations=[emr.CfnCluster.ConfigurationProperty(
        classification="classification",
        configuration_properties={
            "configuration_properties_key": "configurationProperties"
        },
        configurations=[configuration_property_]
    )],
    custom_ami_id="customAmiId",
    ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
        ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
            volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                size_in_gb=123,
                volume_type="volumeType",

                # the properties below are optional
                iops=123
            ),

            # the properties below are optional
            volumes_per_instance=123
        )],
        ebs_optimized=False
    ),
    weighted_capacity=123
)

Attributes

bid_price

The bid price for each EC2 Spot Instance type as defined by InstanceType .

Expressed in USD. If neither BidPrice nor BidPriceAsPercentageOfOnDemandPrice is provided, BidPriceAsPercentageOfOnDemandPrice defaults to 100%.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-bidprice

Return type

Optional[str]

bid_price_as_percentage_of_on_demand_price

The bid price, as a percentage of On-Demand price, for each EC2 Spot Instance as defined by InstanceType .

Expressed as a number (for example, 20 specifies 20%). If neither BidPrice nor BidPriceAsPercentageOfOnDemandPrice is provided, BidPriceAsPercentageOfOnDemandPrice defaults to 100%.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-bidpriceaspercentageofondemandprice

Return type

Union[int, float, None]

configurations

A configuration classification that applies when provisioning cluster instances, which can include configurations for applications and software that run on the cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-configurations

Return type

Union[IResolvable, List[Union[IResolvable, ConfigurationProperty]], None]

custom_ami_id

The custom AMI ID to use for the instance type.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-customamiid

Return type

Optional[str]

ebs_configuration

The configuration of Amazon Elastic Block Store (Amazon EBS) attached to each instance as defined by InstanceType .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-ebsconfiguration

Return type

Union[IResolvable, EbsConfigurationProperty, None]

instance_type

An EC2 instance type, such as m3.xlarge .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-instancetype

Return type

str

weighted_capacity

The number of units that a provisioned instance of this type provides toward fulfilling the target capacities defined in InstanceFleetConfig .

This value is 1 for a master instance fleet, and must be 1 or greater for core and task instance fleets. Defaults to 1 if not specified.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-instancetypeconfig.html#cfn-elasticmapreduce-cluster-instancetypeconfig-weightedcapacity

Return type

Union[int, float, None]

JobFlowInstancesConfigProperty

class CfnCluster.JobFlowInstancesConfigProperty(*, additional_master_security_groups=None, additional_slave_security_groups=None, core_instance_fleet=None, core_instance_group=None, ec2_key_name=None, ec2_subnet_id=None, ec2_subnet_ids=None, emr_managed_master_security_group=None, emr_managed_slave_security_group=None, hadoop_version=None, keep_job_flow_alive_when_no_steps=None, master_instance_fleet=None, master_instance_group=None, placement=None, service_access_security_group=None, task_instance_fleets=None, task_instance_groups=None, termination_protected=None)

Bases: object

JobFlowInstancesConfig is a property of the AWS::EMR::Cluster resource.

JobFlowInstancesConfig defines the instance groups or instance fleets that comprise the cluster. JobFlowInstancesConfig must contain either InstanceFleetConfig or InstanceGroupConfig . They cannot be used together.

You can now define task instance groups or task instance fleets using the TaskInstanceGroups and TaskInstanceFleets subproperties. Using these subproperties reduces delays in provisioning task nodes compared to specifying task nodes with the InstanceFleetConfig and InstanceGroupConfig resources.

Parameters
  • additional_master_security_groups (Optional[Sequence[str]]) – A list of additional Amazon EC2 security group IDs for the master node.

  • additional_slave_security_groups (Optional[Sequence[str]]) – A list of additional Amazon EC2 security group IDs for the core and task nodes.

  • core_instance_fleet (Union[IResolvable, InstanceFleetConfigProperty, Dict[str, Any], None]) – Describes the EC2 instances and instance configurations for the core instance fleet when using clusters with the instance fleet configuration.

  • core_instance_group (Union[IResolvable, InstanceGroupConfigProperty, Dict[str, Any], None]) – Describes the EC2 instances and instance configurations for core instance groups when using clusters with the uniform instance group configuration.

  • ec2_key_name (Optional[str]) – The name of the EC2 key pair that can be used to connect to the master node using SSH as the user called “hadoop.”.

  • ec2_subnet_id (Optional[str]) – Applies to clusters that use the uniform instance group configuration. To launch the cluster in Amazon Virtual Private Cloud (Amazon VPC), set this parameter to the identifier of the Amazon VPC subnet where you want the cluster to launch. If you do not specify this value and your account supports EC2-Classic, the cluster launches in EC2-Classic.

  • ec2_subnet_ids (Optional[Sequence[str]]) – Applies to clusters that use the instance fleet configuration. When multiple EC2 subnet IDs are specified, Amazon EMR evaluates them and launches instances in the optimal subnet. .. epigraph:: The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.

  • emr_managed_master_security_group (Optional[str]) – The identifier of the Amazon EC2 security group for the master node. If you specify EmrManagedMasterSecurityGroup , you must also specify EmrManagedSlaveSecurityGroup .

  • emr_managed_slave_security_group (Optional[str]) – The identifier of the Amazon EC2 security group for the core and task nodes. If you specify EmrManagedSlaveSecurityGroup , you must also specify EmrManagedMasterSecurityGroup .

  • hadoop_version (Optional[str]) – Applies only to Amazon EMR release versions earlier than 4.0. The Hadoop version for the cluster. Valid inputs are “0.18” (no longer maintained), “0.20” (no longer maintained), “0.20.205” (no longer maintained), “1.0.3”, “2.2.0”, or “2.4.0”. If you do not set this value, the default of 0.18 is used, unless the AmiVersion parameter is set in the RunJobFlow call, in which case the default version of Hadoop for that AMI version is used.

  • keep_job_flow_alive_when_no_steps (Union[bool, IResolvable, None]) – Specifies whether the cluster should remain available after completing all steps. Defaults to true . For more information about configuring cluster termination, see Control Cluster Termination in the EMR Management Guide .

  • master_instance_fleet (Union[IResolvable, InstanceFleetConfigProperty, Dict[str, Any], None]) – Describes the EC2 instances and instance configurations for the master instance fleet when using clusters with the instance fleet configuration.

  • master_instance_group (Union[IResolvable, InstanceGroupConfigProperty, Dict[str, Any], None]) – Describes the EC2 instances and instance configurations for the master instance group when using clusters with the uniform instance group configuration.

  • placement (Union[IResolvable, PlacementTypeProperty, Dict[str, Any], None]) – The Availability Zone in which the cluster runs.

  • service_access_security_group (Optional[str]) – The identifier of the Amazon EC2 security group for the Amazon EMR service to access clusters in VPC private subnets.

  • task_instance_fleets (Union[IResolvable, Sequence[Union[IResolvable, InstanceFleetConfigProperty, Dict[str, Any]]], None]) – CfnCluster.JobFlowInstancesConfigProperty.TaskInstanceFleets.

  • task_instance_groups (Union[IResolvable, Sequence[Union[IResolvable, InstanceGroupConfigProperty, Dict[str, Any]]], None]) – CfnCluster.JobFlowInstancesConfigProperty.TaskInstanceGroups.

  • termination_protected (Union[bool, IResolvable, None]) – Specifies whether to lock the cluster to prevent the Amazon EC2 instances from being terminated by API call, user intervention, or in the event of a job-flow error.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

# configuration_property_: emr.CfnCluster.ConfigurationProperty

job_flow_instances_config_property = emr.CfnCluster.JobFlowInstancesConfigProperty(
    additional_master_security_groups=["additionalMasterSecurityGroups"],
    additional_slave_security_groups=["additionalSlaveSecurityGroups"],
    core_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty(
        instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
            instance_type="instanceType",

            # the properties below are optional
            bid_price="bidPrice",
            bid_price_as_percentage_of_on_demand_price=123,
            configurations=[emr.CfnCluster.ConfigurationProperty(
                classification="classification",
                configuration_properties={
                    "configuration_properties_key": "configurationProperties"
                },
                configurations=[configuration_property_]
            )],
            custom_ami_id="customAmiId",
            ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                        size_in_gb=123,
                        volume_type="volumeType",

                        # the properties below are optional
                        iops=123
                    ),

                    # the properties below are optional
                    volumes_per_instance=123
                )],
                ebs_optimized=False
            ),
            weighted_capacity=123
        )],
        launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
            on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
                allocation_strategy="allocationStrategy"
            ),
            spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
                timeout_action="timeoutAction",
                timeout_duration_minutes=123,

                # the properties below are optional
                allocation_strategy="allocationStrategy",
                block_duration_minutes=123
            )
        ),
        name="name",
        target_on_demand_capacity=123,
        target_spot_capacity=123
    ),
    core_instance_group=emr.CfnCluster.InstanceGroupConfigProperty(
        instance_count=123,
        instance_type="instanceType",

        # the properties below are optional
        auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
            constraints=emr.CfnCluster.ScalingConstraintsProperty(
                max_capacity=123,
                min_capacity=123
            ),
            rules=[emr.CfnCluster.ScalingRuleProperty(
                action=emr.CfnCluster.ScalingActionProperty(
                    simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                        scaling_adjustment=123,

                        # the properties below are optional
                        adjustment_type="adjustmentType",
                        cool_down=123
                    ),

                    # the properties below are optional
                    market="market"
                ),
                name="name",
                trigger=emr.CfnCluster.ScalingTriggerProperty(
                    cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                        comparison_operator="comparisonOperator",
                        metric_name="metricName",
                        period=123,
                        threshold=123,

                        # the properties below are optional
                        dimensions=[emr.CfnCluster.MetricDimensionProperty(
                            key="key",
                            value="value"
                        )],
                        evaluation_periods=123,
                        namespace="namespace",
                        statistic="statistic",
                        unit="unit"
                    )
                ),

                # the properties below are optional
                description="description"
            )]
        ),
        bid_price="bidPrice",
        configurations=[emr.CfnCluster.ConfigurationProperty(
            classification="classification",
            configuration_properties={
                "configuration_properties_key": "configurationProperties"
            },
            configurations=[configuration_property_]
        )],
        custom_ami_id="customAmiId",
        ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
            ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                    size_in_gb=123,
                    volume_type="volumeType",

                    # the properties below are optional
                    iops=123
                ),

                # the properties below are optional
                volumes_per_instance=123
            )],
            ebs_optimized=False
        ),
        market="market",
        name="name"
    ),
    ec2_key_name="ec2KeyName",
    ec2_subnet_id="ec2SubnetId",
    ec2_subnet_ids=["ec2SubnetIds"],
    emr_managed_master_security_group="emrManagedMasterSecurityGroup",
    emr_managed_slave_security_group="emrManagedSlaveSecurityGroup",
    hadoop_version="hadoopVersion",
    keep_job_flow_alive_when_no_steps=False,
    master_instance_fleet=emr.CfnCluster.InstanceFleetConfigProperty(
        instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
            instance_type="instanceType",

            # the properties below are optional
            bid_price="bidPrice",
            bid_price_as_percentage_of_on_demand_price=123,
            configurations=[emr.CfnCluster.ConfigurationProperty(
                classification="classification",
                configuration_properties={
                    "configuration_properties_key": "configurationProperties"
                },
                configurations=[configuration_property_]
            )],
            custom_ami_id="customAmiId",
            ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                        size_in_gb=123,
                        volume_type="volumeType",

                        # the properties below are optional
                        iops=123
                    ),

                    # the properties below are optional
                    volumes_per_instance=123
                )],
                ebs_optimized=False
            ),
            weighted_capacity=123
        )],
        launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
            on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
                allocation_strategy="allocationStrategy"
            ),
            spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
                timeout_action="timeoutAction",
                timeout_duration_minutes=123,

                # the properties below are optional
                allocation_strategy="allocationStrategy",
                block_duration_minutes=123
            )
        ),
        name="name",
        target_on_demand_capacity=123,
        target_spot_capacity=123
    ),
    master_instance_group=emr.CfnCluster.InstanceGroupConfigProperty(
        instance_count=123,
        instance_type="instanceType",

        # the properties below are optional
        auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
            constraints=emr.CfnCluster.ScalingConstraintsProperty(
                max_capacity=123,
                min_capacity=123
            ),
            rules=[emr.CfnCluster.ScalingRuleProperty(
                action=emr.CfnCluster.ScalingActionProperty(
                    simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                        scaling_adjustment=123,

                        # the properties below are optional
                        adjustment_type="adjustmentType",
                        cool_down=123
                    ),

                    # the properties below are optional
                    market="market"
                ),
                name="name",
                trigger=emr.CfnCluster.ScalingTriggerProperty(
                    cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                        comparison_operator="comparisonOperator",
                        metric_name="metricName",
                        period=123,
                        threshold=123,

                        # the properties below are optional
                        dimensions=[emr.CfnCluster.MetricDimensionProperty(
                            key="key",
                            value="value"
                        )],
                        evaluation_periods=123,
                        namespace="namespace",
                        statistic="statistic",
                        unit="unit"
                    )
                ),

                # the properties below are optional
                description="description"
            )]
        ),
        bid_price="bidPrice",
        configurations=[emr.CfnCluster.ConfigurationProperty(
            classification="classification",
            configuration_properties={
                "configuration_properties_key": "configurationProperties"
            },
            configurations=[configuration_property_]
        )],
        custom_ami_id="customAmiId",
        ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
            ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                    size_in_gb=123,
                    volume_type="volumeType",

                    # the properties below are optional
                    iops=123
                ),

                # the properties below are optional
                volumes_per_instance=123
            )],
            ebs_optimized=False
        ),
        market="market",
        name="name"
    ),
    placement=emr.CfnCluster.PlacementTypeProperty(
        availability_zone="availabilityZone"
    ),
    service_access_security_group="serviceAccessSecurityGroup",
    task_instance_fleets=[emr.CfnCluster.InstanceFleetConfigProperty(
        instance_type_configs=[emr.CfnCluster.InstanceTypeConfigProperty(
            instance_type="instanceType",

            # the properties below are optional
            bid_price="bidPrice",
            bid_price_as_percentage_of_on_demand_price=123,
            configurations=[emr.CfnCluster.ConfigurationProperty(
                classification="classification",
                configuration_properties={
                    "configuration_properties_key": "configurationProperties"
                },
                configurations=[configuration_property_]
            )],
            custom_ami_id="customAmiId",
            ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
                ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                    volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                        size_in_gb=123,
                        volume_type="volumeType",

                        # the properties below are optional
                        iops=123
                    ),

                    # the properties below are optional
                    volumes_per_instance=123
                )],
                ebs_optimized=False
            ),
            weighted_capacity=123
        )],
        launch_specifications=emr.CfnCluster.InstanceFleetProvisioningSpecificationsProperty(
            on_demand_specification=emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
                allocation_strategy="allocationStrategy"
            ),
            spot_specification=emr.CfnCluster.SpotProvisioningSpecificationProperty(
                timeout_action="timeoutAction",
                timeout_duration_minutes=123,

                # the properties below are optional
                allocation_strategy="allocationStrategy",
                block_duration_minutes=123
            )
        ),
        name="name",
        target_on_demand_capacity=123,
        target_spot_capacity=123
    )],
    task_instance_groups=[emr.CfnCluster.InstanceGroupConfigProperty(
        instance_count=123,
        instance_type="instanceType",

        # the properties below are optional
        auto_scaling_policy=emr.CfnCluster.AutoScalingPolicyProperty(
            constraints=emr.CfnCluster.ScalingConstraintsProperty(
                max_capacity=123,
                min_capacity=123
            ),
            rules=[emr.CfnCluster.ScalingRuleProperty(
                action=emr.CfnCluster.ScalingActionProperty(
                    simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
                        scaling_adjustment=123,

                        # the properties below are optional
                        adjustment_type="adjustmentType",
                        cool_down=123
                    ),

                    # the properties below are optional
                    market="market"
                ),
                name="name",
                trigger=emr.CfnCluster.ScalingTriggerProperty(
                    cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
                        comparison_operator="comparisonOperator",
                        metric_name="metricName",
                        period=123,
                        threshold=123,

                        # the properties below are optional
                        dimensions=[emr.CfnCluster.MetricDimensionProperty(
                            key="key",
                            value="value"
                        )],
                        evaluation_periods=123,
                        namespace="namespace",
                        statistic="statistic",
                        unit="unit"
                    )
                ),

                # the properties below are optional
                description="description"
            )]
        ),
        bid_price="bidPrice",
        configurations=[emr.CfnCluster.ConfigurationProperty(
            classification="classification",
            configuration_properties={
                "configuration_properties_key": "configurationProperties"
            },
            configurations=[configuration_property_]
        )],
        custom_ami_id="customAmiId",
        ebs_configuration=emr.CfnCluster.EbsConfigurationProperty(
            ebs_block_device_configs=[emr.CfnCluster.EbsBlockDeviceConfigProperty(
                volume_specification=emr.CfnCluster.VolumeSpecificationProperty(
                    size_in_gb=123,
                    volume_type="volumeType",

                    # the properties below are optional
                    iops=123
                ),

                # the properties below are optional
                volumes_per_instance=123
            )],
            ebs_optimized=False
        ),
        market="market",
        name="name"
    )],
    termination_protected=False
)

Attributes

additional_master_security_groups

A list of additional Amazon EC2 security group IDs for the master node.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-additionalmastersecuritygroups

Return type

Optional[List[str]]

additional_slave_security_groups

A list of additional Amazon EC2 security group IDs for the core and task nodes.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-additionalslavesecuritygroups

Return type

Optional[List[str]]

core_instance_fleet

Describes the EC2 instances and instance configurations for the core instance fleet when using clusters with the instance fleet configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-coreinstancefleet

Return type

Union[IResolvable, InstanceFleetConfigProperty, None]

core_instance_group

Describes the EC2 instances and instance configurations for core instance groups when using clusters with the uniform instance group configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-coreinstancegroup

Return type

Union[IResolvable, InstanceGroupConfigProperty, None]

ec2_key_name

The name of the EC2 key pair that can be used to connect to the master node using SSH as the user called “hadoop.”.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-ec2keyname

Return type

Optional[str]

ec2_subnet_id

Applies to clusters that use the uniform instance group configuration.

To launch the cluster in Amazon Virtual Private Cloud (Amazon VPC), set this parameter to the identifier of the Amazon VPC subnet where you want the cluster to launch. If you do not specify this value and your account supports EC2-Classic, the cluster launches in EC2-Classic.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-ec2subnetid

Return type

Optional[str]

ec2_subnet_ids

Applies to clusters that use the instance fleet configuration.

When multiple EC2 subnet IDs are specified, Amazon EMR evaluates them and launches instances in the optimal subnet. .. epigraph:

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-ec2subnetids

Return type

Optional[List[str]]

emr_managed_master_security_group

The identifier of the Amazon EC2 security group for the master node.

If you specify EmrManagedMasterSecurityGroup , you must also specify EmrManagedSlaveSecurityGroup .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-emrmanagedmastersecuritygroup

Return type

Optional[str]

emr_managed_slave_security_group

The identifier of the Amazon EC2 security group for the core and task nodes.

If you specify EmrManagedSlaveSecurityGroup , you must also specify EmrManagedMasterSecurityGroup .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-emrmanagedslavesecuritygroup

Return type

Optional[str]

hadoop_version

Applies only to Amazon EMR release versions earlier than 4.0. The Hadoop version for the cluster. Valid inputs are “0.18” (no longer maintained), “0.20” (no longer maintained), “0.20.205” (no longer maintained), “1.0.3”, “2.2.0”, or “2.4.0”. If you do not set this value, the default of 0.18 is used, unless the AmiVersion parameter is set in the RunJobFlow call, in which case the default version of Hadoop for that AMI version is used.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-hadoopversion

Return type

Optional[str]

keep_job_flow_alive_when_no_steps

Specifies whether the cluster should remain available after completing all steps.

Defaults to true . For more information about configuring cluster termination, see Control Cluster Termination in the EMR Management Guide .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-keepjobflowalivewhennosteps

Return type

Union[bool, IResolvable, None]

master_instance_fleet

Describes the EC2 instances and instance configurations for the master instance fleet when using clusters with the instance fleet configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-masterinstancefleet

Return type

Union[IResolvable, InstanceFleetConfigProperty, None]

master_instance_group

Describes the EC2 instances and instance configurations for the master instance group when using clusters with the uniform instance group configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-masterinstancegroup

Return type

Union[IResolvable, InstanceGroupConfigProperty, None]

placement

The Availability Zone in which the cluster runs.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-placement

Return type

Union[IResolvable, PlacementTypeProperty, None]

service_access_security_group

The identifier of the Amazon EC2 security group for the Amazon EMR service to access clusters in VPC private subnets.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-serviceaccesssecuritygroup

Return type

Optional[str]

task_instance_fleets

CfnCluster.JobFlowInstancesConfigProperty.TaskInstanceFleets.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-taskinstancefleets

Return type

Union[IResolvable, List[Union[IResolvable, InstanceFleetConfigProperty]], None]

task_instance_groups

CfnCluster.JobFlowInstancesConfigProperty.TaskInstanceGroups.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-taskinstancegroups

Return type

Union[IResolvable, List[Union[IResolvable, InstanceGroupConfigProperty]], None]

termination_protected

Specifies whether to lock the cluster to prevent the Amazon EC2 instances from being terminated by API call, user intervention, or in the event of a job-flow error.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-jobflowinstancesconfig.html#cfn-elasticmapreduce-cluster-jobflowinstancesconfig-terminationprotected

Return type

Union[bool, IResolvable, None]

KerberosAttributesProperty

class CfnCluster.KerberosAttributesProperty(*, kdc_admin_password, realm, ad_domain_join_password=None, ad_domain_join_user=None, cross_realm_trust_principal_password=None)

Bases: object

KerberosAttributes is a property of the AWS::EMR::Cluster resource.

KerberosAttributes define the cluster-specific Kerberos configuration when Kerberos authentication is enabled using a security configuration. The cluster-specific configuration must be compatible with the security configuration. For more information see Use Kerberos Authentication in the EMR Management Guide .

Parameters
  • kdc_admin_password (str) – The password used within the cluster for the kadmin service on the cluster-dedicated KDC, which maintains Kerberos principals, password policies, and keytabs for the cluster.

  • realm (str) – The name of the Kerberos realm to which all nodes in a cluster belong. For example, EC2.INTERNAL .

  • ad_domain_join_password (Optional[str]) – The Active Directory password for ADDomainJoinUser .

  • ad_domain_join_user (Optional[str]) – Required only when establishing a cross-realm trust with an Active Directory domain. A user with sufficient privileges to join resources to the domain.

  • cross_realm_trust_principal_password (Optional[str]) – Required only when establishing a cross-realm trust with a KDC in a different realm. The cross-realm principal password, which must be identical across realms.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-kerberosattributes.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

kerberos_attributes_property = emr.CfnCluster.KerberosAttributesProperty(
    kdc_admin_password="kdcAdminPassword",
    realm="realm",

    # the properties below are optional
    ad_domain_join_password="adDomainJoinPassword",
    ad_domain_join_user="adDomainJoinUser",
    cross_realm_trust_principal_password="crossRealmTrustPrincipalPassword"
)

Attributes

ad_domain_join_password

The Active Directory password for ADDomainJoinUser .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-kerberosattributes.html#cfn-elasticmapreduce-cluster-kerberosattributes-addomainjoinpassword

Return type

Optional[str]

ad_domain_join_user

Required only when establishing a cross-realm trust with an Active Directory domain.

A user with sufficient privileges to join resources to the domain.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-kerberosattributes.html#cfn-elasticmapreduce-cluster-kerberosattributes-addomainjoinuser

Return type

Optional[str]

cross_realm_trust_principal_password

Required only when establishing a cross-realm trust with a KDC in a different realm.

The cross-realm principal password, which must be identical across realms.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-kerberosattributes.html#cfn-elasticmapreduce-cluster-kerberosattributes-crossrealmtrustprincipalpassword

Return type

Optional[str]

kdc_admin_password

The password used within the cluster for the kadmin service on the cluster-dedicated KDC, which maintains Kerberos principals, password policies, and keytabs for the cluster.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-kerberosattributes.html#cfn-elasticmapreduce-cluster-kerberosattributes-kdcadminpassword

Return type

str

realm

The name of the Kerberos realm to which all nodes in a cluster belong.

For example, EC2.INTERNAL .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-kerberosattributes.html#cfn-elasticmapreduce-cluster-kerberosattributes-realm

Return type

str

KeyValueProperty

class CfnCluster.KeyValueProperty(*, key=None, value=None)

Bases: object

KeyValue is a subproperty of the HadoopJarStepConfig property type.

KeyValue is used to pass parameters to a step.

Parameters
  • key (Optional[str]) – The unique identifier of a key-value pair.

  • value (Optional[str]) – The value part of the identified key.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-keyvalue.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

key_value_property = emr.CfnCluster.KeyValueProperty(
    key="key",
    value="value"
)

Attributes

key

The unique identifier of a key-value pair.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-keyvalue.html#cfn-elasticmapreduce-cluster-keyvalue-key

Return type

Optional[str]

value

The value part of the identified key.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-keyvalue.html#cfn-elasticmapreduce-cluster-keyvalue-value

Return type

Optional[str]

ManagedScalingPolicyProperty

class CfnCluster.ManagedScalingPolicyProperty(*, compute_limits=None)

Bases: object

Managed scaling policy for an Amazon EMR cluster.

The policy specifies the limits for resources that can be added or terminated from a cluster. The policy only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Parameters

compute_limits (Union[IResolvable, ComputeLimitsProperty, Dict[str, Any], None]) – The EC2 unit limits for a managed scaling policy. The managed scaling activity of a cluster is not allowed to go above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-managedscalingpolicy.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

managed_scaling_policy_property = emr.CfnCluster.ManagedScalingPolicyProperty(
    compute_limits=emr.CfnCluster.ComputeLimitsProperty(
        maximum_capacity_units=123,
        minimum_capacity_units=123,
        unit_type="unitType",

        # the properties below are optional
        maximum_core_capacity_units=123,
        maximum_on_demand_capacity_units=123
    )
)

Attributes

compute_limits

The EC2 unit limits for a managed scaling policy.

The managed scaling activity of a cluster is not allowed to go above or below these limits. The limit only applies to the core and task nodes. The master node cannot be scaled after initial configuration.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-managedscalingpolicy.html#cfn-elasticmapreduce-cluster-managedscalingpolicy-computelimits

Return type

Union[IResolvable, ComputeLimitsProperty, None]

MetricDimensionProperty

class CfnCluster.MetricDimensionProperty(*, key, value)

Bases: object

MetricDimension is a subproperty of the CloudWatchAlarmDefinition property type.

MetricDimension specifies a CloudWatch dimension, which is specified with a Key Value pair. The key is known as a Name in CloudWatch. By default, Amazon EMR uses one dimension whose Key is JobFlowID and Value is a variable representing the cluster ID, which is ${emr.clusterId} . This enables the automatic scaling rule for EMR to bootstrap when the cluster ID becomes available during cluster creation.

Parameters
  • key (str) – The dimension name.

  • value (str) – The dimension value.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-metricdimension.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

metric_dimension_property = emr.CfnCluster.MetricDimensionProperty(
    key="key",
    value="value"
)

Attributes

key

The dimension name.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-metricdimension.html#cfn-elasticmapreduce-cluster-metricdimension-key

Return type

str

value

The dimension value.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-metricdimension.html#cfn-elasticmapreduce-cluster-metricdimension-value

Return type

str

OnDemandProvisioningSpecificationProperty

class CfnCluster.OnDemandProvisioningSpecificationProperty(*, allocation_strategy)

Bases: object

The launch specification for On-Demand Instances in the instance fleet, which determines the allocation strategy.

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions. On-Demand Instances allocation strategy is available in Amazon EMR version 5.12.1 and later.

Parameters

allocation_strategy (str) – Specifies the strategy to use in launching On-Demand instance fleets. Currently, the only option is lowest-price (the default), which launches the lowest price first.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ondemandprovisioningspecification.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

on_demand_provisioning_specification_property = emr.CfnCluster.OnDemandProvisioningSpecificationProperty(
    allocation_strategy="allocationStrategy"
)

Attributes

allocation_strategy

Specifies the strategy to use in launching On-Demand instance fleets.

Currently, the only option is lowest-price (the default), which launches the lowest price first.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-ondemandprovisioningspecification.html#cfn-elasticmapreduce-cluster-ondemandprovisioningspecification-allocationstrategy

Return type

str

PlacementTypeProperty

class CfnCluster.PlacementTypeProperty(*, availability_zone)

Bases: object

PlacementType is a property of the AWS::EMR::Cluster resource.

PlacementType determines the Amazon EC2 Availability Zone configuration of the cluster (job flow).

Parameters

availability_zone (str) – The Amazon EC2 Availability Zone for the cluster. AvailabilityZone is used for uniform instance groups, while AvailabilityZones (plural) is used for instance fleets.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-placementtype.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

placement_type_property = emr.CfnCluster.PlacementTypeProperty(
    availability_zone="availabilityZone"
)

Attributes

availability_zone

The Amazon EC2 Availability Zone for the cluster.

AvailabilityZone is used for uniform instance groups, while AvailabilityZones (plural) is used for instance fleets.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-placementtype.html#cfn-elasticmapreduce-cluster-placementtype-availabilityzone

Return type

str

ScalingActionProperty

class CfnCluster.ScalingActionProperty(*, simple_scaling_policy_configuration, market=None)

Bases: object

ScalingAction is a subproperty of the ScalingRule property type.

ScalingAction determines the type of adjustment the automatic scaling activity makes when triggered, and the periodicity of the adjustment.

Parameters
  • simple_scaling_policy_configuration (Union[IResolvable, SimpleScalingPolicyConfigurationProperty, Dict[str, Any]]) – The type of adjustment the automatic scaling activity makes when triggered, and the periodicity of the adjustment.

  • market (Optional[str]) – Not available for instance groups. Instance groups use the market type specified for the group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingaction.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

scaling_action_property = emr.CfnCluster.ScalingActionProperty(
    simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
        scaling_adjustment=123,

        # the properties below are optional
        adjustment_type="adjustmentType",
        cool_down=123
    ),

    # the properties below are optional
    market="market"
)

Attributes

market

Not available for instance groups.

Instance groups use the market type specified for the group.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingaction.html#cfn-elasticmapreduce-cluster-scalingaction-market

Return type

Optional[str]

simple_scaling_policy_configuration

The type of adjustment the automatic scaling activity makes when triggered, and the periodicity of the adjustment.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingaction.html#cfn-elasticmapreduce-cluster-scalingaction-simplescalingpolicyconfiguration

Return type

Union[IResolvable, SimpleScalingPolicyConfigurationProperty]

ScalingConstraintsProperty

class CfnCluster.ScalingConstraintsProperty(*, max_capacity, min_capacity)

Bases: object

ScalingConstraints is a subproperty of the AutoScalingPolicy property type.

ScalingConstraints defines the upper and lower EC2 instance limits for an automatic scaling policy. Automatic scaling activities triggered by automatic scaling rules will not cause an instance group to grow above or shrink below these limits.

Parameters
  • max_capacity (Union[int, float]) – The upper boundary of EC2 instances in an instance group beyond which scaling activities are not allowed to grow. Scale-out activities will not add instances beyond this boundary.

  • min_capacity (Union[int, float]) – The lower boundary of EC2 instances in an instance group below which scaling activities are not allowed to shrink. Scale-in activities will not terminate instances below this boundary.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingconstraints.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

scaling_constraints_property = emr.CfnCluster.ScalingConstraintsProperty(
    max_capacity=123,
    min_capacity=123
)

Attributes

max_capacity

The upper boundary of EC2 instances in an instance group beyond which scaling activities are not allowed to grow.

Scale-out activities will not add instances beyond this boundary.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingconstraints.html#cfn-elasticmapreduce-cluster-scalingconstraints-maxcapacity

Return type

Union[int, float]

min_capacity

The lower boundary of EC2 instances in an instance group below which scaling activities are not allowed to shrink.

Scale-in activities will not terminate instances below this boundary.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingconstraints.html#cfn-elasticmapreduce-cluster-scalingconstraints-mincapacity

Return type

Union[int, float]

ScalingRuleProperty

class CfnCluster.ScalingRuleProperty(*, action, name, trigger, description=None)

Bases: object

ScalingRule is a subproperty of the AutoScalingPolicy property type.

ScalingRule defines the scale-in or scale-out rules for scaling activity, including the CloudWatch metric alarm that triggers activity, how EC2 instances are added or removed, and the periodicity of adjustments. The automatic scaling policy for an instance group can comprise one or more automatic scaling rules.

Parameters
  • action (Union[IResolvable, ScalingActionProperty, Dict[str, Any]]) – The conditions that trigger an automatic scaling activity.

  • name (str) – The name used to identify an automatic scaling rule. Rule names must be unique within a scaling policy.

  • trigger (Union[IResolvable, ScalingTriggerProperty, Dict[str, Any]]) – The CloudWatch alarm definition that determines when automatic scaling activity is triggered.

  • description (Optional[str]) – A friendly, more verbose description of the automatic scaling rule.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingrule.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

scaling_rule_property = emr.CfnCluster.ScalingRuleProperty(
    action=emr.CfnCluster.ScalingActionProperty(
        simple_scaling_policy_configuration=emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
            scaling_adjustment=123,

            # the properties below are optional
            adjustment_type="adjustmentType",
            cool_down=123
        ),

        # the properties below are optional
        market="market"
    ),
    name="name",
    trigger=emr.CfnCluster.ScalingTriggerProperty(
        cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
            comparison_operator="comparisonOperator",
            metric_name="metricName",
            period=123,
            threshold=123,

            # the properties below are optional
            dimensions=[emr.CfnCluster.MetricDimensionProperty(
                key="key",
                value="value"
            )],
            evaluation_periods=123,
            namespace="namespace",
            statistic="statistic",
            unit="unit"
        )
    ),

    # the properties below are optional
    description="description"
)

Attributes

action

The conditions that trigger an automatic scaling activity.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingrule.html#cfn-elasticmapreduce-cluster-scalingrule-action

Return type

Union[IResolvable, ScalingActionProperty]

description

A friendly, more verbose description of the automatic scaling rule.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingrule.html#cfn-elasticmapreduce-cluster-scalingrule-description

Return type

Optional[str]

name

The name used to identify an automatic scaling rule.

Rule names must be unique within a scaling policy.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingrule.html#cfn-elasticmapreduce-cluster-scalingrule-name

Return type

str

trigger

The CloudWatch alarm definition that determines when automatic scaling activity is triggered.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingrule.html#cfn-elasticmapreduce-cluster-scalingrule-trigger

Return type

Union[IResolvable, ScalingTriggerProperty]

ScalingTriggerProperty

class CfnCluster.ScalingTriggerProperty(*, cloud_watch_alarm_definition)

Bases: object

ScalingTrigger is a subproperty of the ScalingRule property type.

ScalingTrigger determines the conditions that trigger an automatic scaling activity.

Parameters

cloud_watch_alarm_definition (Union[IResolvable, CloudWatchAlarmDefinitionProperty, Dict[str, Any]]) – The definition of a CloudWatch metric alarm. When the defined alarm conditions are met along with other trigger parameters, scaling activity begins.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingtrigger.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

scaling_trigger_property = emr.CfnCluster.ScalingTriggerProperty(
    cloud_watch_alarm_definition=emr.CfnCluster.CloudWatchAlarmDefinitionProperty(
        comparison_operator="comparisonOperator",
        metric_name="metricName",
        period=123,
        threshold=123,

        # the properties below are optional
        dimensions=[emr.CfnCluster.MetricDimensionProperty(
            key="key",
            value="value"
        )],
        evaluation_periods=123,
        namespace="namespace",
        statistic="statistic",
        unit="unit"
    )
)

Attributes

cloud_watch_alarm_definition

The definition of a CloudWatch metric alarm.

When the defined alarm conditions are met along with other trigger parameters, scaling activity begins.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scalingtrigger.html#cfn-elasticmapreduce-cluster-scalingtrigger-cloudwatchalarmdefinition

Return type

Union[IResolvable, CloudWatchAlarmDefinitionProperty]

ScriptBootstrapActionConfigProperty

class CfnCluster.ScriptBootstrapActionConfigProperty(*, path, args=None)

Bases: object

ScriptBootstrapActionConfig is a subproperty of the BootstrapActionConfig property type.

ScriptBootstrapActionConfig specifies the arguments and location of the bootstrap script for EMR to run on all cluster nodes before it installs open-source big data applications on them.

Parameters
  • path (str) – Location in Amazon S3 of the script to run during a bootstrap action.

  • args (Optional[Sequence[str]]) – A list of command line arguments to pass to the bootstrap action script.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scriptbootstrapactionconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

script_bootstrap_action_config_property = emr.CfnCluster.ScriptBootstrapActionConfigProperty(
    path="path",

    # the properties below are optional
    args=["args"]
)

Attributes

args

A list of command line arguments to pass to the bootstrap action script.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scriptbootstrapactionconfig.html#cfn-elasticmapreduce-cluster-scriptbootstrapactionconfig-args

Return type

Optional[List[str]]

path

Location in Amazon S3 of the script to run during a bootstrap action.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-scriptbootstrapactionconfig.html#cfn-elasticmapreduce-cluster-scriptbootstrapactionconfig-path

Return type

str

SimpleScalingPolicyConfigurationProperty

class CfnCluster.SimpleScalingPolicyConfigurationProperty(*, scaling_adjustment, adjustment_type=None, cool_down=None)

Bases: object

SimpleScalingPolicyConfiguration is a subproperty of the ScalingAction property type.

SimpleScalingPolicyConfiguration determines how an automatic scaling action adds or removes instances, the cooldown period, and the number of EC2 instances that are added each time the CloudWatch metric alarm condition is satisfied.

Parameters
  • scaling_adjustment (Union[int, float]) – The amount by which to scale in or scale out, based on the specified AdjustmentType . A positive value adds to the instance group’s EC2 instance count while a negative number removes instances. If AdjustmentType is set to EXACT_CAPACITY , the number should only be a positive integer. If AdjustmentType is set to PERCENT_CHANGE_IN_CAPACITY , the value should express the percentage as an integer. For example, -20 indicates a decrease in 20% increments of cluster capacity.

  • adjustment_type (Optional[str]) – The way in which EC2 instances are added (if ScalingAdjustment is a positive number) or terminated (if ScalingAdjustment is a negative number) each time the scaling activity is triggered. CHANGE_IN_CAPACITY is the default. CHANGE_IN_CAPACITY indicates that the EC2 instance count increments or decrements by ScalingAdjustment , which should be expressed as an integer. PERCENT_CHANGE_IN_CAPACITY indicates the instance count increments or decrements by the percentage specified by ScalingAdjustment , which should be expressed as an integer. For example, 20 indicates an increase in 20% increments of cluster capacity. EXACT_CAPACITY indicates the scaling activity results in an instance group with the number of EC2 instances specified by ScalingAdjustment , which should be expressed as a positive integer.

  • cool_down (Union[int, float, None]) – The amount of time, in seconds, after a scaling activity completes before any further trigger-related scaling activities can start. The default value is 0.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-simplescalingpolicyconfiguration.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

simple_scaling_policy_configuration_property = emr.CfnCluster.SimpleScalingPolicyConfigurationProperty(
    scaling_adjustment=123,

    # the properties below are optional
    adjustment_type="adjustmentType",
    cool_down=123
)

Attributes

adjustment_type

The way in which EC2 instances are added (if ScalingAdjustment is a positive number) or terminated (if ScalingAdjustment is a negative number) each time the scaling activity is triggered.

CHANGE_IN_CAPACITY is the default. CHANGE_IN_CAPACITY indicates that the EC2 instance count increments or decrements by ScalingAdjustment , which should be expressed as an integer. PERCENT_CHANGE_IN_CAPACITY indicates the instance count increments or decrements by the percentage specified by ScalingAdjustment , which should be expressed as an integer. For example, 20 indicates an increase in 20% increments of cluster capacity. EXACT_CAPACITY indicates the scaling activity results in an instance group with the number of EC2 instances specified by ScalingAdjustment , which should be expressed as a positive integer.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-simplescalingpolicyconfiguration.html#cfn-elasticmapreduce-cluster-simplescalingpolicyconfiguration-adjustmenttype

Return type

Optional[str]

cool_down

The amount of time, in seconds, after a scaling activity completes before any further trigger-related scaling activities can start.

The default value is 0.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-simplescalingpolicyconfiguration.html#cfn-elasticmapreduce-cluster-simplescalingpolicyconfiguration-cooldown

Return type

Union[int, float, None]

scaling_adjustment

The amount by which to scale in or scale out, based on the specified AdjustmentType .

A positive value adds to the instance group’s EC2 instance count while a negative number removes instances. If AdjustmentType is set to EXACT_CAPACITY , the number should only be a positive integer. If AdjustmentType is set to PERCENT_CHANGE_IN_CAPACITY , the value should express the percentage as an integer. For example, -20 indicates a decrease in 20% increments of cluster capacity.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-simplescalingpolicyconfiguration.html#cfn-elasticmapreduce-cluster-simplescalingpolicyconfiguration-scalingadjustment

Return type

Union[int, float]

SpotProvisioningSpecificationProperty

class CfnCluster.SpotProvisioningSpecificationProperty(*, timeout_action, timeout_duration_minutes, allocation_strategy=None, block_duration_minutes=None)

Bases: object

SpotProvisioningSpecification is a subproperty of the InstanceFleetProvisioningSpecifications property type.

SpotProvisioningSpecification determines the launch specification for Spot instances in the instance fleet, which includes the defined duration and provisioning timeout behavior. .. epigraph:

The instance fleet configuration is available only in Amazon EMR versions 4.8.0 and later, excluding 5.0.x versions.
Parameters
  • timeout_action (str) – The action to take when TargetSpotCapacity has not been fulfilled when the TimeoutDurationMinutes has expired; that is, when all Spot Instances could not be provisioned within the Spot provisioning timeout. Valid values are TERMINATE_CLUSTER and SWITCH_TO_ON_DEMAND . SWITCH_TO_ON_DEMAND specifies that if no Spot Instances are available, On-Demand Instances should be provisioned to fulfill any remaining Spot capacity.

  • timeout_duration_minutes (Union[int, float]) – The spot provisioning timeout period in minutes. If Spot Instances are not provisioned within this time period, the TimeOutAction is taken. Minimum value is 5 and maximum value is 1440. The timeout applies only during initial provisioning, when the cluster is first created.

  • allocation_strategy (Optional[str]) – Specifies the strategy to use in launching Spot Instance fleets. Currently, the only option is capacity-optimized (the default), which launches instances from Spot Instance pools with optimal capacity for the number of instances that are launching.

  • block_duration_minutes (Union[int, float, None]) – The defined duration for Spot Instances (also known as Spot blocks) in minutes. When specified, the Spot Instance does not terminate before the defined duration expires, and defined duration pricing for Spot Instances applies. Valid values are 60, 120, 180, 240, 300, or 360. The duration period starts as soon as a Spot Instance receives its instance ID. At the end of the duration, Amazon EC2 marks the Spot Instance for termination and provides a Spot Instance termination notice, which gives the instance a two-minute warning before it terminates. .. epigraph:: Spot Instances with a defined duration (also known as Spot blocks) are no longer available to new customers from July 1, 2021. For customers who have previously used the feature, we will continue to support Spot Instances with a defined duration until December 31, 2022.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-spotprovisioningspecification.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

spot_provisioning_specification_property = emr.CfnCluster.SpotProvisioningSpecificationProperty(
    timeout_action="timeoutAction",
    timeout_duration_minutes=123,

    # the properties below are optional
    allocation_strategy="allocationStrategy",
    block_duration_minutes=123
)

Attributes

allocation_strategy

Specifies the strategy to use in launching Spot Instance fleets.

Currently, the only option is capacity-optimized (the default), which launches instances from Spot Instance pools with optimal capacity for the number of instances that are launching.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-spotprovisioningspecification.html#cfn-elasticmapreduce-cluster-spotprovisioningspecification-allocationstrategy

Return type

Optional[str]

block_duration_minutes

The defined duration for Spot Instances (also known as Spot blocks) in minutes.

When specified, the Spot Instance does not terminate before the defined duration expires, and defined duration pricing for Spot Instances applies. Valid values are 60, 120, 180, 240, 300, or 360. The duration period starts as soon as a Spot Instance receives its instance ID. At the end of the duration, Amazon EC2 marks the Spot Instance for termination and provides a Spot Instance termination notice, which gives the instance a two-minute warning before it terminates. .. epigraph:

Spot Instances with a defined duration (also known as Spot blocks) are no longer available to new customers from July 1, 2021. For customers who have previously used the feature, we will continue to support Spot Instances with a defined duration until December 31, 2022.
Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-spotprovisioningspecification.html#cfn-elasticmapreduce-cluster-spotprovisioningspecification-blockdurationminutes

Return type

Union[int, float, None]

timeout_action

The action to take when TargetSpotCapacity has not been fulfilled when the TimeoutDurationMinutes has expired;

that is, when all Spot Instances could not be provisioned within the Spot provisioning timeout. Valid values are TERMINATE_CLUSTER and SWITCH_TO_ON_DEMAND . SWITCH_TO_ON_DEMAND specifies that if no Spot Instances are available, On-Demand Instances should be provisioned to fulfill any remaining Spot capacity.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-spotprovisioningspecification.html#cfn-elasticmapreduce-cluster-spotprovisioningspecification-timeoutaction

Return type

str

timeout_duration_minutes

The spot provisioning timeout period in minutes.

If Spot Instances are not provisioned within this time period, the TimeOutAction is taken. Minimum value is 5 and maximum value is 1440. The timeout applies only during initial provisioning, when the cluster is first created.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-spotprovisioningspecification.html#cfn-elasticmapreduce-cluster-spotprovisioningspecification-timeoutdurationminutes

Return type

Union[int, float]

StepConfigProperty

class CfnCluster.StepConfigProperty(*, hadoop_jar_step, name, action_on_failure=None)

Bases: object

StepConfig is a property of the AWS::EMR::Cluster resource.

The StepConfig property type specifies a cluster (job flow) step, which runs only on the master node. Steps are used to submit data processing jobs to the cluster.

Parameters
  • hadoop_jar_step (Union[IResolvable, HadoopJarStepConfigProperty, Dict[str, Any]]) – The JAR file used for the step.

  • name (str) – The name of the step.

  • action_on_failure (Optional[str]) – The action to take when the cluster step fails. Possible values are CANCEL_AND_WAIT and CONTINUE .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-stepconfig.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

step_config_property = emr.CfnCluster.StepConfigProperty(
    hadoop_jar_step=emr.CfnCluster.HadoopJarStepConfigProperty(
        jar="jar",

        # the properties below are optional
        args=["args"],
        main_class="mainClass",
        step_properties=[emr.CfnCluster.KeyValueProperty(
            key="key",
            value="value"
        )]
    ),
    name="name",

    # the properties below are optional
    action_on_failure="actionOnFailure"
)

Attributes

action_on_failure

The action to take when the cluster step fails.

Possible values are CANCEL_AND_WAIT and CONTINUE .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-stepconfig.html#cfn-elasticmapreduce-cluster-stepconfig-actiononfailure

Return type

Optional[str]

hadoop_jar_step

The JAR file used for the step.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-stepconfig.html#cfn-elasticmapreduce-cluster-stepconfig-hadoopjarstep

Return type

Union[IResolvable, HadoopJarStepConfigProperty]

name

The name of the step.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-stepconfig.html#cfn-elasticmapreduce-cluster-stepconfig-name

Return type

str

VolumeSpecificationProperty

class CfnCluster.VolumeSpecificationProperty(*, size_in_gb, volume_type, iops=None)

Bases: object

VolumeSpecification is a subproperty of the EbsBlockDeviceConfig property type.

VolumeSecification determines the volume type, IOPS, and size (GiB) for EBS volumes attached to EC2 instances.

Parameters
  • size_in_gb (Union[int, float]) – The volume size, in gibibytes (GiB). This can be a number from 1 - 1024. If the volume type is EBS-optimized, the minimum value is 10.

  • volume_type (str) – The volume type. Volume types supported are gp2, io1, and standard.

  • iops (Union[int, float, None]) – The number of I/O operations per second (IOPS) that the volume supports.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-volumespecification.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_emr as emr

volume_specification_property = emr.CfnCluster.VolumeSpecificationProperty(
    size_in_gb=123,
    volume_type="volumeType",

    # the properties below are optional
    iops=123
)

Attributes

iops

The number of I/O operations per second (IOPS) that the volume supports.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-volumespecification.html#cfn-elasticmapreduce-cluster-volumespecification-iops

Return type

Union[int, float, None]

size_in_gb

The volume size, in gibibytes (GiB).

This can be a number from 1 - 1024. If the volume type is EBS-optimized, the minimum value is 10.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-volumespecification.html#cfn-elasticmapreduce-cluster-volumespecification-sizeingb

Return type

Union[int, float]

volume_type

The volume type.

Volume types supported are gp2, io1, and standard.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-elasticmapreduce-cluster-volumespecification.html#cfn-elasticmapreduce-cluster-volumespecification-volumetype

Return type

str