AWS SDK Version 3 for .NET
API Reference

AWS services or capabilities described in AWS Documentation may vary by region/location. Click Getting Started with Amazon AWS to see specific differences applicable to the China (Beijing) Region.

This class provides some helper methods for creating a Resize Job Flow step as part of your job flow. The resize step can be used to automatically adjust the composition of your cluster while it is running. For example, if you have a large workflow with different compute requirements, you can use this step to automatically add a task instance group before your most compute intensive step.

AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
IAmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(credentials);
            
var resize = new ResizeJobFlowStep
{
    OnArrested = OnArrested.Continue,
    OnFailure = OnFailure.Continue
};
                    
resize.AddResizeAction(new AddInstanceGroup
{
    InstanceGroup = "core",
    InstanceCount = 10
});
                    
resize.AddResizeAction(new AddInstanceGroup
{
    InstanceGroup = "task",
    InstanceCount = 10,
    WithInstanceType = "m1.small"
});
                    
HadoopJarStepConfig config = resize.ToHadoopJarStepConfig();
            
StepConfig resizeJobFlow = new StepConfig {
     Name = "Resize job flow",
     ActionOnFailure = "TERMINATE_JOB_FLOW",
     HadoopJarStep = config,
};
            
RunJobFlowRequest request = new RunJobFlowRequest {
    Name = "Resize job flow",
    Steps = new List { resizeJobFlow },
    LogUri = "s3://log-bucket/",
    Instances = new JobFlowInstancesConfig {
        Ec2KeyName = "keypair",
        HadoopVersion = "0.20",
        InstanceCount = 5,
        KeepJobFlowAliveWhenNoSteps = true,
        MasterInstanceType = "m1.small",
        SlaveInstanceType = "m1.small"
    }
};
            
RunJobFlowResponse response = emr.RunJobFlow(request);
             

Inheritance Hierarchy

System.Object
  Amazon.ElasticMapReduce.Model.ResizeJobFlowStep

Namespace: Amazon.ElasticMapReduce.Model
Assembly: AWSSDK.ElasticMapReduce.dll
Version: 3.x.y.z

Syntax

C#
public class ResizeJobFlowStep

The ResizeJobFlowStep type exposes the following members

Constructors

NameDescription
Public Method ResizeJobFlowStep()

Creates a new ResizeJobFlowStep using the default Elastic Map Reduce bucket (us-east-1.elasticmapreduce) for the default (us-east-1) region.

Public Method ResizeJobFlowStep(string)

Creates a new ResizeJobFlowStep using the specified Amazon S3 bucket to load resources.

The official bucket format is ".elasticmapreduce", so if you're using the us-east-1 region, you should use the bucket "us-east-1.elasticmapreduce".

Properties

NameTypeDescription
Public Property OnArrested System.Nullable<Amazon.ElasticMapReduce.Model.OnArrested>

What action this step should take if any of the instance group modifications result in the instance group entering Arrested state. This can happen when the bootstrap actions on the newly launched instances are continuously failing.

Public Property OnFailure System.Nullable<Amazon.ElasticMapReduce.Model.OnFailure>

What action this step should take if the modification fails. This can happen when you request to perform an invalid action, such as shrink a core instance group.

Public Property Wait System.Boolean

Whether the step should wait for the modification to complete or if it should just continue onto the next step once the modification request is received. Defaults to true.

Methods

Note:

Asynchronous operations (methods ending with Async) in the table below are for .NET 4.5 or higher. For .NET 3.5 the SDK follows the standard naming convention of BeginMethodName and EndMethodName to indicate asynchronous operations - these method pairs are not shown in the table below.

NameDescription
Public Method AddResizeAction(ResizeAction)

Add a new action for this step to perform. These actions can be to modify or add instance groups. This step supports multiple actions, but requires at least one be specified.

Public Method ToHadoopJarStepConfig()

Creates the final HadoopJarStepConfig once you are done configuring the step. You can use this as you would any other HadoopJarStepConfig.

Version Information

.NET Standard:
Supported in: 1.3

.NET Framework:
Supported in: 4.5, 4.0, 3.5

Portable Class Library:
Supported in: Windows Store Apps
Supported in: Windows Phone 8.1
Supported in: Xamarin Android
Supported in: Xamarin iOS (Unified)
Supported in: Xamarin.Forms