AWS SDK Version 2 for .NET
API Reference

AWS services or capabilities described in AWS Documentation may vary by region/location. Click Getting Started with Amazon AWS to see specific differences applicable to the China (Beijing) Region.

.NET Framework 3.5
 
This class provides some helper methods for creating a Resize Job Flow step as part of your job flow. The resize step can be used to automatically adjust the composition of your cluster while it is running. For example, if you have a large workflow with different compute requirements, you can use this step to automatically add a task instance group before your most compute intensive step.
AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);
IAmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(credentials);
            
var resize = new ResizeJobFlowStep
{
    OnArrested = OnArrested.Continue,
    OnFailure = OnFailure.Continue
};
                    
resize.AddResizeAction(new AddInstanceGroup
{
    InstanceGroup = "core",
    InstanceCount = 10
});
                    
resize.AddResizeAction(new AddInstanceGroup
{
    InstanceGroup = "task",
    InstanceCount = 10,
    WithInstanceType = "m1.small"
});
                    
HadoopJarStepConfig config = resize.ToHadoopJarStepConfig();
            
StepConfig resizeJobFlow = new StepConfig {
     Name = "Resize job flow",
     ActionOnFailure = "TERMINATE_JOB_FLOW",
     HadoopJarStep = config,
};
            
RunJobFlowRequest request = new RunJobFlowRequest {
    Name = "Resize job flow",
    Steps = new List<StepConfig> { resizeJobFlow },
    LogUri = "s3://log-bucket/",
    Instances = new JobFlowInstancesConfig {
        Ec2KeyName = "keypair",
        HadoopVersion = "0.20",
        InstanceCount = 5,
        KeepJobFlowAliveWhenNoSteps = true,
        MasterInstanceType = "m1.small",
        SlaveInstanceType = "m1.small"
    }
};
            
RunJobFlowResponse response = emr.RunJobFlow(request);
             

Inheritance Hierarchy

System.Object
  Amazon.ElasticMapReduce.Model.ResizeJobFlowStep

Namespace: Amazon.ElasticMapReduce.Model
Assembly: AWSSDK.dll
Version: (assembly version)

Syntax

C#
public class ResizeJobFlowStep : Object

The ResizeJobFlowStep type exposes the following members

Constructors

NameDescription
Public Method ResizeJobFlowStep() Creates a new ResizeJobFlowStep using the default Elastic Map Reduce bucket (us-east-1.elasticmapreduce) for the default (us-east-1) region.
Public Method ResizeJobFlowStep(string) Creates a new ResizeJobFlowStep using the specified Amazon S3 bucket to load resources.

The official bucket format is "<region>.elasticmapreduce", so if you're using the us-east-1 region, you should use the bucket "us-east-1.elasticmapreduce".

Properties

NameTypeDescription
Public Property OnArrested System.Nullable<Amazon.ElasticMapReduce.Model.OnArrested> What action this step should take if any of the instance group modifications result in the instance group entering Arrested state. This can happen when the bootstrap actions on the newly launched instances are continuously failing.
Public Property OnFailure System.Nullable<Amazon.ElasticMapReduce.Model.OnFailure> What action this step should take if the modification fails. This can happen when you request to perform an invalid action, such as shrink a core instance group.
Public Property Wait System.Boolean Whether the step should wait for the modification to complete or if it should just continue onto the next step once the modification request is received. Defaults to true.

Methods

NameDescription
Public Method AddResizeAction(ResizeAction) Add a new action for this step to perform. These actions can be to modify or add instance groups. This step supports multiple actions, but requires at least one be specified.
Public Method ToHadoopJarStepConfig() Creates the final HadoopJarStepConfig once you are done configuring the step. You can use this as you would any other HadoopJarStepConfig.

Version Information

.NET Framework:
Supported in: 4.5, 4.0, 3.5

.NET for Windows Store apps:
Supported in: Windows 8