This class provides some helper methods for creating a Resize Job Flow step
as part of your job flow. The resize step can be used to automatically
adjust the composition of your cluster while it is running. For example, if
you have a large workflow with different compute requirements, you can use
this step to automatically add a task instance group before your most compute
intensive step.
CopyC#

AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey); AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(credentials); HadoopJarStepConfig config = new ResizeJobFlowStep() .WithResizeAction(new ModifyInstanceGroup() .WithInstanceGroup("core") .WithInstanceCount(10)) .WithResizeAction(new AddInstanceGroup() .WithInstanceGroup("task") .WithInstanceCount(10) .WithInstanceType("m1.small")) .WithOnArrested(OnArrested.Continue) .WithOnFailure(OnFailure.Continue) .ToHadoopJarStepConfig(); StepConfig resizeJobFlow = new StepConfig { Name = "Resize job flow", ActionOnFailure = "TERMINATE_JOB_FLOW", HadoopJarStep = config }; RunJobFlowRequest request = new RunJobFlowRequest { Name = "Resize job flow", Steps = new List<StepConfig> { resizeJobFlow }, LogUri = "s3://log-bucket/", Instances = new JobFlowInstancesConfig { Ec2KeyName = "keypair", HadoopVersion = "0.20", InstanceCount = 5, KeepJobFlowAliveWhenNoSteps = true, MasterInstanceType = "m1.small", SlaveInstanceType = "m1.small" } }; RunJobFlowResult result = emr.RunJobFlow(request).RunJobFlowResult;

C# |
public class ResizeJobFlowStep

All Members | Constructors | Methods | |||
Icon | Member | Description |
---|---|---|
![]() | ResizeJobFlowStep()()()() |
Creates a new ResizeJobFlowStep using the default Elastic Map Reduce
bucket (us-east-1.elasticmapreduce) for the default (us-east-1) region.
|
![]() | ResizeJobFlowStep(String) |
Creates a new ResizeJobFlowStep using the specified Amazon S3 bucket to
load resources.
The official bucket format is "<region>.elasticmapreduce", so if you're using the us-east-1 region, you should use the bucket "us-east-1.elasticmapreduce". |
![]() | Equals(Object) | (Inherited from Object.) |
![]() | GetHashCode()()()() | Serves as a hash function for a particular type. (Inherited from Object.) |
![]() | GetType()()()() | Gets the type of the current instance. (Inherited from Object.) |
![]() | ToHadoopJarStepConfig()()()() |
Creates the final HadoopJarStepConfig once you are done configuring the step. You can use
this as you would any other HadoopJarStepConfig.
|
![]() | ToString()()()() | Returns a string that represents the current object. (Inherited from Object.) |
![]() | WithOnArrested(OnArrested) | Obsolete.
What action this step should take if any of the instance group modifications result
in the instance group entering Arrested state. This can happen when the bootstrap
actions on the newly launched instances are continuously failing.
|
![]() | WithOnFailure(OnFailure) | Obsolete.
What action this step should take if the modification fails. This can happen when
you request to perform an invalid action, such as shrink a core instance group.
|
![]() | WithResizeAction(ResizeAction) | Obsolete.
Add a new action for this step to perform. These actions can be to modify
or add instance groups. This step supports multiple actions, but requires
at least one be specified.
|
![]() | WithWait(Boolean) | Obsolete.
Specifies whether the step should wait for the modification to complete
or if it should just continue onto the next step once the modification
request is received. Defaults to true.
|

Object | |
![]() | ResizeJobFlowStep |