This class provides helper methods for creating common Elastic MapReduce step
types. To use StepFactory, you should construct it with the appropriate bucket
for your region. The official bucket format is "<region>.elasticmapreduce", so
us-east-1 would use the bucket "us-east-1.elasticmapreduce".
Inheritance Hierarchy
Namespace: Amazon.ElasticMapReduce.Model
Assembly: AWSSDK.dll
Version: (assembly version)
Syntax
public class StepFactory : Object
The StepFactory type exposes the following members
Constructors
| Name | Description | |
|---|---|---|
|
StepFactory(string) | Constructor that gets its resources from the S3 bucket specified |
|
|
StepFactory(RegionEndpoint) | Constructor that gets its resources EMR bucket in the specified region. |
|
|
StepFactory() | Default constructor that gets its resources from the S3 in us-east-1. |
Methods
| Name | Description | |
|---|---|---|
|
|
NewEnableDebuggingStep() | When run as the first step in your job flow, enables the Hadoop debugging UI in the AWS Management Console. |
|
|
NewInstallHiveStep(HiveVersion[]) | |
|
|
NewInstallHiveStep() | Step that installs Hive on your job flow. |
|
|
NewInstallPigStep() | Step that installs Pig on your job flow. |
|
|
NewRunHiveScriptStep(string, String[]) | Step that runs a Hive script on your job flow. |
|
|
NewRunHiveScriptStepVersioned(string, string, String[]) | Step that runs a Hive script on your job flow with a specific verson of Hive. |
|
|
NewRunPigScriptStep(string, String[]) | Step that runs a Pig script on your job flow. |
|
|
NewScriptRunnerStep(string, String[]) | Runs a specified script on the master node of your cluster. |
Examples
Create an interactive Hive job flow with debugging enabled:
IAmazonElasticMapReduce emr = AWSClientFactory.CreateAmazonElasticMapReduceClient(accessKey, secretKey);
StepFactory stepFactory = new StepFactory();
StepConfig enableDebugging = new StepConfig {
Name = "Enable Debugging",
ActionOnFailure = "TERMINATE_JOB_FLOW",
HadoopJarStep = stepFactory.NewEnableDebuggingStep()
};
StepConfig installHive = new StepConfig {
Name = "Install Hive",
ActionOnFailure = "TERMINATE_JOB_FLOW",
HadoopJarStep = stepFactory.NewInstallHiveStep()
};
RunJobFlowRequest request = new RunJobFlowRequest {
Name = "Hive Interactive",
Steps = new List<StepConfig> { enableDebugging, installHive },
LogUri = "s3://log-bucket/",
Instances = new JobFlowInstancesConfig {
Ec2KeyName = "keypair",
HadoopVersion = "0.20",
InstanceCount = 5,
KeepJobFlowAliveWhenNoSteps = true,
MasterInstanceType = "m1.small",
SlaveInstanceType = "m1.small"
}
};
RunJobFlowResponse response = emr.RunJobFlow(request);
Version Information
.NET Framework:
Supported in: 4.5, 4.0, 3.5
.NET for Windows Store apps:
Supported in: Windows 8