AWS SDK Version 3 for .NET
API Reference

AWS services or capabilities described in AWS Documentation may vary by region/location. Click Getting Started with Amazon AWS to see specific differences applicable to the China (Beijing) Region.

This class provides helper methods for creating common Elastic MapReduce step types. To use StepFactory, you should construct it with the appropriate bucket for your region. The official bucket format is ".elasticmapreduce", so us-east-1 would use the bucket "us-east-1.elasticmapreduce".

Inheritance Hierarchy

System.Object
  Amazon.ElasticMapReduce.Model.StepFactory

Namespace: Amazon.ElasticMapReduce.Model
Assembly: AWSSDK.ElasticMapReduce.dll
Version: 3.x.y.z

Syntax

C# 
public class StepFactory

The StepFactory type exposes the following members

Constructors

NameDescription
Public Method StepFactory(string)

Constructor that gets its resources from the S3 bucket specified

Public Method StepFactory(RegionEndpoint)

Constructor that gets its resources EMR bucket in the specified region.

Public Method StepFactory()

Default constructor that gets its resources from the S3 in us-east-1.

Methods

Note:

Asynchronous operations (methods ending with Async) in the table below are for .NET 4.5 or higher. For .NET 3.5 the SDK follows the standard naming convention of BeginMethodName and EndMethodName to indicate asynchronous operations - these method pairs are not shown in the table below.

NameDescription
Public Method NewEnableDebuggingStep()

When run as the first step in your job flow, enables the Hadoop debugging UI in the AWS Management Console.

Public Method NewInstallHiveStep(HiveVersion[])
Public Method NewInstallHiveStep()

Step that installs Hive on your job flow.

Public Method NewInstallPigStep()

Step that installs Pig on your job flow.

Public Method NewRunHiveScriptStep(string, String[])

Step that runs a Hive script on your job flow.

Public Method NewRunHiveScriptStepVersioned(string, string, String[])

Step that runs a Hive script on your job flow with a specific verson of Hive.

Public Method NewRunPigScriptStep(string, String[])

Step that runs a Pig script on your job flow.

Public Method NewScriptRunnerStep(string, String[])

Runs a specified script on the master node of your cluster.

Examples

Create an interactive Hive job flow with debugging enabled: 
 IAmazonElasticMapReduce emr = AWSClientFactory.CreateAmazonElasticMapReduceClient(accessKey, secretKey);
            
 StepFactory stepFactory = new StepFactory();
            
 StepConfig enableDebugging = new StepConfig {
    Name = "Enable Debugging",
    ActionOnFailure = "TERMINATE_JOB_FLOW",
    HadoopJarStep = stepFactory.NewEnableDebuggingStep()
 };
            
 StepConfig installHive = new StepConfig {
    Name = "Install Hive",
    ActionOnFailure = "TERMINATE_JOB_FLOW",
    HadoopJarStep = stepFactory.NewInstallHiveStep()
 };
              
 RunJobFlowRequest request = new RunJobFlowRequest {
    Name = "Hive Interactive",
    Steps = new List<StepConfig> { enableDebugging, installHive },
    LogUri = "s3://log-bucket/",
    Instances = new JobFlowInstancesConfig {
        Ec2KeyName = "keypair",
        HadoopVersion = "0.20",
        InstanceCount = 5,
        KeepJobFlowAliveWhenNoSteps = true,
        MasterInstanceType = "m1.small",
        SlaveInstanceType = "m1.small"
    }
 };
            
RunJobFlowResponse response = emr.RunJobFlow(request);

Version Information

.NET Core App:
Supported in: 3.1

.NET Standard:
Supported in: 2.0

.NET Framework:
Supported in: 4.5, 4.0, 3.5