Class that makes it easy to define Hadoop Streaming steps.
CopyC#
See also: Hadoop Streaming

AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey); AmazonElasticMapReduce emr = new AmazonElasticMapReduceClient(credentials); HadoopJarStepConfig config = new StreamingStep { Inputs = new List<string> { "s3://elasticmapreduce/samples/wordcount/input" }, Output = "s3://my-bucket/output/", Mapper = "s3://elasticmapreduce/samples/wordcount/wordSplitter.py", Reducer = "aggregate" }.ToHadoopJarStepConfig(); StepConfig wordCount = new StepConfig { Name = "Word Count", ActionOnFailure = "TERMINATE_JOB_FLOW", HadoopJarStep = config }; RunJobFlowRequest request = new RunJobFlowRequest { Name = "Word Count", Steps = new List<StepConfig> { wordCount }, LogUri = "s3://log-bucket/", Instances = new JobFlowInstancesConfig { Ec2KeyName = "keypair", HadoopVersion = "0.20", InstanceCount = 5, KeepJobFlowAliveWhenNoSteps = true, MasterInstanceType = "m1.small", SlaveInstanceType = "m1.small" } }; RunJobFlowResult result = emr.RunJobFlow(request).RunJobFlowResult;

C# |
public class StreamingStep

All Members | Constructors | Methods | Properties | ||
Icon | Member | Description |
---|---|---|
![]() | StreamingStep()()()() |
Creates a new default StreamingStep.
|
![]() | Equals(Object) | (Inherited from Object.) |
![]() | GetHashCode()()()() | Serves as a hash function for a particular type. (Inherited from Object.) |
![]() | GetType()()()() | Gets the type of the current instance. (Inherited from Object.) |
![]() | HadoopConfig |
Gets and sets the hadoopConfig.
|
![]() | Inputs |
Gets and sets list of step input paths.
|
![]() | Mapper |
Gets and sets the mapper.
|
![]() | Output |
Gets and sets the output path.
|
![]() | Reducer |
Gets and sets the reducer.
|
![]() | ToHadoopJarStepConfig()()()() |
Creates the final HadoopJarStepConfig once you are done configuring the step. You can use
this as you would any other HadoopJarStepConfig.
|
![]() | ToString()()()() | Returns a string that represents the current object. (Inherited from Object.) |
![]() | WithHadoopConfig(String, String) | Obsolete.
Add a Hadoop config override (-D value)
|
![]() | WithInputs(array<String>[]()[][]) | Obsolete.
Add more input paths to this step.
|
![]() | WithMapper(String) | Obsolete.
Set the mapper.
|
![]() | WithOutput(String) | Obsolete.
Set the output path for this step.
|
![]() | WithReducer(String) | Obsolete.
Set the reducer.
|

Object | |
![]() | StreamingStep |