Amazon Elastic MapReduce
Developer Guide

Using the AWS SDK for .Net to Create an Amazon EMR Cluster

This documentation is for AMI versions 2.x and 3.x of Amazon EMR. See the Amazon EMR Release Guide for information about Amazon EMR releases 4.0.0 and above. For information about managing the Amazon EMR service in 4.x releases, see the Amazon EMR Management Guide.

The following example illustrates how the SDKs can simplify programming with Amazon EMR The code sample below uses the StepFactory object, a helper class for creating common Amazon EMR step types, to create an interactive Hive cluster with debugging enabled.


If you are adding IAM user visibility to a new cluster, call RunJobFlow and set VisibleToAllUsers=true, otherwise IAM users cannot view the cluster.

            var emrClient =AWSClientFactory.CreateAmazonElasticMapReduceClient(RegionEndpoint.USWest2);
            var stepFactory = new StepFactory();
            var enabledebugging = new StepConfig{
                Name = "Enable debugging",
                ActionOnFailure = "TERMINATE_JOB_FLOW",
                HadoopJarStep = stepFactory.NewEnableDebuggingStep()
            var installHive = new StepConfig{
                Name = "Install Hive",
                ActionOnFailure = "TERMINATE_JOB_FLOW",
                HadoopJarStep = stepFactory.NewInstallHiveStep()
            var instanceConfig = new JobFlowInstancesConfig{
                Ec2KeyName = "keypair",
                InstanceCount = 5,
                KeepJobFlowAliveWhenNoSteps = true,
                MasterInstanceType = "m1.small",
                SlaveInstanceType = "m1.small"
            var request = new RunJobFlowRequest{
                Name = "Hive Interactive",
                Steps = {enabledebugging, installHive},
		   AmiVersion = "3.8.0",
                LogUri = "s3://myawsbucket",
                Instances = instanceConfig,
		   ServiceRole = "service_role",
		   JobFlowRole = "jobflow_role"
            var result = emrClient.RunJobFlow(request);

At minimum, you must pass a service role and jobflow role corresponding to EMR_DefaultRole and EMR_EC2_DefaultRole, respectively. You can do this by invoking this AWS CLI command for the same account. First, look to see if the roles already exist:

aws iam list-roles | grep EMR

Both the instance profile (EMR_EC2_DefaultRole) and the service role (EMR_DefaultRole) will be displayed if they exist:

"RoleName": "EMR_DefaultRole", 
            "Arn": "arn:aws:iam::AccountID:role/EMR_DefaultRole"
            "RoleName": "EMR_EC2_DefaultRole", 
            "Arn": "arn:aws:iam::AccountID:role/EMR_EC2_DefaultRole"

If the default roles do not exist, you can use the following AWS CLI command to create them:

aws emr create-default-roles