Amazon Elastic MapReduce
Developer Guide (API Version 2009-03-31)
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.Did this page help you?  Yes | No |  Tell us about it...

Configure IAM User Permissions

Amazon EMR supports AWS Identity and Access Management (IAM) policies. IAM is a web service that enables AWS customers to manage users and user permissions. For more information on IAM, go to Using IAM in the Using IAM guide.

IAM enables you to create users under your AWS account. You can define policies that limit the actions those users can take with your AWS resources. For example, you can choose to give an IAM user the ability to view, but not to create or terminate, Amazon S3 buckets in your AWS account. IAM is available at no charge to all AWS account holders; you do not need to sign up for IAM. You can use IAM through the Amazon EMR console, the Amazon EMR CLI, and programmatically through the Amazon EMR API and the AWS SDKs.

Instead of giving permissions to individual users, it can be convenient to use IAM roles and group users with certain permissions. For more information, see Configure IAM Roles for Amazon EMR.

Hidden Clusters

By default, if an IAM user launches a cluster, that cluster is hidden from other IAM users on the AWS account. For example, if an IAM user uses the CLI to run the --list command, the CLI only lists hidden clusters launched by that IAM user, not hidden clusters launched by other IAM users on the AWS account. This filtering occurs on all Amazon EMR interfaces—the console, CLI, API, and SDKs—and prevents IAM users from accessing and inadvertently changing clusters created by other IAM users. It is useful for clusters that are intended to be viewed by only a single IAM user and the main AWS account.

Note

This filtering does not prevent IAM users from viewing the underlying resources of the cluster, such as EC2 instances, by using AWS interfaces outside of Amazon EMR.

Visible Clusters

You also have the option to make a cluster visible and accessible to all IAM users under a single AWS account. This visibility can be set when you launch the cluster, or it can be added to a cluster that is already running.

Using this feature, you can make it possible for all IAM users on your account to access the cluster and, by configuring the policies of the IAM groups they belong to, control how those users interact with the cluster. For example, Devlin, a developer, belongs to a group that has an IAM policy that grants full access to all Amazon EMR functionality. He could launch a cluster that is visible to all other IAM users on his company's AWS account. A second IAM user, Ann, a data analyst with the same company, could then run queries on that cluster. Because Ann does not launch or terminate clusters, the IAM policy for the group she is in would only contain the permissions necessary for her to run her queries.

To make a cluster visible to all IAM users using the Amazon EMR console

IAM user visibility
  1. Open the Amazon Elastic MapReduce console at https://console.aws.amazon.com/elasticmapreduce/.

  2. Click Create cluster.

  3. In the Security and Access section, in the IAM User Access field, choose All other IAM Users.

    This makes the cluster visible and accessible to all IAM users on the AWS account. For more information, see Configure IAM User Permissions.

  4. Click Done and proceed to create the cluster as described in Plan an Amazon EMR Cluster.

To make a cluster visible to all IAM users using the Amazon EMR CLI

  • If you are adding IAM user visibility to a new cluster, add the --visible-to-all-users flag to the cluster call as shown in the following example.

    In the directory where you installed the Amazon EMR CLI, run the following from the command line. For more information, see the Command Line Interface Reference for Amazon EMR.

    • Linux, UNIX, and Mac OS X users:

      ./elastic-mapreduce --create --alive /
      --instance-type m1.xlarge --num-instances 2 /
      --visible-to-all-users 
    • Windows users:

      ruby elastic-mapreduce --create --alive --instance-type m1.xlarge --num-instances 2 --visible-to-all-users 

    If you are adding IAM user visibility to an existing cluster, you can use the --set-visible-to-all-users option of the Amazon EMR CLI, and specify identifier of the cluster to modify. This is shown in the following example, where job-flow-identifier would be replaced by the cluster identifier of your cluster. The visibility of a running cluster can be changed only by the IAM user that created the cluster or the AWS account that owns the cluster.

    In the directory where you installed the Amazon EMR CLI, run the following from the command line. For more information, see the Command Line Interface Reference for Amazon EMR.

    • Linux, UNIX, and Mac OS X users:

      ./elastic-mapreduce --set-visible-to-all-users true --jobflow job-flow-identifier
    • Windows users:

      ruby elastic-mapreduce --set-visible-to-all-users true --jobflow job-flow-identifier

To make a cluster visible to all IAM users using the Amazon EMR API

  • If you are adding IAM user visibility to a new cluster, call RunJobFlow and set VisibleToAllUsers=true, as shown in the following example.

    https://elasticmapreduce.amazonaws.com?Operation=RunJobFlow
    &Name=MyJobFlowName
    &VisibleToAllUsers=true
    &LogUri=s3n%3A%2F%2Fmybucket%2Fsubdir
    &Instances.MasterInstanceType=m1.small
    &Instances.SlaveInstanceType=m1.small
    &Instances.InstanceCount=4
    &Instances.Ec2KeyName=myec2keyname
    &Instances.Placement.AvailabilityZone=us-east-1a
    &Instances.KeepJobFlowAliveWhenNoSteps=true
    &Instances.TerminationProtected=true
    &Steps.member.1.Name=MyStepName
    &Steps.member.1.ActionOnFailure=CONTINUE
    &Steps.member.1.HadoopJarStep.Jar=MyJarFile
    &Steps.member.1.HadoopJarStep.MainClass=MyMainClass
    &Steps.member.1.HadoopJarStep.Args.member.1=arg1
    &Steps.member.1.HadoopJarStep.Args.member.2=arg2
    &AuthParams
    				

    If you are adding IAM user visibility to an existing cluster, call SetVisibleToAllUsers and set VisibleToAllUsers to true, as shown in the following example. The visibility of a running cluster can be changed only by the IAM user that created the cluster or the AWS account that owns the cluster.

    https://elasticmapreduce.amazonaws.com?Operation=SetVisibleToAllUsers
    &VisibleToAllUsers=true
    &JobFlowIds.member.1=j-3UN6WX5RRO2AG 
    &AuthParams