Amazon EMR supports AWS Identity and Access Management (IAM) policies. IAM is a web service that enables AWS customers to manage users and user permissions. For more information on IAM, go to Using IAM in the Using IAM guide.
IAM enables you to create users under your AWS account. You can define policies that limit the actions those users can take with your AWS resources. For example, you can choose to give an IAM user the ability to view, but not to create or delete, Amazon S3 buckets in your AWS account. IAM is available at no charge to all AWS account holders; you do not need to sign up for IAM. You can use IAM through the Amazon EMR console, the AWS CLI, and programmatically through the Amazon EMR API and the AWS SDKs.
Instead of giving permissions to individual users, it can be convenient to use IAM roles and group users with certain permissions. For more information, see Configure IAM Roles for Amazon EMR.
By default, clusters created using the console and AWS CLI are visible to all IAM users. You
can change this setting when you launch a cluster or after the cluster is created. If an
IAM user launches a cluster, and that cluster is hidden from other IAM users on the
AWS account, only that user will see the cluster. For example, if an IAM user uses the
AWS CLI to run the
list-clusters command, clusters created by other IAM
users with IAM user access set to "No other IAM users" are not listed. This filtering
occurs on all Amazon EMR interfaces—the console, CLI, API, and SDKs—and prevents
IAM users from accessing and inadvertently changing clusters created by other IAM
users. It is useful for clusters that are intended to be viewed by only a single IAM
user and the main AWS account.
This filtering does not prevent IAM users from viewing the underlying resources of the cluster, such as EC2 instances, by using AWS interfaces outside of Amazon EMR.
The default option, launching a cluster with IAM user access set to "All other IAM users," makes a cluster visible and accessible to all IAM users under a single AWS account. Using this feature, all IAM users on your account have access to the cluster and, by configuring the policies of the IAM groups they belong to, you control how those users interact with the cluster. For example, Devlin, a developer, belongs to a group that has an IAM policy that grants full access to all Amazon EMR functionality. He could launch a cluster that is visible to all other IAM users on his company's AWS account. A second IAM user, Ann, a data analyst with the same company, could then run queries on that cluster. Because Ann does not launch or terminate clusters, the IAM policy for the group she is in would only contain the permissions necessary for her to run her queries.
To configure cluster access using the Amazon EMR console
Open the Amazon EMR console at https://console.aws.amazon.com/elasticmapreduce/.
Click Create cluster.
In the Security and Access section, in the IAM User Access field, choose All other IAM users to make the cluster visible and accessible to all IAM users on the AWS account (the default option). Choose No other IAM users to restrict access. For more information, see Configure IAM User Permissions.
Proceed with creating the cluster as described in Plan an Amazon EMR Cluster.
To configure cluster access using the AWS CLI
By default, clusters created using the AWS CLI are visible to all users. You may optionally
--visible-to-all-users parameter to make a cluster visible
to all IAM users. To restrict cluster access on a new cluster using the AWS CLI, type
create-cluster command with the
To restrict access to a new cluster, type the following command and replace
myKey with the name of your EC2 key pair.
Linux, UNIX, and Mac OS X users:
aws emr create-cluster --name "
Test cluster" --ami-version
3.3--no-visible-to-all-users \ --applications Name=
Hue\ --use-default-roles --ec2-attributes KeyName=
aws emr create-cluster --name "
Test cluster" --ami-version
3.3--no-visible-to-all-users --applications Name=
Hue--use-default-roles --ec2-attributes KeyName=
When you specify the instance count without using the
--instance-groups parameter, a single Master node is launched,
and the remaining instances are launched as core nodes. All nodes will use the
instance type specified in the command.
If you have not previously created the default EMR service role and EC2
instance profile, type aws
emr create-default-roles to create
them before typing the
If you are configuring cluster access to an existing cluster, type the
modify-cluster-attributes subcommand with the
--no-visible-to-all-users parameter or the
--visible-to-all-users parameter. The visibility of a running
cluster can be changed only by the IAM user that created the cluster or the
AWS account owner. To use this subcommand, you need the cluster identifier
available via the console or the
To restrict access to a running cluster, type the following command.
aws emr modify-cluster-attributes --cluster-id
For more information on using Amazon EMR commands in the AWS CLI, see http://docs.aws.amazon.com/cli/latest/reference/emr.
To configure cluster access using the Amazon EMR API
To configure IAM user access on a new cluster, call
RunJobFlow and set
false, as shown in the
https://elasticmapreduce.amazonaws.com?Operation=RunJobFlow &Name=MyJobFlowName &VisibleToAllUsers=
true&LogUri=s3n%3A%2F%2Fmybucket%2Fsubdir &Instances.MasterInstanceType=m1.large &Instances.SlaveInstanceType=m1.large &Instances.InstanceCount=4 &Instances.Ec2KeyName=myec2keyname &Instances.Placement.AvailabilityZone=us-east-1a &Instances.KeepJobFlowAliveWhenNoSteps=true &Instances.TerminationProtected=true &Steps.member.1.Name=MyStepName &Steps.member.1.ActionOnFailure=CONTINUE &Steps.member.1.HadoopJarStep.Jar=MyJarFile &Steps.member.1.HadoopJarStep.MainClass=MyMainClass &Steps.member.1.HadoopJarStep.Args.member.1=arg1 &Steps.member.1.HadoopJarStep.Args.member.2=arg2 &AuthParams