Customize IAM roles - Amazon EMR

Customize IAM roles

You may want to customize the IAM service roles and permissions to limit privileges according to your security requirements. To customize permissions, we recommend that you create new roles and policies. Begin with the permissions in the managed policies for the default roles (for example, AmazonElasticMapReduceforEC2Role and AmazonElasticMapReduceRole). Then, copy and paste the contents to new policy statements, modify the permissions as appropriate, and attach the modified permissions policies to the roles that you create. You must have the appropriate IAM permissions to work with roles and policies. For more information, see Allow users and groups to create and modify roles.

If you create a custom EMR role for EC2, follow the basic work flow, which automatically creates an instance profile of the same name. Amazon EC2 allows you to create instance profiles and roles with different names, but Amazon EMR does not support this configuration, and it results in an "invalid instance profile" error when you create the cluster.

Important

Inline policies are not automatically updated when service requirements change. If you create and attach inline policies, be aware that service updates might occur that suddenly cause permissions errors. For more information, see Managed Policies and Inline Policies in the IAM User Guide and Specify custom IAM roles when you create a cluster.

For more information about working with IAM roles, see the following topics in the IAM User Guide:

Specify custom IAM roles when you create a cluster

You specify the service role for Amazon EMR and the role for the Amazon EC2 instance profile when you create a cluster. The user who is creating clusters needs permissions to retrieve and assign roles to Amazon EMR and EC2 instances. Otherwise, a account is not authorized to call EC2 error occurs. For more information, see Allow users and groups to create and modify roles.

Use the console to specify custom roles

When you create a cluster, you can specify a custom service role for Amazon EMR, a custom role for the EC2 instance profile, and a custom Auto Scaling role using Advanced options. When you use Quick options, the default service role and the default role for the EC2 instance profile are specified. For more information, see IAM service roles used by Amazon EMR.

Note

We’ve redesigned the Amazon EMR console to make it easier to use. See What's new with the console? to learn about the differences between the old and new console experiences.

New console
To specify custom IAM roles with the new console

When you create a cluster with the new console, you must specify a custom service role for Amazon EMR and a custom role for the EC2 instance profile. For more information, see IAM service roles used by Amazon EMR.

  1. Sign in to the AWS Management Console, and open the Amazon EMR console at https://console.aws.amazon.com/emr.

  2. Under EMR on EC2 in the left navigation pane, choose Clusters, and then choose Create cluster.

  3. Under Security configuration and permissions, find the IAM role for instance profile and Service role for Amazon EMR fields. For each role type, select a role from the list. Only roles within your account that have the appropriate trust policy for that role type are listed.

  4. Choose any other options that apply to your cluster.

  5. To launch your cluster, choose Create cluster.

Old console
To specify custom IAM roles with the old console

When you create a cluster with the old console, you can specify a custom service role for Amazon EMR, a custom role for the EC2 instance profile, and a custom Auto Scaling role using Advanced options. When you use Quick options, the default service role and the default role for the EC2 instance profile are specified. For more information, see IAM service roles used by Amazon EMR.

  1. Navigate to the new Amazon EMR console and select Switch to the old console from the side navigation. For more information on what to expect when you switch to the old console, see Using the old console.

  2. Choose Create cluster, Go to advanced options.

  3. Choose the cluster settings appropriate for your application until you reach Security Options. Under Permissions, the Default roles for Amazon EMR are selected.

  4. Choose Custom.

  5. For each role type, select a role from the list. Only roles within your account that have the appropriate trust policy for that role type are listed.

  6. Choose other options as appropriate for your cluster and then choose Create Cluster.

Use the AWS CLI to specify custom roles

You can specify a service role for Amazon EMR and a service role for cluster EC2 instances explicitly using options with the create-cluster command from the AWS CLI. Use the --service-role option to specify the service role. Use the InstanceProfile argument of the --ec2-attributes option to specify the role for the EC2 instance profile.

The Auto Scaling role is specified using a separate option, --auto-scaling-role. For more information, see Using automatic scaling with a custom policy for instance groups.

To specify custom IAM roles using the AWS CLI
  • The following command specifies the custom service role, MyCustomServiceRoleForEMR, and a custom role for the EC2 instance profile, MyCustomServiceRoleForClusterEC2Instances, when launching a cluster. This example uses the default Amazon EMR role.

    Note

    Linux line continuation characters (\) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^).

    aws emr create-cluster --name "Test cluster" --release-label emr-7.0.0 \ --applications Name=Hive Name=Pig --service-role MyCustomServiceRoleForEMR \ --ec2-attributes InstanceProfile=MyCustomServiceRoleForClusterEC2Instances,\ KeyName=myKey --instance-type m5.xlarge --instance-count 3

You can use these options to specify default roles explicitly rather than using the --use-default-roles option. The --use-default-roles option specifies the service role and the role for the EC2 instance profile defined in the config file for the AWS CLI.

The following example demonstrates the contents of a config file for the AWS CLI the specifies custom roles for Amazon EMR. With this configuration file, when the --use-default-roles option is specified, the cluster is created using the MyCustomServiceRoleForEMR and MyCustomServiceRoleForClusterEC2Instances. By default, the config file specifies the default service_role as AmazonElasticMapReduceRole and the default instance_profile as EMR_EC2_DefaultRole.

[default] output = json region = us-west-1 aws_access_key_id = myAccessKeyID aws_secret_access_key = mySecretAccessKey emr = service_role = MyCustomServiceRoleForEMR instance_profile = MyCustomServiceRoleForClusterEC2Instances