Setting up EMR Serverless - Amazon EMR

Setting up EMR Serverless

Sign up for your AWS account

When you sign up for AWS, you automatically sign up your AWS account for all services, including the generally available Amazon EMR deployment options. If you have an AWS account already, skip to Create a user and grant permissions. If you don't have an AWS account, use the following procedure to create one.

To create an AWS account

  1. Open https://portal.aws.amazon.com/billing/signup.

  2. Follow the online instructions. Part of the sign-up procedure involves receiving a phone call and entering a verification code on the phone keypad.

Create a user and grant permissions

As a best practice, create an AWS Identity and Access Management (IAM) user with administrator permissions, and then use that IAM user for all work that doesn't require root credentials. Navigate to the IAM console at https://console.aws.amazon.com/iam/, create a password for console access, and create access keys to use command line tools. For instructions, see Creating your first IAM admin user and group in the IAM User Guide.

After you create an IAM user or role to work with EMR Serverless, attach an IAM policy to the user so that the user has sufficient permissions to invoke EMR Serverless actions. A policy similar to the following policy is ideal to get started.

{ "Version": "2012-10-17", "Statement": [ { "Sid": "EMRStudioCreate", "Effect": "Allow", "Action": [ "elasticmapreduce:CreateStudioPresignedUrl", "elasticmapreduce:DescribeStudio", "elasticmapreduce:CreateStudio", "elasticmapreduce:ListStudios" ], "Resource": "*" }, { "Sid": "EMRServerlessFullAccess", "Effect": "Allow", "Action": [ "emr-serverless:*" ], "Resource": "*" }, { "Sid": "AllowEC2ENICreationWithEMRTags", "Effect": "Allow", "Action": [ "ec2:CreateNetworkInterface" ], "Resource": [ "arn:aws:ec2:*:*:network-interface/*" ], "Condition": { "StringEquals": { "aws:CalledViaLast": "ops.emr-serverless.amazonaws.com" } } }, { "Sid": "AllowEMRServerlessServiceLinkedRoleCreation", "Effect": "Allow", "Action": "iam:CreateServiceLinkedRole", "Resource": "arn:aws:iam::*:role/aws-service-role/*" } ] }

In production environments, we recommend that you use finer-grained policies. For examples of such policies, see User access policy examples for EMR Serverless. To learn more about access management, see Access management for AWS resources in the IAM User Guide.

You can use this same process to create more groups and users and to give your users access to your AWS account resources.

Install and configure the AWS CLI

If you want to use EMR Serverless APIs, you must install the latest version of the AWS Command Line Interface (AWS CLI). You don't need the AWS CLI to use EMR Serverless from the EMR Studio console, and you can get started without the CLI by following the steps in Getting started from the console.

To set up the AWS CLI

  1. To install the latest version of the AWS CLI for macOS, Linux, or Windows, see Installing or updating the latest version of the AWS CLI.

  2. To configure the AWS CLI and secure setup of your access to AWS services, including EMR Serverless, see Quick configuration with aws configure.

  3. To verify the setup, enter the following DataBrew command at the command prompt.

    aws emr-serverless help

    AWS CLI commands use the default AWS Region from your configuration, unless you set it with a parameter or a profile. To set your AWS Region with a parameter, you can add the --region parameter to each command.

    To set your AWS Region with a profile, first add a named profile in the ~/.aws/config file or the %UserProfile%/.aws/config file (for Microsoft Windows). Follow the steps in Named profiles for the AWS CLI. Next, set your AWS Region and other settings with a command similar to the one in the following example.

    [profile emr-serverless] aws_access_key_id = ACCESS-KEY-ID-OF-IAM-USER aws_secret_access_key = SECRET-ACCESS-KEY-ID-OF-IAM-USER region = us-east-1 output = text

Open the console

Most of the console-oriented topics in this section start from the Amazon EMR console. If you aren't already signed in to your AWS account, sign in, then open the Amazon EMR console and continue to the next section to continue getting started with Amazon EMR.