Enable cluster access for Amazon EMR on EKS

You must allow Amazon EMR on EKS access to a specific namespace in your cluster by taking the following actions: creating a Kubernetes role, binding the role to a Kubernetes user, and mapping the Kubernetes user with the service linked role AWSServiceRoleForAmazonEMRContainers. These actions are automated in eksctl when the IAM identity mapping command is used with emr-containers as the service name. You can perform these operations easily by using the following command.

eksctl create iamidentitymapping \
    --cluster my_eks_cluster \
    --namespace kubernetes_namespace \
    --service-name "emr-containers"

Replace my_eks_cluster with the name of your Amazon EKS cluster and replace kubernetes_namespace with the Kubernetes namespace created to run Amazon EMR workloads.

Important

You must download the latest eksctl using the previous step Install eksctl to use this functionality.

Manual steps to enable cluster access for Amazon EMR on EKS

You can also use the following manual steps to enable cluster access for Amazon EMR on EKS.

1. Create a Kubernetes role in a specific namespace

   Amazon EKS 1.22

   With Amazon EKS 1.22, run the following command to create a Kubernetes role in a specific namespace. This role grants the necessary RBAC permissions to Amazon EMR on EKS.

   namespace=my-namespace
   cat - >>EOF | kubectl apply -f - >>namespace "${namespace}"
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: emr-containers
     namespace: ${namespace}
   rules:
     - apiGroups: [""]
       resources: ["namespaces"]
       verbs: ["get"]
     - apiGroups: [""]
       resources: ["serviceaccounts", "services", "configmaps", "events", "pods", "pods/log"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
     - apiGroups: [""]
       resources: ["secrets"]
       verbs: ["create", "patch", "delete", "watch"]
     - apiGroups: ["apps"]
       resources: ["statefulsets", "deployments"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
     - apiGroups: ["batch"]
       resources: ["jobs"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
     - apiGroups: ["extensions", "networking.k8s.io"]
       resources: ["ingresses"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
     - apiGroups: ["rbac.authorization.k8s.io"]
       resources: ["roles", "rolebindings"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
     - apiGroups: [""],
       resources: ["persistentvolumeclaims"],
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
   EOF
         

   Amazon EKS 1.21 and below

   With Amazon EKS 1.21 and below, run the following command to create a Kubernetes role in a specific namespace. This role grants the necessary RBAC permissions to Amazon EMR on EKS.

   namespace=my-namespace
   cat - >>EOF | kubectl apply -f - >>namespace "${namespace}"
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: emr-containers
     namespace: ${namespace}
   rules:
     - apiGroups: [""]
       resources: ["namespaces"]
       verbs: ["get"]
     - apiGroups: [""]
       resources: ["serviceaccounts", "services", "configmaps", "events", "pods", "pods/log"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
     - apiGroups: [""]
       resources: ["secrets"]
       verbs: ["create", "patch", "delete", "watch"]
     - apiGroups: ["apps"]
       resources: ["statefulsets", "deployments"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
     - apiGroups: ["batch"]
       resources: ["jobs"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
     - apiGroups: ["extensions"]
       resources: ["ingresses"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
     - apiGroups: ["rbac.authorization.k8s.io"]
       resources: ["roles", "rolebindings"]
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"]
     - apiGroups: [""],
       resources: ["persistentvolumeclaims"],
       verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"]
   EOF
         

2. Create a Kubernetes role binding scoped to the namespace

   Run the following command to create a Kubernetes role binding in the given namespace. This role binding grants the permissions defined in the role created in the previous step to a user named emr-containers. This user identifies service-linked roles for Amazon EMR on EKS and thus allows Amazon EMR on EKS to perform actions as defined by the role you created.

   namespace=my-namespace
   
   cat - <<EOF | kubectl apply -f - --namespace "${namespace}"
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: emr-containers
     namespace: ${namespace}
   subjects:
   - kind: User
     name: emr-containers
     apiGroup: rbac.authorization.k8s.io
   roleRef:
     kind: Role
     name: emr-containers
     apiGroup: rbac.authorization.k8s.io
   EOF

3. Update Kubernetes aws-auth configuration map

   You can use one of the following options to map the Amazon EMR on EKS service-linked role with the emr-containers user that was bound with the Kubernetes role in the previous step.

   Option 1: Using eksctl

   Run the following eksctl command to map the Amazon EMR on EKS service-linked role with the emr-containers user.

   eksctl create iamidentitymapping \
       --cluster my-cluster-name \
       --arn "arn:aws:iam::my-account-id:role/AWSServiceRoleForAmazonEMRContainers" \
       --username emr-containers

   Option 2: Without using eksctl

   1. Run the following command to open the aws-auth configuration map in text editor.

      kubectl edit -n kube-system configmap/aws-auth

      Note

      If you receive an error stating Error from server (NotFound): configmaps "aws-auth" not found, see the steps in Add user roles in the Amazon EKS User Guide to apply the stock ConfigMap.

   2. Add Amazon EMR on EKS service-linked role details to the mapRoles section of the ConfigMap, under data. Add this section if it does not already exist in the file. The updated mapRoles section under data looks like the following example.

      apiVersion: v1
      data:
        mapRoles: |
          - rolearn: arn:aws:iam::<your-account-id>:role/AWSServiceRoleForAmazonEMRContainers
            username: emr-containers
          - ... <other previously existing role entries, if there's any>.

   3. Save the file and exit your text editor.