Enable cluster access for Amazon EMR on EKS - Amazon EMR

Enable cluster access for Amazon EMR on EKS

You must allow Amazon EMR on EKS access to a specific namespace in your cluster by taking the following actions: creating a Kubernetes role, binding the role to a Kubernetes user, and mapping the Kubernetes user with the service linked role AWSServiceRoleForAmazonEMRContainers. These actions are automated in eksctl when the IAM identity mapping command is used with emr-containers as the service name. You can perform these operations easily by using the following command.

eksctl create iamidentitymapping \ --cluster my_eks_cluster \ --namespace kubernetes_namespace \ --service-name "emr-containers"

Replace my_eks_cluster with the name of your Amazon EKS cluster and replace kubernetes_namespace with the Kubernetes namespace created to run Amazon EMR workloads.

Important

You must download the latest eksctl using the previous step Install eksctl to use this functionality.

Manual steps to enable cluster access for Amazon EMR on EKS

You can also use the following manual steps to enable cluster access for Amazon EMR on EKS.

  1. Create a Kubernetes role in a specific namespace

    Run the following command to create a Kubernetes role in a specific namespace. This role grants the necessary RBAC permissions to Amazon EMR on EKS.

    namespace=my-namespace cat - <<EOF | kubectl apply -f - --namespace "${namespace}" apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: emr-containers namespace: ${namespace} rules: - apiGroups: [""] resources: ["namespaces"] verbs: ["get"] - apiGroups: [""] resources: ["serviceaccounts", "services", "configmaps", "events", "pods", "pods/log"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"] - apiGroups: [""] resources: ["secrets"] verbs: ["create", "patch", "delete", "watch"] - apiGroups: ["apps"] resources: ["statefulsets", "deployments"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["batch"] resources: ["jobs"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["extensions"] resources: ["ingresses"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "annotate", "patch", "label"] - apiGroups: ["rbac.authorization.k8s.io"] resources: ["roles", "rolebindings"] verbs: ["get", "list", "watch", "describe", "create", "edit", "delete", "deletecollection", "annotate", "patch", "label"] EOF
  2. Create a Kubernetes role binding scoped to the namespace

    Run the following command to create a Kubernetes role binding in the given namespace. This role binding grants the permissions defined in the role created in the previous step to a user named emr-containers. This user identifies service-linked roles for Amazon EMR on EKS and thus allows Amazon EMR on EKS to perform actions as defined by the role you created.

    namespace=my-namespace cat - <<EOF | kubectl apply -f - --namespace "${namespace}" apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: emr-containers namespace: ${namespace} subjects: - kind: User name: emr-containers apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: emr-containers apiGroup: rbac.authorization.k8s.io EOF
  3. Update Kubernetes aws-auth conīŦguration map

    You can use one of the following options to map the Amazon EMR on EKS service-linked role with the emr-containers user that was bound with the Kubernetes role in the previous step.

    Option 1: Using eksctl

    Run the following eksctl command to map the Amazon EMR on EKS service-linked role with the emr-containers user.

    eksctl create iamidentitymapping \ --cluster my-cluster-name \ --arn "arn:aws:iam::my-account-id:role/AWSServiceRoleForAmazonEMRContainers" \ --username emr-containers

    Option 2: Without using eksctl

    1. Run the following command to open the aws-auth configuration map in text editor.

      kubectl edit -n kube-system configmap/aws-auth
      Note

      If you receive an error stating Error from server (NotFound): configmaps "aws-auth" not found, see the steps in Add user roles in the Amazon EKS User Guide to apply the stock ConfigMap.

    2. Add Amazon EMR on EKS service-linked role details to the mapRoles section of the ConfigMap, under data. Add this section if it does not already exist in the file. The updated mapRoles section under data looks like the following example.

      apiVersion: v1 data: mapRoles: | - rolearn: arn:aws:iam::<your-account-id>:role/AWSServiceRoleForAmazonEMRContainers username: emr-containers - ... <other previously existing role entries, if there's any>.
    3. Save the file and exit your text editor.