Using Parameter Store parameters in Amazon Elastic Kubernetes Service
To show secrets from Secrets Manager and parameters from Parameter Store as files mounted in Amazon EKS
pods, you can use the AWS Secrets and Configuration Provider (ASCP) for the Kubernetes Secrets Store CSI
Driver
With the ASCP, you can retrieve parameters that are stored and managed in Parameter Store. Then you can use the parameters in your workloads running on Amazon EKS. If your parameter contains multiple key value pairs in JSON format, you can optionally choose to mount them in Amazon EKS. The ASCP can use JMESPath syntax to query the key value pairs in your parameter.
You can use AWS Identity and Access Management (IAM) roles and policies to limit access to your parameters to specific Amazon EKS pods in a cluster. The ASCP retrieves the pod identity and exchanges the identity for an IAM role. ASCP assumes the IAM role of the pod. Then it can retrieve parameters from Parameter Store that are authorized for that role.
To learn how to integrate Secrets Manager with Amazon EKS, see Using Secrets Manager secrets in Amazon Elastic Kubernetes Service.
Installing the ASCP
The ASCP is available on GitHub in the secrets-store-csi-driver-provider-aws
To install the Kubernetes Secrets Store CSI Driver and the ASCP
-
To install the Kubernetes Secrets Store CSI Driver, run the following commands. For full installation instructions, see Installation
in the Kubernetes Secrets Store CSI Driver Book. For information about installing Helm, see Using Helm with Amazon EKS. helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts helm install -n kube-system csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver
-
To install the ASCP, use the YAML file in the GitHub repository deployment directory. For information about installing
kubectl
, see Installingkubectl
.kubectl apply -f https://raw.githubusercontent.com/aws/secrets-store-csi-driver-provider-aws/main/deployment/aws-provider-installer.yaml
Step 1: Set up access control
To grant your Amazon EKS pod access to parameters in Parameter Store, you first create a policy that limits access to the parameters that the pod needs to access. Then you create an IAM role for service account and attach the policy to it. For more information about restricting access to Systems Manager parameters using IAM policies, see Restricting access to Parameter Store parameters using IAM policies.
Note
When using Parameter Store parameters, the permission
ssm:GetParameters
is needed in the policy.
The ASCP retrieves the pod identity and exchanges it for the IAM role. ASCP assumes the IAM role of the pod, which gives it access to the parameters you authorized. Other containers can't access the parameters unless you also associate them with the IAM role.
Step 2: Mount parameters in Amazon EKS
To show parameters in Amazon EKS as though they are files on the filesystem, you
create a SecretProviderClass
YAML file that contains information
about your parameters and how to mount them in the Amazon EKS pod.
The SecretProviderClass
must be in the same namespace as the
Amazon EKS pod it references.
SecretProviderClass
The SecretProviderClass
YAML has the following format.
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1 kind: SecretProviderClass metadata: name: <NAME> spec: provider: aws parameters:
- parameters
-
Contains the details of the mount request.
- objects
-
A string containing a YAML declaration of the parameters to be mounted. We recommend using a YAML multi-line string or pipe (|) character.
- objectName
-
The friendly name of the parameter. This becomes the file name of the parameter in the Amazon EKS pod unless you specify
objectAlias
. For Parameter Store this must be theName
of the parameter, and cannot be a full Amazon Resource Name (ARN). - jmesPath
-
(Optional) A map of the keys in the JSON encoded parameter to the files to be mounted in Amazon EKS. The following example shows what a JSON encoded parameter looks like.
{ "username" : "myusername", "password" : "mypassword" }
The keys are
username
andpassword
. The value associated withusername
ismyusername
, and the value associated withpassword
ismypassword
.- path
-
The key in the parameter.
- objectAlias
-
The file name to be mounted in the Amazon EKS pod.
- objectType
-
For Parameter Store, this field is required. Use
ssmparameter
. - objectAlias
-
(Optional) The file name of the parameter in the Amazon EKS pod. If you don't specify this field, the
objectName
appears as the file name. - objectVersion
-
(Optional) The version number of the parameter. We recommend that you don't use this field because you must update it every time you update the parameter. By default, the most recent version is used. For Parameter Store parameters, you can use
objectVersion
orobjectVersionLabel
but not both. - objectVersionLabel
-
(Optional) The parameter label for the version. The default is the most recent version. For Parameter Store parameters, you can use
objectVersion
orobjectVersionLabel
but not both.
- region
-
(Optional) The AWS Region of the parameter. If you don't use this field, the ASCP looks up the Region from the annotation on the node. This lookup adds overhead to mount requests, so we recommend that you provide the Region for clusters that use large numbers of pods.
- pathTranslation
-
(Optional) A single substitution character to use if the file name (either
objectName
orobjectAlias
) contains the path separator character, such as slash (/) on Linux. If a parameter name contains the path separator, the ASCP can't create a mounted file with that name. Instead, you can replace the path separator character with a different character by entering it in this field. If you don't use this field, the default is underscore (_), so for example,My/Path/Parameter
mounts asMy_Path_Parameter
.To prevent character substitution, enter the string
False
.
Example
The following example configuration shows a
SecretProviderClass
with a Parameter Store parameter
resource.
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1 kind: SecretProviderClass metadata: name: aws-secrets spec: provider: aws parameters: objects: | - objectName: "MyParameter" objectType: "ssmparameter"
Step 3: Update the deployment YAML
Update your deployment YAML to use the secrets-store.csi.k8s.io
driver and reference the SecretProviderClass
resource created in
the previous step. This ensures your cluster is using the Secrets Store CSI
driver.
Below is a sample deployment YAML using a SecretProviderClass
named aws-secrets
.
volumes: - name: secrets-store-inline csi: driver: secrets-store.csi.k8s.io readOnly: true volumeAttributes: secretProviderClass: "aws-secrets"
Tutorial: Creating and mounting a parameter in an Amazon EKS pod
In this tutorial, you create an example parameter in Parameter Store, and then you mount the parameter in an Amazon EKS pod and deploy it.
Before you begin, install the ASCP. For more information, see Installing the ASCP.
To create and mount a secret
-
Set the AWS Region and the name of your cluster as shell variables so you can use them in
bash
commands. Forregion
, enter the AWS Region where your Amazon EKS cluster runs. Forclustername
, enter the name of your cluster.REGION=
region
CLUSTERNAME=clustername
-
Create a test parameter.
aws ssm put-parameter --name "MyParameter" --value "EKS parameter" --type String --region "$REGION"
-
Create a resource policy for the pod that limits its access to the parameter you created in the previous step. For
, use the ARN of the parameter. Save the policy ARN in a shell variable. To retrieve the parameter ARN, useparameter-arn
get-parameter
.POLICY_ARN=$(aws --region "$REGION" --query Policy.Arn --output text iam create-policy --policy-name nginx-parameter-deployment-policy --policy-document '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["ssm:GetParameter", "ssm:GetParameters"], "Resource": ["
parameter-arn
"] } ] }') -
Create an IAM OpenID Connect (OIDC) provider for the cluster if you don't already have one. For more information, see Create an IAM OIDC provider for your cluster.
eksctl utils associate-iam-oidc-provider --region="$REGION" --cluster="$CLUSTERNAME" --approve # Only run this once
-
Create the service account the pod uses, and associate the resource policy you created in step 3 with that service account. For this tutorial, for the service account name, you use nginx-deployment-sa. For more information, see Create an IAM role for a service account.
eksctl create iamserviceaccount --name nginx-deployment-sa --region="$REGION" --cluster "$CLUSTERNAME" --attach-policy-arn "$POLICY_ARN" --approve --override-existing-serviceaccounts
-
Create the
SecretProviderClass
to specify which parameter to mount in the pod. The following command uses the file location of aSecretProviderClass
file namedExampleSecretProviderClass.yaml
. For information about creating your ownSecretProviderClass
, see SecretProviderClass.kubectl apply -f ./ExampleSecretProviderClass.yaml
-
Deploy your pod. The following command uses a deployment file named
ExampleDeployment.yaml
. For information about creating your ownSecretProviderClass
, see Step 3: Update the deployment YAML.kubectl apply -f ./ExampleDeployment.yaml
-
To verify the parameter has been mounted properly, use the following command and confirm that your parameter value appears.
kubectl exec -it $(kubectl get pods | awk '/nginx-deployment/{print $1}' | head -1) cat /mnt/secrets-store/MyParameter; echo
The parameter value appears.
"EKS parameter"
Troubleshooting
You can view most errors by describing the pod deployment.
To see error messages for your container
-
Get a list of pod names with the following command. If you aren't using the default namespace, use
-n <NAMESPACE>
.kubectl get pods
-
To describe the pod, in the following command, for
pod-id
use the pod ID from the pods you found in the previous step. If you aren't using the default namespace, use-n <NAMESPACE>
.kubectl describe pod/
pod-id
To see errors for the ASCP
-
To find more information in the provider logs, in the following command, for
pod-id
use the ID of the csi-secrets-store-provider-aws pod.kubectl -n kube-system get pods kubectl -n kube-system logs pod/
pod-id