Set up automatic rotation for AWS Secrets Manager secrets using the AWS CLI - AWS Secrets Manager

Set up automatic rotation for AWS Secrets Manager secrets using the AWS CLI

Rotation is the process of periodically updating a secret. When you rotate a secret, you update the credentials in both the secret and the database or service that the secret is for.

Secrets Manager uses Lambda functions to rotate secrets. For an overview, see How rotation works.

You can also use the console to set up rotation. For more information, see Automatic rotation (console).

To set up rotation using the AWS CLI, if you are rotating an Amazon RDS, Amazon Redshift, or Amazon DocumentDB secret, you first need to choose a Rotation strategy. If you choose the alternating users strategy, you must store a separate secret with credentials for a database superuser. Next, you write the rotation function code. Secrets Manager provides templates you can base your function on. Then you create a Lambda function with your code and set permissions for both the Lambda function and the Lambda execution role. The next step is to make sure that the Lambda rotation function can access both Secrets Manager and your database or service through the network. Finally, you configure the secret for rotation.

To turn on automatic rotation, you must have permission to create the IAM execution role and attach a permission policy to it. You need both iam:CreateRole and iam:AttachRolePolicy permissions.

Warning

Granting an identity both iam:CreateRole and iam:AttachRolePolicy permissions allows the identity to grant themselves any permissions.

(Optional) Step 1: Create a superuser secret

For Amazon RDS, Amazon Redshift, and Amazon DocumentDB, Secrets Manager offers two rotation strategies:

Single user rotation strategy

This strategy updates credentials for one user in one secret. For Amazon RDS Db2 instances, because users can't change their own passwords, you must provide admin credentials in a separate secret. This is the simplest rotation strategy, and it is appropriate for most use cases. In particular, we recommend you use this strategy for credentials for one-time (ad hoc) or interactive users.

When the secret rotates, open database connections are not dropped. While rotation is happening, there is a short period of time between when the password in the database changes and when the secret is updated. During this time, there is a low risk of the database denying calls that use the rotated credentials. You can mitigate this risk with an appropriate retry strategy. After rotation, new connections use the new credentials.

Alternating users rotation strategy

This strategy updates credentials for two users in one secret. You create the first user, and during the first rotation, the rotation function clones it to create the second user. Every time the secret rotates, the rotation function alternates which user's password it updates. Because most users don't have permission to clone themselves, you must provide the credentials for a superuser in another secret. We recommend using the single-user rotation strategy when cloned users in your database don't have the same permissions as the original user, and for credentials for one-time (ad hoc) or interactive users.

This strategy is appropriate for databases with permission models where one role owns the database tables and a second role has permission to access the database tables. It is also appropriate for applications that require high availability. If an application retrieves the secret during rotation, the application still gets a valid set of credentials. After rotation, both user and user_clone credentials are valid. There is even less chance of applications getting a deny during this type of rotation than single user rotation. If the database is hosted on a server farm where the password change takes time to propagate to all servers, there is a risk of the database denying calls that use the new credentials. You can mitigate this risk with an appropriate retry strategy.

Secrets Manager creates the cloned user with the same permissions as the original user. If you change the original user's permissions after the clone is created, you must also change the cloned user's permissions.

Important

If you choose the alternating users strategy, you must Create a database secret and store database superuser credentials in it. You need a secret with superuser credentials because rotation clones the first user, and most users do not have that permission.

Step 2: Write the rotation function code

To rotate a secret, you need a rotation function. A rotation function is a Lambda function that Secrets Manager calls to rotate your secret.

For a function that can rotate an Amazon RDS, Amazon Aurora, Amazon Redshift, Amazon DocumentDB, or Amazon ElastiCache secret, you can copy the code from the appropriate template supplied by Secrets Manager.

For all other types of secrets, use the generic rotation template as a starting point to write your own rotation function.

Save your rotation function in a ZIP file my-function.zip along with any required dependencies.

As you write your function, be cautious about including debugging or logging statements. These statements can cause information in your function to be written to Amazon CloudWatch, so you need to make sure the log doesn't include any sensitive information collected during development.

For security, Secrets Manager only permits a Lambda rotation function to rotate the secret directly. The rotation function can't call a second Lambda function to rotate the secret.

For examples of log statements, see the AWS Secrets Manager rotation function templates source code.

If you use external binaries and libraries, for example to connect to a resource, you need to manage patching them and keeping them up-to-date.

For debugging suggestions, see Testing and debugging serverless applications.

To open your Lambda rotation function for editing
  1. In the Secrets Manager console, choose your secret.

  2. In the Rotation configuration section, under Lambda rotation function, choose your rotation function.

    The Lambda console opens.

    • To change the code in the function, scroll down to the Code source section.

    • For MySQL version 5.7 and higher, for alternating users rotation, to change the maximum username length, under Environment variables, change USERNAME_CHARACTER_LIMIT.

If your function doesn't already have it, copy the code from the SecretsManagerRotationTemplate.

There are four steps to rotating a secret, which correspond to the following four methods of a Lambda rotation function.

create_secret

In create_secret, you first check if a secret exists by calling get_secret_value with the passed-in ClientRequestToken. If there's no secret, you create a new secret with create_secret and the token as the VersionId. Then you can generate a new secret value with get_random_password. You must ensure the new secret value only includes characters that are valid for the database or service. Exclude characters by using the ExcludeCharacters parameter. Call put_secret_value to store it with the staging label AWSPENDING. Storing the new secret value in AWSPENDING helps ensure idempotency. If rotation fails for any reason, you can refer to that secret value in subsequent calls. See How do I make my Lambda function idempotent.

As you test your function, use the AWS CLI to see version stages: call describe-secret and look at VersionIdsToStages.

set_secret

In set_secret, you change the credential in the database or service to match the new secret value in the AWSPENDING version of the secret.

If you pass statements to a service that interprets statements, like a database, use query parameterization For more information, see Query Parameterization Cheat Sheet on the OWASP web site.

The rotation function is a privileged deputy that has the authorization to access and modify customer credentials in both the Secrets Manager secret and the target resource. To prevent a potential confused deputy attack, you need to make sure that an attacker cannot use the function to access other resources. Before you update the credential:

  • Check that the credential in the AWSCURRENT version of the secret is valid. If the AWSCURRENT credential isn't valid, abandon the rotation attempt.

  • Check that the AWSCURRENT and AWSPENDING secret values are for the same resource. For a username and password, check that the AWSCURRENT and AWSPENDING usernames are the same.

  • Check that the destination service resource is the same. For a database, check that the AWSCURRENT and AWSPENDING host names are the same.

test_secret

In test_secret, you test the AWSPENDING version of the secret by using it to access the database or service.

finish_secret

In finish_secret, you use update_secret_version_stage to move the staging label AWSCURRENT from the previous secret version to the new secret version. Secrets Manager automatically adds the AWSPREVIOUS staging label to the previous version, so that you retain the last known good version of the secret.

Step 3: Create the Lambda function and execution role

A Lambda execution role is a role that Lambda assumes when the function is invoked.

To create a Lambda rotation function and execution role
  1. Create a trust policy for the Lambda execution role and save it as a JSON file. For examples, see Permissions for rotation. The policy must:

    • Allow the role to call Secrets Manager operations on the secret.

    • Allow the role to use the KMS key if the secret is encrypted with a key other than aws/secretsmanager.

    • Allow the role to call the service that the secret is for.

  2. Create the Lambda execution role and apply the trust policy by calling iam create-role.

    aws iam create-role \ --role-name rotation-lambda-role \ --assume-role-policy-document file://trust-policy.json
  3. (Optional) For a secret that contains Amazon RDS or Aurora credentials, if you are using the alternating users strategy and the superuser secret is managed by Amazon RDS, then you must allow the rotation function to call read-only APIs on Amazon RDS so that it can get the connection information for the database. To do this, attach the AWS managed policy AmazonRDSReadOnlyAccess to the Lambda function execution role by calling iam attach-role-policy.

    aws iam attach-role-policy \ --policy-arn arn:aws:iam::aws:policy/AmazonRDSReadOnlyAccess \ --role-name rotation-lambda-role
  4. Create the Lambda function from the ZIP file by calling lambda create-function.

    aws lambda create-function \ --function-name my-rotation-function \ --runtime python3.9 \ --zip-file fileb://my-function.zip \ --handler my-handler \ --role arn:aws:iam::123456789012:role/service-role/rotation-lambda-role
  5. Set a resource policy on the Lambda function to allow Secrets Manager to invoke it by calling lambda add-permission. The example command includes source-account to help prevent Lambda from being used as a confused deputy.

    aws lambda add-permission \ --function-name my-rotation-function \ --action lambda:InvokeFunction \ --statement-id SecretsManager \ --principal secretsmanager.amazonaws.com \ --source-account 123456789012

Step 4: Set up network access

To be able to rotate a secret, the Lambda rotation function must be able to access both the secret and the database or service.

To access a secret

Your Lambda rotation function must be able to access a Secrets Manager endpoint. If your Lambda function can access the internet, then you can use a public endpoint. To find an endpoint, see AWS Secrets Manager endpoints.

If your Lambda function runs in a VPC that doesn't have internet access, we recommend you configure Secrets Manager service private endpoints within your VPC. Your VPC can then intercept requests addressed to the public regional endpoint and redirect them to the private endpoint. For more information, see VPC endpoint.

Alternatively, you can enable your Lambda function to access a Secrets Manager public endpoint by adding a NAT gateway or an internet gateway to your VPC, which allows traffic from your VPC to reach the public endpoint. This exposes your VPC to more risk because an IP address for the gateway can be attacked from the public Internet.

To access the database or service

If your database or service is running on an Amazon EC2 instance in a VPC, we recommend that you configure your Lambda function to run in the same VPC. Then the rotation function can communicate directly with your service. For more information, see Configuring VPC access.

To allow the Lambda function to access the database or service, you must make sure that the security groups attached to your Lambda rotation function allow outbound connections to the database or service. You must also make sure that the security groups attached to your database or service allow inbound connections from the Lambda rotation function.

For alternating users rotation where the superuser secret is managed by another AWS service, the Lambda rotation function must be able to call the service endpoint to get the database connection information. We recommend that you configure a VPC endpoint for the database service. For more information, see:

Step 5: Configure the secret for rotation

To turn on automatic rotation for your secret, call rotate-secret. You can set a rotation schedule with a cron() or rate() schedule expression, and you can set a rotation window duration. You can rotate a secret as often as every four hours. For more information, see Schedule expressions.

aws secretsmanager rotate-secret \ --secret-id MySecret \ --rotation-lambda-arn arn:aws:lambda:Region:123456789012:function:my-rotation-function \ --rotation-rules "{\"ScheduleExpression\": \"cron(0 16 1,15 * ? *)\", \"Duration\": \"2h\"}"

Next steps

See Troubleshoot AWS Secrets Manager rotation.