Cache secrets using AWS Lambda extensions
Created by HARI OHM PRASATH RAJAGOPAL (AWS)
Environment: PoC or pilot | Technologies: Modernization; Cloud-native; Serverless; Software development & testing | Workload: Open-source |
AWS services: AWS Lambda; AWS Secrets Manager |
Summary
This pattern demonstrates how to host an HTTP server that caches secrets required for an AWS Lambda function. The Lambda extension does the following:
Hosts a local HTTP server, which is invoked by a Lambda function to retrieve the secret instead of retrieving it directly from AWS Secrets Manager.
Uses a config.yaml file inside the Lambda function to get the list of secrets that must be cached in memory.
Refreshes the secrets cache based on the value (in minutes) set in the environment variable
CACHE_TIMEOUT
. If no value is specified, it defaults to 10.
This extension is helpful in the following use cases:
If the Lambda function needs access to Secrets Manager for reading a secret, the secret can get cached as part of the extension startup instead of being fetched as part of Lambda's first request. This helps to avoid a cold start.
The Lambda function specifies which secrets are cached by using the configuration file (
config.yaml
), which is part of the deployment package.The Lambda function allows the expiration of in-memory cache, which can be configured as a Lambda environment variable.
The business can maintain the cache layer in-memory instead of using a /tmp storage for reading and writing values in Secrets Manager.
Prerequisites and limitations
Prerequisites
An active AWS account
AWS Command Line Interface (AWS CLI) version 1.7 or later, installed and configured on macOS, Linux, or Windows
Zip utility installed in the local system
The code that is in the Attachments section
Architecture
Architecture
The following diagram provides a high-level view of all the components.

Secrets are cached based on the
config.yaml
file defined inside the deployment package of the Lambda function, and the cache is refreshed based on environment variables.Based on the keys specified inside
config.yaml
, the secrets are read and cached in-memory from Secrets Manager.The Lambda function reads the secret by making an HTTP GET call to the extension, and the extension returns the value from the in-memory cache.
Initializing the extension and reading secrets from the cache
The following sequence diagram shows the initialization of the Lambda extension and how the Lambda function reads cached secrets by using the HTTP server that is hosted inside the extension.

The sequence of events inside the Lambda function and the extension includes the following.
Part of the initialization extension reads the
config.yaml
file from the Lambda function under the default path (/var/task
) and gets the list of secrets that need to be cached.When the extension has a list of keys that needs to be fetched, it invokes Secrets Manager to get all the secrets (specified in
config.yaml
) and stores them in memory.The extension starts a local HTTP server.
To get the secret value, the Lambda function makes an HTTP GET:/cache/<secretname> call to the extension.
The extension returns the cached value to the Lambda function based on the
secretname
path parameter.
Technology
AWS Lambda
AWS Secrets Manager
Tools
Tools
AWS CLI – AWS Command Line Interface (AWS CLI) is a unified tool to manage your AWS services.
AWS Lambda – AWS Lambda supports running code without provisioning or managing servers.
AWS Secrets Manager – AWS Secrets Manager enables the replacement of hardcoded credentials, including passwords, with an API call to Secrets Manager to retrieve the secret. Because Secrets Manager can automatically rotate the secret according to a schedule, you can replace long-term secrets with short-term ones, reducing the risk of compromise.
Git client
– Git provides GUI tools, or you can use the command line or a desktop tool to check out the required artifacts from GitHub. Node.js
– Node.js is an event-driven JavaScript runtime environment designed for building scalable network applications. The runtime for Node.js should be installed on the local system.
Epics
Task | Description | Skills required |
---|---|---|
Create a secret in Secrets Manager. | Run the command that is in the Additional information section. The command assumes that you placed your secret, such as this example JavaScript Object Notation (JSON) text structure | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Create a new Lambda function. | On the AWS Management Console, choose Lambda and then choose Create function. Create a new Node.js runtime Lambda function with the name | Developer, AWS systems administrator |
Assign a policy to the Lambda IAM role. | Choose the Permission tab. Under Execution role, choose the Role name link and attach the | Developer, AWS systems administrator |
Update the handler code. | Choose the Configuration tab and update the function with the code in the Additional information section. The code invokes the local server hosted inside the Lambda extension to read the value of the secret instead of directly going to Secrets Manager. | Developer, AWS systems administrator |
Create a secrets cache configuration file. | Create a new file named config.yaml under the root directory with the contents specified in the Additional information section. The value | Developer, AWS systems administrator |
Create the cache, rebuild timeout, and update the Lambda runtime. | Create a new environment variable CACHE_TIMEOUT, and set the value in minutes. The cache will be refreshed based on this value. Note: If the environment value is not found, the cache gets refreshed every 10 minutes. | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Deploy the Lambda extension. | To deploy the extension and associate the layer to the Lambda function, run the command that is in the Additional information section. | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Invoke the Lambda function. | To invoke the Lambda function, use the command that is in the Additional information section. The function should return | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Verify Lambda extension invocation. | On the Amazon CloudWatch console, navigate to Logs, Log Groups. Select the log group | Developer, AWS systems administrator |
Related resources
Additional information
Create secret
The following command assumes that you placed your secret, such as this example JSON text structure {"username":"janedoe","password":" aBC1D2*!3EE"}
, in a file named mycreds.json
.
aws secretsmanager create-secret --name secret_now --secret-string file://mycreds.json
Update the handler code
A sample with the following code is in example-function/index.js
in the zipped files that are provided in the Attachments section.
exports.handler = function(event, context, callback) { const https = require('http') const options = { hostname: 'localhost', port: 8080, path: '/cache/secret_now', method: 'GET' } const req = https.request(options, res => { res.on('data', d => { console.log("Response from cache: "+d); return d; }) }) req.on('error', error => { console.error(error) }) req.end() };
Create a secrets cache configuration file
A sample of the following code is in the example-function/config.yaml
file in the Attachments section. In the example, secret_now
is the name of the secret that was created in the previous step and will be cached by the extension. To cache additional secrets, add them to this file.
SecretManagers: – secrets: – secret_now
Deploy the Lambda extension
> chmod +x deploy.sh > ./deploy.sh
Invoke the Lambda function
aws lambda invoke \ --function-name "Secrets-Extension-Lambda-Test" \ --payload '{"payload": "hello"}' /tmp/invoke-result \ --cli-binary-format raw-in-base64-out \ --log-type Tail
The function should return "StatusCode": 200
.
Attachments
To access additional content that is associated with this document, unzip the following file: attachment.zip