Cache secrets using AWS Lambda extensions
Created by HARI OHM PRASATH RAJAGOPAL (AWS), Karan Shah (AWS), and Giridhar Shyam Sankararaman (AWS)
Environment: PoC or pilot | Technologies: Modernization; Cloud-native; Serverless; Software development & testing | Workload: Open-source |
AWS services: AWS Lambda; AWS Secrets Manager |
Summary
This pattern demonstrates how to host an HTTP server that caches secrets required for an AWS Lambda function. The Lambda extension does the following:
Hosts a local HTTP server, which is invoked by a Lambda function to retrieve the secret instead of retrieving it directly from AWS Secrets Manager.
Uses a config.yaml file inside the Lambda function to get the list of secrets that must be cached in memory.
Refreshes the secrets cache based on the value (in minutes) set in the environment variable
CACHE_TIMEOUT
. If no value is specified, it defaults to 10.
This extension is helpful in the following use cases:
If the Lambda function needs access to Secrets Manager for reading a secret, the secret can get cached as part of the extension startup instead of being fetched as part of Lambda's first request. This helps to avoid a cold start.
The Lambda function specifies which secrets are cached by using the configuration file (
config.yaml
), which is part of the deployment package.The Lambda function allows the expiration of in-memory cache, which can be configured as a Lambda environment variable.
The business can maintain the cache layer in-memory instead of using a /tmp storage for reading and writing values in Secrets Manager.
Prerequisites and limitations
Prerequisites
An active AWS account.
AWS Command Line Interface (AWS CLI) version 1.7 or later, installed and configured on macOS, Linux, or Windows.
Zip utility installed in the local system.
The code that is in the Attachments section, which contains the code for creating a Lambda extension as a layer. Details are in the Additional information section.
Architecture
Architecture
The following diagram provides a high-level view of all the components.

Secrets are cached based on the
config.yaml
file defined inside the deployment package of the Lambda function, and the cache is refreshed based on environment variables.Based on the keys specified inside
config.yaml
, the secrets are read and cached in-memory from Secrets Manager.The Lambda function reads the secret by making an HTTP GET call to the extension, and the extension returns the value from the in-memory cache.
Initializing the extension and reading secrets from the cache
The following sequence diagram shows the initialization of the Lambda extension and how the Lambda function reads cached secrets by using the HTTP server that is hosted inside the extension.

The sequence of events inside the Lambda function and the extension includes the following.
The extension is initialized first before the Lambda function, based on the lifecycle. As part of the initialization, the extension reads the
config.yaml
file from the Lambda function under the default path (/var/task
) and gets the list of secrets that need to be cached.When the extension has a list of keys that needs to be fetched, it invokes Secrets Manager to get all the secrets (specified in
config.yaml
) and stores them in memory.The extension starts a local HTTP server.
To get the secret value, the Lambda function makes an HTTP GET:/cache/<secretname> call to the extension.
Note: The web server is made available locally to the Lambda runtime environment. The communication between the Lambda function and the web server is in HTTP. It is assumed that this is fine, as this example is to demonstrate the capability to communicate to the cache locally.
The extension returns the cached value to the Lambda function based on the
secretname
path parameter.
Technology
AWS Lambda
AWS Secrets Manager
HTTP web server
Tools
Tools
AWS CLI – AWS Command Line Interface (AWS CLI) is a unified tool to manage your AWS services.
AWS Lambda – AWS Lambda supports running code without provisioning or managing servers.
AWS Secrets Manager – AWS Secrets Manager enables the replacement of hardcoded credentials, including passwords, with an API call to Secrets Manager to retrieve the secret. Because Secrets Manager can automatically rotate the secret according to a schedule, you can replace long-term secrets with short-term ones, reducing the risk of compromise.
Git client
– Git provides GUI tools, or you can use the command line or a desktop tool to check out the required artifacts from GitHub. Node.js
– Node.js is an event-driven JavaScript runtime environment designed for building scalable network applications. The runtime for Node.js should be installed on the local system.
Epics
Task | Description | Skills required |
---|---|---|
Create a secret in Secrets Manager. | Run the command that is in the Additional information section. The command assumes that you placed your secret, such as this example JavaScript Object Notation (JSON) text structure | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Create a new Lambda function. | On the AWS Management Console, choose Lambda and then choose Create function. Create a new Node.js runtime Lambda function with the name | Developer, AWS systems administrator |
Assign a policy to the Lambda IAM role. | Choose the Configuration tab and navigate to the Permission tab. Under Execution role, choose the Role name link to open the attached role in AWS Identity and Access Management (IAM). Attach the Note: As a best practice, create an IAM policy with read-only permission on the secret that needs to be cached, and assign it to the IAM role. | Developer, AWS systems administrator |
Update the handler code. | Choose the Code tab and update the function with the code included in the Additional information section. The code invokes the local server hosted inside the Lambda extension to read the value of the secret instead of directly going to Secrets Manager. | Developer, AWS systems administrator |
Create a secrets cache configuration file. | In the Lambda function Code tab, create a new file named The value | Developer, AWS systems administrator |
Create the cache, rebuild timeout, and update the Lambda runtime. | Create a new environment variable Note: If the environment value is not found, the cache gets refreshed every 10 minutes. | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Deploy the Lambda extension. | To deploy the extension and associate the layer to the Lambda function, run the command that is in the Additional information section. | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Invoke the Lambda function. | To invoke the Lambda function, use the command that is in the Additional information section. The function should return | Developer, AWS systems administrator |
Task | Description | Skills required |
---|---|---|
Verify Lambda extension invocation. | On the Amazon CloudWatch console, navigate to Logs, Log Groups. Select the log group | Developer, AWS systems administrator |
Related resources
Additional information
Create secret
The following command assumes that you placed your secret, such as this example JSON text structure {"username":"janedoe","password":" aBC1D2*!3EE"}
, in a file named mycreds.json
.
aws secretsmanager create-secret --name secret_now --secret-string file://mycreds.json
Update the handler code
An example with the following code is in example-function/index.js
in the zipped files that are provided in the Attachments section.
exports.handler = function(event, context, callback) { const https = require('http') const options = { hostname: 'localhost', port: 8080, path: '/cache/secret_now', method: 'GET' } const req = https.request(options, res => { res.on('data', d => { console.log("Response from cache: "+d); return d; }) }) req.on('error', error => { console.error(error) }) req.end() };
Create a secrets cache configuration file
Create a config.yaml
file in the Lambda function Code tab under the root directory. In the example, secret_now
is the name of the secret that was created in the previous step and will be cached by the extension. To cache additional secrets, add them to this file.
SecretManagers: - secrets: - secret_now
Deploy the Lambda extension
> chmod +x deploy.sh > ./deploy.sh
Invoke the Lambda function
aws lambda invoke \ --function-name "Secrets-Extension-Lambda-Test" \ --payload '{"payload": "hello"}' /tmp/invoke-result \ --cli-binary-format raw-in-base64-out \ --log-type Tail
The function should return "StatusCode": 200
.
Attachments
To access additional content that is associated with this document, unzip the following file: attachment.zip