View a markdown version of this page

Getting started with GPU-accelerated Event Functions - AWS Lambda

Getting started with GPU-accelerated Event Functions

To run GPU-accelerated Event Functions on AWS Lambda Managed Instances, you create a capacity provider with GPU instance types, build a container image from a GPU base image, create a function with accelerator memory on that capacity provider, and publish a version. This page walks through those four steps.

Step 1: Create a GPU-instance capacity provider

Note

This tutorial assumes you already have a capacity provider operator role and VPC resources (subnets and security groups). To create them, see Getting started with Lambda Managed Instances.

Create a capacity provider for your GPU-accelerated functions by listing at least one supported GPU instance type in AllowedInstanceTypes. A capacity provider cannot mix GPU and non-GPU instance types, so use a dedicated capacity provider for GPU-accelerated functions. To maximize availability, specify multiple GPU instance types across generations. Lambda attempts to use newer generations and falls back to older generations if capacity for newer generations is unavailable.

aws lambda create-capacity-provider \ --capacity-provider-name my-gpu-capacity-provider \ --vpc-config SubnetIds=[$SUBNET_ID],SecurityGroupIds=[$SECURITY_GROUP_ID] \ --permissions-config CapacityProviderOperatorRoleArn=arn:aws:iam::${ACCOUNT_ID}:role/MyOperatorRole \ --instance-requirements '{ "Architectures": ["x86_64"], "AllowedInstanceTypes": ["g6.4xlarge", "g5.4xlarge", "g4dn.4xlarge"] }'

Step 2: Build and push your function's container image

GPU-accelerated functions require a container image deployment package. Lambda provides GPU-enabled container base images in the Amazon ECR Public Gallery, with preinstalled GPU libraries including CUDA, PyTorch, vLLM, and SGLang. You can start from a Lambda GPU base image and add your handler code and dependencies, or build your own image from scratch using the Lambda runtime interface client. For more information, see GPU base images.

The following Dockerfile uses the PyTorch base image, downloads ResNet-18 model weights at build time, and copies in the handler.

To reduce startup time, we recommend storing large artifacts, such as model weights, outside your container image. For more information, see Optimizing startup time for GPU-accelerated functions.

Dockerfile:

FROM public.ecr.aws/lambda/deep-learning-containers/pytorch:cuda RUN python3 -c "from torchvision import models; models.resnet18(weights=models.ResNet18_Weights.DEFAULT)" COPY handler.py /var/task/ CMD ["handler.handler"]

The handler.py file loads the ResNet-18 model onto the GPU during initialization and runs one inference per invocation on a sample tensor:

import torch from torchvision import models model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT).cuda().eval() def handler(event, context): x = torch.randn(1, 3, 224, 224, device="cuda") # image-shaped tensor with torch.inference_mode(): logits = model(x) return {"device": str(logits.device), "top_class_index": int(logits.argmax(1))}

Then, follow the instructions below to create an Amazon Elastic Container Registry (Amazon ECR) repository, build the image, and push it to the repository.

aws ecr create-repository --repository-name my-gpu-image --region ${REGION} aws ecr get-login-password --region ${REGION} | \ docker login --username AWS --password-stdin ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com docker build -t my-gpu-image . docker tag my-gpu-image:latest ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/my-gpu-image:latest docker push ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/my-gpu-image:latest

Step 3: Create your GPU-accelerated Lambda event function

When creating a GPU-accelerated function, specify the accelerator memory (GPU VRAM) your workload needs using the AcceleratorConfig field:

aws lambda create-function \ --function-name my-gpu-function \ --package-type Image \ --code ImageUri=${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/my-gpu-image:latest \ --role arn:aws:iam::${ACCOUNT_ID}:role/MyLambdaExecutionRole \ --architectures x86_64 \ --memory-size 10240 \ --timeout 900 \ --ephemeral-storage Size=32768 \ --accelerator-config '{"AcceleratorMemorySize":16}' \ --capacity-provider-config LambdaManagedInstancesCapacityProviderConfig={CapacityProviderArn=arn:aws:lambda:${REGION}:${ACCOUNT_ID}:capacity-provider:my-gpu-capacity-provider}

When you set these parameters, keep the following in mind:

  • Timeout – --timeout sets the maximum duration of an invocation. On Lambda Managed Instances it also determines the initialization timeout, which is the larger of 130 seconds and the function timeout, up to 15 minutes. Set it long enough for your workload's initialization and invocation. For more information, see Understanding the Lambda Managed Instances execution environment.

  • Ephemeral storage – Set --ephemeral-storage (/tmp) large enough for your workload's storage needs. GPU-accelerated functions run on EC2 instances with NVMe-backed storage, included at no additional charge. The default is 512 MB and the maximum is 32,768 MB. For more information, see the Limits section.

  • Memory size – Set --memory-size to the host memory your function requires. Lambda reserves part of each instance's system memory for management operations. Choose an instance type with enough memory for your function plus this reserved portion.

Step 4: Publish and invoke

As with other functions on Lambda Managed Instances, publish a version to deploy your GPU-accelerated function to your capacity provider. This example uses $LATEST.PUBLISHED, which deploys your latest code without numbered versions. For workloads that require controlled rollouts, or whose invocations run for many minutes, you can use numbered versions with weighted aliases to shift traffic between versions. For more information about deployment best practices, see Deployment best practices for GPU-accelerated functions.

Note

Lambda maintains GPU capacity for each active version. To free capacity for new versions, delete or deactivate versions you no longer use.

aws lambda publish-version --function-name my-gpu-function --publish-to LATEST_PUBLISHED

After the publish operation completes and the version is active, invoke your function:

aws lambda invoke \ --function-name my-gpu-function \ --cli-binary-format raw-in-base64-out \ --payload '{}' \ response.json
Note

The command does not specify a version. When you invoke a function on Lambda Managed Instances without a version or alias, Lambda invokes the $LATEST.PUBLISHED version.

Clean up

To avoid incurring charges, delete the resources you created:

  1. Delete the function:

    aws lambda delete-function --function-name my-gpu-function
  2. Delete the capacity provider:

    aws lambda delete-capacity-provider --capacity-provider-name my-gpu-capacity-provider
  3. Delete the Amazon ECR repository:

    aws ecr delete-repository --repository-name my-gpu-image --force --region ${REGION}