Testing serverless applications on AWS
Dan Fox, Rohan Mehta, and Rob Hill, Amazon Web Services (AWS)
December 2022 (document history)
This guide discusses methodologies for testing serverless applications, describes the challenges that you might encounter during testing, and introduces best practices. These testing techniques are intended to help you iterate more quickly and release your code more confidently.
This guide is for developers who are looking to establish testing strategies for their
serverless applications. You can use the guide as a starting point to learn about testing
strategies, and then visit the Serverless Test Samples
repository
Overview
Automated tests are critical investments that help ensure application quality and development speed. Testing also accelerates developer feedback. As a developer, you want to be able to iterate rapidly on your application and get feedback on the quality of your code. Many developers are used to writing applications that they deploy to an environment on their desktop, either directly to their operating system or within a container-based environment. When you work in desktop or container-based environments, you typically write tests against code that is hosted entirely on your desktop. However, in serverless applications, architecture components might not be deployable to a desktop environment but might instead exist only in the cloud. A cloud-based architecture might include persistence layers, messaging systems, APIs, and other components. When you write application code that relies on these components, it might be difficult to determine the best way to design and run tests.
This guide helps you align to a testing strategy that reduces friction and confusion, and increases code quality.
Prerequisites
This guide assumes that you are familiar with the basics of automated tests, including how automated software tests are used to ensure software quality. The guide provides a high-level introduction to a serverless application testing strategy, and doesn’t require any hands-on experience writing tests.
Definitions
This guide uses the following terms:
-
Unit tests are tests that are run against code for a single architectural component in isolation.
-
Integration tests are run against two or more architectural components, typically in a cloud environment.
-
End-to-end tests verify behaviors across an entire application.
-
Emulators are applications (often provided by a third party) that are designed to mimic a cloud service without provisioning or invoking any resources.
-
Mocks (also called fakes) are implementations in a testing application that replace a dependency with a simulation of that dependency.
Targeted business outcomes
The best practices in this guide are intended to help you achieve two main objectives:
-
Increase quality of serverless applications
-
Decrease time to implement or change features
Increase software quality
An application’s quality depends to a large extent on the ability of developers to test a variety of scenarios to verify functionality. When you don’t implement automated tests, or, more typically, if your tests don’t cover the required scenarios adequately, the quality of your application can’t be determined or guaranteed.
In a server-based architecture, teams are able to easily define a scope for testing: Any code that runs on the application server has to be tested. Other components that call in to the server, or dependencies that the server calls, are often considered external and out of scope for testing by the team responsible for the application on the server.
Serverless applications often consist of smaller units of work, such as AWS Lambda functions, that run in their own environment. Teams will likely be responsible for multiples of these smaller units within a single application. Some application functionality can be delegated entirely to managed services such as Amazon Simple Storage Service (Amazon S3) or Amazon Simple Queue Service (Amazon SQS) without using any internally developed code. Traditional server-based models for software testing might exclude managed services by considering them external to the application. This can lead to inadequate coverage, where critical scenarios might be limited to manual exploratory testing or to a few integration test cases where the outcome varies by environment. Therefore, adopting testing strategies that encompass managed service behaviors and cloud configurations can improve software quality.
Decrease time to implement or change features
Software bugs and configuration problems have the least effect on costs and schedules when you’re able to catch these issues during an iterative development cycle. When these issues remain undetected by a developer, identifying problems requires additional effort from more people.
A serverless architecture might include managed services that provide critical application functionality through API calls. For this reason, your development cycle should include tests that accurately demonstrate the proper functionality when interacting with these services. If these tests are not in place, you might encounter issues that arise from the differences between your environment and the deployed environment. As a result, you might spend even more time attempting to reproduce and verify a fix, because the iteration now requires checking the changes against an environment that’s different from your preferred setup.
A proper serverless testing strategy improves your iteration time by providing accurate results for tests that include calls to other services.
