Best practices for testing serverless applications

The following sections outline best practices for achieving effective coverage when testing serverless applications.

Prioritize testing in the cloud

For well-designed applications, you can employ a variety of testing techniques to satisfy a range of requirements and conditions. However, based on current tooling, we recommend that you focus on testing in the cloud as much as possible. Although testing in the cloud can create developer latency, increase costs, and sometimes require investments in additional DevOps controls, this technique provides the most reliable, accurate, and complete test coverage.

You should have access to isolated environments in which to perform testing. Ideally, each developer should have a dedicated AWS account to avoid any issues with resource naming that can occur when multiple developers who are working in the same code try to deploy or invoke API calls on resources that have identical names. These environments should be configured with the appropriate alerts and controls to avoid unnecessary spending. For example, you can limit the type, tier, or size of resources that can be created, and set up email alerts when estimated costs exceed a given threshold.

If you must share a single AWS account with other developers, automated test processes should name resources to be unique for each developer. For example, update scripts or TOML configuration files that cause AWS SAM CLI sam deploy or sam sync commands can automatically specify a stack name that includes the local developer’s user name.

Testing in the cloud is valuable for all phases of testing, including unit tests, integration tests, and end-to-end tests.

Use mocks if necessary

Mock frameworks are a valuable tool for writing fast unit tests. They are especially valuable when tests need to cover complex internal business logic, such as mathematical or financial calculations or simulations. Look for unit tests that have a large number of test cases or input variations, where the inputs do not change the pattern or the content of calls to other cloud services. Creating mock tests for these scenarios can improve developer iteration times.

Code that is covered by unit tests that use mock testing should also be covered by testing in the cloud. This is necessary because the mocks are still running on a developer’s laptop or build machine, and the environment might be configured differently than it will be in the cloud. For example, your code might include Lambda functions that use more memory or take more time than Lambda is configured to allocate when it’s run with certain input parameters. Or your code might include environment variables that aren’t configured in the same way (or at all), and the differences might cause the code to behave differently or to fail.

Don’t use mocks of cloud services to validate the proper implementation of those service integrations. Although it might be acceptable to mock a cloud service when you’re testing other functionality, you should test cloud service calls in the cloud to validate the correct configuration and functional implementation.

Mocks can add value to unit testing, especially when you’re testing a large number of cases frequently. This benefit is reduced for integration tests, because the level of effort to implement the necessary mocks increases with the number of connection points. End-to-end testing should not use mocks, because these tests generally deal with states and complex logic that cannot be easily simulated with mock frameworks.

Avoid the use of emulators

Emulators might be convenient for some use cases. For example, a development team might have limited, inconsistent, or slow access to the internet. In this case, testing on an emulator might be the only way to reliably iterate on code before moving to a cloud environment.

For most other circumstances, use emulators sparingly. When you use an emulator, it might become difficult to innovate and include new AWS service features in your testing until the emulation vendor releases an update to provide feature parity. Emulators also require upfront and ongoing expenses for purchasing and configuration on multiple development systems and build machines. Moreover, many cloud services don’t have emulators available, and selecting an emulation testing strategy will either preclude the use of those services (leading to potentially more expensive workarounds) or produce code and configurations that aren’t well tested.

If emulation testing is unavoidable, take advantage of testing in the cloud as much as possible to ensure that proper cloud configurations are in place and to test interactions with other cloud services that can be only simulated or mocked in an emulated environment.

If needed, emulation testing can provide feedback for unit tests. Some types of integration tests and end-to-end tests might be available as well, depending on the features and behavioral parity of the emulation software.

Accelerate feedback loops

When you test in the cloud, use tools and techniques to accelerate development feedback loops. For example, use AWS SAM Accelerate and AWS CDK watch mode to decrease the time it takes to push code modifications to a cloud environment. The samples in the GitHub Serverless Test Samples repository explore some of these techniques.

We also recommend that you create and test cloud resources from your local machine as early as possible during development—not only after a check-in to source control. This practice enables quicker exploration and experimentation when developing solutions. In addition, the ability to automate deployment from a development machine helps you discover cloud configuration problems more quickly and reduces wasted effort from updating and approving modifications to source control.