Automatically generate mocks using mockall Static replay

Test AWS SDK for Rust projects

While there are many ways you can implement unit testing in your AWS SDK for Rust project, there are two that are particularly useful:

Use automock from the mockall crate to create and execute your tests.
Use the AWS Smithy runtime's StaticReplayClient to create a fake HTTP client can be used instead of the standard HTTP client normally used by AWS services. This client returns the HTTP responses that you specify rather than communicating with the service over the network, so tests get known data for testing purposes.

There are other approaches you can take, but these are the recommended best practices.

You can view the complete code for these examples on GitHub.

Automatically generate mocks using mockall

You can (mostly) automatically generate the mock implementations your tests need by using the popular mockall crate and its automock attribute.

Start by including the needed modules, including the automock module from the mockall crate:


use aws_sdk_s3 as s3;
#[allow(unused_imports)]
use mockall::automock;

use s3::operation::list_objects_v2::{ListObjectsV2Error, ListObjectsV2Output};

Next, add code that automatically determines which implementation of the application's Amazon S3 wrapper structure to use: the one written to access Amazon S3 over the network, or the mock implementation generated by mockall. In this example, the one that's selected is given the name S3:
```
#[cfg(test)]
pub use MockS3Impl as S3;
#[cfg(not(test))]
pub use S3Impl as S3;
```

The implementation of the Amazon S3 wrapper structure that actually sends requests to AWS is called S3Impl. The dead_code attribute is used to tell the linter not to report a problem if the S3Impl type isn't used, which happens when testing is enabled. The conditional #[cfg_attr(test, automock)] is used to indicate that when testing is enabled, the automock attribute should be set. This tells mockall to generate a mock of S3Impl that will be named MockS3Impl:


#[allow(dead_code)]
pub struct S3Impl {
    inner: s3::Client,
}

#[cfg_attr(test, automock)]
impl S3Impl {
    #[allow(dead_code)]
    pub fn new(inner: s3::Client) -> Self {
        Self { inner }
    }

    #[allow(dead_code)]
    pub async fn list_objects(
        &self,
        bucket: &str,
        prefix: &str,
        continuation_token: Option<String>,
    ) -> Result<ListObjectsV2Output, s3::error::SdkError<ListObjectsV2Error>> {
        self.inner
            .list_objects_v2()
            .bucket(bucket)
            .prefix(prefix)
            .set_continuation_token(continuation_token)
            .send()
            .await
    }
}

The test functions are in a module named test, which is conditionally built if testing is enabled by checking the test attribute:


#[cfg(test)]
mod test {
    use super::*;
    use mockall::predicate::eq;

    #[tokio::test]
    async fn test_single_page() {
        let mut mock = MockS3Impl::default();
        mock.expect_list_objects()
            .with(eq("test-bucket"), eq("test-prefix"), eq(None))
            .return_once(|_, _, _| {
                Ok(ListObjectsV2Output::builder()
                    .set_contents(Some(vec![
                        // Mock content for ListObjectsV2 response
                        s3::types::Object::builder().size(5).build(),
                        s3::types::Object::builder().size(2).build(),
                    ]))
                    .build())
            });

        // Run the code we want to test with it
        let size = determine_prefix_file_size(mock, "test-bucket", "test-prefix")
            .await
            .unwrap();

        // Verify we got the correct total size back
        assert_eq!(7, size);
    }

    #[tokio::test]
    async fn test_multiple_pages() {
        // Create the Mock instance with two pages of objects now
        let mut mock = MockS3Impl::default();
        mock.expect_list_objects()
            .with(eq("test-bucket"), eq("test-prefix"), eq(None))
            .return_once(|_, _, _| {
                Ok(ListObjectsV2Output::builder()
                    .set_contents(Some(vec![
                        // Mock content for ListObjectsV2 response
                        s3::types::Object::builder().size(5).build(),
                        s3::types::Object::builder().size(2).build(),
                    ]))
                    .set_next_continuation_token(Some("next".to_string()))
                    .build())
            });
        mock.expect_list_objects()
            .with(
                eq("test-bucket"),
                eq("test-prefix"),
                eq(Some("next".to_string())),
            )
            .return_once(|_, _, _| {
                Ok(ListObjectsV2Output::builder()
                    .set_contents(Some(vec![
                        // Mock content for ListObjectsV2 response
                        s3::types::Object::builder().size(3).build(),
                        s3::types::Object::builder().size(9).build(),
                    ]))
                    .build())
            });

        // Run the code we want to test with it
        let size = determine_prefix_file_size(mock, "test-bucket", "test-prefix")
            .await
            .unwrap();

        assert_eq!(19, size);
    }
}

The test creates an instance of MockS3Impl by calling its default() method. Then it uses the mock's expect_list_objects() method (which is created automatically by automock) to set the expected result when calling the list_objects() method for given values of its input parameters.

After the expectations are established, determine_prefix_file_size() is called to test the function using the expectations that have been set. The returned value is checked to confirm that it's correct, using an assertion.

The determine_prefix_file_size() function uses the Amazon S3 wrapper to get the size of the prefix file:


#[allow(dead_code)]
pub async fn determine_prefix_file_size(
    // Now we take a reference to our trait object instead of the S3 client
    // s3_list: ListObjectsService,
    s3_list: S3,
    bucket: &str,
    prefix: &str,
) -> Result<usize, s3::Error> {
    let mut next_token: Option<String> = None;
    let mut total_size_bytes = 0;
    loop {
        let result = s3_list
            .list_objects(bucket, prefix, next_token.take())
            .await?;

        // Add up the file sizes we got back
        for object in result.contents() {
            total_size_bytes += object.size().unwrap_or(0) as usize;
        }

        // Handle pagination, and break the loop if there are no more pages
        next_token = result.next_continuation_token.clone();
        if next_token.is_none() {
            break;
        }
    }
    Ok(total_size_bytes)
}

The type S3 is used to call the wrapped SDK for Rust functions to support both S3Impl and MockS3Impl when making HTTP requests. The mock automatically generated by mockall reports any test failures when testing is enabled.

Simulate HTTP traffic using static replay

The aws-smithy-runtime crate includes a test utility class called StaticReplayClient. This HTTP client class can be specified instead of the default HTTP client when creating an AWS service object.

When initializing the StaticReplayClient, you provide a list of HTTP request and response pairs as ReplayEvent objects. While the test is running, each HTTP request is recorded and the client returns the next HTTP response found in the next ReplayEvent in the event list as the HTTP client's response. This lets the test run using known data and without a network connection.

Using static replay

To use static replay, you don't need to use a wrapper. Instead, determine what the actual network traffic should look like for the data your test will use, and provide that traffic data to the StaticReplayClient to use each time the SDK issues a request from the AWS service.

Note

There are several ways to collect the expected network traffic, including the AWS CLI and many network traffic analyzers and packet sniffer tools.

Create a list of ReplayEvent objects that specify the expected HTTP requests and the responses that should be returned for them.
Create a StaticReplayClient with the HTTP transaction list created in the previous step.
Create a configuration object for the AWS client, specifying the StaticReplayClient as the Config object's http_client.
Create the AWS service client object, specifying the configuration created in the previous step.
Perform the operations that you want to test, using the service object that's configured to use the StaticReplayClient. Each time the SDK sends an API request to AWS, the next response in the list is used.

Note
The next response in the list is always returned, even if the sent request doesn't match the one in the vector of ReplayEvent objects.
When all the desired requests have been made, call the StaticReplayClient.assert_requests_match() function to verify that the requests sent by the SDK match the ones in the list of ReplayEvent objects.

Example

Let's look at the tests for the same determine_prefix_file_size() function in the previous example, but using static replay instead of mocking.

The test begins by creating the ReplayEvent structures representing each of the HTTP transactions that should take place during the test. Each event contains an HTTP request object and an HTTP response object representing the information that the AWS service would normally reply with. These events are passed into a call to StaticReplayClient::new():


        let page_1 = ReplayEvent::new(
                http::Request::builder()
                    .method("GET")
                    .uri("https://test-bucket.s3.us-east-1.amazonaws.com/?list-type=2&prefix=test-prefix")
                    .body(SdkBody::empty())
                    .unwrap(),
                http::Response::builder()
                    .status(200)
                    .body(SdkBody::from(include_str!("./testing/response_multi_1.xml")))
                    .unwrap(),
            );
        let page_2 = ReplayEvent::new(
                http::Request::builder()
                    .method("GET")
                    .uri("https://test-bucket.s3.us-east-1.amazonaws.com/?list-type=2&prefix=test-prefix&continuation-token=next")
                    .body(SdkBody::empty())
                    .unwrap(),
                http::Response::builder()
                    .status(200)
                    .body(SdkBody::from(include_str!("./testing/response_multi_2.xml")))
                    .unwrap(),
            );
        let replay_client = StaticReplayClient::new(vec![page_1, page_2]);

The result is stored in replay_client. This represents an HTTP client that can then be used by the SDK for Rust by specifying it in the client's configuration.

To create the Amazon S3 client, call the client class's from_conf() function to create the client using a configuration object:


        let client: s3::Client = s3::Client::from_conf(
            s3::Config::builder()
                .behavior_version(BehaviorVersion::latest())
                .credentials_provider(make_s3_test_credentials())
                .region(s3::config::Region::new("us-east-1"))
                .http_client(replay_client.clone())
                .build(),
        );

The configuration object is specified using the builder's http.client() method, and the credentials are specified using the credentials_provider() method. The credentials are created using a function called make_s3_test_credentials(), which returns a fake credentials structure:


fn make_s3_test_credentials() -> s3::config::Credentials {
    s3::config::Credentials::new(
        "ATESTCLIENT",
        "astestsecretkey",
        Some("atestsessiontoken".to_string()),
        None,
        "",
    )
}

These credentials don't need to be valid because they won't actually be sent to AWS.

Run the test by calling the function that needs testing. In this example, that function's name is determine_prefix_file_size(). Its first parameter is the Amazon S3 client object to use for its requests. Therefore, specify the client created using the StaticReplayClient so requests are handled by that rather than going out over the network:
```
        let size = determine_prefix_file_size(client, "test-bucket", "test-prefix")
            .await
            .unwrap();

        assert_eq!(19, size);

        replay_client.assert_requests_match(&[]);
```
When the call to determine_prefix_file_size() is finished, an assert is used to confirm that the returned value matches the expected value. Then, the StaticReplayClient method assert_requests_match() function is called. This function scans the recorded HTTP requests and confirms that they all match the ones specified in the array of ReplayEvent objects provided when creating the replay client.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Logging

Code examples