Automatically generate mocks using mockall Static replay

Unit testing

While there are many ways you can implement unit testing in your AWS SDK for Rust project, there are a few that we recommend:

Use automock from the mockall crate to create and execute your tests.
Use the AWS Smithy runtime's StaticReplayClient to create a fake HTTP client that can be used instead of the standard HTTP client that is normally used by AWS services. This client returns the HTTP responses that you specify rather than communicating with the service over the network, so that tests get known data for testing purposes.

Automatically generate mocks using mockall

You can automatically generate the majority of the mock implementations that your tests need by using the popular automock from the mockall crate .

This example tests a custom method called determine_prefix_file_size(). This method calls a custom list_objects() wrapper method that calls Amazon S3. By mocking list_objects(), the determine_prefix_file_size() method can be tested without actually contacting Amazon S3.

In a command prompt for your project directory, add the mockall crate as a dependency:
```
$ cargo add mockall
```
This adds the crate to the [dependencies] section of your Cargo.toml file.
Include the automock module from the mockall crate.

Also include any other libraries related to the AWS service that you are testing, in this case, Amazon S3.
```
use aws_sdk_s3 as s3;
#[allow(unused_imports)]
use mockall::automock;

use s3::operation::list_objects_v2::{ListObjectsV2Error, ListObjectsV2Output};
```
Next, add code that determines which of two implementation of the application's Amazon S3 wrapper structure to use.
- The real one written to access Amazon S3 over the network.
- The mock implementation generated by mockall.
In this example, the one that's selected is given the name S3. The selection is conditional based on the test attribute:
```
#[cfg(test)]
pub use MockS3Impl as S3;
#[cfg(not(test))]
pub use S3Impl as S3;
```

The S3Impl struct is the implementation of the Amazon S3 wrapper structure that actually sends requests to AWS.

When testing is enabled, this code isn't used because the request is sent to the mock and not AWS. The dead_code attribute tells the linter not to report a problem if the S3Impl type isn't used.
The conditional #[cfg_attr(test, automock)] indicates that when testing is enabled, the automock attribute should be set. This tells mockall to generate a mock of S3Impl that will be named MockS3Impl.
In this example, the list_objects() method is the call you want mocked. automock will automatically create an expect_list_objects() method for you.


#[allow(dead_code)]
pub struct S3Impl {
    inner: s3::Client,
}

#[cfg_attr(test, automock)]
impl S3Impl {
    #[allow(dead_code)]
    pub fn new(inner: s3::Client) -> Self {
        Self { inner }
    }

    #[allow(dead_code)]
    pub async fn list_objects(
        &self,
        bucket: &str,
        prefix: &str,
        continuation_token: Option<String>,
    ) -> Result<ListObjectsV2Output, s3::error::SdkError<ListObjectsV2Error>> {
        self.inner
            .list_objects_v2()
            .bucket(bucket)
            .prefix(prefix)
            .set_continuation_token(continuation_token)
            .send()
            .await
    }
}

Create the test functions in a module named test.

The conditional #[cfg(test)] indicates that mockall should build the test module if the test attribute is true.


#[cfg(test)]
mod test {
    use super::*;
    use mockall::predicate::eq;

    #[tokio::test]
    async fn test_single_page() {
        let mut mock = MockS3Impl::default();
        mock.expect_list_objects()
            .with(eq("test-bucket"), eq("test-prefix"), eq(None))
            .return_once(|_, _, _| {
                Ok(ListObjectsV2Output::builder()
                    .set_contents(Some(vec![
                        // Mock content for ListObjectsV2 response
                        s3::types::Object::builder().size(5).build(),
                        s3::types::Object::builder().size(2).build(),
                    ]))
                    .build())
            });

        // Run the code we want to test with it
        let size = determine_prefix_file_size(mock, "test-bucket", "test-prefix")
            .await
            .unwrap();

        // Verify we got the correct total size back
        assert_eq!(7, size);
    }

    #[tokio::test]
    async fn test_multiple_pages() {
        // Create the Mock instance with two pages of objects now
        let mut mock = MockS3Impl::default();
        mock.expect_list_objects()
            .with(eq("test-bucket"), eq("test-prefix"), eq(None))
            .return_once(|_, _, _| {
                Ok(ListObjectsV2Output::builder()
                    .set_contents(Some(vec![
                        // Mock content for ListObjectsV2 response
                        s3::types::Object::builder().size(5).build(),
                        s3::types::Object::builder().size(2).build(),
                    ]))
                    .set_next_continuation_token(Some("next".to_string()))
                    .build())
            });
        mock.expect_list_objects()
            .with(
                eq("test-bucket"),
                eq("test-prefix"),
                eq(Some("next".to_string())),
            )
            .return_once(|_, _, _| {
                Ok(ListObjectsV2Output::builder()
                    .set_contents(Some(vec![
                        // Mock content for ListObjectsV2 response
                        s3::types::Object::builder().size(3).build(),
                        s3::types::Object::builder().size(9).build(),
                    ]))
                    .build())
            });

        // Run the code we want to test with it
        let size = determine_prefix_file_size(mock, "test-bucket", "test-prefix")
            .await
            .unwrap();

        assert_eq!(19, size);
    }
}

Each test uses let mut mock = MockS3Impl::default(); to create a mock instance of MockS3Impl.
It uses the mock's expect_list_objects() method (which was created automatically by automock) to set the expected result for when the list_objects() method is used elsewhere in the code.
After the expectations are established, it uses these to test the function by calling determine_prefix_file_size(). The returned value is checked to confirm that it's correct, using an assertion.

The determine_prefix_file_size() function uses the Amazon S3 wrapper to get the size of the prefix file:


#[allow(dead_code)]
pub async fn determine_prefix_file_size(
    // Now we take a reference to our trait object instead of the S3 client
    // s3_list: ListObjectsService,
    s3_list: S3,
    bucket: &str,
    prefix: &str,
) -> Result<usize, s3::Error> {
    let mut next_token: Option<String> = None;
    let mut total_size_bytes = 0;
    loop {
        let result = s3_list
            .list_objects(bucket, prefix, next_token.take())
            .await?;

        // Add up the file sizes we got back
        for object in result.contents() {
            total_size_bytes += object.size().unwrap_or(0) as usize;
        }

        // Handle pagination, and break the loop if there are no more pages
        next_token = result.next_continuation_token.clone();
        if next_token.is_none() {
            break;
        }
    }
    Ok(total_size_bytes)
}

The type S3 is used to call the wrapped SDK for Rust functions to support both S3Impl and MockS3Impl when making HTTP requests. The mock automatically generated by mockall reports any test failures when testing is enabled.

You can view the complete code for these examples on GitHub.

Simulate HTTP traffic using static replay

The aws-smithy-runtime crate includes a test utility class called StaticReplayClient. This HTTP client class can be specified instead of the default HTTP client when creating an AWS service object.

When initializing the StaticReplayClient, you provide a list of HTTP request and response pairs as ReplayEvent objects. While the test is running, each HTTP request is recorded and the client returns the next HTTP response found in the next ReplayEvent in the event list as the HTTP client's response. This lets the test run using known data and without a network connection.

Using static replay

To use static replay, you don't need to use a wrapper. Instead, determine what the actual network traffic should look like for the data your test will use, and provide that traffic data to the StaticReplayClient to use each time the SDK issues a request from the AWS service client.

Note

There are several ways to collect the expected network traffic, including the AWS CLI and many network traffic analyzers and packet sniffer tools.

Create a list of ReplayEvent objects that specify the expected HTTP requests and the responses that should be returned for them.
Create a StaticReplayClient using the HTTP transaction list created in the previous step.
Create a configuration object for the AWS client, specifying the StaticReplayClient as the Config object's http_client.
Create the AWS service client object, using the configuration created in the previous step.
Perform the operations that you want to test, using the service object that's configured to use the StaticReplayClient. Each time the SDK sends an API request to AWS, the next response in the list is used.

Note
The next response in the list is always returned, even if the sent request doesn't match the one in the vector of ReplayEvent objects.
When all the desired requests have been made, call the StaticReplayClient.assert_requests_match() function to verify that the requests sent by the SDK match the ones in the list of ReplayEvent objects.

Example

Let's look at the tests for the same determine_prefix_file_size() function in the previous example, but using static replay instead of mocking.

In a command prompt for your project directory, add the aws-smithy-runtime crate as a dependency:
```
$ cargo add aws-smithy-runtime --features test-util
```
This adds the crate to the [dependencies] section of your Cargo.toml file.

In your source file, include the aws_smithy_runtime types that you'll need.


use aws_smithy_runtime::client::http::test_util::{ReplayEvent, StaticReplayClient};
use aws_smithy_types::body::SdkBody;

The test begins by creating the ReplayEvent structures representing each of the HTTP transactions that should take place during the test. Each event contains an HTTP request object and an HTTP response object representing the information that the AWS service would normally reply with. These events are passed into a call to StaticReplayClient::new():


        let page_1 = ReplayEvent::new(
                http::Request::builder()
                    .method("GET")
                    .uri("https://test-bucket.s3.us-east-1.amazonaws.com/?list-type=2&prefix=test-prefix")
                    .body(SdkBody::empty())
                    .unwrap(),
                http::Response::builder()
                    .status(200)
                    .body(SdkBody::from(include_str!("./testing/response_multi_1.xml")))
                    .unwrap(),
            );
        let page_2 = ReplayEvent::new(
                http::Request::builder()
                    .method("GET")
                    .uri("https://test-bucket.s3.us-east-1.amazonaws.com/?list-type=2&prefix=test-prefix&continuation-token=next")
                    .body(SdkBody::empty())
                    .unwrap(),
                http::Response::builder()
                    .status(200)
                    .body(SdkBody::from(include_str!("./testing/response_multi_2.xml")))
                    .unwrap(),
            );
        let replay_client = StaticReplayClient::new(vec![page_1, page_2]);

The result is stored in replay_client. This represents an HTTP client that can then be used by the SDK for Rust by specifying it in the client's configuration.

To create the Amazon S3 client, call the client class's from_conf() function to create the client using a configuration object:


        let client: s3::Client = s3::Client::from_conf(
            s3::Config::builder()
                .behavior_version(BehaviorVersion::latest())
                .credentials_provider(make_s3_test_credentials())
                .region(s3::config::Region::new("us-east-1"))
                .http_client(replay_client.clone())
                .build(),
        );

The configuration object is specified using the builder's http_client() method, and the credentials are specified using the credentials_provider() method. The credentials are created using a function called make_s3_test_credentials(), which returns a fake credentials structure:


fn make_s3_test_credentials() -> s3::config::Credentials {
    s3::config::Credentials::new(
        "ATESTCLIENT",
        "astestsecretkey",
        Some("atestsessiontoken".to_string()),
        None,
        "",
    )
}

These credentials don't need to be valid because they won't actually be sent to AWS.

Run the test by calling the function that needs testing. In this example, that function's name is determine_prefix_file_size(). Its first parameter is the Amazon S3 client object to use for its requests. Therefore, specify the client created using the StaticReplayClient so requests are handled by that rather than going out over the network:
```
        let size = determine_prefix_file_size(client, "test-bucket", "test-prefix")
            .await
            .unwrap();

        assert_eq!(19, size);

        replay_client.assert_requests_match(&[]);
```
When the call to determine_prefix_file_size() is finished, an assert is used to confirm that the returned value matches the expected value. Then, the StaticReplayClient method assert_requests_match() function is called. This function scans the recorded HTTP requests and confirms that they all match the ones specified in the array of ReplayEvent objects provided when creating the replay client.

You can view the complete code for these examples on GitHub.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Create presigned URLs

Waiters