Workload example: Serverless data lake - AWS Prescriptive Guidance

Workload example: Serverless data lake

This workload is an example of Theme 1: Use managed services.

The data lake uses Amazon S3 for storage and AWS Lambda for ETL. These resources are defined in an AWS Cloud Development Kit (AWS CDK) app. Changes to the system are deployed through AWS CodePipeline. This pipeline is restricted to the application team. When the application team makes a pull request for the code repository, the two-person rule is used.

For this workload, the application team takes the following actions to address the Essential Eight strategies.

Application control

Patch applications

  • The application team enables Lambda scanning in Amazon Inspector and configures alerts for deprecated or vulnerable libraries.

  • The application team enable AWS Config to track AWS resources for asset discovery.

Restrict administrative privileges

  • As described in the Core architecture section, the application team already restricts access to production deployments through an approval rule on their deployment pipeline.

  • The application team relies on the centralised identity federation and centralised logging solutions that are described in the Core architecture section.

  • The application team creates an AWS CloudTrail trail and Amazon CloudWatch filters.

  • The application team sets up Amazon Simple Notification Service (Amazon SNS) alerts for CodePipeline deployments and AWS CloudFormation stack deletions.

Patch operating systems

  • The application team enables Lambda scanning in Amazon Inspector and configures alerts for deprecated or vulnerable libraries.

Multi-factor authentication

  • The application team relies on the centralised identity federation solution described in the Core architecture section. This solution enforces MFA, logs authentications, and alerts on or automatically responds to suspicious MFA events.

Regular backups

  • The application team stores code, such as AWS CDK apps and Lambda functions and configurations, in a code repository.

  • The application team enables versioning and Amazon S3 Object Lock to help prevent objects from deletion or modification.

  • The application team relies on built-in Amazon S3 durability rather than replicating their entire dataset to another AWS Region.

  • The application team runs a copy of the workload in another AWS Region that meets their data sovereignty requirements. They use Amazon DynamoDB global tables and Amazon S3 Cross-Region Replication to replicate data automatically from the primary Region to the secondary Region.