Architecture overview

Deploying this solution builds the following environment in the AWS Cloud.

Scale-Out Computing on AWS architecture

The solution uses AWS CloudFormation templates to deploy the infrastructure components, AWS services, operating systems and management software, and custom logic scripts to create a scale-out design and engineering workflow reference implementation. The following overview describes the eight components and their associated AWS services deployed with the solution. For additional details about each component, refer to the Solution components section.

1. Amazon EC2 Auto Scaling to automatically provision the resources necessary to run cluster user tasks such as scale-out compute jobs.

2. This solution also deploys Amazon Elastic File System for persistent storage, Amazon Simple Storage Service (Amazon S3) for persistent logs, and optional parallel file system FSx for Lustre.

3. At its core, the Amazon Elastic Compute Cloud (Amazon EC2) instance implements a scheduler, which dynamically provisions AWS resources required for jobs submitted by users. The scheduler instance also hosts web interface which allows users and administrators to interact with the environment.

4. Launch a 2D or 3D Workstation that uses NICE Desktop Cloud Visualization (DCV), that can be used to submit batch jobs and run GUI tools.

5. Security services and resources that are used include AWS Secrets Manager, AWS Certificate Manager, Security Groups, and AWS Identity and Access Management(IAM).

6. AWS Lambda functions to verify the required prerequisites and create a default signed certificate for an Application Load Balancer (ALB) to manage access to DCV workstation sessions.

7. An Amazon OpenSearch Service cluster to store job and host information.

8. Elastic Load Balancing is used to ensure accessibility across Availability Zones, and Cost Allocation Tags are used with AWS Cost Explorer.