Solution components

Authentication mechanism

Workload Discovery on AWS uses an Amazon Cognito User Pool for both the web user interface (UI) and Amazon API Gateway authentication. Once authenticated, Amazon Cognito provides a JSON Web Token (JWT) to the web UI that will be provided with all subsequent API requests. If a valid JWT is not provided, the API request will fail and return a HTTP 403 Forbidden response.

Web UI and storage management

The web UI was developed using React and provides a front-end console to allow users to interact with Workload Discovery on AWS.

Amazon CloudFront appends secure headers to every HTTP request to the web UI. This provides an additional layer of security, protecting against attacks such as Cross-site scripting (XSS).

Figure 4: Workload Discovery on AWS web UI and storage management components

The web UI resources are hosted in the WebUIBucket Amazon Simple Storage Service (Amazon S3) bucket and distributed by Amazon CloudFront. AWS Amplify provides an abstraction layer to simplify the integrations to API Gateway, AWS AppSync, and Amazon S3. Amazon Cognito authenticates users at the login stage. On successful login, a JSON Web Token (JWT) is provided in the authentication response from Amazon Cognito. The JWT must be sent with all subsequent API requests. If the JWT is not provided, then the API request will fail and return a HTTP 403 Forbidden response.

AWS AppSync is used to facilitate interaction with various configurations available to Workload Discovery on AWS, including managing imported Regions. AWS AppSync integrates with Amazon DynamoDB for create, read, update, and delete (CRUD) operations, but utilizes the Settings AWS Lambda function to handle more complex requests, such as importing a new Region, which require an API call to AWS Config to authorize the new Region.

AWS AppSync endpoints are also used to allow the web UI to retrieve resource relationship data from the data component using an Amazon Resource Name (ARN) and querying estimated resource cost data from AWS CURs in the cost component.

Amazon API Gateway builds the PerspectiveWebRestAPI endpoint and provides access to the relationship data that Workload Discovery on AWS collects. This API endpoint is called when you build out your architecture diagram.

Refer to Web UI features and common tasks for an overview of UI features and common tasks.

Data component

Figure 5: Workload Discovery on AWS data component

The web UI sends requests to the PerspectiveWebRestAPI and AWSPerspectiveAppSyncAPI API Gateway endpoints serving requests to the Gremlin AWS Lambda functions. The Lambda functions process the requests and query Amazon Neptune to retrieve data about the provided resources. AWS AppSync supports requests for resource data using an ID or Amazon Resource Name (ARN) and retrieves the estimated cost data from the AWS CURs.

The discovery component sends requests to the PerspectiveWebRestAPI API Gateway endpoint when it requires the latest data about the resources already discovered. This is to ensure that the discovery component aligns with the current state of the Neptune relationship graph.

The ServerGremlinAPI API Gateway endpoint receives requests from the AWS Fargate task in the discovery component and is authenticated using an Identity and Access Management (IAM) role that provides access to the Amazon OpenSearch Service cluster. The API Gateway endpoint is backed by the Search Lambda function that processes incoming requests and communicates with the OpenSearch Service cluster. The OpenSearch Service cluster provides an index of the relationship data discovered by Workload Discovery on AWS.

Image deployment component

Figure 6: Workload Discovery on AWS image deployment component

The image deployment component builds the container image that is used by the discovery component. The code is hosted in the DiscoveryBucket Amazon S3 bucket and downloaded at deployment time by AWS CodePipeline. CodePipeline initiates an AWS CodeBuild job that builds the container image and uploads it to Amazon Elastic Container Registry (Amazon ECR).

Discovery component

The discovery component is the main data-gathering element of the Workload Discovery on AWS architecture. It is responsible for querying AWS Config and making describe API calls to maintain the inventory of resources and their relationships between one another.

Figure 7: Workload Discovery on AWS discovery component

This solution configures Amazon ECS to run an AWS Fargate task using the container image downloaded from Amazon ECR. The AWS Fargate task is scheduled to run at 15-minute intervals. The resource relationship data that is collected is inserted into an Amazon Neptune graph database and Amazon OpenSearch Service.

The discovery component workflow consists of three steps:

1. Amazon ECS invokes an AWS Fargate task at 15 minutes intervals.

2. The Fargate task gathers resource data from AWS Config and AWS API describe calls.

3. The Fargate task runs HTTP POST requests to the ServerGremlinAPI API Gateway endpoint to aggregate resource relationship data and persist it into Amazon Neptune and Amazon OpenSearch Service.

Cost component

Figure 8: Workload Discovery on AWS cost component

You can create an AWS CUR in AWS Billing and Cost Management and Cost Management. This publishes a Parquet formatted file to the CostAndUsageReportBucket S3 bucket. The web UI makes requests to the AWS AppSync endpoint that invokes the Cost Lambda function. The function sends predefined queries to Amazon Athena that return estimated cost information from AWS CURs.

Due to the size of the AWS CURs, the responses from Amazon Athena can be very large. The solution stores the results in the AthenaResultsBucket Amazon S3 bucket and paginates the results back to the web UI. The lifecycle policy configured on this bucket removes items that are more than seven days old.

Supported resources

For a list of AWS resource types that Workload Discovery on AWS can discover within your accounts and Regions, refer to Supported resources.

Workload Discovery on AWS architecture diagram management

Workload Discovery on AWS architecture diagrams can be saved using the web UI where create, read, update, and delete (CRUD) operations can be performed. The AWS Amplify storage API allows Workload Discovery on AWS to store architecture diagrams in an Amazon S3 bucket. There are two levels of permissions available:

• All users - Allows Workload Discovery on AWS architecture diagrams to be visible to Workload Discovery on AWS users in your deployment. Users can download and edit these diagrams.

• You - Allows Workload Discovery on AWS architecture diagrams to be visible only to the creator. Other users will not view them.