Architecture overview

This section provides a reference implementation architecture diagram for the components deployed with this solution.

Architecture diagram

This solution is built with two primary components:

1. The Orchestrator component, created by deploying the solution’s AWS CloudFormation template.

2. The AWS CodePipeline instance deployed from either calling the solution’s Amazon API Gateway, or by committing a configuration file into an AWS CodeCommit repository. The solution’s pipelines are implemented as AWS CloudFormation templates, which allows you to extend the solution and add custom pipelines.

To support multiple use cases and business needs, the solution provides two AWS CloudFormation templates: option 1 for single account deployment, and option 2 for multi-account deployment. In both templates, the solution provides the option to use Amazon SageMaker model registry to deploy versioned models. The model registry allows you to catalog models for production, manage model versions, associate metadata with models, manage the approval status of a model, deploy models to production, and automate model deployment with continuous integration and continuous delivery (CI/CD).

Template option 1: Single-account deployment

The solution’s single-account architecture provides the following components and workflows, which are shown as numbered steps in the following diagram.

MLOps Workload Orchestrator solution architecture (single account)

This solution’s single-account template provides the following components and workflows:

1. The orchestrator (solution owner or DevOps engineer) launches the solution in the AWS account and selects the preferred options, for example, using Amazon SageMaker registry or providing an existing Amazon Simple Storage Service (Amazon S3) bucket.

2. The orchestrator uploads the required assets for the target pipeline (for example, model artifact, training data, or custom algorithm zip file) into the Amazon S3 assets bucket. If using Amazon SageMaker model registry, the orchestrator (or an automated pipeline) must register the model with the model registry.

3. A single account AWS CodePipeline instance is provisioned by either sending an API call to the API Gateway, or by committing the mlops-config.json file to the Git repository. Depending on the pipeline type, the Orchestrator AWS Lambda function packages the target AWS CloudFormation template and its parameter and configurations using the body of the API call or the mlops-config.json file, and uses it as the source stage for the AWS CodePipeline instance.

   Note

   If you are provisioning the model monitor pipeline, the orchestrator must first provision the real-time inference pipeline, and then provision the model monitor pipeline.

   If a custom algorithm (for example, not a built-in Amazon SageMaker algorithm) was used to train the model, the orchestrator must provide the Amazon ECR custom algorithm’s image URI, or build and register the Docker image using the custom algorithm image builder pipeline.

4. The DeployPipeline stage takes the packaged CloudFormation template and its parameters/configurations and deploys the target pipeline into the same account.

5. After the target pipeline is provisioned, users can access its functionalities. An Amazon Simple Notification Service (Amazon SNS) notification is sent to the email provided in the solution’s launch parameters.

   Note

   The single-account AWS CodePipeline’s AWS CloudFormation action is granted admin permissions to deploy different resources by different MLOps pipelines. Roles are defined by the pipelines' CloudFormation templates. This makes it easy to add new pipelines. To restrict the types of resources a template can deploy, customers can create an AWS Identity and Access Management (IAM) role, with limited permissions, and pass it to the CloudFormation action as the deployment role.

Template option 2: Multi-account deployment

This solution uses AWS Organizations and AWS CloudFormation StackSets to allow you to provision or update ML pipelines across AWS accounts and Regions. Using an AWS Organizations delegated administrator account (also referred to as the orchestrator account in this guide) allows you to deploy ML pipelines implemented as AWS CloudFormation templates into selected target accounts (for example, development, staging, and production accounts).

We recommend using AWS Organizations to govern across account deployments with the following structure:

• Orchestrator account (the AWS Organizations delegated administrator account) - The MLOps Workload Orchestrator solution is deployed into this account.

• Development Organizational Unit - Contains development account(s).

• Staging Organizational Unit - Contains staging account(s).

• Production Organizational Unit - Contains production account(s).

This solution uses the AWS Organizations service-managed permissions model to allow the orchestrator account to deploy pipelines into the target accounts (for example, development, staging, and production account).

Note

You must set up the recommended AWS Organizations structure, enable trusted access with AWS Organizations, and register a delegated administrator account before implementing the solution’s multi-account deployment option into the orchestrator account.

Important

By default, the solution expects the orchestrator account to be an AWS Organizations delegated administrator account. This follows best practices to limit the access to the AWS Organizations management account. However, if you want to use your management account as the orchestrator account, the solution allows you to switch to the management account by modifying the AWS CloudFormation template parameter: Are you using a delegated administrator account (AWS Organizations)? to No.

MLOps Workload Orchestrator solution architecture (multi-account)

This solution’s multi-account template provides the following components and workflows:

1. The orchestrator (solution owner or DevOps engineer with admin access to the orchestrator account) provides the AWS Organizations information (for example, development, staging, and production organizational unit IDs and account numbers). They also specify the desired options (for example, using Amazon SageMaker Registry, or providing an existing S3 bucket), and then launch the solution in their AWS account.

2. The orchestrator uploads the required assets for the target pipeline (for example, model artifact, training data, and/or custom algorithm zip file) into the S3 assets bucket in the AWS orchestrator account. If Amazon SageMaker model registry is used, the orchestrator (or an automated pipeline) must register the model with the model registry.

3. A multi-account AWS CodePipeline instance is provisioned by either sending an API call to the API Gateway, or by committing the mlops-config.json file to the Git repository. Depending on the pipeline type, the Orchestrator Lambda function packages the target AWS CloudFormation template and its parameters/configurations for each stage using the body of the API call or the mlops-config.json file, and uses it as the source stage for the AWS CodePipeline instance.

4. The DeployDev stage takes the packaged CloudFormation template and its parameters/configurations and deploys the target pipeline into the development account.

5. After the target pipeline is provisioned into the development account, the developer can then iterate on the pipeline.

6. After the development is finished, the orchestrator (or another authorized account) manually approves the DeployStaging action to move to the DeployStaging Stage.

7. The DeployStaging stage deploys the target pipeline into the staging account, using the staging configuration.

8. Testers perform different tests on the deployed pipeline.

9. After the pipeline passes quality tests, the orchestrator can approve the DeployProd action.

10. The DeployProd stage deploys the target pipeline (with production configurations) into the production account.

11. Finally, the target pipeline is live in production. An Amazon SNS notification is sent to the email provided in the solution’s launch parameters.

Note

This solution uses the model’s name (provided in the API call or mlops-config.json file) as part of the provisioned AWS CloudFormation stack name, which creates the multi-account CodePipeline instance. When a request is made to provision a pipeline, the Orchestrator Lambda function first checks to determine whether a stack exists with the specified name. If the stack does not exist, the Lambda function provisions a new stack. If a stack with the same name already exists, the function assumes that you want to update the existing pipeline using the new parameters.

Shared resources and data between accounts

In the multi-account template option, the development, staging, and production accounts each have access to the S3 assets bucket, blueprint bucket, Amazon ECR repository, and Amazon SageMaker model registry in the orchestrator account. If the S3 assets bucket, Amazon ECR repository, and Amazon SageMaker model registry are created by the solution (for example, if the customer did not provide existing resources when installing the solution), the solution will grant permissions to the development, staging, and production accounts access to these resources. If you provided an existing S3 assets bucket and Amazon ECR repository, or are using an Amazon SageMaker model registry that was not created by the solution, then you must set up permissions to allow other accounts to access these resources.

The following data is shared across accounts:

• Model artifact

• Baseline datasets used to create baselines for Amazon SageMaker data quality, model quality, model bias, and model explainability monitors

• Custom algorithm Amazon ECR image’s URL, used to train the model

To allow data separation and security, the following data is not shared between accounts:

• Location of captured data - You must provide the full S3 path for each account to store data captured by the real-time inference Amazon SageMaker endpoint.

• Batch inference data - You must provide the full S3 path to the inference data for each account.

• Location of the batch transform’s output - You must provide the full S3 path for each account where the output of the batch transform job will be stored.

• Location of baseline job’s output - You must provide the full S3 path for each account where the output of the baseline job for model monitor will be stored.

• Location of monitoring schedule job’s output - You must provide the full S3 path for each account where the output of the monitoring schedule will be stored.