MLOE-12: Automate operations through MLOps and CI/CD - Machine Learning Lens
Implementation planDocumentsBlogs

MLOE-12: Automate operations through MLOps and CI/CD

Automate ML workload operations using infrastructure as code (IaC) and configuration as code (CaC). Select appropriate MLOps mechanisms to orchestrate your ML workflows and integrate with CI/CD pipelines for automated deployments. This approach ensures consistency across your staging and production deployment environments. Enable model observability and version control across your hosting infrastructure.

Implementation plan

You can choose either AWS CloudFormation or AWS Cloud Development Kit (AWS CDK):

  • Use AWS CloudFormation -AWS CloudFormation enables you to create and provision AWS deployments predictably and repeatedly by using a template file to create and delete a collection of resources together as a single unit (a stack). You can manage and provision stacks across multiple AWS accounts and AWS Regions.

  • Use AWS Cloud Development Kit (AWS CDK) - Use AWS Cloud Development Kit (AWS CDK) (AWS CDK) as a software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. You can define your cloud resources in AWS CDK using familiar programming languages.

You can choose any of the following MLOps strategies based on your ML workflows:

  • Use SageMaker Pipelines to orchestrate your workflows

Using Amazon SageMaker Pipelines, you can create ML workflows with Python SDK, and then visualize and manage your workflow using Amazon SageMaker Studio. Amazon SageMaker Pipelines logs every step of your workflow, creating an audit trail of model components such as training data, platform configurations, model parameters, and learning gradients. 

Amazon SageMaker to automate training of a machine learning model. Define all the

steps in the workflow and set up alerts to start the flow.

  • Use third-party tools - Use third-party deployment orchestration tools, such as

Apache Airflow, that integrate with AWS service APIs to automate model training and deployment. Amazon Managed Workflows for Apache Airflow (MWAA) orchestrates your workflows using Directed Acyclic Graphs (DAGs) written in Python.

data is available.

Documents

Blogs