SageMaker Pipelines Overview

An Amazon SageMaker Model Building Pipelines pipeline is a series of interconnected steps that are defined using the Pipelines SDK. You can also build your pipeline without the SDK using the pipeline definition JSON schema. This pipeline definition encodes a pipeline using a directed acyclic graph (DAG) that can be exported as a JSON definition. This DAG gives information on the requirements for and relationships between each step of your pipeline. The structure of a pipeline's DAG is determined by the data dependencies between steps. These data dependencies are created when the properties of a step's output are passed as the input to another step. The following image is an example of a pipeline DAG:

The following topics describe fundamental SageMaker Pipelines concepts. For a tutorial describing the implementation of these concepts, see Create and Manage SageMaker Pipelines.

Topics

• Pipeline Structure and Execution
• IAM Access Management
• Cross-Account Support for SageMaker Pipelines
• Pipeline Parameters
• Pipeline Steps
• Lift-and-shift Python code with the @step decorator
• Pass Data Between Steps
• Caching Pipeline Steps
• Retry Policy for Pipeline Steps
• Selective execution of pipeline steps
• Baseline calculation, drift detection and lifecycle with ClarifyCheck and QualityCheck steps in Amazon SageMaker Model Building Pipelines
• Schedule Pipeline Runs
• Amazon SageMaker Experiments Integration
• Local Mode
• Troubleshooting Amazon SageMaker Model Building Pipelines