Deploy preprocessing logic into an ML model in a single endpoint using an inference pipeline in Amazon SageMaker - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpicsRelated resources

Deploy preprocessing logic into an ML model in a single endpoint using an inference pipeline in Amazon SageMaker

Created by Mohan Gowda Purushothama (AWS), Gabriel Rodriguez Garcia (AWS), and Mateusz Zaremba (AWS)

Environment: Production

Technologies: Machine learning & AI; Containers & microservices

AWS services: Amazon SageMaker; Amazon ECR

Summary

This pattern explains how to deploy multiple pipeline model objects in a single endpoint by using an inference pipeline in Amazon SageMaker. The pipeline model object represents different machine learning (ML) workflow stages, such as preprocessing, model inference, and postprocessing. To illustrate the deployment of serially connected pipeline model objects, this pattern shows you how to deploy a preprocessing Scikit-learn container and a regression model based on the linear learner algorithm built into SageMaker. The deployment is hosted behind a single endpoint in SageMaker.

Note: The deployment in this pattern uses the ml.m4.2xlarge instance type. We recommend using an instance type that aligns with your data size requirements and the complexity of your workflow. For more information, see Amazon SageMaker Pricing. This pattern uses prebuilt Docker images for Scikit-learn, but you can use your own Docker containers and integrate them into your workflow.

Prerequisites and limitations

Prerequisites

Product versions

Architecture

Target technology stack

  • Amazon Elastic Container Registry (Amazon ECR)

  • Amazon SageMaker

  • Amazon SageMaker Studio

  • Amazon Simple Storage Service (Amazon S3)

  • Real-time inference endpoint for Amazon SageMaker

Target architecture

The following diagram shows the architecture for the deployment of an Amazon SageMaker pipeline model object.

Architecture for deployment of SageMaker pipeline model object

The diagram shows the following workflow:

  1. A SageMaker notebook deploys a pipeline model.

  2. An S3 bucket stores the model artifacts.

  3. Amazon ECR gets the source container images from the S3 bucket.

Tools

AWS tools

Code

The code for this pattern is available in the GitHub Inference Pipeline with Scikit-learn and Linear Learner repository.

Epics

TaskDescriptionSkills required

Prepare the dataset for your regression task.

Open a notebook in Amazon SageMaker Studio.

To import all necessary libraries and initialize your working environment, use the following example code in your notebook:

import sagemaker
from sagemaker import get_execution_role

sagemaker_session = sagemaker.Session()

# Get a SageMaker-compatible role used by this Notebook Instance.
role = get_execution_role()

# S3 prefix
bucket = sagemaker_session.default_bucket()
prefix = "Scikit-LinearLearner-pipeline-abalone-example"

To download a sample dataset, add the following code to your notebook:

! mkdir abalone_data
! aws s3 cp s3://sagemaker-sample-files/datasets/tabular/uci_abalone/abalone.csv ./abalone_data

Note: The example in this pattern uses the Abalone Data Set from the UCI Machine Learning Repository.

Data scientist

Upload the dataset to an S3 bucket.

In the notebook where you prepared your dataset earlier, add the following code to upload your sample data to an S3 bucket:

WORK_DIRECTORY = "abalone_data"

train_input = sagemaker_session.upload_data(
    path="{}/{}".format(WORK_DIRECTORY, "abalone.csv"),
    bucket=bucket,
    key_prefix="{}/{}".format(prefix, "train"),
)
Data scientist
TaskDescriptionSkills required

Prepare the preprocessor.py script.

  1. Copy the preprocessing logic from the Python file in the GitHub sklearn_abalone_featurizer.py repository, and then paste the code into a separate Python file called sklearn_abalone_featurizer.py. You can modify the code to fit your custom dataset and custom workflow.

  2. Save the sklearn_abalone_featurizer.py file in the root directory of your project (that is, in the same location where you run your SageMaker notebook).

Data scientist

Create the SKLearn preprocessor object.

To create an SKLearn preprocessor object (called SKLearn Estimator) that you can incorporate into your final inference pipeline, run the following code in your SageMaker notebook:

from sagemaker.sklearn.estimator import SKLearn

FRAMEWORK_VERSION = "0.23-1"
script_path = "sklearn_abalone_featurizer.py"

sklearn_preprocessor = SKLearn(
    entry_point=script_path,
    role=role,
    framework_version=FRAMEWORK_VERSION,
    instance_type="ml.c4.xlarge",
    sagemaker_session=sagemaker_session,
)
sklearn_preprocessor.fit({"train": train_input})
Data scientist

Test the preprocessor's inference.

To confirm that your preprocessor is defined correctly, launch a batch transform job by entering the following code in your SageMaker notebook:

# Define a SKLearn Transformer from the trained SKLearn Estimator
transformer = sklearn_preprocessor.transformer(
    instance_count=1, instance_type="ml.m5.xlarge", assemble_with="Line", accept="text/csv"
)


# Preprocess training input
transformer.transform(train_input, content_type="text/csv")
print("Waiting for transform job: " + transformer.latest_transform_job.job_name)
transformer.wait()
preprocessed_train = transformer.output_path
TaskDescriptionSkills required

Create a model object.

To create a model object based on the linear learner algorithm, enter the following code in your SageMaker notebook:

import boto3
from sagemaker.image_uris import retrieve

ll_image = retrieve("linear-learner", boto3.Session().region_name)
s3_ll_output_key_prefix = "ll_training_output"
s3_ll_output_location = "s3://{}/{}/{}/{}".format(
    bucket, prefix, s3_ll_output_key_prefix, "ll_model"
)

ll_estimator = sagemaker.estimator.Estimator(
    ll_image,
    role,
    instance_count=1,
    instance_type="ml.m4.2xlarge",
    volume_size=20,
    max_run=3600,
    input_mode="File",
    output_path=s3_ll_output_location,
    sagemaker_session=sagemaker_session,
)

ll_estimator.set_hyperparameters(feature_dim=10, predictor_type="regressor", mini_batch_size=32)

ll_train_data = sagemaker.inputs.TrainingInput(
    preprocessed_train,
    distribution="FullyReplicated",
    content_type="text/csv",
    s3_data_type="S3Prefix",
)

data_channels = {"train": ll_train_data}
ll_estimator.fit(inputs=data_channels, logs=True)

The preceding code retrieves the relevant Amazon ECR Docker image from the public Amazon ECR Registry for the model, creates an estimator object, and then uses that object to train the regression model.

Data scientist
TaskDescriptionSkills required

Deploy the pipeline model.

To create a pipeline model object (that is, a preprocessor object) and deploy the object, enter the following code in your SageMaker notebook:

from sagemaker.model import Model
from sagemaker.pipeline import PipelineModel
import boto3
from time import gmtime, strftime

timestamp_prefix = strftime("%Y-%m-%d-%H-%M-%S", gmtime())

scikit_learn_inferencee_model = sklearn_preprocessor.create_model()
linear_learner_model = ll_estimator.create_model()

model_name = "inference-pipeline-" + timestamp_prefix
endpoint_name = "inference-pipeline-ep-" + timestamp_prefix
sm_model = PipelineModel(
    name=model_name, role=role, models= [scikit_learn_inferencee_model, linear_learner_model]
)

sm_model.deploy(initial_instance_count=1, instance_type="ml.c4.xlarge", endpoint_name=endpoint_name)

Note: You can adjust the instance type used in the model object to meet your needs.

Data scientist

Test the inference.

To confirm the endpoint is working correctly, run the following sample inference code in your SageMaker notebook:

from sagemaker.predictor import Predictor
from sagemaker.serializers import CSVSerializer

payload = "M, 0.44, 0.365, 0.125, 0.516, 0.2155, 0.114, 0.155"
actual_rings = 10
predictor = Predictor(
    endpoint_name=endpoint_name, sagemaker_session=sagemaker_session, serializer=CSVSerializer()
)

print(predictor.predict(payload))
Data scientist

Related resources