Configure inference output in Autopilot-generated containers - Amazon SageMaker

Configure inference output in Autopilot-generated containers

Amazon SageMaker Autopilot generates an ordered ContainerDefinition list that can be used to build a model to deploy in a machine learning pipeline. This model can be used for online hosting and inference. Customers can access the list of inference container definitions with the ListCandidateForAutoMLJob API. The list of inference container definitions representing the best candidate is also available as part of the DescribeAutoMLJob response.

Inference container definitions for regression and classification problem types

The inference containers generated depend on the problem type of the job.

  • Regression: Generates two containers:

    1. First is the feature engineering container that transforms the original features to features that the regression algorithms can train on.

    2. Second is the algorithm container that transforms features and generates the regression score for the dataset.

  • Classification: Generates three containers:

    1. The feature engineering container that transforms the original features to features that the classification algorithms can train on.

    2. The algorithm container that generates the winning predicted_label. It can also produce the various probabilities associated with the classification outcomes in the inference response.

    3. A feature engineering container that performs post-processing of the algorithm prediction. For example, an inverse transform of the predicted label to original label.

Select inference responses for classification models

With classification inference containers, you can select the content of the inference responses. There are four predefined keys:

  • predicted_label: The winning label determined by Autopilot.

  • probability: The probability of the True class for binary classification. The probability of winning class for multiclass classification.

  • probabilities: The list of probabilities for all corresponding .

  • labels: List of all labels

By default, inference containers are configured to generate predicted_label only.

Three environment variables are used to select the optional inference content:

  • SAGEMAKER_INFERENCE_SUPPORTED: this is set to provide hints to you about what content each container supports.

  • SAGEMAKER_INFERENCE_INPUT: should be set to the keys that the container expects in input payload.

  • SAGEMAKER_INFERENCE_OUTPUT: should be populated with the set of keys the container outputs.

In order to choose the inference response content, we need to add the SAGEMAKER_INFERENCE_INPUT, SAGEMAKER_INFERENCE_OUTPUT appropriately in the second and the third containers in the list of containers for classification problem.

The keys supported by the third classification model container are predicted_label, labels, probability and probabilities Hence the SAGEMAKER_INFERENCE_SUPPORTED environment includes the names of all these keys.

The keys supported by the second container (Algorithm) are predicted_label, probability, and probabilities. Note that the `labels` is deliberately not added to the SAGEMAKER_INFERENCE_SUPPORTED.

Here is how to update the definition of the inference containers to receive predicted_label and probability.

containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label, probability'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})

Here is how to update the definition of the inference containers to receive predicted_label and probabilities and labels. Note that you do not need to pass the labels to the second container, the algorithm container. That is redundant because it can be generated by the third container independently. This reduces the latency.

containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label,probabilities'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label,probabilities'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probabilities,labels'})

You can use the Amazon SageMaker Python SDK to accomplish the following:

from sagemaker import AutoML aml = AutoML.attach(auto_ml_job_name='AUTOML_JOB_NAME') aml_best_model = aml.create_model(name='SELECT_MODEL_NAME', candidate=None, inference_response_keys=['probabilities', 'labels']) aml_transformer = aml_best_model.transformer(accept='text/csv', assemble_with='Line', instance_type='ml.m5.xlarge', instance_count=1,) aml_transformer.transform(test_data_s3_path, content_type='text/csv', split_type='Line', job_name=<Add jobName>, wait=True)