Amazon SageMaker Autopilot container output - Amazon SageMaker

Amazon SageMaker Autopilot container output

Amazon SageMaker Autopilot generates an ordered ContainerDefinition list that can be used to build a model to deploy in a machine learning pipeline. This model can be used for online hosting and inference. Customers can access the list of inference container definitions with the ListCandidateForAutoMLJob API. The list of inference container definitions representing the best candidate is also available as part of the DescribeAutoMLJob response.

Inference container definitions for regression and classification problem types

The inference containers generated depend on the problem type of the job.

  • Regression: generates two containers:

    1. First is the feature engineering container that transforms the original features to features that the regression algorithms can train on.

    2. Second is the algorithm container that transforms features and generates the regression score for the dataset.

  • Classification: generates three containers:

    1. First is the feature engineering container that transforms the original features to features that the classification algorithms can train on.

    2. Second is the algorithm container that generates the winning predicted_label and that can also produce the various probabilities associated with the classification outcomes in the inference response.

    3. Third is a feature engineering container that performs post-processing of the algorithm prediction, for example, an inverse transform of the predicted label to original label.

Select inference response for classification models

Classification inference containers allow you to select the content of the inference responses. There are four predefined keys:

  • predicted_label: The winning label determined by Autopilot.

  • probability: The probability of the True class for binary classification. The probability of winning class for multiclass classification.

  • probabilities: The list of probabilities for all corresponding labels.

  • labels: List of all labels

By default, inference containers are configured to generate predicted_label only.

Three environment variables are used to select the optional inference content:

  • SAGEMAKER_INFERENCE_SUPPORTED: this is set to provide hints to the users about what content each container supports.

  • SAGEMAKER_INFERENCE_INPUT: should be set to the keys that the container expects in its input payload.

  • SAGEMAKER_INFERENCE_OUTPUT: should be populated with the set of keys the container outputs.

In order to choose the inference response content, we need to add the SAGEMAKER_INFERENCE_INPUT, SAGEMAKER_INFERENCE_OUTPUT appropriately in the second and the third containers in the list of containers for classification problem.

The keys supported by the third classification model container are predicted_label, labels, probability and probabilities Hence the SAGEMAKER_INFERENCE_SUPPORTED environment includes the names of all these keys.

The keys supported by the second container (Algorithm) are predicted_label, probability and probabilities. Note that the `labels` is deliberately not added to the SAGEMAKER_INFERENCE_SUPPORTED.

Here is how to update the definition of the inference containers to receive predicted_labels and probability.

containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': ‘predicted_labels, probability’}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': ‘predicted_labels, probability’}) container[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': ‘predicted_labels, probability’})

Here is how to update the definition of the inference containers to receive predicted_labels and probabilities and labels. Note that you do not need to pass the label to the second container, the algorithm container, as it’s redundant and can be generated by the third container independently. This reduces the latency.

containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': ‘predicted_labels,probabilities’}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': ‘predicted_labels,probabilities’}) container[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': ‘predicted_labels, probabilities,labels’})

You can use the SageMaker Python SDK. to accomplish this as follows:

from sagemaker import AutoML aml = AutoML.attach(auto_ml_job_name='AUTOML_JOB_NAME') aml_best_model = aml.create_model(name='SELECT_MODEL_NAME', candidate=None, inference_response_keys=['probabilities', 'labels']) aml_transformer = aml_best_model.transformer(accept='text/csv', assemble_with='Line', instance_type='ml.m5.xlarge', instance_count=1,) aml_transformer.transform(test_data_s3_path, content_type='text/csv', split_type='Line', job_name=<Add jobName>, wait=True)