Configure inference output in Autopilotgenerated containers
Amazon SageMaker Autopilot generates an ordered ContainerDefinition
list that can be used to build a model to deploy
in a machine learning pipeline. This model can be used for online hosting and inference.
Customers can access the list of inference container definitions with the ListCandidateForAutoMLJob
API. The list of inference container
definitions representing the best candidate is also available as part of the DescribeAutoMLJob
response.
Topics
Inference container definitions for regression and classification problem types
The inference containers generated depend on the problem type of the job.

Regression: generates two containers:

First is the feature engineering container that transforms the original features to features that the regression algorithms can train on.

Second is the algorithm container that transforms features and generates the regression score for the dataset.


Classification: generates three containers:

First is the feature engineering container that transforms the original features to features that the classification algorithms can train on.

Second is the algorithm container that generates the winning
predicted_label
and that can also produce the various probabilities associated with the classification outcomes in the inference response. 
Third is a feature engineering container that performs postprocessing of the algorithm prediction, for example, an inverse transform of the predicted label to original label.

Select inference response for classification models
Classification inference containers allow you to select the content of the inference responses. There are four predefined keys:

predicted_label
: The winning label determined by Autopilot. 
probability
: The probability of theTrue
class for binary classification. The probability of winning class for multiclass classification. 
probabilities
: The list of probabilities for all corresponding labels. 
labels
: List of all labels
By default, inference containers are configured to generate predicted_label
only.
Three environment variables are used to select the optional inference content:

SAGEMAKER_INFERENCE_SUPPORTED
: this is set to provide hints to the users about what content each container supports. 
SAGEMAKER_INFERENCE_INPUT
: should be set to the keys that the container expects in its input payload. 
SAGEMAKER_INFERENCE_OUTPUT
: should be populated with the set of keys the container outputs.
In order to choose the inference response content, we need to add the
SAGEMAKER_INFERENCE_INPUT
, SAGEMAKER_INFERENCE_OUTPUT
appropriately in the second and the third containers in the list of containers for
classification problem.
The keys supported by the third classification model container are
predicted_label
, labels
, probability
and
probabilities
Hence the SAGEMAKER_INFERENCE_SUPPORTED
environment includes the names of all these keys.
The keys supported by the second container (Algorithm) are predicted_label, probability and probabilities. Note that the `labels` is deliberately not added to the SAGEMAKER_INFERENCE_SUPPORTED.
Here is how to update the definition of the inference containers to receive
predicted_label
and probability
.
containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label, probability'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probability'})
Here is how to update the definition of the inference containers to receive
predicted_label
and probabilities
and labels
. Note
that you do not need to pass the labels
to the second container, the algorithm
container, as it’s redundant and can be generated by the third container independently.
This
reduces the latency.
containers[1]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label,probabilities'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_INPUT': 'predicted_label,probabilities'}) containers[2]['Environment'].update({'SAGEMAKER_INFERENCE_OUTPUT': 'predicted_label, probabilities,labels'})
You can use the Amazon SageMaker Python SDK
from sagemaker import AutoML aml = AutoML.attach(auto_ml_job_name='AUTOML_JOB_NAME') aml_best_model = aml.create_model(name='SELECT_MODEL_NAME', candidate=None, inference_response_keys=['probabilities', 'labels']) aml_transformer = aml_best_model.transformer(accept='text/csv', assemble_with='Line', instance_type='ml.m5.xlarge', instance_count=1,) aml_transformer.transform(test_data_s3_path, content_type='text/csv', split_type='Line', job_name=<Add jobName>, wait=True)