Amazon SageMaker
Developer Guide

Tune an IP Insights Model

Automatic model tuning, also called hyperparameter tuning, finds the best version of a model by running many jobs that test a range of hyperparameters on your dataset. You choose the tunable hyperparameters, a range of values for each, and an objective metric. You choose the objective metric from the metrics that the algorithm computes. Automatic model tuning searches the hyperparameters chosen to find the combination of values that result in the model that optimizes the objective metric.

For more information about model tuning, see Automatic Model Tuning.

Metrics Computed by the IP Insights Algorithm

The Amazon SageMaker IP Insights algorithm is an unsupervised learning algorithm that learns associations between IP addresses and entities. The algorithm trains a discriminator model , which learns to separate observed data points (positive samples) from randomly generated data points (negative samples). Automatic model tuning on IP Insights helps you find the model that can most accurately distinguish between unlabeled validation data and automatically generated negative samples. The model accuracy on the validation dataset is measured by the area under the receiver operating characteristic (ROC) curve. This validation:discriminator_auc metric can take values between 0.0 and 1.0, where 1.0 indicates perfect accuracy.

The IP Insights algorithm computes a validation:discriminator_auc metric during validation, the value of which is used as the objective function to optimize for hyperparameter tuning.

Metric Name Description Optimization Direction
validation:discriminator_auc

Area under the ROC curve on the validation dataset. The validation dataset is not labeled. AUC is a metric that describes the model's ability to discriminate validation data points from randomly generated data points.

Maximize

Tunable IP Insights Hyperparameters

You can tune the following hyperparameters for the Amazon SageMaker IP Insights algorithm.

Parameter Name Parameter Type Recommended Ranges
epochs

IntegerParameterRange

MinValue: 1, MaxValue: 100

learning_rate

ContinuousParameterRange

MinValue: 1e-4, MaxValue: 0.1

mini_batch_size

IntegerParameterRanges

MinValue: 100, MaxValue: 50000

num_entity_vectors

IntegerParameterRanges

MinValue: 10000, MaxValue: 1000000

num_ip_encoder_layers

IntegerParameterRanges

MinValue: 1, MaxValue: 10

random_negative_sampling_rate

IntegerParameterRanges

MinValue: 0, MaxValue: 10

shuffled_negative_sampling_rate

IntegerParameterRanges

MinValue: 0, MaxValue: 10

vector_dim

IntegerParameterRanges

MinValue: 8, MaxValue: 256

weight_decay

ContinuousParameterRange

MinValue: 0.0, MaxValue: 1.0