Amazon SageMaker
Developer Guide

Tuning a Factorization Machines Model

Automatic model tuning, also known as hyperparameter tuning, finds the best version of a model by running many jobs that test a range of hyperparameters on your dataset. You choose the tunable hyperparameters, a range of values for each, and an objective metric. You choose the objective metric from the metrics that the algorithm computes. Automatic model tuning searches the hyperparameters chosen to find the combination of values that result in the model that optimizes the objective metric.

For more information about model tuning, see Automatic Model Tuning.

Metrics Computed by the Factorization Machines Algorithm

The factorization machines algorithm has both binary classification and regression predictor types. The predictor type determines which metric you can use for automatic model tuning. The algorithm reports a test:rmse regressor metric, which is computed during training. When tuning the model for regression tasks, choose this metric as the objective.

Metric Name Description Optimization Direction
test:rmse

Root Mean Square Error

Minimize

The factorization machines algorithm reports three binary classification metrics, which are computed during training. When tuning the model for binary classification tasks, choose one of these as the objective.

Metric Name Description Optimization Direction
test:binary_classification_accuracy

Accuracy

Maximize

test:binary_classification_cross_entropy

Cross Entropy

Minimize

test:binary_f_beta

Beta

Maximize

Tunable Hyperparameters

You can tune the following hyperparameters for the factorization machines algorithm. The initialization parameters that contain the terms bias, linear, and factorization depend on their initialization method. There are three initialization methods: uniform, normal, and constant. The initialization methods themselves are not tunable. You must set the method before the optimization procedure. For example, if the initialization method is uniform, then only the scale parameters are tunable. FSpecifically, if bias_init_method==uniform, bias_init_scale, linear_init_scale, and factors_init_scale are tunable. Similarly, if the initialization method is normal, then only sigma parameters are tunable. If the initialization method is constant, then only value parameters are tunable. These dependencies are listed in the following table.

Parameter Name Parameter Type Recommended Ranges Dependency
mini_batch_size

IntegerParameterRange

MinValue: 100, MaxValue: 10000

None

epoch

IntegerParameterRange

MinValue: 1, MaxValue: 1000

None

bias_lr

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

None

linear_lr

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

None

factors_lr

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

None

bias_wd

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

None

linear_wd

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

None

factors_wd

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512]

None

bias_init_scale

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==uniform

bias_init_sigma

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==normal

bias_init_value

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==constant

linear_init_scale

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==uniform

linear_init_sigma

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==normal

linear_init_value

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==constant

factors_init_scale

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==uniform

factors_init_sigma

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==normal

factors_init_value

ContinuousParameterRange

MinValue: 1e-8, MaxValue: 512

bias_init_method==constant