This is prerelease documentation for a service in preview release. It is subject to change.

The Mixture Density Networks (MDN) Recipe

The Amazon Forecast Mixture Density Networks (MDN) recipe is a probabilistic deep neural network algorithm used for time-series forecasting. It uses multiple time-series and categorical attributes associated with a particular item to forecast a target variable. The algorithm generates output as a mixture of Gaussians for all points in the forecast horizon at once.

How MDN Works

Use Amazon Forecast MDN for time-series data with features of seasonality and a large number of peaks and valleys. These spikes can be caused by multiple associative variables. For a predictor of such a time series, the algorithm takes as input all associative time-series features generated through Amazon Forecast, samples each time series of a specified training length (train_length), and learns one global model across the time series from all of the item, instead of relying on just one time series.

The typical loss functions from a neural network aren't suited for modeling variations in the demand curve. In this case, the modeled output distribution is assumed to be multi-modal. As such, a mixture of Gaussians is kept in an output layer of this model. The MDN algorithm is similar to the DeepAR algorithm in the way that it samples the time series and feeds it into the network.

MDN Hyperparameters

Parameter Name Description
train_length The number of time points in the past that are used in the model to make predictions. According to certain heuristics, Amazon Forecast computes this parameter value to be generally 2 to 3 times the prediction length as the default. Typically, this value should be large and cover multiple seasons in the past. For example, for daily time series, this value must be at least 365 days.
Required

No

Valid values

Positive integer

Default value

Length of the training input time series

num_of_gaussians The number of Gaussian outputs for each time point in the forecast horizon.
Required

No

Valid values

Positive integer between 1 and 5

Default value

2

epochs The number of dataset passes for the algorithm.
Required

No

Valid values

Positive integer

Default value

50

embedding_dimension The size of the embeddings to transform all categorical and one-hot features before passing through the recurrent layer.
Required

No

Valid values

Positive integer

Default value

50

mini-batch_size The batch size used for training the network. A batch consists of sampled time series. The number of instances are the number of items.
Required

No

Valid values

Positive integer

Default value

128

Tune MDN Models

You can tune an MDN recipe by adjusting the num_of_gaussians parameter value. In most situations, more than one Gaussian helps reduce the quantile losses and narrow the confidence interval of the predictions. For most applications, you can include between 1 and 10 Gaussians.