Using Delta Compression in Lumberyard Networking - Lumberyard User Guide

Using Delta Compression in Lumberyard Networking

You can use Lumberyard networking's delta compression feature to reduce the amount of network traffic generated by dataset updates.

By default, when a change is made to a non-compressed dataset, GridMate propagates the change by sending all of the data in the dataset across the network. Such a dataset is declared in the following way:

GridMate::DataSet<float> m_myFloat;

An update to such a dataset uses four bytes on the network. To reduce this, you can use a compression marshaler like the HalfMarshaler located in lumberyard_version\dev\Code\Framework\GridMate\GridMate\Serialize\CompressionMarshal.h.

The following syntax specifies a dataset that uses the HalfMarshaler compression marshaler:

GridMate::DataSet<float, HalfMarshaler> m_myFloat;

This compression marshaler reduces the network payload to two bytes, but at the cost of some precision.

Delta Compression Overview

Starting in Lumberyard version 1.18, you can use GridMate's delta compression feature to further reduce the payload to only one byte for each dataset update.

Delta compression reduces the network payload by sending only the data that changed (the "delta") instead of all data in the dataset. For example, if a float changes from 0.2 to 0.3, you can represent the change as +0.1 and encode the change in a single byte. Compared to a regular dataset, this reduces the average payload for float values by 66 percent compared to a regular dataset, or to 50 percent compared to a half marshaler compressed dataset.

GridMate supports delta compressed datasets for the two most frequently used types: float and AZ::Vector3.

The following syntax shows how to declare delta compressed datasets.

#include <GridMate/Replica/DeltaCompressedDataSet.h> DeltaCompressedDataSet<float, 5> m_myDeltaCompressedFloat; DeltaCompressedDataSet<AZ::Vector3, 1> m_myDeltaCompressedVector3;

DeltaCompressedDataSet is a template that takes at least two parameters: a type and the delta range value as an unsigned integer. The delta range parameter is described in the next section.

Delta Range

The delta range parameter, which is unique to delta compression datasets, specifies the precision of delta compression. This value acts like a threshold. When the dataset has changed by an amount greater than the specified delta range, all data in the dataset is sent.

In the previous example, if the starting value is 0, data changes that remain within the range of -5 and +5 are compressed into one byte.

For calculation purposes, the precision of delta compression is double the delta range divided by 255, which is the size of one byte. For a delta range of 5, the precision is 5*2/255=0.04.


Pay close attention to the required precision of your values. The highest precision for a DeltaCompressedDataSet is declared with a delta range of 1, as in the following example:

DeltaCompressedDataSet<float, 1> m_highestPrecisionExample;

The precision in this case is 1*2/255=0.0078.


You can work around this precision limit by changing the measurement unit that you use. For example, instead of sending meters, you can send millimeters.

Calculating Payload Reduction

Suppose a delta compressed dataset is declared in the following way:

DeltaCompressedDataSet<float, 5> m_myDeltaCompressedFloat;

If a change is less than five (the configurable value specified in the second template parameter) then the update is delta-encoded in one byte. Otherwise, a full value is sent.

The following table illustrates the behavior of a sample delta compressed dataset in chronological order.

Chronological Order Value Bytes Sent Description
1 0.0 4

Because this is the first value, it is sent as a full update.

2 0.1 1

The change is less than five, so a one-byte delta is sent.

3 4.9 1

The change is still less than five, so a one-byte delta is sent.

4 5.1 4

The change is greater than five, so a full update is sent.

5 5.3 1

The change since the last full update is less than five (5.3-5.1=0.2), so a one-byte delta is sent.

In practice, a slowly changing float value averages 1.4 bytes for each update.

AZ::Vector3 Values

The behavior for compressed float datasets is also applicable to the delta compression of AZ::Vector3 values. However, each coordinate value in the vector is compressed into one byte, for a minimum total of three bytes. This compares favorably to the twelve bytes required for an uncompressed AZ::Vector3 value.