Monitoring data transfers with Amazon CloudWatch metrics - AWS DataSync

Monitoring data transfers with Amazon CloudWatch metrics

Amazon CloudWatch provides metrics to track DataSync transfer performance and troubleshoot issues with your transfer task.

You can monitor AWS DataSync transfer performance by using Amazon CloudWatch metrics. DataSync metrics are automatically sent to CloudWatch in 5-minute intervals. The metrics are retained for a period of 15 months.

To see CloudWatch metrics for DataSync, you can use the following tools:

  • The CloudWatch console

  • The CloudWatch CLI

  • The CloudWatch API

  • The DataSync console (on the task execution's details page)

For more information, see the Amazon CloudWatch User Guide.

CloudWatch metrics for DataSync

DataSync metrics use the aws/datasync namespace and provide metrics for the following dimensions:

  • AgentId – The unique ID of the agent (if your task uses an agent).

  • TaskId – The unique ID of the task. It takes the form of task-01234567890abcdef.

The aws/datasync namespace includes the following metrics. Some metrics aren't available with every task mode.

CloudWatch metric Task mode support Description

BytesCompressed

Basic

The number of physical bytes that DataSync transfers over the network after compression (if compression is possible). This number is typically less than BytesTransferred unless the data isn't compressible.

Unit: Bytes

BytesPreparedDestination

Basic

The number of logical bytes that DataSync prepares at the destination location.

Unit: Bytes

BytesPreparedSource

Basic

The number of logical bytes that DataSync prepares at the source location.

Unit: Bytes

BytesTransferred

Basic

The number of bytes that DataSync sends to the network before compression (if compression is possible). For the number of bytes transferred over the network, see the BytesCompressed metric.

Unit: Bytes

BytesVerifiedDestination

Basic

The number of logical bytes that DataSync verifies at the destination location.

Unit: Bytes

BytesVerifiedSource

Basic

The number of logical bytes that DataSync verifies at the source location.

Units: Bytes

BytesWritten

Enhanced, Basic

The number of logical bytes that DataSync writes to the destination location.

Unit: Bytes

FilesDeleted

Enhanced, Basic

The number of files, objects, and directories that DataSync deletes in your destination location. If you don't configure your task to delete data in the destination that isn't in the source, the value is always 0.

Unit: Count

FilesListedSource

Enhanced

The number of objects that DataSync finds at your source location.

Unit: Count

FilesPrepared

Enhanced

The number of objects that DataSync will attempt to transfer after comparing your source and destination locations. For more information, see How DataSync prepares your data transfer.

This metric isn't applicable if you configure your task to transfer all data. In that scenario, DataSync copies everything from the source to the destination without comparing differences between the locations.

Unit: Count

FilesPreparedDestination

Basic

The number of files, objects, and directories that DataSync prepares at the destination location.

Unit: Count

FilesPreparedSource

Basic

The number of files, objects, and directories that DataSync prepares at the source location.

Unit: Count

FilesSkipped

Basic

The number of files, objects, and directories that DataSync skips during your transfer.

Unit: Count

FilesTransferred

Enhanced, Basic

The number of files, objects, and directories that DataSync transfers over the network. This value is updated periodically during the task execution when something is read from the source and sent over the network.

Note

This value can be less than EstimatedFilesToTransfer in a DescribeTaskExecution response if DataSync fails to transfer something. In some cases, this value can also be greater than EstimatedFilesToTransfer. This metric is implementation-specific for some location types, so don't use it as an exact indication of what transferred or to monitor your task execution.

Unit: Count

FilesVerified

Enhanced

The number of objects that DataSync verifies during your transfer.

Unit: Count

FilesVerifiedDestination

Basic

The number of files, objects, and directories that DataSync verifies at the destination location.

Unit: Count

FilesVerifiedSource

Basic

The number of files, objects, and directories that DataSync verifies at the source location.

Unit: Count