Configuring how AWS DataSync verifies data integrity - AWS DataSync

On December 7, 2023, we discontinued version 1 DataSync agents. Check the Agents page on the DataSync console to see if you have affected agents. If you do, replace those agents or delete them if they aren't in use. If you need more help, contact AWS Support.

Configuring how AWS DataSync verifies data integrity

During a transfer, AWS DataSync always checks the integrity of your data, but you can specify how and when this verification happens with the following options:

  • Verify only the data transferred (recommended) – DataSync calculates the checksum of transferred files and metadata at the source location. At the end of the transfer, DataSync then compares this checksum to the checksum calculated on those files at the destination.

    We recommend this option when transferring to S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive storage classes. For more information, see Storage class considerations with Amazon S3 transfers.

  • Verify all data in the destination – At the end of the transfer, DataSync scans the entire source and destination to verify that both locations are fully synchronized.

    You can't use this option when transferring to S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive storage classes. For more information, see Storage class considerations with Amazon S3 transfers.

  • Check integrity during transfer – DataSync doesn't run additional verification at the end of the transfer. Data integrity is still checked during the transfer with checksum verification.

The following instructions describe how to configure data verification when creating a task. You also can configure this when editing a task or starting a task execution.

To configure data verification by using the console
  1. Open the AWS DataSync console at https://console.aws.amazon.com/datasync/.

  2. In the left navigation pane, expand Data transfer, then choose Tasks, and then choose Create task.

  3. Configure your task's source and destination locations.

    For more information, see Where can I transfer my data with AWS DataSync?

  4. For Verification, choose one of the following:

    • Verify only the data transferred (recommended)

    • Verify all data in the destination

    • Check integrity during transfer

You can configure how DataSync verifies data by using the VerifyMode parameter with any of the following operations: