EFS-to-EFS Backup Solution
EFS-to-EFS Backup Solution

Appendix D: Large File Systems

The EFS-to-EFS backup solution uses fpsync, a tool that synchronizes directories in parallel using fpart (sorts and packs files into partitions) and rsync (a file-copying tool), to copy the source file system to the backup file system. File systems with a large number of files (around one million files or more) can cause the fpart process to fail. If the process fails, the solution will generate an incomplete backup error message.


      EFS-to-EFS backup solution sample incomplete backup process error message

Figure 3: Sample incomplete backup process error message

You can also review the fpsync output logs in your Amazon Simple Storage Service (Amazon S3) bucket to determine if the fpsync synchronization ran successfully. The following log message is an example of a successful fpsync synchronization:


      EFS-to-EFS backup solution sample successful fpsync synchronization message

Figure 4: Sample successful fpsync synchronization message

If the fpsync synchronization is not successful, the solution will generate an incomplete log message.


      EFS-to-EFS backup solution sample incomplete fpsync synchronization message

Figure 5: Sample incomplete fpsync synchronization message

For customers with Amazon EFS file systems with a large number of files, we recommend that you launch multiple deployments of the solution with different source prefixes that point to different locations in your source file system. For more information, see Sizing and Capacity.