Amazon FSx for Lustre
Lustre User Guide

Using Data Repositories

Amazon FSx for Lustre is designed for short-term, compute-intensive workloads where your long-term data is stored in a durable data repository, such as Amazon S3 or an on-premises data store. With Amazon FSx for Lustre, your file system is generally brought up for the duration of your compute job (typically several hours or days). Amazon FSx for Lustre is cost-optimized for these workloads by providing high-performance, nonreplicated file storage that’s designed to be spun up and down as needed. An Amazon FSx for Lustre repository isn't meant to store durable, long-term data.

When you use Amazon FSx for Lustre with a durable storage repository, you can ingest and process large volumes of file data in a high-performance file system. At the same time, you can periodically write intermediate results to your data repository. By taking this approach, you can restart your workload at any time from the latest data you've stored in your data repository. When your workload is done, you can write final results from your file system to your data repository, and delete your file system.

You have the option to link your Amazon FSx for Lustre file system to an Amazon S3 data repository when you create the file system. For more information, see Step 1: Create Your Amazon FSx for Lustre File System.

Amazon FSx for Lustre is deeply integrated with Amazon S3. This integration means that you can seamlessly access the objects stored in your Amazon S3 buckets from applications mounting your Amazon FSx for Lustre file system. Amazon FSx for Lustre also supports cloud bursting workloads with on-premises data repositories by enabling you to copy data from on-premises clients using AWS Direct Connect or VPN. When you use Amazon FSx for Lustre with either type of data repository, you can have your data copied into your Amazon FSx for Lustre file system as needed. You can also run your compute-intensive workloads on Amazon EC2 instances in the AWS Cloud and copy your results to your data repository when your workload is done.


In many cases, you link one or more Amazon FSx for Lustre file systems to an Amazon S3 bucket. If so, don't delete the Amazon S3 bucket until all linked file systems have been deleted.


Amazon FSx for Lustre does not support using Amazon S3 buckets that are encrypted using SSE-KMS and SSE-C for data respositories. For more information, see Protecting Data Using Server-Side Encryption in the Amazon Simple Storage Service Developer Guide.