Deployment options for FSx for Lustre file systems
Amazon FSx for Lustre provides two file system deployment options: scratch and persistent.
Note
Both deployment options support solid state drive (SSD) storage. However, hard disk drive (HDD) storage is supported only in one of the persistent deployment types.
You choose the file system deployment type when you create a new file system, using the AWS Management Console, the AWS Command Line Interface (AWS CLI), or the Amazon FSx for Lustre API. For more information, see Step 1: Create your FSx for Lustre file system and CreateFileSystem in the Amazon FSx API Reference.
Encryption of data at rest is automatically enabled when you create an Amazon FSx for Lustre file system, regardless of the deployment type you use. Scratch 2 and persistent file systems automatically encrypt data in transit when they are accessed from Amazon EC2 instances that support encryption in transit. For more information on encryption, see Data encryption in Amazon FSx for Lustre.
Scratch file systems
Scratch file systems are designed for temporary storage and shorter-term processing of data. Data isn't replicated and doesn't persist if a file server fails. Scratch file systems provide high burst throughput of up to six times the baseline throughput of 200 MBps per TiB of storage capacity. For more information, see Aggregate file system performance.
Use scratch file systems when you need cost-optimized storage for short-term, processing-heavy workloads.
On a scratch file system, file servers aren't replaced if they fail and data isn't replicated. If a file server or a storage disk becomes unavailable on a scratch file system, files stored on other servers are still accessible. If clients try to access data that is on the unavailable server or disk, clients experience an immediate I/O error.
The following table illustrates the availability or durability that scratch file systems of example sizes are designed for, over the course of a day and a week. Because larger file systems have more file servers and more disks, the probabilities of failure are increased.
| File system size (TiB) | Number of file servers | Availability/durability over one day | Availability/durability over one week |
|---|---|---|---|
|
1.2 |
2 |
99.9% |
99.4% |
2.4 |
2 |
99.9% |
99.4% |
4.8 |
3 |
99.8% |
99.2% |
|
9.6 |
5 |
99.8% |
98.6% |
50.4 |
22 |
99.1% |
93.9% |
Persistent file systems
Persistent file systems are designed for longer-term storage and workloads. The file servers are highly available, and data is automatically replicated within the same Availability Zone in which the file system is located. The data volumes attached to the file servers are replicated independently from the file servers to which they are attached.
Amazon FSx continuously monitors persistent file systems for hardware failures, and automatically replaces infrastructure components in the event of a failure. On a persistent file system, if a file server becomes unavailable, it's replaced automatically within minutes of failure. During that time, client requests for data on that server transparently retry and eventually succeed after the file server is replaced. Data on persistent file systems is replicated on disks, and any failed disks are automatically replaced transparently.
Use persistent file systems for longer-term storage and for throughput-focused workloads that run for extended periods or indefinitely, and that might be sensitive to disruptions in availability.
Persistent deployment types automatically encrypt data in transit when they are accessed from Amazon EC2 instances that support encryption in transit.
Amazon FSx for Lustre supports two persistent deployment types: Persistent_1 and Persistent_2.
Persistent_2 deployment type
Persistent_2 is the latest generation of Persistent deployment type, and is best-suited for use cases that require longer-term storage, and have latency-sensitive workloads that require the highest levels of IOPS and throughput. Persistent_2 deployment types support higher levels of throughput per unit storage as compared to Persistent_1 file systems, and offer four levels of throughput per unit of storage: 125, 250, 500, and 1000 MB/s/TiB.
If you specify a metadata configuration when you create a Persistent_2 file system, you can choose to increase your metadata performance over time—independently of your file system's storage capacity—to satisfy growing performance requirements and to support larger workloads.
You can create Persistent_2 file systems with a metadata configuration mode using the Amazon FSx console, AWS Command Line Interface, and API.
Persistent_1 deployment type
Persistent_1 deployment types can be built on Lustre 2.10 or 2.12, and support SSD (solid state drive) and HDD (hard disk drive) storage types. The Persistent_1 deployment type is well-suited for use cases that require longer-term storage, and have throughput-focused workloads that aren't latency-sensitive.
For a Persistent_1 file system with SSD storage, the throughput per unit of storage is either 50, 100, or 200 MB/s per tebibyte (TiB). For HDD storage, Persistent_1 throughput per unit of storage is 12 or 40 MB/s per TiB.
You can create Persistent_1 deployment types only by using the AWS CLI and the Amazon FSx API.
Deployment type availability
Scratch_2, Persistent_1, and Persistent_2 deployment types are available in the following AWS Regions:
| AWS Region | Scratch_2 | Persistent_1 | Persistent_2 |
|---|---|---|---|
|
US East (Ohio) |
✓ |
✓ |
✓ |
|
US East (N. Virginia) |
✓ |
✓ |
✓ |
|
US East (Atlanta) Local Zone |
✓ (Persistent 125 and 250 only) |
||
|
US East (Dallas) Local Zone |
✓ (Persistent 125 and 250 only) |
||
|
US West (N. California) |
✓ |
✓ |
|
|
US West (Los Angeles) Local Zone |
✓ |
✓ |
|
|
US West (Oregon) |
✓ |
✓ |
✓ |
|
Africa (Cape Town) |
✓ |
✓ |
|
|
Asia Pacific (Hong Kong) |
✓ |
✓ |
✓ |
|
Asia Pacific (Hyderabad) |
✓ |
✓ |
|
|
Asia Pacific (Jakarta) |
✓ |
✓ |
|
|
Asia Pacific (Melbourne) |
✓ |
✓ |
|
|
Asia Pacific (Mumbai) |
✓ |
✓ |
✓ |
|
Asia Pacific (Osaka) |
✓ |
✓ |
|
|
Asia Pacific (Seoul) |
✓ |
✓ |
✓ |
|
Asia Pacific (Singapore) |
✓ |
✓ |
✓ |
|
Asia Pacific (Sydney) |
✓ |
✓ |
✓ |
|
Asia Pacific (Tokyo) |
✓ |
✓ |
✓ |
|
Canada (Central) |
✓ |
✓ |
✓ |
|
Canada West (Calgary) |
✓ (Persistent 125 and 250 only) |
||
|
Europe (Frankfurt) |
✓ |
✓ |
✓ |
|
Europe (Ireland) |
✓ |
✓ |
✓ |
|
Europe (London) |
✓ |
✓ |
✓ |
|
Europe (Milan) |
✓ |
✓ |
|
|
Europe (Paris) |
✓ |
✓ |
|
|
Europe (Spain) |
✓ |
✓ |
|
|
Europe (Stockholm) |
✓ |
✓ |
✓ |
|
Europe (Zurich) |
✓ |
✓ |
|
|
Israel (Tel Aviv) |
✓ |
✓ (Persistent 125 and 250 only) |
|
|
Middle East (Bahrain) |
✓ |
✓ |
|
|
Middle East (UAE) |
✓ |
✓ |
|
|
South America (São Paulo) |
✓ |
✓ |
|
|
AWS GovCloud (US-East) |
✓ |
✓ |
|
|
AWS GovCloud (US-West) |
✓ |
✓ |
