Amazon EFS performance tips - Amazon Elastic File System

Amazon EFS performance tips

When using Amazon EFS, keep the following performance tips in mind:

Average I/O size

The distributed nature of Amazon EFS enables high levels of availability, durability, and scalability. This distributed architecture results in a small latency overhead for each file operation. Due to this per-operation latency, overall throughput generally increases as the average I/O size increases, because the overhead is amortized over a larger amount of data.

Simultaneous connections

You can mount Amazon EFS file systems on up to thousands of Amazon EC2 and other AWS compute instances concurrently. You can drive higher throughput levels on your file system in aggregate across compute instances if you can parallelize your application across more instances.

Request model

By enabling asynchronous writes to your file system, pending write operations are buffered on the Amazon EC2 instance before they are written to Amazon EFS asynchronously. Asynchronous writes typically have lower latencies. When performing asynchronous writes, the kernel uses additional memory for caching. A file system that has enabled synchronous writes, or one that opens files using an option that bypasses the cache (for example, O_DIRECT), issues synchronous requests to Amazon EFS. Every operation goes through a round trip between the client and Amazon EFS.

Note

Your chosen request model has tradeoffs in consistency (if you're using multiple Amazon EC2 instances) and speed.

NFS client mount settings

Verify that you’re using the recommended mount options as outlined in Mounting EFS file systems and in Additional mounting considerations. Amazon EFS supports the Network File System versions 4.0 and 4.1 (NFSv4) protocols when mounting your file systems on Amazon EC2 instances. NFSv4.1 provides better performance. Only NFSv4.0 is supported for Amazon EC2 Mac instances running macOS Big Sur.

Note

You might want to increase the size of the read and write buffers for your NFS client to 1 MB when you mount your file system.

Optimizing the NFS read_ahead_kb size

Note

For Linux kernel versions 5.4.*, the Linux NFS client uses a default read_ahead_kb value of 128 KB. We recommend increasing this value to 15 MB.

The NFS read_ahead_kb attribute defines the number of kilobytes for the Linux kernel to read ahead or prefetch during a sequential read operation. For Linux kernel versions prior to 5.4.*, the read_ahead_kb value is set by multiplying NFS_MAX_READAHEAD by the value for rsize (the client configured read buffer size set in the mount options). When using the recommended mount options, this formula sets read_ahead_kb to 15 MB. However, starting with Linux kernel version 5.4.*, the Linux NFS client uses a default read_ahead_kb value of 128 KB.

We recommend setting the read_ahead_kb size to 15 MB.

The Amazon EFS mount helper that is availaible in amazon-efs-utils version 1.30.2 and higher, automatically modifies read_ahead_kb to equal 15 * rsize, or 15 MB, after mounting the file system.

For Linux kernels 5.4 or newer, if you do not use the mount helper to mount your file systems, you might consider manually setting read_ahead_kb to 15 MB for improved performance. Do so after mounting the file system by using the following command. Replace:

  • read-ahead-value-kb with the desired size in kilobytes

  • efs-mount-point with the file system's mount point.

sudo bash -c "echo read-ahead-value-in-kb > /sys/class/bdi/0:$(stat -c '%d' efs-mount-point)/read_ahead_kb"

The following example sets the read_ahead_kb size to 15 MB.

sudo bash -c "echo 15000 > /sys/class/bdi/0:$(stat -c '%d' efs)/read_ahead_kb"

Optimizing Amazon EC2 instances

Applications that perform a large number of read and write operations likely need more memory or computing capacity than applications that don't. When launching your Amazon EC2 instances, choose instance types that have the amount of these resources that your application needs. The performance characteristics of Amazon EFS file systems don't depend on the use of EBS-optimized instances.

Encryption and performance

Amazon EFS supports two forms of encryption, encryption in transit and encryption at rest. Choosing to enable either or both types of encryption for your file system has a minimal effect on I/O latency and throughput.

For information about the Amazon EFS quotas for total file system throughput, per-instance throughput, and operations per second in General Purpose performance mode, see Amazon EFS quotas and limits.