Menu
Amazon Elastic Compute Cloud
User Guide for Linux Instances

SSD Instance Store Volumes

The following instances support instance store volumes that use solid state drives (SSD) to deliver very high random I/O performance: C3, G2, HI1, I2, I3, M3, R3, and X1. For more information about the instance store volumes support by each instance type, see Instance Store Volumes.

To ensure the best IOPS performance from your SSD instance store volumes on Linux, we recommend that you use the most recent version of the Amazon Linux AMI, or another Linux AMI with a kernel version of 3.8 or later. If you do not use a Linux AMI with a kernel version of 3.8 or later, your instance will not achieve the maximum IOPS performance available for these instance types.

Like other instance store volumes, you must map the SSD instance store volumes for your instance when you launch it, and the data on an SSD instance volume persists only for the life of its associated instance. For more information, see Add Instance Store Volumes to Your EC2 Instance.

NVMe SSD Volumes

I3 instances offer non-volatile memory express (NVMe) SSD instance store volumes. To access the NVMe volumes, you must use an operating system that supports NVMe. The following are the minimum operating system requirements:

  • The current Amazon Linux AMI

  • Ubuntu version 16.10, or version 16.04 LTS. Note that version 14.04 has an older version of NVMe that we do not recommend.

  • Red Hat Enterprise Linux versions 6.5

  • CentOS version 7

  • SUSE Linux Enterprise Server version 12, or version 11 with SP3

  • Windows Server 2016, Windows Server 2012 R2, or Windows Server 2008 R2. Note that Windows Server 2012 and Windows Server 2008 are not supported.

After you connect to your instance, you can list the NVMe devices using the lspci command. The following is example output for an i3.8xlarge instance, which supports 4 NVMe devices.

Copy
$ lspci 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 00:03.0 Ethernet controller: Device 1d0f:ec20 00:17.0 Non-Volatile memory controller: Device 1d0f:cd01 00:18.0 Non-Volatile memory controller: Device 1d0f:cd01 00:19.0 Non-Volatile memory controller: Device 1d0f:cd01 00:1a.0 Non-Volatile memory controller: Device 1d0f:cd01 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)

If you are using a supported operating system but you do not see the NVMe devices, verify that the NVMe module is loaded using the following lsmod command.

Copy
$ lsmod | grep nvme nvme 48813 0

The NVMe volumes are compliant with the NVMe 1.0a specification. You can use the NVMe commands with your NVMe volumes. With the Amazon Linux AMI, you can install the nvme-cli package from the repo using the yum install command. With other supported versions of Linux, you can download the nvme-cli package if it's not available in the image.

Instance Store Volume TRIM Support

The following instances support SSD volumes with TRIM: I2, I3, and R3.

With instance store volumes that support TRIM, you can use the TRIM command to notify the SSD controller when you no longer need data that you've written. This provides the controller with more free space, which can reduce write amplification and increase performance. For more information about using TRIM commands, see the documentation for the operating system for your instance.

Instance store volumes that support TRIM are fully trimmed before they are allocated to your instance. These volumes are not formatted with a file system when an instance launches, so you must format them before they can be mounted and used. For faster access to these volumes, you should specify the file system-specific option that skips the TRIM operation when you format them. On Linux, you should also add the discard option to your mount command or /etc/fstab file entries for the devices that support TRIM so that they use this feature effectively. On Windows, use the following command: fsutil behavior set DisableDeleteNotify 1.

To make an instance store volume with TRIM support available for use on Linux

  1. Map the instance store volume when you launch the instance. For more information, see Add Instance Store Volumes to Your EC2 Instance.

  2. From the instance, list the available devices using the lsblk command or view the instance store volumes using instance metadata.

  3. Verify that your operating system and device support TRIM using the following command (replacing xvdb with the name of your device):

    Copy
    [ec2-user ~]$ sudo cat /sys/block/xvdb/queue/discard_max_bytes 322122547200

    If this command returns a value other than 0, then your operating system and device support TRIM.

  4. Format the volume with the file system of your choice.

    • (EXT4) To format the volume with the ext4 file system, use the following command (replacing xvdc with the name of your device):

      Copy
      [ec2-user ~]$ sudo mkfs.ext4 -E nodiscard /dev/xvdc
    • (XFS) To format the volume with the xfs file system, use the following command (replacing xvdb with the name of your device):

      Copy
      [ec2-user ~]$ sudo mkfs.xfs -K /dev/xvdb

      Note

      You might need to install XFS file system support on your operating system for this command to work. For Amazon Linux, use the sudo yum install -y xfsprogs command.

  5. Mount the device using the discard option. Be sure to specify the device name of the volume. You can select an existing directory or create a new one using the mkdir command.

    Copy
    [ec2-user ~]$ sudo mount -o discard /dev/xvdb /mnt/my-data
  6. (Optional) If you want the device to mount at boot time, you can add or modify an entry in the /etc/fstab file with the discard option.

    Copy
    /dev/xvdb /mnt/xvdb xfs defaults,nofail,discard 0 2 /dev/xvdc /mnt/xvdc ext4 defaults,nofail,discard 0 2

    Important

    After you edit the /etc/fstab file, verify that there are no errors running the sudo mount -a command. If there are any errors in this file, the system may not boot properly or at all.

HI1 SSD Storage

With SSD storage on HI1 instances:

  • The primary data source is an instance store with SSD storage.

  • Read performance is consistent and write performance can vary.

  • Write amplification can occur.

  • The TRIM command is not currently supported.

Instance Store with SSD Storage

The hi1.4xlarge instances use an Amazon EBS-backed root device. However, their primary data storage is provided by the SSD volumes in the instance store. Like other instance store volumes, these instance store volumes persist only for the life of the instance. Because the root device of the hi1.4xlarge instance is Amazon EBS-backed, you can still start and stop your instance. When you stop an instance, your application persists, but your production data in the instance store does not persist. For more information about instance store volumes, see Amazon EC2 Instance Store.

Variable Write Performance

Write performance depends on how your applications utilize logical block addressing (LBA) space. If your applications use the total LBA space, write performance can degrade by about 90 percent. Benchmark your applications and monitor the queue length (the number of pending I/O requests for a volume) and I/O size.

Write Amplification

Write amplification refers to an undesirable condition associated with flash memory and SSDs, where the actual amount of physical information written is a multiple of the logical amount intended to be written. Because flash memory must be erased before it can be rewritten, the process to perform these operations results in moving (or rewriting) user data and metadata more than once. This multiplying effect increases the number of writes required over the life of the SSD, which shortens the time that it can reliably operate. The hi1.4xlarge instances are designed with a provisioning model intended to minimize write amplification.

Random writes have a much more severe impact on write amplification than serial writes. If you are concerned about write amplification, allocate less than the full tebibyte of storage for your application (also known as over provisioning).

The TRIM Command

The TRIM command enables the operating system to notify an SSD that blocks of previously saved data are considered no longer in use. TRIM limits the impact of write amplification.

TRIM support is not available for HI1 instances. For information about instances that support TRIM, see Instance Store Volume TRIM Support.