Amazon Elastic Compute Cloud
User Guide for Linux (API Version 2014-10-01)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Amazon EC2 Instance Store

Many Amazon EC2 instance types can access disk storage from disks that are physically attached to the host computer. This disk storage is referred to as instance store.

Instance Storage Concepts

An instance store provides temporary block-level storage for use with an instance. The size of an instance store ranges from 900 MiB to up to 48 TiB, and varies by instance type. Larger instance types have larger instance stores. Some smaller instance families, such as T2 and T1, do not support instance store volumes at all and they use Amazon EBS exclusively for storage. For more information, see Instance Stores Available on Instance Types.

An instance store consists of one or more instance store volumes. When you launch an instance store-backed AMI, each instance store volume available to the instance is automatically mapped. When you launch an Amazon EBS-backed AMI, instance store volumes must be configured using block device mapping at launch time (with either the default block device mapping for the chosen AMI or manually using the console or the CLI or SDK tools). Volumes must be formatted and mounted on the running instance before they can be used. Some AMIs (such as Ubuntu and Amazon Linux) that use the cloud-init utilities during the initial boot cycle may format and mount a single instance store volume, but this varies by AMI. By default, instances launched from an Amazon EBS-backed AMI have no mounted instance store volumes. Instances launched from an instance store-backed AMI have a mounted instance store volume for the virtual machine's root device volume (the size of this volume varies by AMI, but the maximum size is 10 GiB) in addition to the instance store volumes included with the instance type. For more information about instance store-backed AMIs and Amazon EBS-backed AMIs, see Storage for the Root Device.

Instance store volumes are usable only from a single instance during its lifetime; they can't be detached and then attached to another instance. If you create an AMI from an instance, the data on its instance store volumes isn't preserved and isn't present on the instance store volumes for the instances that you launch from this AMI. While an instance store is dedicated to a particular instance, the disk subsystem is shared among instances on a host computer, as shown in the following figure.

Amazon EC2 instance storage

The data in an instance store persists only during the lifetime of its associated instance. If an instance reboots (intentionally or unintentionally), data in the instance store persists. However, data on instance store volumes is lost under the following circumstances:

  • Failure of an underlying drive

  • Stopping an Amazon EBS-backed instance

  • Terminating an instance

Therefore, do not rely on instance store volumes for valuable, long-term data. Instead, keep your data safe by using a replication strategy across multiple instances, storing data in Amazon S3, or using Amazon EBS volumes. For more information, see Amazon Elastic Block Store (Amazon EBS).

When you launch an instance, whether it's launched from an Amazon EBS-backed AMI or an instance store-backed AMI, you can attach instance store volumes to the instance using block device mapping. For more information, see Adding Instance Store Volumes to an AMI.

Instance Stores Available on Instance Types

Amazon EC2 instances are divided into different instance types, which determine the size of the instance store available on the instance by default. When you launch an instance, you can specify an instance type or use the default instance type, which is an m1.small instance.

The instance type also determines the type of hardware for your instance store volumes. Some instance types use solid state drives (SSD) to deliver very high random I/O performance. This is a good option when you need storage with very low latency, but you don't need it to persist when the instance terminates, or you can take advantage of fault tolerant architectures. For more information see HI1 Instances.

The following table shows the instance types along with the size and quantity of the instance store volumes available to each instance type; these instance store volumes are included as part of the instance's hourly cost.

Instance TypeInstance Store Volumes

c1.medium

1 x 350 GB

c1.xlarge

4 x 420 GB (1680 GB)

c3.large

2 x 16 GB SSD (32 GB)

c3.xlarge

2 x 40 GB SSD (80 GB)

c3.2xlarge

2 x 80 GB SSD (160 GB)

c3.4xlarge

2 x 160 GB SSD (320 GB)

c3.8xlarge

2 x 320 GB SSD (640 GB)

cc2.8xlarge

4 x 840 GB (3360 GB)

cg1.4xlarge

2 x 840 GB (1680 GB)

cr1.8xlarge

2 x 120 GB SSD (240 GB)

g2.2xlarge1 x 60 GB SSD

hi1.4xlarge

2 x 1024 GB SSD (2048 GB)

hs1.8xlarge

24 x 2048 GB (49 TB)

i2.xlarge

1 x 800 GB SSD

i2.2xlarge

2 x 800 GB SSD (1600 GB)

i2.4xlarge

4 x 800 GB SSD (3200 GB)

i2.8xlarge

8 x 800 GB SSD (6400 GB)

m1.small

1 x 160 GB

m1.medium

1 x 410 GB

m1.large

2 x 420 GB (840 GB)

m1.xlarge

4 x 420 GB (1680 GB)

m2.xlarge

1 x 420 GB

m2.2xlarge

1 x 850 GB

m2.4xlarge

2 x 840 GB (1680 GB)

m3.medium

1 x 4 GB SSD

m3.large

1 x 32 GB SSD

m3.xlarge

2 x 40 GB SSD (80 GB)

m3.2xlarge

2 x 80 GB SSD (160 GB)

r3.large

1 x 32 GB SSD

r3.xlarge

1 x 80 GB SSD

r3.2xlarge

1 x 160 GB SSD

r3.4xlarge

1 x 320 GB SSD

r3.8xlarge

2 X 320 GB SSD (640 GB)

t1.micro

None (use Amazon EBS volumes)

t2.micro

None (use Amazon EBS volumes)

t2.small

None (use Amazon EBS volumes)

t2.medium

None (use Amazon EBS volumes)

† The c1.medium and m1.small instance types also include a 900 MB instance store swap volume, which may not be automatically enabled at boot time. For more information, see Instance Store Swap Volumes.

Instance Store Swap Volumes

Swap space in Linux can be used when a system requires more memory than it has been physically allocated. When swap space is enabled, Linux systems can swap infrequently used memory pages from physical memory to swap space (either a dedicated partition or a swap file in an existing file system) and free up that space for memory pages that require high speed access.

Note

Using swap space for memory paging is not as fast or efficient as using RAM. If your workload is regularly paging memory into swap space, you should consider migrating to a larger instance type with more RAM. For more information, see Resizing Your Instance.

The c1.medium and m1.small instance types have a limited amount of physical memory to work with, and they are given a 900 MB swap volume at launch time to act as virtual memory for Linux AMIs. Although the Linux kernel sees this swap space as a partition on the root device, it is actually a separate instance store volume, regardless of your root device type.

Amazon Linux AMIs automatically enable and use this swap space, but your AMI may require some additional steps to recognize and use this swap space. To see if your instance is using swap space, you can use the swapon -s command.

[ec2-user@ip-12-34-56-78 ~]$ swapon -s
Filename                                Type            Size    Used    Priority
/dev/xvda3                              partition       917500  0       -1

The above instance has a 900 MB swap volume attached and enabled. If you don't see a swap volume listed with this command, you may need to enable swap space for the device. Check your available disks using the lsblk command.

[ec2-user@ip-12-34-56-78 ~]$ lsblk
NAME  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda1 202:1    0    8G  0 disk /
xvda3 202:3    0  896M  0 disk

Here, the swap volume xvda3 is available to the instance, but it is not enabled (notice that the MOUNTPOINT field is empty). You can enable the swap volume with the swapon command.

Note

You need to prepend /dev/ to the device name listed by lsblk. Your device may be named differently, such as sda3, sde3, or xvde3. Use the device name for your system in the command below.

[ec2-user@ip-12-34-56-78 ~]$ sudo swapon /dev/xvda3

Now the swap space should show up in lsblk and swapon -s output.

[ec2-user@ip-12-34-56-78 ~]$ lsblk
NAME  MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda1 202:1    0    8G  0 disk /
xvda3 202:3    0  896M  0 disk [SWAP]
[ec2-user@ip-12-34-56-78 ~]$ swapon -s
Filename                                Type            Size    Used    Priority
/dev/xvda3                              partition       917500  0       -1

You will also need to edit your /etc/fstab file so that this swap space is automatically enabled at every system boot.

[ec2-user@ip-12-34-56-78 ~]$ sudo vim /etc/fstab

Append the following line to your /etc/fstab file (using the swap device name for your system):

/dev/xvda3       none    swap    sw  0       0

To use an instance store volume as swap space

Any instance store volume can be used as swap space. For example, the m3.medium instance type includes a 4 GB SSD instance store volume that is appropriate for swap space. If your instance store volume is much larger (for example, 350 GB), you may consider partitioning the volume with a smaller swap partition of 4-8 GB and the rest for a data volume.

  1. List the block devices attached to your instance to get the device name for your instance store volume.

    [ec2-user ~]$ lsblk -p
    NAME       MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
    /dev/xvdb  202:16   0   4G  0 disk /media/ephemeral0
    /dev/xvda1 202:1    0   8G  0 disk /

    In this example, the instance store volume is /dev/xdvb. Because this is an Amazon Linux instance, the instance store volume is formatted and mounted at /media/ephemeral0; not all Linux operating systems do this automatically.

  2. (Optional) If your instance store volume is mounted (it will list a MOUNTPOINT in the lsblk command output), you need to unmount it with the following command.

    [ec2-user ~]$ sudo umount /dev/xvdb
  3. Set up a Linux swap area on the device with the mkswap command.

    [ec2-user ~]$ sudo mkswap /dev/xvdb
    mkswap: /dev/xvdb: warning: wiping old ext3 signature.
    Setting up swapspace version 1, size = 4188668 KiB
    no label, UUID=b4f63d28-67ed-46f0-b5e5-6928319e620b
  4. Enable the new swap space.

    [ec2-user ~]$ sudo swapon /dev/xvdb
  5. Verify that the new swap space is being used.

    [ec2-user ~]$ swapon -s
    Filename				Type		Size	Used	Priority
    /dev/xvdb                              	partition	4188668	0	-1
  6. Edit your /etc/fstab file so that this swap space is automatically enabled at every system boot.

    [ec2-user ~]$ sudo vim /etc/fstab

    If your /etc/fstab file has an entry for /dev/xvdb (or /dev/sdb) change it to match the line below; if it does not have an entry for this device, append the following line to your /etc/fstab file (using the swap device name for your system):

    /dev/xvdb       none    swap    sw  0       0

Instance Store Device Names

Within an instance store, instance store volumes are exposed as block devices. The virtual devices for instance store volumes are ephemeral[0-23]. Instance types that support one instance store volume have ephemeral0. Instance types that support two instance store volumes have ephemeral0 and ephemeral1. Instance types that support four instance store volumes have ephemeral0, ephemeral1, ephemeral2, and ephemeral3, and so on.

Many instance store volumes are pre-formatted with the ext3 file system. SSD-based instance store volumes that support TRIM instruction are not pre-formatted with any file system. However, you can format volumes with the file system of your choice after you launch your instance.

Each entry in a block device mapping consists of a device name and the volume that it's mapped to. The instance store volumes are available to the instance, but you can't access them until they are mounted. On Linux, the instance type determines which instance store volumes are mounted for you and which are available for you to mount yourself. The block device driver for the instance assigns the actual volume name when mounting the volume, and the name assigned can be different than the name that Amazon EC2 recommends.

An instance can have multiple instance store volumes mapped to a device. However, the number and size of these volumes must not exceed the instance store available for the instance type. For more information, see Instance Stores Available on Instance Types.

Instance Store Usage Scenarios

Instance store volumes are ideal for temporary storage of information that changes frequently, such as buffers, caches, scratch data, and other temporary content, or for data that is replicated across a fleet of instances, such as a load-balanced pool of web servers.

Making Instance Stores Available on Your Instances

Instances that use Amazon EBS for the root device do not, by default, have instance store available at boot time. Also, you can't attach instance store volumes after you've launched an instance. Therefore, if you want your Amazon EBS-backed instance to use instance store volumes, you must specify them using a block device mapping when you create your AMI or launch your instance. Examples of block device mapping entries are: /dev/sdb=ephemeral0 and /dev/sdc=ephemeral1. For more information about block device mapping, see Block Device Mapping

The following procedure describes how to launch an Amazon EBS-backed m1.large Linux instance with instance store volumes.

Accessing Instance Stores on Amazon EBS-backed Linux Instances

  1. Locate an Amazon EBS-backed Linux/Unix AMI.

  2. Launch an instance that supports at least two instance store volumes with this AMI and add block device mapping entries for ephemeral0 and ephemeral1.

    For more information, see To add volumes to an instance.

  3. Connect to the instance.

  4. Verify the instance stores currently mounted on the disk.

  5. Notice a 10GiB root partition mounted at / and a 420 GiB mounted on an /media/ephemeral0 volume. Your m1.large instance comes with 2 420 GiB instance store volumes; the second volume is available but must be mounted before it can be used.

  6. To format and mount the other 420 GiB instance store volume:

    1. Create a file system of your choice on the device /dev/sdc (requires root privileges).

    2. Create a directory on which to mount the device.

    3. Mount the device on the newly created directory.

  7. Verify that the device has been mounted.

  8. Optionally, list the files on the root device.

You can also map instance store volumes to block devices when you create an AMI. The instances launched from such an AMI have instance store volumes at boot time. For information about adding a block device mapping while creating an AMI, see Creating an Amazon EBS-Backed Linux AMI.

Depending on the instance type, some instance store volumes on Amazon EC2 instance store-backed Linux and Unix instances are not mounted when the instance is launched. For example, on an m1.large Linux and Unix instance, the device /dev/sdc, although formatted and available, must be mounted before it can be used.

The following procedure describes how to access the instance store from within Amazon EC2 instance store-backed m1.large Linux instance.

Accessing Instance Stores on Amazon EC2 instance store-backed Linux Instances

1

Locate an Amazon EC2 instance store-backed Linux/Unix AMI.

2

Launch an m1.large instance.

3

Connect to the instance.

4

Check out the file systems currently mounted on the disk.

5

Notice 10 GiB root partition mounted on the root and 420 GiB mounted on an ephemeral0 device. Your m1.large instance comes with 2 420 GiB instance store volumes; the second volume is available but must be mounted before it can be used.

6

To mount the other 420GiB:

a. Create a directory on which to mount the device.
b. Mount the device on the newly created directory.

7

Check to see whether the device has been mounted.

8

Optionally, list the files on the root device.


Suppressing Instance Stores at Launch Time

You can prevent a particular instance storage volume from attaching to the instance. You can do this for both Amazon EC2 instance store-backed instances and Amazon EBS-backed instances. For example, specifying the mapping /dev/sdc=none when launching an instance prevents /dev/sdc from attaching to the instance. For more information about block device mapping, see Block Device Mapping.

Adding Instance Store Volumes to an AMI

Amazon EBS-backed AMIs don't include an instance store by default. However, you might want instances launched from your Amazon EBS-backed AMIs to include instance store volumes.

After you add instance store volumes to an AMI, any instance you launch from the AMI includes these instance store volumes. You can confirm that the instance store devices are available from within the instance itself using instance metadata. For more information, see Viewing the Instance Block Device Mapping for Instance Store Volumes.

Note

For M3 instances, you must specify instance store volumes in the block device mapping for the instance when you launch it. When you launch an M3 instance, instance store volumes specified in the block device mapping for the AMI may be ignored if they are not specified as part of the instance block device mapping.

You can create an AMI that includes instance store volumes using the console or the command line.

To add instance store volumes to an AMI using the console

  1. Open the Amazon EC2 console.

  2. In the navigation pane, click Instances.

  3. Select an instance, click Actions, select Image, and then select Create Image.

  4. In the Create Image dialog, add a meaningful name and description to your image.

  5. For each instance store volume, Click Add New Volume, select an instance store volume from the Type list and a device name from Device.

  6. Click Create Image.

To add instance store volumes to an AMI using the command line

You can use one of the following commands. For more information about these command line interfaces, see Accessing Amazon EC2.

For more information, see Block Device Mapping.

Optimizing Disk Performance

Because of the way that Amazon EC2 virtualizes disks, the first write to any location on an instance store volume performs more slowly than subsequent writes. For most applications, amortizing this cost over the lifetime of the instance is acceptable. However, if you require high disk performance, we recommend that you pre-warm your drives by writing once to every drive location before production use.

Note

The I2, R3, and HI1 instance types use direct-attached solid state drives that provide maximum performance at launch time, without pre-warming.

If you require greater flexibility in latency or throughput, we recommend using Amazon EBS.

To pre-warm the instance storage volumes, use the following Linux/Unix dd commands, depending on which store you want to initialize (/dev/sdb, etc.).

Note

Make sure to unmount the drive before performing this command.

Initialization can take a long time (about 8 hours for an extra large instance).

To initialize the instance store volumes, use the following commands on the m1.large, m1.xlarge, c1.xlarge, m2.xlarge, m2.2xlarge, and m2.4xlarge instance types:

dd if=/dev/zero of=/dev/sdb bs=1M          
dd if=/dev/zero of=/dev/sdc bs=1M          
dd if=/dev/zero of=/dev/sdd bs=1M          
dd if=/dev/zero of=/dev/sde bs=1M 

For information about the instance storage that is available for each instance type, see Instance Stores Available on Instance Types.

To perform initialization on all instance store volumes at the same time, use the following command:

dd if=/dev/zero bs=1M|tee /dev/sdb|tee /dev/sdc|tee /dev/sde > /dev/sdd

Configuring drives for RAID initializes them by writing to every drive location. When configuring software-based RAID, make sure to change the minimum reconstruction speed:

echo $((30*1024)) > /proc/sys/dev/raid/speed_limit_min