Amazon Elastic Compute Cloud
User Guide for Linux (API Version 2014-10-01)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Creating an Instance Store-Backed Linux AMI

To create an instance store-backed Linux AMI, start from an instance that you've launched from an existing instance store-backed Linux AMI. After you've customized the instance to suit your needs, bundle the volume and register a new AMI, which you can use to launch new instances with these customizations.

If you need to create an instance store-backed Windows AMI, see Creating an Instance Store-Backed Windows AMI in the Amazon EC2 User Guide for Microsoft Windows Instances.

The AMI creation process is different for instance store-backed AMIs. For more information about the differences between Amazon EBS-backed and instance store-backed instances, and how to determine the root device type for your instance, see Storage for the Root Device. If you need to create an Amazon EBS-backed Linux AMI, see Creating an Amazon EBS-Backed Linux AMI.

Overview of the Creation Process for Instance Store-Backed AMIs

The following diagram summarizes the process of creating an AMI from an instance store-backed instance.

Creating an instance store-backed AMI

First, launch an instance from an AMI that's similar to the AMI that you'd like to create. You can connect to your instance and customize it. When the instance is set up the way you want it, you can bundle it. It takes several minutes for the bundling process to complete. After the process completes, you have a bundle, which consists of an image manifest (image.manifest.xml) and files (image.part.xx) that contain a template for the root volume. Next you upload the bundle to your Amazon S3 bucket and then register your AMI.

When you launch an instance using the new AMI, we create the root volume for the instance using the bundle that you uploaded to Amazon S3. The storage space used by the bundle in Amazon S3 incurs charges to your account until you delete it. For more information, see Deregistering Your AMI.

If you add instance store volumes to your instance in addition to the root device volume, the block device mapping for the new AMI contains information for these volumes, and the block device mappings for instances that you launch from the new AMI automatically contain information for these volumes. For more information, see Block Device Mapping.

Prerequisites

Before you can create the AMI, you must complete the following tasks:

  • Install the AMI tools. For more information, see Set Up the AMI Tools.

  • Install the API tools. For more information, see Setting Up the Amazon EC2 Command Line Interface Tools on Linux.

  • Ensure that you have an Amazon S3 bucket for the bundle. To create an Amazon S3 bucket, open the Amazon S3 console and click Create Bucket.

    Note

    You can also use the AWS CLI mb command to create a bucket. To get started with the AWS CLI, see AWS Command Line Interface User Guide.

  • Ensure that you have the following credentials:

    • Your AWS account ID. To retrieve your account ID, go to Your Security Credentials and expand Account Identifiers.

    • An X.509 certificate and private key. If you need to create an X.509 certificate, go to Your Security Credentials, expand X.509 Certificates, and click Create New Certificate. The X.509 certificate and private key are used to encrypt and decrypt your AMI.

    • Your access key ID. If you need to retrieve or create an access key ID, go to Your Security Credentials, and expand Access Keys.

    • Your secret access key. You can't retrieve your secret access key. Therefore, if you can't find your secret access key, you'll need to create a new one. To create a secret access key, go to Your Security Credentials, expand Access Keys, and click Create New Access Key.

  • Connect to your instance and customize it. For example, you can install software and applications, copy data, delete temporary files, and modify the Linux configuration.

Creating an AMI from an Instance Store-Backed Linux Instance

To prepare to use the Amazon EC2 AMI Tools (HVM instances only)

The Amazon EC2 AMI tools require GRUB Legacy to boot properly. Some AMIs (most notably, Ubuntu) are configured to use GRUB 2. You must check to see that your instance uses GRUB Legacy, and if not, you need to install and configure it.

HVM instances also require partitioning tools to be installed for the AMI tools to work properly.

  1. Check to see if your instance uses GRUB Legacy.

    1. List the block devices to find the root block device.

      [ec2-user ~]$ lsblk
      NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
      xvda    202:0    0   8G  0 disk
      └─xvda1 202:1    0   8G  0 part /
      xvdb    202:16   0  30G  0 disk /media/ephemeral0

      In this example, the root device (indicated by a MOUNTPOINT of /) is /dev/xvda1. The root block device is its parent, /dev/xvda.

    2. Determine the GRUB version on the root block device.

      [ec2-user ~]$ sudo file -s /dev/xvda
      /dev/xvda: x86 boot sector; GRand Unified Bootloader, stage1 version 0x3, stage2 address 0x2000, 1st sector stage2 0x800, stage2 segment 0x200, GRUB version 0.94; partition 1: ID=0xee, starthead 254, startsector 1, 16777215 sectors, extended partition table (last)\011, code offset 0x48

      In the above example, the GRUB version is 0.94, which is GRUB Legacy. If your GRUB version is 0.9x or less, you may move on to Step 3. If you do not see a GRUB version in this output, try the grub-install -v command.

      ubuntu:~$ grub-install -v
      grub-install (GRUB) 1.99-21ubuntu3.10

      In this example, the GRUB version is greater than 0.9x, so GRUB Legacy must be installed. Proceed to Step 2.

  2. Install the grub package using the package manager for your distribution to install GRUB Legacy. For Ubuntu instances, use the following command.

    ubuntu:~$ sudo apt-get install -y grub

    You can verify that your instance is using GRUB Legacy with the grub --version command.

    ubuntu:~$ grub --version
    grub (GNU GRUB 0.97)
  3. Install the following partition management packages using the package manager for your distribution.

    • gdisk (some distributions may call this package gptfdisk instead)

    • kpartx

    For Ubuntu instances, use the following command.

    ubuntu:~$ sudo apt-get install -y gdisk kpartx
  4. Check the kernel parameters for your instance.

    ubuntu:~$ cat /proc/cmdline
    BOOT_IMAGE=/boot/vmlinuz-3.2.0-54-virtual root=UUID=4f392932-ed93-4f8f-aee7-72bc5bb6ca9d ro console=ttyS0 xen_emul_unplug=unnecessary

    Note the options following the kernel and root device parameters, console=ttyS0 and xen_emul_unplug=unnecessary.

  5. Check the kernel entries in /boot/grub/menu.lst.

    ubuntu:~$ grep ^kernel /boot/grub/menu.lst
    kernel		/boot/vmlinuz-3.2.0-54-virtual root=LABEL=cloudimg-rootfs ro console=hvc0
    kernel		/boot/vmlinuz-3.2.0-54-virtual root=LABEL=cloudimg-rootfs ro single
    kernel		/boot/memtest86+.bin

    Note that the console parameter is pointing to hvc0 instead of ttyS0 and that the xen_emul_unplug=unnecessary parameter is missing.

  6. Edit the /boot/grub/menu.lst file with your favorite text editor (such as vim or nano) to change the console and add the parameters you identified earlier to the boot entries.

    title           Ubuntu 12.04.3 LTS, kernel 3.2.0-54-virtual
    root            (hd0)
    kernel          /boot/vmlinuz-3.2.0-54-virtual root=LABEL=cloudimg-rootfs ro console=ttyS0 xen_emul_unplug=unnecessary
    initrd          /boot/initrd.img-3.2.0-54-virtual
    
    title           Ubuntu 12.04.3 LTS, kernel 3.2.0-54-virtual (recovery mode)
    root            (hd0)
    kernel          /boot/vmlinuz-3.2.0-54-virtual root=LABEL=cloudimg-rootfs ro single console=ttyS0 xen_emul_unplug=unnecessary
    initrd          /boot/initrd.img-3.2.0-54-virtual
    
    title           Ubuntu 12.04.3 LTS, memtest86+
    root            (hd0)
    kernel          /boot/memtest86+.bin
  7. Verify that your kernel entries now contain the correct parameters.

    ubuntu:~$ grep ^kernel /boot/grub/menu.lst
    kernel		/boot/vmlinuz-3.2.0-54-virtual root=LABEL=cloudimg-rootfs ro console=ttyS0 xen_emul_unplug=unnecessary
    kernel		/boot/vmlinuz-3.2.0-54-virtual root=LABEL=cloudimg-rootfs ro single console=ttyS0 xen_emul_unplug=unnecessary
    kernel		/boot/memtest86+.bin

To create an AMI from an instance store-backed Linux instance

This procedure assumes that you have satisfied the prerequisites in Prerequisites.

  1. Upload your credentials to your instance. We use these credentials to ensure that only you and Amazon EC2 can access your AMI.

    1. Create a temporary directory on your instance for your credentials as follows:

      [ec2-user ~]$ mkdir /tmp/cert

      This enables you to exclude your credentials from the created image.

    2. Copy your X.509 certificate and private key from your computer to the /tmp/cert directory on your instance, using a secure copy tool such as scp. The -i my-private-key.pem option in the following scp command is the private key you use to connect to your instance with SSH, not the X.509 private key. For example:

      you@your_computer:~ $ scp -i my-private-key.pem /path/to/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem /path/to/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem ec2-user@ec2-203-0-113-25.compute-1.amazonaws.com:/tmp/cert/
      pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem  100%  717     0.7KB/s   00:00
      cert-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem  100%  685     0.7KB/s   00:00
  2. Prepare the bundle to upload to Amazon S3 using the ec2-bundle-vol command. Be sure to specify the -e option to exclude the directory where your credentials are stored. By default, the bundle process excludes files that might contain sensitive information. These files include *.sw, *.swo, *.swp, *.pem, *.priv, *id_rsa*, *id_dsa* *.gpg, *.jks, */.ssh/authorized_keys, and */.bash_history. To include all of these files, use the --no-filter option. To include some of these files, use the --include option.

    Important

    By default, the AMI bundling process creates a compressed, encrypted collection of files in the /tmp directory that represent your root volume. If you do not have enough free disk space in /tmp to store the bundle, you need to specify a different location for the bundle to be stored with the -d /path/to/bundle/storage option. Some instances have ephemeral storage mounted at /mnt or /media/ephemeral0 that you can use, or you can also create, attach, and mount a new Amazon EBS volume to store the bundle.

    1. The ec2-bundle-vol command needs to run as root. For most commands, you can use sudo to gain elevated permissions, but in this case, you should run sudo -E su to keep your environment variables.

      [ec2-user ~]$ sudo -E su
    2. Run the ec2-bundle-vol command with the following arguments. If you do not have enough available disk space in /tmp to store your bundle, specify a location that has available space with the -d /path/to/bundle/storage option. For HVM instances, be sure to add the --partition flag; otherwise, your AMI will not boot. For more information on this command and its available options, see ec2-bundle-vol in the Amazon EC2 Command Line Reference.

      [root ec2-user]# $EC2_AMITOOL_HOME/bin/ec2-bundle-vol -k /tmp/cert/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem -c /tmp/cert/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem -u your_aws_account_id -r x86_64 -e /tmp/cert

      It can take a few minutes to create the image. When this command completes, your tmp directory contains the bundle (image.manifest.xml, plus multiple image.part.xx files).

    3. Exit from the root shell.

      [root ec2-user]# exit
  3. Upload your bundle to Amazon S3 using the ec2-upload-bundle command. Note that if the bundle prefixes (directories) don't exist in the bucket, this command creates them.

    Note

    If you specified a path with the -d /path/to/bundle/storage option in Step 2.b, use that same path in the -m option below, instead of /tmp.

    [ec2-user ~]$ ec2-upload-bundle -b my-s3-bucket/bundle_folder/bundle_name -m /tmp/image.manifest.xml -a your_access_key_id -s your_secret_access_key --region us-west-2
  4. (Optional) After the bundle is uploaded to Amazon S3, you can remove the bundle from the /tmp directory on the instance using the following rm command:

    Note

    If you specified a path with the -d /path/to/bundle/storage option in Step 2.b, use that same path below, instead of /tmp.

    [ec2-user ~]$ sudo rm /tmp/image.manifest.xml /tmp/image.part.* /tmp/image
  5. Register your AMI using the ec2-register command. Note that you don't need to specify the -O and -W options if you've set the AWS_ACCESS_KEY and AWS_SECRET_KEY environment variables.

    Important

    For HVM AMIs, add the --virtualization-type hvm flag.

    [ec2-user ~]$ ec2-register my-s3-bucket/bundle_folder/bundle_name/image.manifest.xml -n AMI_name -O your_access_key_id -W your_secret_access_key --region us-west-2

Converting your Instance Store-Backed AMI to an Amazon EBS-Backed AMI

You can convert an instance store-backed Linux AMI that you own to an Amazon EBS-backed Linux AMI.

Important

You can't convert an instance store-backed Windows AMI to an Amazon EBS-backed Windows AMI and you cannot convert an AMI that you do not own.

To convert an instance store-backed AMI to an Amazon EBS-backed AMI

  1. Launch an Amazon Linux instance from an Amazon EBS-backed AMI. For more information, see Launching an Instance. Amazon Linux instances have the Amazon EC2 command line and AMI tools pre-installed.

  2. Upload the X.509 private key that you used to bundle your instance store-backed AMI to your instance. We use this key to ensure that only you and Amazon EC2 can access your AMI.

    1. Create a temporary directory on your instance for your X.509 private key as follows:

      [ec2-user ~]$ mkdir /tmp/cert
    2. Copy your X.509 private key from your computer to the /tmp/cert directory on your instance, using a secure copy tool such as scp. The my-private-key parameter in the following command is the private key you use to connect to your instance with SSH. For example:

      you@your_computer:~ $ scp -i my-private-key.pem /path/to/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem ec2-user@ec2-203-0-113-25.compute-1.amazonaws.com:/tmp/cert/
      pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem  100%  717     0.7KB/s   00:00
  3. Set environment variables for your AWS access key and secret key.

    [ec2-user ~]$ export AWS_ACCESS_KEY=your_access_key_id
    [ec2-user ~]$ export AWS_SECRET_KEY=your_secret_access_key
  4. Prepare an Amazon EBS volume for your new AMI.

    1. Create an empty Amazon EBS volume in the same Availability Zone as your instance using the ec2-create-volume command. Note the volume ID in the command output.

      Important

      This Amazon EBS volume must be the same size or larger than the original instance store root volume.

      [ec2-user ~]$ ec2-create-volume --size 10 --region us-west-2 --availability-zone us-west-2b
      VOLUME	volume_id	10		us-west-2b	creating	2014-01-24T23:11:45+0000	standard
    2. Attach the volume to your Amazon EBS-backed instance using the ec2-attach-volume command.

      [ec2-user ~]$ ec2-attach-volume volume_id -i instance_id --device /dev/sdb --region us-west-2
      ATTACHMENT	volume_id	instance_id	/dev/sdb	attaching	2014-01-24T23:15:34+0000
  5. Create a folder for your bundle.

    [ec2-user ~]$ mkdir /tmp/bundle
  6. Download the bundle for your instance store-based AMI to /tmp/bundle using the ec2-download-bundle command.

    [ec2-user ~]$ ec2-download-bundle -b my-s3-bucket/bundle_folder/bundle_name -m image.manifest.xml -a $AWS_ACCESS_KEY -s $AWS_SECRET_KEY --privatekey /path/to/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem -d /tmp/bundle
  7. Reconstitute the image file from the bundle using the ec2-unbundle command.

    1. Change directories to the bundle folder.

      [ec2-user ~]$ cd /tmp/bundle/
    2. Run the ec2-unbundle command.

      [ec2-user bundle]$ ec2-unbundle -m image.manifest.xml --privatekey /path/to/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem
  8. Copy the files from the unbundled image to the new Amazon EBS volume.

    [ec2-user bundle]$ sudo dd if=/tmp/bundle/image of=/dev/sdb bs=1M
  9. Create a mount point for the new Amazon EBS volume and mount the volume.

    [ec2-user bundle]$ sudo mkdir /mnt/ebs
    [ec2-user bundle]$ sudo mount /dev/sdb /mnt/ebs
  10. Open the /etc/fstab file on the EBS volume with your favorite text editor (such as vim or nano) and remove any entries for instance store (ephemeral) volumes. Because the Amazon EBS volume is mounted on /mnt/ebs, the fstab file is located at /mnt/ebs/etc/fstab.

    [ec2-user bundle]$ sudo nano /mnt/ebs/etc/fstab
    #
    LABEL=/     /           ext4    defaults,noatime  1   1
    tmpfs       /dev/shm    tmpfs   defaults        0   0
    devpts      /dev/pts    devpts  gid=5,mode=620  0   0
    sysfs       /sys        sysfs   defaults        0   0
    proc        /proc       proc    defaults        0   0
    /dev/sdb        /media/ephemeral0       auto    defaults,comment=cloudconfig    0       2

    In the above example, the last line should be removed.

  11. Unmount the volume and detach it from the instance.

    [ec2-user bundle]$ sudo umount /mnt/ebs
    [ec2-user bundle]$ ec2-detach-volume volume_id --region us-west-2
    ATTACHMENT	volume_id	instance_id	/dev/sdb	detaching	2014-01-24T23:15:34+0000
  12. Create an AMI from the new Amazon EBS volume as follows.

    1. Create a snapshot of the new Amazon EBS volume.

      [ec2-user bundle]$ ec2-create-snapshot --region us-west-2 -d "your_snapshot_description" -O $AWS_ACCESS_KEY -W $AWS_SECRET_KEY volume_id
      SNAPSHOT	snapshot_id	volume_id	pending	2014-01-25T00:18:48+000		1234567891011	10	your_snapshot_description
    2. Check to see that your snapshot is complete.

      [ec2-user bundle]$ ec2-describe-snapshots --region us-west-2 snapshot_id
      SNAPSHOT	snapshot_id	volume_id	completed	2014-01-25T00:18:48+0000	100%	1234567891011	10	your_snapshot_description
    3. Identify the architecture and the kernel image (aki) used on the original AMI with the ec2-describe-images command. You need the AMI ID of the original instance store-backed AMI for this step.

      [ec2-user bundle]$ ec2-describe-images --region us-west-2 ami-id
      IMAGE	ami-8ef297be	amazon/amzn-ami-pv-2013.09.2.x86_64-s3	amazon	available	public		x86_64	machine	aki-fc8f11cc			instance-store	paravirtual	xen

      In this example, the architecture is x86_64 and the kernel image ID is aki-fc8f11cc. Use these values in the following step. If the output of the above command also lists an ari ID, take note of that as well.

    4. Register your new AMI with the snapshot ID of your new Amazon EBS volume and the values from the previous step. If the previous command output listed an ari ID, include that in the following command with --ramdisk ari_id.

      [ec2-user bundle]$ ec2-register --region us-west-2 -n your_new_ami_name -s snapshot_id -a x86_64 --kernel aki-fc8f11cc
      IMAGE	new-ami-id
  13. (Optional) After you have tested that you can launch an instance from your new AMI, you can delete the Amazon EBS volume that you created for this procedure.

    $ ec2-delete-volume volume_id