Menu
Amazon Elastic Compute Cloud
User Guide for Linux Instances

User Provided Kernels

If you have a need for a custom kernel on your Amazon EC2 instances, you can start with an AMI that is close to what you want, compile the custom kernel on your instance, and modify the menu.lst file to point to the new kernel. This process varies depending on the virtualization type that your AMI uses. For more information, see Linux AMI Virtualization Types.

HVM AMIs (GRUB)

HVM instance volumes are treated like actual physical disks. The boot process is similar to that of a bare metal operating system with a partitioned disk and bootloader, which allows it to work with all currently supported Linux distributions. The most common bootloader is GRUB, and the following section describes configuring GRUB to use a custom kernel.

Configuring GRUB for HVM AMIs

The following is an example of a menu.lst configuration file for an HVM AMI. In this example, there are two kernel entries to choose from: Amazon Linux 2016.03 (the original kernel for this AMI), and Vanilla Linux 4.7.4 (a newer version of the Vanilla Linux kernel from https://www.kernel.org/). The Vanilla entry was copied from the original entry for this AMI, and the kernel and initrd paths were updated to the new locations. The default 0 parameter points the bootloader to the first entry that it sees (in this case, the Vanilla entry), and the fallback 1 parameter points the bootloader to the next entry if there is a problem booting the first.

By default, GRUB does not send its output to the instance console because it creates an extra boot delay. For more information, see Instance Console Output. If you are installing a custom kernel, you should consider enabling GRUB output by deleting the hiddenmenu line and adding serial and terminal lines to /boot/grub/menu.lst as shown in the example below.

Important

Avoid printing large amounts of debug information during the boot process; the serial console does not support high rate data transfer.

default=0
fallback=1
timeout=5
serial --unit=0 --speed=9600
terminal --dumb --timeout=5 serial console

title Vanilla Linux 4.7.4
root (hd0)
kernel /boot/vmlinuz-4.7.4 root=LABEL=/ console=tty1 console=ttyS0
initrd /boot/initrd.img-4.7.4

title Amazon Linux 2016.03 (4.4.11-23.53.amzn1.x86_64)
root (hd0,0)
kernel /boot/vmlinuz-4.4.11-23.53.amzn1.x86_64 root=LABEL=/ console=tty1 console=ttyS0
initrd /boot/initramfs-4.4.11-23.53.amzn1.x86_64.img

You don't need to specify a fallback kernel in your menu.lst file, but we recommend that you have a fallback when you test a new kernel. GRUB can fall back to another kernel in the event that the new kernel fails. Having a fallback kernel allows the instance to boot even if the new kernel isn't found.

If your new Vanilla Linux kernel fails, the output will be similar to the example below.

^M Entry 0 will be booted automatically in 3 seconds. ^M Entry 0 will be booted automatically in 2 seconds. ^M Entry 0 will be booted automatically in 1 seconds.

Error 13: Invalid or unsupported executable format
[ 0.000000] Initializing cgroup subsys cpuset

Paravirtual AMIs (PV-GRUB)

Amazon Machine Images that use paravirtual (PV) virtualization use a system called PV-GRUB during the boot process. PV-GRUB is a paravirtual bootloader that runs a patched version of GNU GRUB 0.97. When you start an instance, PV-GRUB starts the boot process and then chain loads the kernel specified by your image's menu.lst file.

PV-GRUB understands standard grub.conf or menu.lst commands, which allows it to work with all currently supported Linux distributions. Older distributions such as Ubuntu 10.04 LTS, Oracle Enterprise Linux or CentOS 5.x require a special "ec2" or "xen" kernel package, while newer distributions include the required drivers in the default kernel package.

Most modern paravirtual AMIs use a PV-GRUB AKI by default (including all of the paravirtual Linux AMIs available in the Amazon EC2 Launch Wizard Quick Start menu), so there are no additional steps that you need to take to use a different kernel on your instance, provided that the kernel you want to use is compatible with your distribution. The best way to run a custom kernel on your instance is to start with an AMI that is close to what you want and then to compile the custom kernel on your instance and modify the menu.lst file as shown in Configuring GRUB for Paravirtual AMIs to boot with that kernel.

You can verify that the kernel image for an AMI is a PV-GRUB AKI by executing the following describe-images command with the Amazon EC2 command line tools (substituting the kernel image ID you want to check:

$ aws ec2 describe-images --filters Name=image-id,Values=aki-880531cd

Check whether the Name field starts with pv-grub.

Limitations of PV-GRUB

PV-GRUB has the following limitations:

  • You can't use the 64-bit version of PV-GRUB to start a 32-bit kernel or vice versa.

  • You can't specify an Amazon ramdisk image (ARI) when using a PV-GRUB AKI.

  • AWS has tested and verified that PV-GRUB works with these file system formats: EXT2, EXT3, EXT4, JFS, XFS, and ReiserFS. Other file system formats might not work.

  • PV-GRUB can boot kernels compressed using the gzip, bzip2, lzo, and xz compression formats.

  • Cluster AMIs don't support or need PV-GRUB, because they use full hardware virtualization (HVM). While paravirtual instances use PV-GRUB to boot, HVM instance volumes are treated like actual disks, and the boot process is similar to the boot process of a bare metal operating system with a partitioned disk and bootloader.

  • PV-GRUB versions 1.03 and earlier don't support GPT partitioning; they support MBR partitioning only.

  • If you plan to use a logical volume manager (LVM) with Amazon EBS volumes, you need a separate boot partition outside of the LVM. Then you can create logical volumes with the LVM.

Configuring GRUB for Paravirtual AMIs

To boot PV-GRUB, a GRUB menu.lst file must exist in the image; the most common location for this file is /boot/grub/menu.lst.

The following is an example of a menu.lst configuration file for booting an AMI with a PV-GRUB AKI. In this example, there are two kernel entries to choose from: Amazon Linux 2016.03 (the original kernel for this AMI), and Vanilla Linux 4.7.4 (a newer version of the Vanilla Linux kernel from https://www.kernel.org/). The Vanilla entry was copied from the original entry for this AMI, and the kernel and initrd paths were updated to the new locations. The default 0 parameter points the bootloader to the first entry it sees (in this case, the Vanilla entry), and the fallback 1 parameter points the bootloader to the next entry if there is a problem booting the first.

default 0
fallback 1
timeout 0
hiddenmenu

title Vanilla Linux 4.7.4
root (hd0)
kernel /boot/vmlinuz-4.7.4 root=LABEL=/ console=hvc0
initrd /boot/initrd.img-4.7.4

title Amazon Linux 2016.03 (4.4.11-23.53.amzn1.x86_64)
root (hd0)
kernel /boot/vmlinuz-4.4.11-23.53.amzn1.x86_64 root=LABEL=/ console=hvc0
initrd /boot/initramfs-4.4.11-23.53.amzn1.x86_64.img

You don't need to specify a fallback kernel in your menu.lst file, but we recommend that you have a fallback when you test a new kernel. PV-GRUB can fall back to another kernel in the event that the new kernel fails. Having a fallback kernel allows the instance to boot even if the new kernel isn't found.

PV-GRUB checks the following locations for menu.lst, using the first one it finds:

  • (hd0)/boot/grub

  • (hd0,0)/boot/grub

  • (hd0,0)/grub

  • (hd0,1)/boot/grub

  • (hd0,1)/grub

  • (hd0,2)/boot/grub

  • (hd0,2)/grub

  • (hd0,3)/boot/grub

  • (hd0,3)/grub

Note that PV-GRUB 1.03 and earlier only check one of the first two locations in this list.

Amazon PV-GRUB Kernel Image IDs

PV-GRUB AKIs are available in all Amazon EC2 regions. There are AKIs for both 32-bit and 64-bit architecture types. Most modern AMIs use a PV-GRUB AKI by default.

We recommend that you always use the latest version of the PV-GRUB AKI, as not all versions of the PV-GRUB AKI are compatible with all instance types. Use the following describe-images command to get a list of the PV-GRUB AKIs for the current region:

$ aws ec2 describe-images --owners amazon --filters Name=name,Values=pv-grub-*.gz

Note that PV-GRUB is the only AKI available in the ap-southeast-2 region. You should verify that any AMI you want to copy to this region is using a version of PV-GRUB that is available in this region.

The following are the current AKI IDs for each region. Register new AMIs using an hd0 AKI.

Note

We continue to provide hd00 AKIs for backward compatibility in regions where they were previously available.

ap-northeast-1, Asia Pacific (Tokyo)

Image IDImage Name
aki-136bf512 pv-grub-hd0_1.04-i386.gz
aki-176bf516 pv-grub-hd0_1.04-x86_64.gz

ap-northeast-2, Asia Pacific (Seoul)

Image IDImage Name
aki-01a66b6f pv-grub-hd0_1.04-x86_64.gz

ap-southeast-1, Asia Pacific (Singapore) Region

Image IDImage Name
aki-ae3973fc pv-grub-hd0_1.04-i386.gz
aki-503e7402 pv-grub-hd0_1.04-x86_64.gz

ap-southeast-2, Asia Pacific (Sydney)

Image IDImage Name
aki-cd62fff7 pv-grub-hd0_1.04-i386.gz
aki-c362fff9 pv-grub-hd0_1.04-x86_64.gz

ap-south-1, Asia Pacific (Mumbai)

Image IDImage Name
aki-a7305ac8 pv-grub-hd0_1.04-x86_64.gz

eu-central-1, EU (Frankfurt)

Image IDImage Name
aki-3e4c7a23 pv-grub-hd0_1.04-i386.gz
aki-184c7a05 pv-grub-hd0_1.04-x86_64.gz

eu-west-1, EU (Ireland)

Image IDImage Name
aki-68a3451f pv-grub-hd0_1.04-i386.gz
aki-52a34525 pv-grub-hd0_1.04-x86_64.gz

sa-east-1, South America (São Paulo)

Image IDImage Name
aki-5b53f446 pv-grub-hd0_1.04-i386.gz
aki-5553f448 pv-grub-hd0_1.04-x86_64.gz

us-east-1, US East (N. Virginia)

Image IDImage Name
aki-8f9dcae6 pv-grub-hd0_1.04-i386.gz
aki-919dcaf8 pv-grub-hd0_1.04-x86_64.gz

us-gov-west-1, AWS GovCloud (US)

Image IDImage Name
aki-1fe98d3c pv-grub-hd0_1.04-i386.gz
aki-1de98d3e pv-grub-hd0_1.04-x86_64.gz

us-west-1, US West (N. California)

Image IDImage Name
aki-8e0531cb pv-grub-hd0_1.04-i386.gz
aki-880531cd pv-grub-hd0_1.04-x86_64.gz

us-west-2, US West (Oregon)

Image IDImage Name
aki-f08f11c0 pv-grub-hd0_1.04-i386.gz
aki-fc8f11cc pv-grub-hd0_1.04-x86_64.gz

Updating PV-GRUB

We recommend that you always use the latest version of the PV-GRUB AKI, as not all versions of the PV-GRUB AKI are compatible with all instance types. Also, older versions of PV-GRUB are not available in all regions, so if you copy an AMI that uses an older version to a region that does not support that version, you will be unable to boot instances launched from that AMI until you update the kernel image. Use the following procedures to check your instance's version of PV-GRUB and update it if necessary.

To check your PV-GRUB version

  1. Find the kernel ID for your instance.

    $ ec2-describe-instance-attribute instance_id --kernel --region region
    kernel	instance_id	aki-fc8f11cc

    The kernel ID for this instance is aki-fc8f11cc.

  2. View the version information of that kernel ID.

    $ ec2-describe-images aki-fc8f11cc --region region
    IMAGE	aki-fc8f11cc	amazon/pv-grub-hd0_1.04-x86_64.gz ...

    This kernel image is PV-GRUB 1.04. If your PV-GRUB version is not the newest version (as shown in Amazon PV-GRUB Kernel Image IDs), you should update it using the following procedure.

To update your PV-GRUB version

If your instance is using an older version of PV-GRUB, you should update it to the latest version.

  1. Identify the latest PV-GRUB AKI for your region and processor architecture from Amazon PV-GRUB Kernel Image IDs.

  2. Stop your instance. Your instance must be stopped to modify the kernel image used.

    $ ec2-stop-instances instance_id --region region
    INSTANCE	instance_id	stopped	stopped
  3. Modify the kernel image used for your instance.

    $ ec2-modify-instance-attribute --kernel kernel_id --region region instance_id
  4. Restart your instance.

    $ ec2-start-instances --region region instance_id