Enabling Your Own Linux Kernels

To enable user-provided kernels, Amazon has published Amazon Kernel Images (AKIs) that use a system called PV-GRUB. PV-GRUB is a paravirtual “mini-OS” that runs a version of GNU GRUB, the standard Linux boot loader. PV-GRUB selects the kernel to boot by reading /boot/grub/menu.lst from your image. It will load the kernel specified by your image and then shut down the “mini-OS,” so that it no longer consumes any resources. One of the advantages of this solution is that PV-GRUB understands standard grub.conf or menu.lst commands, which allows it to work with most existing Linux distributions.

The following task list describes what you need to do to enable an AMI to use PV-GRUB AKI to run a user-provided kernel.

default 0
timeout 3
fallback 1

title Vanilla EC2 Kernel 2.6.32.10
	root (hd0)
	kernel /boot/vmlinux-2.6.32.10-ACME_SYS_EC2 root=/dev/sda1
	initrd /boot/initrd-2.6.32.10-ACME_SYS_EC2

title Ubuntu EC2 2.6.32.302-EC
	root (hd0)
	kernel /boot/ubuntu-ec2 root=/dev/sda1
	initrd /boot/initrd-ec2 

The following information is provided to assist those who are familiar with compiling kernels. Many Linux distributions provide documentation on how to compile a kernel in kernel source packages. Amazon is unable to offer support for compiling your own kernel.

It is recommended that you use a pyops kernel, release 2.6.39 or later. Release 2.6.39 or later, with pyops and Xen options enabled, will work out of the box.

If you are working with 2.6.32 through 2.6.38, boot with the XSAVE instruction in the guest as shown below.

openSUSE

The openSUSE distribution kernel provides native source code that enables compiling an Amazon EC2-capable kernel. For openSUSE, the configuration options are part of the mainline 11.2 and higher. Please use the following kernel configuration options:

Patching vanilla or other distribution kernels (PVOps)

For PVOps kernels, you can elect to disable the XSAVE hypercall in the guest. The following patch works against 2.6.32 through 2.6.38 kernels.

---
 arch/x86/xen/enlighten.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 52f8e19..6db3d67 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -802,6 +802,7 @@ static void xen_write_cr4(unsigned long cr4)
 {
 	cr4 &= ~X86_CR4_PGE;
 	cr4 &= ~X86_CR4_PSE;
+	cr4 &= ~X86_CR4_OSXSAVE;
 
 	native_write_cr4(cr4);
 }
-- 
1.6.6.1

Q: What are user selectable kernels?

Amazon EC2 provides user selectable kernels which enables you to select a kernel when bundling an AMI or launching an instance. User selectable kernels are useful for keeping your instances up to date with security fixes and updates, being able to use functionality provided by new distributions, and for using specialty applications that have unique timing requirements.

Q: How do I find user selectable kernels?

Use ec2-describe-images -o amazon --filter "image-type=kernel". This lists all public kernels that are currently available.

Q: What type of dependencies do kernels have?

Kernels are most likely to require a RAM disk that contains required drivers (for example, Xen drivers, video drivers, and so on). If you launch a kernel without a required RAM disk, it will not work properly.

Q: How do I know a kernel/AMI combination will work together?

If you are concerned about whether the kernel/image combination will work well together, Amazon provides several AMIs that have tested combinations that you can use as a starting point for your AMIs or AMIs that you can use as a foundation for a public AMIs. If you require a certified kernel/ AMI combination, you can find them as paid AMIs through organizations such as RedHat. For more information, see Paid AMIs.

Q: Can I use my own kernel?

Yes. For more information, see Enabling Your Own Linux Kernels.

Q: How do I know which kernels are compatible with PV-GRUB?

For a list of compatible PV-GRUB kernels, see Distribution Kernels Compatible with EC2. Some Linux distributions provide kernels that are not compatible with Amazon EC2. We are working with vendors to ensure that the most popular AMIs provide kernels that work with Amazon EC2, and we have tested a number of these AMIs and found them to be compatible with PV-GRUB. Unfortunately, it is not possible to support every kernel that is or can be compiled. To avoid the situation in which a kernel does not work consistently or at all, we recommend that you use a known good kernel, select a non-PV-GRUB AKI, or seek support from your AMI vendor.

Q: In what runlevel do instances start?

If you use one of the Xen provided kernels to boot your EC2 instance it will default to run level 4. However if you use PV-GRUB to boot your own kernel inside of the instance, the instance will then default to the internally configured run level.

Q: Which PV-GRUB kernel should I use?

It is recommended that you always use the latest version of the PV-GRUB AKI, as not all versions of the PV-GRUB AKI will be compatible with all instance types. Use the following command to get a list of the PV-GRUB AKIs:

$ ec2-describe-images -o amazon --filter "manifest-location=*pv-grub-hd0*"