Amazon Elastic Compute Cloud
User Guide for Linux (API Version 2014-10-01)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Enabling Enhanced Networking on Linux Instances in a VPC

With C3, R3, and I2 instances, you can enable enhanced networking capabilities. Amazon EC2 supports enhanced networking capabilities using single root I/O virtualization (SR-IOV). Enabling enhanced networking on your instance results in higher performance (packets per second), lower latency, and lower jitter.

To enable enhanced networking on your instance, you must ensure that its kernel has the ixgbevf module installed and that you set the sriovNetSupport attribute for the instance. For the best performance, we recommend that the ixgbevf module is version 2.14.2 or higher.

Important

The latest Amazon Linux HVM AMIs have the module required for enhanced networking installed and have the required attribute set. Therefore, if you launch an Amazon EBS–backed C3, R3, or I2 instance using a current Amazon Linux HVM AMI, enhanced networking is already enabled for your instance.

Note that you can get directions for Windows from Enabling Enhanced Networking on Windows Instances in a VPC in the Amazon EC2 User Guide for Microsoft Windows Instances.

Requirements

Before enabling enhanced networking, make sure you do the following:

  • Launch the instance from an HVM AMI with a minimum Linux kernel version of 2.6.32.

  • Launch the instance using one of the following instance types: c3.large, c3.xlarge, c3.2xlarge, c3.4xlarge, c3.8xlarge, i2.xlarge, i2.2xlarge, i2.4xlarge, i2.8xlarge, r3.large, r3.xlarge, r3.2xlarge, r3.4xlarge, or r3.8xlarge. For more information about instance types, see Amazon EC2 Instances.

  • Launch the instance in a VPC. (You can't enable enhanced networking if the instance is in EC2-Classic.)

  • Install and configure either the AWS CLI or Amazon EC2 CLI tools to any computer you choose, preferably your local desktop or laptop. For more information, see Accessing Amazon EC2. If you choose the Amazon EC2 CLI tools, install version 1.6.12.0 or later. You can use the ec2-version command to verify the version of your CLI tools.

  • If you have important data on the instance that you want to preserve, you should back that data up now by creating a snapshot for Amazon EBS-backed instances or creating an AMI for instance store-backed instances. Updating kernels and kernel modules as well as enabling the sriovNetSupport attribute may make incompatible instances or operating systems unreachable; if you have a recent backup, your data will still be retained if this happens.

Testing Whether Enhanced Networking Is Enabled

To test whether enhanced networking is already enabled, verify that the ixgbevf module is installed on your instance and that the sriovNetSupport attribute is set. If your instance satisfies these two conditions, then the ethtool -i ethn command should show that the module is in use on the network interface.

Kernel Module (ixgbevf)

To verify that the ixgbevf module is installed and that the version is compatible with enhanced networking, use the modinfo command as follows:

[ec2-user ~]$ modinfo ixgbevf
filename:       /lib/modules/3.10.48-55.140.amzn1.x86_64/kernel/drivers/amazon/ixgbevf/ixgbevf.ko
version:        2.14.2
license:        GPL
description:    Intel(R) 82599 Virtual Function Driver
author:         Intel Corporation, <linux.nics@intel.com>
srcversion:     50CBF6F36B99FE70E56C95A
alias:          pci:v00008086d00001515sv*sd*bc*sc*i*
alias:          pci:v00008086d000010EDsv*sd*bc*sc*i*
depends:
intree:         Y
vermagic:       3.10.48-55.140.amzn1.x86_64 SMP mod_unload modversions
parm:           InterruptThrottleRate:Maximum interrupts per second, per vector, (956-488281, 0=off, 1=dynamic), default 1 (array of int)

In the above Amazon Linux case, the ixgbevf module is already installed and it is at the minimum recommended version (2.14.2).

ubuntu:~$ modinfo ixgbevf
filename:       /lib/modules/3.13.0-29-generic/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko
version:        2.11.3-k
license:        GPL
description:    Intel(R) 82599 Virtual Function Driver
author:         Intel Corporation, <linux.nics@intel.com>
srcversion:     0816EA811025C8062A9C269
alias:          pci:v00008086d00001515sv*sd*bc*sc*i*
alias:          pci:v00008086d000010EDsv*sd*bc*sc*i*
depends:
intree:         Y
vermagic:       3.13.0-29-generic SMP mod_unload modversions
signer:         Magrathea: Glacier signing key
sig_key:        66:02:CB:36:F1:31:3B:EA:01:C4:BD:A9:65:67:CF:A7:23:C9:70:D8
sig_hashalgo:   sha512
parm:           debug:Debug level (0=none,...,16=all) (int)

In the above Ubuntu instance, the module is installed, but the version is 2.11.3-k, which does not have all of the latest bug fixes that the recommended version 2.14.2 does. In this case, the ixgevf module would work, but a newer version can still be installed and loaded on the instance for the best experience.

Instance Attribute (sriovNetSupport)

To check whether an instance has the enhanced networking attribute set, use one of the following commands:

  • describe-instance-attribute (AWS CLI)

    $ aws ec2 describe-instance-attribute --instance-id instance_id --attribute sriovNetSupport

    If the enhanced networking attribute isn't set, SriovNetSupport is empty. Otherwise, it is set as follows:

    "SriovNetSupport": {
        "Value": "simple"
    },
  • ec2-describe-instance-attribute (Amazon EC2 CLI)

    $ ec2-describe-instance-attribute instance_id --sriov

    If the enhanced networking attribute isn't set, you'll see no output for this command. Otherwise, the output is as follows:

    sriovNetSupport instance_id   simple

Image Attribute (sriovNetSupport)

To check whether an AMI already has the enhanced networking attribute set, use one of the following commands:

  • describe-image-attribute (AWS CLI)

    $ aws ec2 describe-image-attribute --image-id ami_id --attribute sriovNetSupport

    Note

    This command only works for images that you own. You receive an AuthFailure error for images that do not belong to your account.

    If the enhanced networking attribute isn't set, SriovNetSupport is empty. Otherwise, it is set as follows:

    "SriovNetSupport": {
        "Value": "simple"
    },
  • ec2-describe-image-attribute (Amazon EC2 CLI)

    $ ec2-describe-image-attribute ami_id --sriov

    Note

    This command only works for images that you own. You will receive an AuthFailure error for images that do not belong to your account.

    If the enhanced networking attribute isn't set, you'll see no output for this command. Otherwise, the output is as follows:

    sriovNetSupport ami_id        simple

Network Interface Driver

Use the following command to verify that the module is being used on a particular interface, substituting the interface name that you wish to check. If you are using a single interface (default), it will be eth0.

[ec2-user ~]$ ethtool -i eth0
driver: vif
version:
firmware-version:
bus-info: vif-0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

In the above case, the ixgbevf module is not loaded, because the listed driver is vif.

[ec2-user ~]$ ethtool -i eth0
driver: ixgbevf
version: 2.14.2
firmware-version: N/A
bus-info: 0000:00:03.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

In this case, the ixgbevf module is loaded and at the minimum recommended version. This instance has enhanced networking properly configured.

Enabling Enhanced Networking on Amazon Linux

The latest Amazon Linux HVM AMIs have the module required for enhanced networking installed and have the required attribute set. Therefore, if you launch an Amazon EBS–backed C3, R3, or I2 instance using a current Amazon Linux HVM AMI, enhanced networking is already enabled for your instance. For more information, see Testing Whether Enhanced Networking Is Enabled.

If you launched your instance using an older Amazon Linux AMI and it does not have enhanced networking enabled already, use the following procedure to enable enhanced networking.

To enable enhanced networking (EBS-backed instances)

  1. Connect to your instance.

  2. From the instance, run the following command to update your instance with the newest kernel and kernel modules, including ixgbevf:

    [ec2-user ~]$ sudo yum update
  3. From your local computer, reboot your instance using the Amazon EC2 console or one of the following commands: reboot-instances (AWS CLI) or ec2-reboot-instances (Amazon EC2 CLI).

  4. Connect to your instance again and verify that the ixgbevf module is installed and at the minimum recommended version using the modinfo ixgbevf command from Testing Whether Enhanced Networking Is Enabled.

  5. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: stop-instances (AWS CLI) or ec2-stop-instances (Amazon EC2 CLI).

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enable enhanced networking (instance store-backed instances).

  6. From your local computer, enable the enhanced networking attribute using one of the following commands.

    Warning

    There is no way to disable the enhanced networking attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  7. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch another C3, R3, or I2 instance with the enhanced networking enabled by default.

  8. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: start-instances (AWS CLI) or ec2-start-instances (Amazon EC2 CLI).

  9. Connect to your instance and verify that the ixgbevf module is installed and loaded on your network interface using the ethtool -i ethn command from Testing Whether Enhanced Networking Is Enabled.

To enable enhanced networking (instance store-backed instances)

If your instance is an instance store–backed instance, follow Step 1 through Step 4 in the previous procedure, and then create a new AMI as described in Creating an Instance Store-Backed Linux AMI. Be sure to enable the enhanced networking attribute when you register the AMI.

  • register-image (AWS CLI)

    $ aws ec2 register-image --sriov-net-support simple ...
  • ec2-register (Amazon EC2 CLI)

    $ ec2-register --sriov simple ...

Enabling Enhanced Networking on Ubuntu

The following procedure provides the general steps that you'll take when enabling enhanced networking on an Ubuntu instance.

To enable enhanced networking on Ubuntu (EBS-backed instances)

  1. Connect to your instance.

  2. Update the package cache and packages.

    ubuntu:~$ sudo apt-get update && sudo apt-get upgrade -y

    Important

    If during the update process you are prompted to install grub, use /dev/xvda to install grub onto, and then choose to keep the current version of /boot/grub/menu.lst.

  3. Install the dkms package so that your ixgbevf module is rebuilt every time your kernel is updated.

    ubuntu:~$ sudo apt-get install -y dkms
  4. Download the latest source for the ixgbevf module on your instance from Sourceforge at http://sourceforge.net/projects/e1000/files/ixgbevf%20stable/. At the time of this writing, that version was 2.14.2. If the latest version available is not 2.14.2, substitute your version of ixgbevf in the following commands.

    ubuntu:~$ wget "sourceforge.net/projects/e1000/files/ixgbevf stable/2.14.2/ixgbevf-2.14.2.tar.gz"
  5. Decompress and unarchive the ixgbevf package.

    ubuntu:~$ tar -xzf ixgbevf-2.14.2.tar.gz
  6. Move the ixgbevf package to the /usr/src/ directory so dkms can find it and build it for each kernel update.

    ubuntu:~$ sudo mv ixgbevf-2.14.2 /usr/src/
  7. Create the dkms configuration file with the following values, substituting your version of ixgbevf.

    1. Create the file.

      ubuntu:~$ sudo touch /usr/src/ixgbevf-2.14.2/dkms.conf
    2. Edit the file and add the following values.

      ubuntu:~$ sudo vim /usr/src/ixgbevf-2.14.2/dkms.conf
      PACKAGE_NAME="ixgbevf"
      PACKAGE_VERSION="2.14.2"
      CLEAN="cd src/; make clean"
      MAKE="cd src/; make BUILD_KERNEL=${kernelver}"
      BUILT_MODULE_LOCATION[0]="src/"
      BUILT_MODULE_NAME[0]="ixgbevf"
      DEST_MODULE_LOCATION[0]="/updates"
      DEST_MODULE_NAME[0]="ixgbevf"
      AUTOINSTALL="yes"
  8. Add, build, and install the ixgbevf module on your instance with dkms.

    1. Add the module to dkms.

      ubuntu:~$ sudo dkms add -m ixgbevf -v 2.14.2
    2. Build the module with dkms.

      ubuntu:~$ sudo dkms build -m ixgbevf -v 2.14.2
    3. Install the module with dkms.

      ubuntu:~$ sudo dkms install -m ixgbevf -v 2.14.2
  9. Rebuild the initramfs so the correct module is loaded at boot time.

    ubuntu:~$ sudo update-initramfs -c -k all
  10. Verify that the ixgbevf module is installed and at the minimum recommended version using the modinfo ixgbevf command from Testing Whether Enhanced Networking Is Enabled.

    ubuntu:~$ modinfo ixgbevf
    filename:       /lib/modules/3.13.0-29-generic/updates/dkms/ixgbevf.ko
    version:        2.14.2
    license:        GPL
    description:    Intel(R) 82599 Virtual Function Driver
    author:         Intel Corporation, <linux.nics@intel.com>
    srcversion:     50CBF6F36B99FE70E56C95A
    alias:          pci:v00008086d00001515sv*sd*bc*sc*i*
    alias:          pci:v00008086d000010EDsv*sd*bc*sc*i*
    depends:
    vermagic:       3.13.0-29-generic SMP mod_unload modversions
    parm:           InterruptThrottleRate:Maximum interrupts per second, per vector, (956-488281, 0=off, 1=dynamic), default 1 (array of int)
  11. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: stop-instances (AWS CLI) or ec2-stop-instances (Amazon EC2 CLI).

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enable enhanced networking on Ubuntu (instance store-backed instances).

  12. From your local computer, enable the enhanced networking attribute using one of the following commands. Note that there is no way to disable the networking attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  13. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch a C3, R3, or I2 instance with the enhanced networking enabled by default.

  14. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: start-instances (AWS CLI) or ec2-start-instances (Amazon EC2 CLI).

  15. (Optional) Connect to your instance and verify that the module is installed.

To enable enhanced networking on Ubuntu (instance store-backed instances)

If your instance is an instance store-backed instance, follow Step 1 through Step 10 in the previous procedure, and then create a new AMI as described in Creating an Instance Store-Backed Linux AMI. Be sure to enable the enhanced networking attribute when you register the AMI.

Warning

Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  • register-image (AWS CLI)

    $ aws ec2 register-image --sriov-net-support simple ...
  • ec2-register (Amazon EC2 CLI)

    $ ec2-register --sriov simple ...

Enabling Enhanced Networking on Other Linux Distributions

The following procedure provides the general steps that you'll take when enabling enhanced networking on a Linux distribution other than Amazon Linux or Ubuntu. For more information, such as detailed syntax for commands, file locations, or package and tool support, see the specific documentation for your Linux distribution.

To enable enhanced networking on Linux (EBS-backed instances)

  1. Connect to your instance.

  2. Download the latest source for the ixgbevf module on your instance from Sourceforge at http://sourceforge.net/projects/e1000/files/ixgbevf%20stable/. At the time of this writing, that version was 2.14.2; this is the minimum version recommended for enhanced networking.

  3. Compile and install the ixgbevf module on your instance.

    If your distribution supports dkms, then you should consider configuring dkms to recompile the ixgbevf module whenever your system's kernel is updated. If your distribution does not support dkms natively, you can find it in the EPEL repository (https://fedoraproject.org/wiki/EPEL) for Red Hat Enterprise Linux variants, or you can download the software at http://linux.dell.com/dkms/. Use Step 6 through Step 8 in To enable enhanced networking on Ubuntu (EBS-backed instances) for help configuring dkms.

    Warning

    If you compile the ixgbevf module for your current kernel and then upgrade your kernel without rebuilding the driver for the new kernel, your system may revert to the distribution-specific ixgbevf module at the next reboot, which could make your system unreachable if the distribution-specific version is incompatible with enhanced networking.

  4. Run the sudo depmod command to update module dependencies.

  5. Update the initramfs on your instance to ensure that the new module loads at boot time.

  6. Determine if your system uses predictable network interface names by default. Systems that use systemd or udev versions 197 or greater can rename Ethernet devices and they do not guarantee that a single network interface will be named eth0. This behavior can cause problems connecting to your instance. For more information on predictable network interface names, and to see other configuration options, go to http://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/.

    1. You can check the systemd or udev versions on RPM-based systems with the following command:

      [ec2-user ~]$ rpm -qa | grep -e '^systemd-[0-9]\+\|^udev-[0-9]\+'
      systemd-208-11.el7_0.2.x86_64

      In the above Red Hat 7 example, the systemd version is 208, so predictable network interface names must be disabled.

    2. Disable predictable network interface names by adding the net.ifnames=0 option to the GRUB_CMDLINE_LINUX line in /etc/default/grub.

      [ec2-user ~]$ sudo sed -i '/^GRUB\_CMDLINE\_LINUX/s/\"$/\ net\.ifnames\=0\"/' /etc/default/grub
    3. Rebuild the grub configuration file.

      [ec2-user ~]$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
  7. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: stop-instances (AWS CLI) or ec2-stop-instances (Amazon EC2 CLI).

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enabled enhanced networking (instance store–backed instances)

  8. From your local computer, enable the enhanced networking attribute using one of the following commands. Note that there is no way to disable the networking attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  9. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch a C3, R3, or I2 instance with the enhanced networking enabled by default.

  10. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: start-instances (AWS CLI) or ec2-start-instances (Amazon EC2 CLI).

  11. (Optional) Connect to your instance and verify that the module is installed.

To enabled enhanced networking (instance store–backed instances)

If your instance is an instance store–backed instance, follow Step 1 through Step 5 in the previous procedure, and then create a new AMI as described in Creating an Instance Store-Backed Linux AMI. Be sure to enable the enhanced networking attribute when you register the AMI.

Warning

Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  • register-image (AWS CLI)

    $ aws ec2 register-image --sriov-net-support simple ...
  • ec2-register (Amazon EC2 CLI)

    $ ec2-register --sriov simple ...

Troubleshooting Connectivity Issues

If you lose connectivity while enabling enhanced networking, the ixgbevf module might be incompatible with the kernel. Try installing the version of the ixgbevf module included with the distribution of Linux for your instance.

If you enable enhanced networking for a PV instance or AMI, this can make your instance unreachable.