Menu
Amazon Elastic Compute Cloud
User Guide for Linux Instances

Enabling Enhanced Networking with the Intel 82599 VF Interface on Linux Instances in a VPC

Amazon EC2 provides enhanced networking capabilities to C3, C4, D2, I2, R3, and M4 (excluding m4.16xlarge) instances with the Intel 82599 VF interface, which uses the Intel ixgbevf driver.

To prepare for enhanced networking with the Intel 82599 VF interface, set up your instance as follows:

  • Launch the instance from an HVM AMI using Linux kernel version of 2.6.32 or later. The latest Amazon Linux HVM AMIs have the modules required for enhanced networking installed and have the required attributes set. Therefore, if you launch an Amazon EBS–backed, enhanced networking–supported instance using a current Amazon Linux HVM AMI, enhanced networking is already enabled for your instance.

  • Launch the instance in a VPC. (You can't enable enhanced networking if the instance is in EC2-Classic.)

  • Install and configure the AWS CLI or the AWS Tools for Windows PowerShell on any computer you choose, preferably your local desktop or laptop. For more information, see Accessing Amazon EC2. Enhanced networking cannot be managed from the Amazon EC2 console.

  • If you have important data on the instance that you want to preserve, you should back that data up now by creating an AMI from your instance. Updating kernels and kernel modules, as well as enabling the sriovNetSupport attribute, may render incompatible instances or operating systems unreachable; if you have a recent backup, your data will still be retained if this happens.

Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled

To test whether enhanced networking with the Intel 82599 VF interface is already enabled, verify that the ixgbevf module is installed on your instance and that the sriovNetSupport attribute is set. If your instance satisfies these two conditions, then the ethtool -i ethn command should show that the module is in use on the network interface.

Kernel Module (ixgbevf)

To verify that the ixgbevf module is installed and that the version is compatible with enhanced networking, use the modinfo command as follows:

Copy
[ec2-user ~]$ modinfo ixgbevf filename: /lib/modules/3.10.48-55.140.amzn1.x86_64/kernel/drivers/amazon/ixgbevf/ixgbevf.ko version: 2.14.2 license: GPL description: Intel(R) 82599 Virtual Function Driver author: Intel Corporation, <linux.nics@intel.com> srcversion: 50CBF6F36B99FE70E56C95A alias: pci:v00008086d00001515sv*sd*bc*sc*i* alias: pci:v00008086d000010EDsv*sd*bc*sc*i* depends: intree: Y vermagic: 3.10.48-55.140.amzn1.x86_64 SMP mod_unload modversions parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (956-488281, 0=off, 1=dynamic), default 1 (array of int)

In the above Amazon Linux case, the ixgbevf module is already installed and it is at the minimum recommended version (2.14.2).

Copy
ubuntu:~$ modinfo ixgbevf filename: /lib/modules/3.13.0-29-generic/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko version: 2.11.3-k license: GPL description: Intel(R) 82599 Virtual Function Driver author: Intel Corporation, <linux.nics@intel.com> srcversion: 0816EA811025C8062A9C269 alias: pci:v00008086d00001515sv*sd*bc*sc*i* alias: pci:v00008086d000010EDsv*sd*bc*sc*i* depends: intree: Y vermagic: 3.13.0-29-generic SMP mod_unload modversions signer: Magrathea: Glacier signing key sig_key: 66:02:CB:36:F1:31:3B:EA:01:C4:BD:A9:65:67:CF:A7:23:C9:70:D8 sig_hashalgo: sha512 parm: debug:Debug level (0=none,...,16=all) (int)

In the above Ubuntu instance, the module is installed, but the version is 2.11.3-k, which does not have all of the latest bug fixes that the recommended version 2.14.2 does. In this case, the ixgbevf module would work, but a newer version can still be installed and loaded on the instance for the best experience.

Instance Attribute (sriovNetSupport)

To check whether an instance has the enhanced networking sriovNetSupport attribute set, use one of the following commands:

  • describe-instance-attribute (AWS CLI)

    Copy
    aws ec2 describe-instance-attribute --instance-id instance_id --attribute sriovNetSupport
  • Get-EC2InstanceAttribute (AWS Tools for Windows PowerShell)

    Copy
    Get-EC2InstanceAttribute -InstanceId instance-id -Attribute sriovNetSupport

If the attribute isn't set, SriovNetSupport is empty; otherwise, it is set as follows:

"SriovNetSupport": {
    "Value": "simple"
},

Image Attribute (sriovNetSupport)

To check whether an AMI already has the enhanced networking sriovNetSupport attribute set, use one of the following commands:

  • describe-image-attribute (AWS CLI)

    Copy
    aws ec2 describe-image-attribute --image-id ami_id --attribute sriovNetSupport

    Note that this command only works for images that you own. You receive an AuthFailure error for images that do not belong to your account.

  • Get-EC2ImageAttribute (AWS Tools for Windows PowerShell)

    Copy
    Get-EC2ImageAttribute -ImageId ami-id -Attribute sriovNetSupport

If the attribute isn't set, SriovNetSupport is empty; otherwise, it is set as follows:

"SriovNetSupport": {
    "Value": "simple"
},

Network Interface Driver

Use the following command to verify that the module is being used on a particular interface, substituting the interface name that you wish to check. If you are using a single interface (default), it will be eth0.

Copy
[ec2-user ~]$ ethtool -i eth0 driver: vif version: firmware-version: bus-info: vif-0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no

In the above case, the ixgbevf module is not loaded, because the listed driver is vif.

Copy
[ec2-user ~]$ ethtool -i eth0 driver: ixgbevf version: 2.14.2 firmware-version: N/A bus-info: 0000:00:03.0 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: yes supports-priv-flags: no

In this case, the ixgbevf module is loaded and at the minimum recommended version. This instance has enhanced networking properly configured.

Enabling Enhanced Networking with the Intel 82599 VF Interface on Amazon Linux

The latest Amazon Linux HVM AMIs have the ixgbevf module required for enhanced networking installed and have the required sriovNetSupport attribute set. Therefore, if you launch a C3, C4, D2, I2, R3, or M4 (excluding m4.16xlarge) instance using a current Amazon Linux HVM AMI, enhanced networking is already enabled for your instance. For more information, see Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

If you launched your instance using an older Amazon Linux AMI and it does not have enhanced networking enabled already, use the following procedure to enable enhanced networking.

To enable enhanced networking (EBS-backed instances)

  1. Connect to your instance.

  2. From the instance, run the following command to update your instance with the newest kernel and kernel modules, including ixgbevf:

    Copy
    [ec2-user ~]$ sudo yum update
  3. From your local computer, reboot your instance using the Amazon EC2 console or one of the following commands: reboot-instances (AWS CLI), Restart-EC2Instance (AWS Tools for Windows PowerShell).

  4. Connect to your instance again and verify that the ixgbevf module is installed and at the minimum recommended version using the modinfo ixgbevf command from Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

  5. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: stop-instances (AWS CLI), Stop-EC2Instance (AWS Tools for Windows PowerShell). If your instance is managed by AWS OpsWorks, you should stop the instance in the AWS OpsWorks console so that the instance state remains in sync.

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enable enhanced networking (instance store-backed instances).

  6. From your local computer, enable the enhanced networking attribute using one of the following commands.

    Warning

    There is no way to disable the enhanced networking attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

    • modify-instance-attribute (AWS CLI)

      Copy
      aws ec2 modify-instance-attribute --instance-id instance_id --sriov-net-support simple
    • Edit-EC2InstanceAttribute (AWS Tools for Windows PowerShell)

      Copy
      Edit-EC2InstanceAttribute -InstanceId instance_id -SriovNetSupport "simple"
  7. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.

  8. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: start-instances (AWS CLI), Start-EC2Instance (AWS Tools for Windows PowerShell). If your instance is managed by AWS OpsWorks, you should start the instance in the AWS OpsWorks console so that the instance state remains in sync.

  9. Connect to your instance and verify that the ixgbevf module is installed and loaded on your network interface using the ethtool -i ethn command from Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

To enable enhanced networking (instance store-backed instances)

If your instance is an instance store–backed instance, follow Step 1 through Step 4 in the previous procedure, and then create a new AMI as described in Creating an Instance Store-Backed Linux AMI. Be sure to enable the enhanced networking attribute when you register the AMI.

  • register-image (AWS CLI)

    Copy
    aws ec2 register-image --sriov-net-support simple ...
  • Register-EC2Image (AWS Tools for Windows PowerShell)

    Copy
    Register-EC2Image -SriovNetSupport "simple" ...

Enabling Enhanced Networking with the Intel 82599 VF Interface on Ubuntu

The following procedure provides the general steps that you'll take when enabling enhanced networking with the Intel 82599 VF interface on an Ubuntu instance.

To enable enhanced networking on Ubuntu (EBS-backed instances)

  1. Connect to your instance.

  2. Update the package cache and packages.

    Copy
    ubuntu:~$ sudo apt-get update && sudo apt-get upgrade -y

    Important

    If during the update process, you are prompted to install grub, use /dev/xvda to install grub onto, and then choose to keep the current version of /boot/grub/menu.lst.

  3. Install the dkms package so that your ixgbevf module is rebuilt every time your kernel is updated.

    Copy
    ubuntu:~$ sudo apt-get install -y dkms
  4. Download the source for version 2.16.4 of the ixgbevf module on your instance from Sourceforge at http://sourceforge.net/projects/e1000/files/ixgbevf%20stable/.

    Note that earlier versions of ixgbevf, including the minimum recommended version, 2.14.2, do not build properly on some versions of Ubuntu. The 2.16.4 version of ixgbevf should be used for Ubuntu instances.

    Copy
    ubuntu:~$ wget "sourceforge.net/projects/e1000/files/ixgbevf stable/2.16.4/ixgbevf-2.16.4.tar.gz"
  5. Decompress and unarchive the ixgbevf package.

    Copy
    ubuntu:~$ tar -xzf ixgbevf-2.16.4.tar.gz
  6. Move the ixgbevf package to the /usr/src/ directory so that dkms can find it and build it for each kernel update.

    Copy
    ubuntu:~$ sudo mv ixgbevf-2.16.4 /usr/src/
  7. Create the dkms configuration file with the following values, substituting your version of ixgbevf.

    1. Create the file.

      Copy
      ubuntu:~$ sudo touch /usr/src/ixgbevf-2.16.4/dkms.conf
    2. Edit the file and add the following values.

      Copy
      ubuntu:~$ sudo vim /usr/src/ixgbevf-2.16.4/dkms.conf PACKAGE_NAME="ixgbevf" PACKAGE_VERSION="2.16.4" CLEAN="cd src/; make clean" MAKE="cd src/; make BUILD_KERNEL=${kernelver}" BUILT_MODULE_LOCATION[0]="src/" BUILT_MODULE_NAME[0]="ixgbevf" DEST_MODULE_LOCATION[0]="/updates" DEST_MODULE_NAME[0]="ixgbevf" AUTOINSTALL="yes"
  8. Add, build, and install the ixgbevf module on your instance using dkms.

    1. Add the module to dkms.

      Copy
      ubuntu:~$ sudo dkms add -m ixgbevf -v 2.16.4
    2. Build the module with dkms.

      Copy
      ubuntu:~$ sudo dkms build -m ixgbevf -v 2.16.4
    3. Install the module with dkms.

      Copy
      ubuntu:~$ sudo dkms install -m ixgbevf -v 2.16.4
  9. Rebuild initramfs so the correct module is loaded at boot time.

    Copy
    ubuntu:~$ sudo update-initramfs -c -k all
  10. Verify that the ixgbevf module is installed and at the minimum recommended version using the modinfo ixgbevf command from Testing Whether Enhanced Networking with the Intel 82599 VF Interface is Enabled.

    Copy
    ubuntu:~$ modinfo ixgbevf filename: /lib/modules/3.13.0-74-generic/updates/dkms/ixgbevf.ko version: 2.16.4 license: GPL description: Intel(R) 10 Gigabit Virtual Function Network Driver author: Intel Corporation, <linux.nics@intel.com> srcversion: 759A432E3151C8F9F6EA882 alias: pci:v00008086d00001515sv*sd*bc*sc*i* alias: pci:v00008086d000010EDsv*sd*bc*sc*i* depends: vermagic: 3.13.0-74-generic SMP mod_unload modversions parm: InterruptThrottleRate:Maximum interrupts per second, per vector, (956-488281, 0=off, 1=dynamic), default 1 (array of int)
  11. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: stop-instances (AWS CLI), Stop-EC2Instance (AWS Tools for Windows PowerShell). If your instance is managed by AWS OpsWorks, you should stop the instance in the AWS OpsWorks console so that the instance state remains in sync.

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enable enhanced networking on Ubuntu (instance store-backed instances).

  12. From your local computer, enable the enhanced networking sriovNetSupport attribute using one of the following commands. Note that there is no way to disable this attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

    • modify-instance-attribute (AWS CLI)

      Copy
      aws ec2 modify-instance-attribute --instance-id instance_id --sriov-net-support simple
    • Edit-EC2InstanceAttribute (AWS Tools for Windows PowerShell)

      Copy
      Edit-EC2InstanceAttribute -InstanceId instance_id -SriovNetSupport "simple"
  13. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking sriovNetSupport attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.

  14. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: start-instances (AWS CLI), Start-EC2Instance (AWS Tools for Windows PowerShell). If your instance is managed by AWS OpsWorks, you should start the instance in the AWS OpsWorks console so that the instance state remains in sync.

  15. (Optional) Connect to your instance and verify that the module is installed.

To enable enhanced networking on Ubuntu (instance store-backed instances)

If your instance is an instance store-backed instance, follow Step 1 through Step 10 in the previous procedure, and then create a new AMI as described in Creating an Instance Store-Backed Linux AMI. Be sure to enable the enhanced networking attribute when you register the AMI.

Warning

Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  • register-image (AWS CLI)

    Copy
    aws ec2 register-image --sriov-net-support simple ...
  • Register-EC2Image (AWS Tools for Windows PowerShell)

    Copy
    Register-EC2Image -SriovNetSupport "simple" ...

Enabling Enhanced Networking with the Intel 82599 VF Interface on Other Linux Distributions

The following procedure provides the general steps that you'll take when enabling enhanced networking with the Intel 82599 VF interface on a Linux distribution other than Amazon Linux or Ubuntu. For more information, such as detailed syntax for commands, file locations, or package and tool support, see the specific documentation for your Linux distribution.

To enable enhanced networking on Linux (EBS-backed instances)

  1. Connect to your instance.

  2. Download the source for version 2.14.2 of the ixgbevf module on your instance from Sourceforge at http://sourceforge.net/projects/e1000/files/ixgbevf%20stable/. This is the minimum version recommended for enhanced networking.

    Earlier versions of ixgbevf, including the minimum recommended version, 2.14.2, do not build properly on some Linux distributions, including certain versions of Ubuntu. If you receive build errors, you may try a newer version, such as 2.16.4 (which fixes the build issue on affected Ubuntu versions).

  3. Compile and install the ixgbevf module on your instance.

    If your distribution supports dkms, then you should consider configuring dkms to recompile the ixgbevf module whenever your system's kernel is updated. If your distribution does not support dkms natively, you can find it in the EPEL repository (https://fedoraproject.org/wiki/EPEL) for Red Hat Enterprise Linux variants, or you can download the software at http://linux.dell.com/dkms/. Use Step 6 through Step 8 in To enable enhanced networking on Ubuntu (EBS-backed instances) for help configuring dkms.

    Warning

    If you compile the ixgbevf module for your current kernel and then upgrade your kernel without rebuilding the driver for the new kernel, your system may revert to the distribution-specific ixgbevf module at the next reboot, which could make your system unreachable if the distribution-specific version is incompatible with enhanced networking.

  4. Run the sudo depmod command to update module dependencies.

  5. Update initramfs on your instance to ensure that the new module loads at boot time.

  6. Determine if your system uses predictable network interface names by default. Systems that use systemd or udev versions 197 or greater can rename Ethernet devices and they do not guarantee that a single network interface will be named eth0. This behavior can cause problems connecting to your instance. For more information and to see other configuration options, see Predictable Network Interface Names on the freedesktop.org website.

    1. You can check the systemd or udev versions on RPM-based systems with the following command:

      Copy
      [ec2-user ~]$ rpm -qa | grep -e '^systemd-[0-9]\+\|^udev-[0-9]\+' systemd-208-11.el7_0.2.x86_64

      In the above Red Hat 7 example, the systemd version is 208, so predictable network interface names must be disabled.

    2. Disable predictable network interface names by adding the net.ifnames=0 option to the GRUB_CMDLINE_LINUX line in /etc/default/grub.

      Copy
      [ec2-user ~]$ sudo sed -i '/^GRUB\_CMDLINE\_LINUX/s/\"$/\ net\.ifnames\=0\"/' /etc/default/grub
    3. Rebuild the grub configuration file.

      Copy
      [ec2-user ~]$ sudo grub2-mkconfig -o /boot/grub2/grub.cfg
  7. From your local computer, stop the instance using the Amazon EC2 console or one of the following commands: stop-instances (AWS CLI), Stop-EC2Instance (AWS Tools for Windows PowerShell). If your instance is managed by AWS OpsWorks, you should stop the instance in the AWS OpsWorks console so that the instance state remains in sync.

    Important

    If you are using an instance store-backed instance, you can't stop the instance. Instead, proceed to To enabled enhanced networking (instance store–backed instances)

  8. From your local computer, enable the enhanced networking attribute using one of the following commands. Note that there is no way to disable the networking attribute after you've enabled it.

    Warning

    Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

    • modify-instance-attribute (AWS CLI)

      Copy
      aws ec2 modify-instance-attribute --instance-id instance_id --sriov-net-support simple
    • Edit-EC2InstanceAttribute (AWS Tools for Windows PowerShell)

      Copy
      Edit-EC2InstanceAttribute -InstanceId instance_id -SriovNetSupport "simple"
  9. (Optional) Create an AMI from the instance, as described in Creating an Amazon EBS-Backed Linux AMI . The AMI inherits the enhanced networking attribute from the instance. Therefore, you can use this AMI to launch another instance with enhanced networking enabled by default.

    Important

    If your instance operating system contains an /etc/udev/rules.d/70-persistent-net.rules file, you must delete it before creating the AMI. This file contains the MAC address for the Ethernet adapter of the original instance. If another instance boots with this file, the operating system will be unable to find the device and eth0 may fail, causing boot issues. This file is regenerated at the next boot cycle, and any instances launched from the AMI create their own version of the file.

  10. From your local computer, start the instance using the Amazon EC2 console or one of the following commands: start-instances (AWS CLI), Start-EC2Instance (AWS Tools for Windows PowerShell). If your instance is managed by AWS OpsWorks, you should start the instance in the AWS OpsWorks console so that the instance state remains in sync.

  11. (Optional) Connect to your instance and verify that the module is installed.

To enabled enhanced networking (instance store–backed instances)

If your instance is an instance store–backed instance, follow Step 1 through Step 5 in the previous procedure, and then create a new AMI as described in Creating an Instance Store-Backed Linux AMI. Be sure to enable the enhanced networking attribute when you register the AMI.

Warning

Enhanced networking is supported only for HVM instances. Enabling enhanced networking with a PV instance can make it unreachable. Setting this attribute without the proper module or module version can also make your instance unreachable.

  • register-image (AWS CLI)

    Copy
    aws ec2 register-image --sriov-net-support simple ...
  • Register-EC2Image (AWS Tools for Windows PowerShell)

    Copy
    Register-EC2Image -SriovNetSupport "simple" ...

Troubleshooting Connectivity Issues

If you lose connectivity while enabling enhanced networking, the ixgbevf module might be incompatible with the kernel. Try installing the version of the ixgbevf module included with the distribution of Linux for your instance.

If you enable enhanced networking for a PV instance or AMI, this can make your instance unreachable.