Menu
Amazon Elastic Compute Cloud
User Guide for Linux Instances

Linux Accelerated Computing Instances

If you require high processing capability, you'll benefit from using accelerated computing instances, which provide access to hardware-based compute accelerators such as Graphics Processing Units (GPUs) or Field Programmable Gate Arrays (FPGAs). Accelerated computing instances enable more parallelism for higher throughput on compute-intensive workloads.

GPU-based instances provide access to NVIDIA GPUs with thousands of compute cores. You can use GPU-based accelerated computing instances to accelerate scientific, engineering, and rendering applications by leveraging the CUDA or Open Computing Language (OpenCL) parallel computing frameworks. You can also use them for graphics applications, including game streaming, 3-D application streaming, and other graphics workloads.

FPGA-based instances provide access to large FPGAs with millions of parallel system logic cells. You can use FPGA-based accelerated computing instances to accelerate workloads such as genomics, financial analysis, real-time video processing, big data analysis, and security workloads by leveraging custom hardware accelerations. You can develop these accelerations using hardware description languages such as Verilog or VHDL, or by using higher-level languages such as OpenCL parallel computing frameworks. You can either develop your own hardware acceleration code or purchase hardware accelerations through the AWS Marketplace.

Important

FPGA-based instances do not support Microsoft Windows.

You can cluster accelerated computing instances into a placement group. Placement groups provide low latency and high-bandwidth connectivity between the instances within a single Availability Zone. For more information, see Placement Groups.

For information about Windows accelerated computing instances, see Windows Accelerated Computing Instances in the Amazon EC2 User Guide for Windows Instances.

Accelerated Computing Instance Families

Accelerated computing instance families use hardware accelerators, or co-processors, to perform some functions, such as floating point number calculations, graphics processing, or data pattern matching, more efficiently than is possible in software running on CPUs. The following accelerated computing instance families are available for you to launch in Amazon EC2.

F1 Instances

F1 instances use Xilinx UltraScale+ VU9P FPGAs and are designed to accelerate computationally intensive algorithms, such as data-flow or highly-parallel operations not suited to general purpose CPUs. Each FPGA in an F1 instance contains approximately 2.5 million logic elements and approximately 6,800 Digital Signal Processing (DSP) engines, along with 64 GiB of local DDR ECC protected memory, connected to the instance by a dedicated PCIe Gen3 x16 connection. F1 instances support enhanced networking with the Elastic Network Adapter (ENA), are EBS-optimized by default, and provide local NVMe SSD volumes.

Developers can use the FPGA Developer AMI and AWS Hardware Developer Kit to create custom hardware accelerations for use on F1 instances. The FPGA Developer AMI includes development tools for full-cycle FPGA development in the cloud. Using these tools, developers can create and share Amazon FPGA Images (AFIs) that can be loaded onto the FPGA of an F1 instance.

For more information, see Amazon EC2 F1 Instances.

P2 Instances

P2 instances use NVIDIA Tesla K80 GPUs and are designed for general purpose GPU computing using the CUDA or OpenCL programming models. P2 instances provide high bandwidth networking, powerful single and double precision floating-point capabilities, and 12 GiB of memory per GPU, which makes them ideal for deep learning, graph databases, high performance databases, computational fluid dynamics, computational finance, seismic analysis, molecular modeling, genomics, rendering, and other server-side GPU compute workloads.

G3 Instances

G3 instances use NVIDIA Tesla M60 GPUs and provide a cost-effective, high-performance platform for graphics applications using DirectX or OpenGL. G3 instances also provide NVIDIA GRID Virtual Workstation features, supporting 4 monitors with resolutions up to 4096x2160. Example applications include 3D visualizations, graphics-intensive remote workstations, 3D rendering, video encoding, virtual reality, and other server-side graphics workloads requiring massively parallel processing power.

G2 Instances

G2 instances use NVIDIA GRID K520 GPUs and provide a cost-effective, high-performance platform for graphics applications using DirectX or OpenGL. NVIDIA GRID GPUs also support NVIDIA’s fast capture and encode API operations. Example applications include video creation services, 3D visualizations, streaming graphics-intensive applications, and other server-side graphics workloads.

CG1 Instances

CG1 instances use NVIDIA Tesla M2050 GPUs and are designed for general purpose GPU computing using the CUDA or OpenCL programming models. CG1 instances provide customers with high bandwidth networking, double precision floating-point capabilities, and error-correcting code (ECC) memory, making them ideal for high performance computing (HPC) applications.

Hardware Specifications

For more information about the hardware specifications for each Amazon EC2 instance type, see Amazon EC2 Instances.

Accelerated Computing Instance Limitations

Accelerated computing instances have the following limitations:

AMIs for GPU-based Accelerated Computing Instances

To help you get started, NVIDIA provides AMIs for GPU-based accelerated computing instances. These reference AMIs include the NVIDIA driver, which enables full functionality and performance of the NVIDIA GPUs.

For a list of AMIs with the NVIDIA driver, see AWS Marketplace (NVIDIA GRID).

You can launch accelerated computing instances using any HVM AMI.

Installing the NVIDIA Driver on Linux Instances

A GPU-based accelerated computing instance must have the appropriate NVIDIA driver. The NVIDIA driver that you install must be compiled against the kernel that you plan to run on your instance.

Amazon provides AMIs with updated and compatible builds of the NVIDIA kernel drivers for each official kernel upgrade in the AWS Marketplace. If you decide to use a different NVIDIA driver version than the one that Amazon provides, or decide to use a kernel that's not an official Amazon build, you must uninstall the Amazon-provided NVIDIA packages from your system to avoid conflicts with the versions of the drivers that you are trying to install.

Use this command to uninstall Amazon-provided NVIDIA packages:

Copy
sudo yum erase nvidia cuda

The Amazon-provided CUDA toolkit package has dependencies on the NVIDIA drivers. Uninstalling the NVIDIA packages erases the CUDA toolkit. You must reinstall the CUDA toolkit after installing the NVIDIA driver.

Obtaining the NVIDIA Driver

For G3 instances, you can download the driver from Amazon S3 using the AWS CLI or SDKs. To install the AWS CLI, see Installing the AWS Command Line Interface in the AWS Command Line Interface User Guide. Use the following AWS CLI command to download the driver:

Important

This download is available to AWS customers only. By downloading, you agree that you will only use the downloaded software to develop AMIs for use with the NVIDIA Tesla M60 hardware. Upon installation of the software, you will be bound by the terms of the NVIDIA GRID Cloud End User License Agreement.

Copy
aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/ .

For P2, G2, and CG1 instances, you can download NVIDIA drivers from http://www.nvidia.com/Download/Find.aspx. Select the appropriate driver for your instance:

P2 Instances

Product Type Tesla
Product Series K-Series
Product K-80
Operating System Linux 64-bit
Recommended/Beta Recommended/Certified

G2 Instances

Product Type GRID
Product Series GRID Series
Product GRID K520
Operating System Linux 64-bit
Recommended/Beta Recommended/Certified

CG1 Instances

Product Type Tesla
Product Series M-Class
Product M2050
Operating System Linux 64-bit
Recommended/Beta Recommended/Certified

For more information about installing and configuring the driver, choose the ADDITIONAL INFORMATION tab on the download page for the driver on the NVIDIA website and choose the README link.

Installing the NVIDIA Driver Manually

To install the driver on a Linux instance

  1. Update your package cache and get necessary package updates for your instance.

    • For Amazon Linux, CentOS, and Red Hat Enterprise Linux:

      Copy
      sudo yum update -y
    • For Ubuntu and Debian:

      Copy
      sudo apt-get update -y
  2. (Ubuntu 16.04 and later, with the linux-aws package) Upgrade the linux-aws package to receive the latest version.

    Copy
    sudo apt-get upgrade -y linux-aws
  3. Reboot your instance to load the latest kernel version.

    Copy
    sudo reboot
  4. Reconnect to your instance after it has rebooted.

  5. Install the gcc compiler and the kernel headers package for the version of the kernel you are currently running.

    • For Amazon Linux, CentOS, and Red Hat Enterprise Linux:

      Copy
      sudo yum install -y gcc kernel-devel-$(uname -r)
    • For Ubuntu and Debian:

      Copy
      sudo apt-get install -y gcc linux-headers-$(uname -r)
  6. (Graphical instances only) Disable the nouveau open source driver for NVIDIA graphics cards.

    1. Add nouveau to the modprobe blacklist file. Copy the following code block and paste it into a terminal.

      Copy
      cat << EOF | sudo tee --append /etc/modprobe.d/blacklist.conf blacklist vga16fb blacklist nouveau blacklist rivafb blacklist nvidiafb blacklist rivatv EOF
    2. Edit /etc/default/grub and add the following text to the GRUB_CMDLINE_LINUX line:

      Copy
      modprobe.blacklist=nouveau
    3. Rebuild the Grub configuration.

      • CentOS and Red Hat Enterprise Linux:

        Copy
        sudo grub2-mkconfig -o /boot/grub2/grub.cfg
      • For Ubuntu and Debian:

        Copy
        sudo update-grub
  7. Download the driver package that you identified earlier. For P2 instances, the following command downloads the 375.66 version of the NVIDIA driver.

    Copy
    wget http://us.download.nvidia.com/XFree86/Linux-x86_64/375.66/NVIDIA-Linux-x86_64-375.66.run

    For G3 instances, you can download the driver from Amazon S3 using the AWS CLI or SDKs. To install the AWS CLI, see Installing the AWS Command Line Interface in the AWS Command Line Interface User Guide. Use the following AWS CLI command to download the driver and the NVIDIA GRID Cloud End User License Agreement:

    Important

    This download is available to AWS customers only. By downloading, you agree that you will only use the downloaded software to develop AMIs for use with the NVIDIA Tesla M60 hardware. Upon installation of the software, you will be bound by the terms of the NVIDIA GRID Cloud End User License Agreement.

    Copy
    aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/ .
  8. Run the self-install script to install the NVIDIA driver that you downloaded in the previous step. For example:

    Copy
    sudo /bin/bash ./NVIDIA-Linux-x86_64-375.66.run
  9. Reboot the instance.

    Copy
    sudo reboot
  10. Confirm that the driver is functional. The response for the following command lists the installed NVIDIA driver version and details about the GPUs.

    Note

    This command may take several minutes to run.

    Copy
    nvidia-smi -q | head
  11. (G3 instances only) If you are using a G3 instance, complete the GRID activation steps in Activate GRID Workstation Features (G3 Instances Only)

  12. (P2 and G3 instances only) If you are using a P2 or G3 instance, complete the optimization steps in Optimizing GPU Settings (P2 and G3 Instances Only) to achieve the best performance from your GPU.

Activate GRID Workstation Features (G3 Instances Only)

To activate the GRID features on G3 instances, you must define the product type for the driver in the /etc/nvidia/gridd.conf file.

To activate GRID features on G3 Linux instances

  1. Create the /etc/nvidia/gridd.conf file from the provided template file.

    Copy
    sudo cp /etc/nvidia/gridd.conf.template /etc/nvidia/gridd.conf
  2. Open the /etc/nvidia/gridd.conf file in your favorite text editor.

  3. Find the FeatureType line, and set it equal to 2.

    Copy
    FeatureType=2
  4. Save the file and exit.

  5. Restart the nvidia-gridd licensing daemon to pick up the new configuration.

    Copy
    sudo service nvidia-gridd restart

Optimizing GPU Settings (P2 and G3 Instances Only)

There are several GPU setting optimizations that you can perform to achieve the best performance on P2 and G3 instances. By default, the NVIDIA driver uses an autoboost feature, which varies the GPU clock speeds. By disabling the autoboost feature and setting the GPU clock speeds to their maximum frequency, you can consistently achieve the maximum performance with your P2 and G3 instances. The following procedure helps you to configure the GPU settings to be persistent, disable the autoboost feature, and set the GPU clock speeds to their maximum frequency.

To optimize P2 or G3 GPU settings

  1. Configure the GPU settings to be persistent.

    Note

    This command may take several minutes to run.

    Copy
    sudo nvidia-smi -pm 1
  2. Disable the autoboost feature for all GPUs on the instance.

    Copy
    sudo nvidia-smi --auto-boost-default=0
  3. Set all GPU clock speeds to their maximum frequency. Use the memory and graphics clock speeds listed below, based on your instance type.

    • For P2 instances: 2505,875

    • For G3 instances: 2505,1177

    Copy
    sudo nvidia-smi -ac 2505,1177

Getting Started with FPGA Development

The FPGA Developer AMI provides the tools you need to develop, test, and build AFIs. You can use the FPGA Developer AMI on any EC2 instance with at least 32 GB of system memory (for example, C4 instances).

For more information, see the documentation for the AWS FPGA Hardware Development Kit.