Setting Up to Launch Amazon EC2 with Elastic Inference

The most convenient way to set up Amazon EC2 with Elastic Inference uses the Elastic Inference setup script described in https://aws.amazon.com/blogs/machine-learning/launch-ei-accelerators-in-minutes-with-the-amazon-elastic-inference-setup-tool-for-ec2/. To manually launch an instance and associate it with an Elastic Inference accelerator, first configure your security groups and AWS PrivateLink endpoint services. Then, configure an instance role with the Elastic Inference policy.

Configuring Your Security Groups for Elastic Inference

You need two security groups. One for inbound and outbound traffic for the new Elastic Inference VPC endpoint. A second one for outbound traffic for the associated Amazon EC2 instances that you launch.

Configure Your Security Groups for Elastic Inference

To configure a security group for an Elastic Inference accelerator (console)

1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

2. In the left navigation pane, choose Security, Security Groups.

3. Choose Create Security Group

4. Under Create Security Group, specify a name and description for the security group and choose the ID of the VPC. Choose Create and then choose Close.

5. Select the check box next to your security group and choose Actions, Edit inbound rules. Add a rule to allow HTTPS traffic on port 443 as follows:

   1. Choose Add Rule.

   2. For Type, select HTTPS.

   3. For Source, specify a CIDR block (for example, 0.0.0.0/0) or the security group for your instance.

   4. To allow traffic for port 22 to the EC2 instance, repeat the procedure. For Type, select SSH.

   5. Choose Save rules and then choose Close.

6. Choose Edit outbound rules. Choose Add rule. To allow traffic for all ports, for Type, select All Traffic.

7. Choose Save rules.

To configure a security group for an Elastic Inference accelerator (AWS CLI)

1. Create a security group using the create-security-group command:

   aws ec2 create-security-group
   --description insert a description for the security group
   --group-name assign a name for the security group
   [--vpc-id enter the VPC ID]

2. Create inbound rules using the authorize-security-group-ingress command:

   aws ec2 authorize-security-group-ingress --group-id insert the security group ID --protocol tcp --port 443 --cidr 0.0.0.0/0

   aws ec2 authorize-security-group-ingress --group-id insert the security group ID --protocol tcp --port 22 --cidr 0.0.0.0/0

3. The default setting for outbound rules allows all traffic from all ports for this instance.

Configuring AWS PrivateLink Endpoint Services

Elastic Inference uses VPC endpoints to privately connect the instance in your VPC with their associated Elastic Inference accelerator. Create a VPC endpoint for Elastic Inference before you launch instances with accelerators. This needs to be done just one time per VPC. For more information, see Interface VPC Endpoints (AWS PrivateLink).

To configure an AWS PrivateLink endpoint service (console)

1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

2. In the left navigation pane, choose Endpoints, Create Endpoint.

3. For Service category, choose Find service by name.

4. For Service Name, select com.amazonaws.<your-region>.elastic-inference.runtime.

   For example, for the us-west-2 Region, select com.amazonaws.us-west-2.elastic-inference.runtime.

5. For Subnets, select one or more Availability Zones where the endpoint should be created. Where you plan to launch instances with accelerators, you must select subnets for the Availability Zone.

6. Enable the private DNS name and enter the security group for your endpoint. Choose Create endpoint. Note the VPC endpoint ID for later.

7. The security group that we configured for the endpoint in previous steps must allow inbound traffic to port 443.

To configure an AWS PrivateLink endpoint service (AWS CLI)

• Use the https://docs.aws.amazon.com/cli/latest/reference/ec2/create-vpc-endpoint.html command and specify the following: VPC ID, type of VPC endpoint (interface), service name, subnets to use the endpoint, and security groups to associate with the endpoint network interfaces. For information about how to set up a security group for your VPC endpoint, see Configuring Your Security Groups for Elastic Inference.

  aws ec2 create-vpc-endpoint --vpc-id vpc-insert VPC ID --vpc-endpoint-type Interface --service-name com.amazonaws.us-west-2.elastic-inference.runtime --subnet-id subnet-insert subnet --security-group-id sg-insert security group ID

Configuring an Instance Role with an Elastic Inference Policy

To launch an instance with an Elastic Inference accelerator, you must provide an IAM role that allows actions on Elastic Inference accelerators.

To configure an instance role with an Elastic Inference policy (console)

1. Open the IAM console at https://console.aws.amazon.com/iam/.

2. In the left navigation pane, choose Policies, Create Policy.

3. Choose JSON and paste the following policy:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "elastic-inference:Connect",
                   "iam:List*",
                   "iam:Get*",
                   "ec2:Describe*",
                   "ec2:Get*"
               ],
               "Resource": "*"
           }
       ]
   }

   Note

   You may get a warning message about the Elastic Inference service not being recognizable. This is a known issue and does not block creation of the policy.

4. Choose Review policy and enter a name for the policy, such as ec2-role-trust-policy.json, and a description.

5. Choose Create policy.

6. In the left navigation pane, choose Roles, Create role.

7. Choose AWS service, EC2, Next: Permissions.

8. Select the name of the policy that you just created (ec2-role-trust-policy.json). Choose Next: Tags.

9. Provide a role name and choose Create Role.

When you create your instance, select the role under Configure Instance Details in the launch wizard.

To configure an instance role with an Elastic Inference policy (AWS CLI)

• To configure an instance role with an Elastic Inference policy, follow the steps in Creating an IAM Role. Add the following policy to your instance:

  {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": [
                  "elastic-inference:Connect",
                  "iam:List*",
                  "iam:Get*",
                  "ec2:Describe*",
                  "ec2:Get*"
              ],
              "Resource": "*"
          }
      ]
  }

  Note

  You may get a warning message about the Elastic Inference service not being recognizable. This is a known issue and does not block creation of the policy.

Launching an Instance with Elastic Inference

You can now configure Amazon EC2 instances with accelerators to launch within your subnet. Choose any supported Amazon EC2 instance type and Elastic Inference accelerator size. Elastic Inference accelerators are available to all current generation instance types. There are two accelerator types.

EIA2 is the second generation of Elastic Inference accelerators. It offers improved performance and increased memory. With up to 8 GB of GPU memory, EIA2 is a cost-effective resource for deploying machine learning (ML) models. Use it for applications such as image classification, object detection, automated speech recognition, and language translation. Your accelerator memory choices depend on the size of your input and models. You can choose from the following Elastic Inference accelerators:

• eia2.medium with 2 GB of accelerator memory

• eia2.large with 4 GB of accelerator memory

• eia2.xlarge with 8 GB of accelerator memory

Note: We continue to support EIA1 in three sizes: eia1.medium, eia1.large, and eia1.xlarge

You can launch an instance with Elastic Inference automatically by using the Amazon Elastic Inference setup tool for EC2, or manually using the console or AWS Command Line Interface.

To launch an instance with Elastic Inference (console)

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

2. Choose Launch Instance.

3. Under Choose an Amazon Machine Image, select an Amazon Linux or Ubuntu AMI. We recommend one of the Deep Learning AMIs.

   Note

   Attaching multiple Elastic Inference accelerators to a single Amazon EC2 instance requires that the instance has AWS Deep Learning AMI (DLAMI) Version 25 or later.

4. Under Choose an Instance Type, select the hardware configuration of your instance.

5. Choose Next: Configure Instance Details.

6. Under Configure Instance Details, check the configuration settings. Ensure that you are using the VPC with the security groups for the instance and the Elastic Inference accelerator that you set up earlier. For more information, see Configuring Your Security Groups for Elastic Inference.

7. For IAM role, select the role that you created in the Configuring an Instance Role with an Elastic Inference Policy procedure.

8. Select Add an Elastic Inference accelerator.

9. Select the size and amount of Elastic Inference accelerators. Your options are: eia2.medium, eia2.large, and eia2.xlarge.

10. To add another Elastic Inference accelerator, choose Add. Then select the size and amount of accelerators to add.

11. (Optional) You can choose to add storage and tags by choosing Next at the bottom of the page. Or, you can let the instance wizard complete the remaining configuration steps for you.

12. In the Add Security Group step, choose the security group created previously.

13. Review the configuration of your instance and choose Launch.

14. You are prompted to choose an existing key pair for your instance or to create a new key pair. For more information, see Amazon EC2 Key Pairs..

    Warning

    Don’t select the Proceed without a key pair option. If you launch your instance without a key pair, then you can’t connect to it.

15. After making your key pair selection, choose Launch Instances.

16. A confirmation page lets you know that your instance is launching. To close the confirmation page and return to the console, choose View Instances.

17. Under Instances, you can view the status of the launch. It takes a short time for an instance to launch. When you launch an instance, its initial state is pending. After the instance starts, its state changes to running.

18. It can take a few minutes for the instance to be ready so that you can connect to it. Check that your instance has passed its status checks. You can view this information in the Status Checks column.

To launch an instance with Elastic Inference (AWS CLI)

To launch an instance with Elastic Inference at the command line, you need your key pair name, subnet ID, security group ID, AMI ID, and the name of the instance profile that you created in the section Configuring an Instance Role with an Elastic Inference Policy. For the security group ID, use the one you created for your instance that contains the AWS PrivateLink endpoint. For more information, see Configuring Your Security Groups for Elastic Inference). For more information about the AMI ID, see Finding a Linux AMI.

1. Use the run-instances command to launch your instance and accelerator:

   aws ec2 run-instances --image-id ami-image ID --instance-type m5.large --subnet-id subnet-subnet ID --elastic-inference-accelerator Type=eia2.large --key-name key pair name --security-group-ids sg-security group ID --iam-instance-profile Name="accelerator profile name"

   To launch an instance with multiple accelerators, you can add multiple Type parameters to --elastic-inference-accelerator.

   aws ec2 run-instances --image-id ami-image ID --instance-type m5.large --subnet-id subnet-subnet ID --elastic-inference-accelerator Type=eia2.large,Count=2 Type=eia2.xlarge  --key-name key pair name --region region name --security-group-ids sg-security group ID 

2. When the run-instances operation succeeds, your output is similar to the following. The ElasticInferenceAcceleratorArn identifies the Elastic Inference accelerator.

   "ElasticInferenceAcceleratorAssociations": [
    {
    "ElasticInferenceAcceleratorArn": "arn:aws:elastic-inference:us-west-2:204044812891:elastic-inference-accelerator/eia-3e1de7c2f64a4de8b970c205e838af6b",
    "ElasticInferenceAcceleratorAssociationId": "eia-assoc-031f6f53ddcd5f260",
    "ElasticInferenceAcceleratorAssociationState": "associating",
    "ElasticInferenceAcceleratorAssociationTime": "2018-10-05T17:22:20.000Z"
    }
    ],

You are now ready to run your models using TensorFlow, MXNet, or PyTorch on the provided AMI.

Once your Elastic Inference accelerator is running, you can use the describe-accelerators AWS CLI. This command returns information about the accelerator, such as the region it is in and the name of the accelerator. For more information about the usage of this command, see the Elastic Inference AWS CLI Command Reference.