Amazon Elastic Inference
Developer Guide

Setting Up to Launch Amazon EC2 with Elastic Inference

The most convenient way to setup Amazon EC2 with Elastic Inference uses the EI setup script described in https://aws.amazon.com/blogs/machine-learning/launch-ei-accelerators-in-minutes-with-the-amazon-elastic-inference-setup-tool-for-ec2/. To manually launch an instance and associate it with an Elastic Inference accelerator, you must first configure your security groups and AWS PrivateLink endpoint services. Then, you must configure an instance role with the Elastic Inference policy.

Configuring Your Security Groups for Elastic Inference

You need two security groups: one for inbound and outbound traffic for the new Elastic Inference VPC endpoint and another for outbound traffic for the associated EC2 instances that you launch.

Configure Your Security Groups for Elastic Inference

To configure a security group for an Elastic Inference accelerator (console)

  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the left navigation pane, choose Security, Security Groups.

  3. Choose Create Security Group

  4. Under Create Security Group, specify a name and description for the security group and choose the ID of the VPC. Choose Create and then choose Close.

  5. Select the check box next to your security group and choose Actions, Edit inbound rules. Add a rule to allow HTTPS traffic on port 443 as follows:

    1. Choose Add Rule.

    2. For Type, select HTTPS.

    3. For Source, specify a CIDR block (for example, 0.0.0.0/0) or the security group for your instance.

    4. To allow traffic for port 22 to the EC2 instance, repeat the procedure. For Type, select SSH.

    5. Choose Save rules and then choose Close.

  6. Choose Edit outbound rules. Choose Add rule. To allow traffic for all ports, for Type, select All Traffic.

  7. Choose Save rules.

To configure a security group for an Elastic Inference accelerator (AWS CLI)

  1. Create a security group using the create-security-group command:

    aws ec2 create-security-group --description insert a description for the security group --group-name assign a name for the security group [--vpc-id enter the VPC ID]
  2. Create inbound rules using the authorize-security-group-ingress command:

    aws ec2 authorize-security-group-ingress --group-id insert the security group ID --protocol tcp --port 443 --cidr 0.0.0.0/0
    aws ec2 authorize-security-group-ingress --group-id insert the security group ID --protocol tcp --port 22 --cidr 0.0.0.0/0
  3. The default setting for outbound rules allows all traffic from all ports for this instance.

Elastic Inference uses VPC endpoints to privately connect the instance in your VPC with their associated Elastic Inference accelerator. You must create a VPC endpoint for Elastic Inference before you launch instances with accelerators. This needs to be done just one time per VPC. For more information, see Interface VPC Endpoints (AWS PrivateLink).

To configure an AWS PrivateLink endpoint service (console)

  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the left navigation pane, choose Endpoints, Create Endpoint.

  3. For Service category, choose Find service by name.

  4. For Service Name, select com.amazonaws.<your-region>.elastic-inference.runtime.

    For example, for the us-west-2 region, select com.amazonaws.us-west-2.elastic-inference.runtime.

  5. For Subnets, select one or more Availability Zones where the endpoint should be created. Where you plan to launch instances with accelerators, you must select subnets for the Availability Zone.

  6. Enable the private DNS name and enter the security group for your endpoint. Choose Create endpoint. Note the VPC endpoint ID for later.

  7. The security group that we configured for the endpoint in previous steps must allow inbound traffic to port 443.

To configure a AWS PrivateLink endpoint service (AWS CLI)

Configuring an Instance Role with an Elastic Inference Policy

To launch an instance with an Elastic Inference accelerator, you must provide an IAM role that allows actions on Elastic Inference accelerators.

To configure an instance role with an Elastic Inference policy (console)

  1. Open the IAM console at https://console.aws.amazon.com/iam/.

  2. In the left navigation pane, choose Policies, Create Policy.

  3. Choose JSON and paste the following policy:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elastic-inference:Connect", "iam:List*", "iam:Get*", "ec2:Describe*", "ec2:Get*" ], "Resource": "*" } ] }

    Note

    You may get a warning message about the elastic-inference service not being recognizable. This is a known issue and does not block creation of the policy.

  4. Choose Review policy and enter a name for the policy, such as ec2-role-trust-policy.json, and a description.

  5. Choose Create policy.

  6. In the left navigation pane, choose Roles, Create role.

  7. Choose AWS service, EC2, Next: Permissions.

  8. Select the name of the policy that you just created (ec2-role-trust-policy.json). Choose Next: Tags.

  9. Provide a role name and choose Create Role.

When you create your instance, select the role under Configure Instance Details in the launch wizard.

To configure an instance role with an Elastic Inference policy (AWS CLI)

  • To configure an instance role with an Elastic Inference policy, follow the steps in Creating an IAM Role. Add the following policy to your instance:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elastic-inference:Connect", "iam:List*", "iam:Get*", "ec2:Describe*", "ec2:Get*" ], "Resource": "*" } ] }

    Note

    You may get a warning message about the elastic-inference service not being recognizable. This is a known issue and does not block creation of the policy.

Launching an Instance with Elastic Inference

You can now configure EC2 instances with accelerators to launch within your subnet. You can choose any supported Amazon EC2 instance type and Elastic Inference accelerator size. Elastic Inference accelerators are available to all current generation instance types. There are three Elastic Inference accelerator sizes to choose from:

  • eia1.medium with 1 GB of accelerator memory

  • eia1.large with 2 GB of accelerator memory

  • eia1.xlarge with 4 GB of accelerator memory

You can launch an instance with Elastic Inference automatically by using the Amazon Elastic Inference setup tool for EC2, or manually using the console or Amazon CLI.

To launch an instance with Elastic Inference (console)

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. Choose Launch Instance.

  3. Under Choose an Amazon Machine Image, select an Amazon Linux or Ubuntu AMI. We recommend one of the Deep Learning AMIs.

  4. Under Choose an Instance Type, select the hardware configuration of your instance.

  5. Choose Next: Configure Instance Details.

  6. Under Configure Instance Details, check the configuration settings. Ensure that you are using the VPC with the security groups for the instance and the Elastic Inference accelerator that you set up earlier. For more information, see Configuring Your Security Groups for Elastic Inference.

  7. For IAM role, select the role that you created in the Configuring an Instance Role with an Elastic Inference Policy procedure.

  8. Select Add an Elastic Inference accelerator.

  9. Select the size of the Elastic Inference accelerator. Your options are: eia1.medium, eia1.large, and eia1.xlarge.

  10. (Optional) You can choose to add storage and tags by choosing Next at the bottom of the page. Or, you can let the instance wizard complete the remaining configuration steps for you.

  11. In the Add Security Group step, choose the security group created previously.

  12. Review the configuration of your instance and choose Launch.

  13. You are prompted to choose an existing key pair for your instance or to create a new key pair. For more information, see Amazon EC2 Key Pairs..

    Warning

    Don’t select the Proceed without a key pair option. If you launch your instance without a key pair, then you can’t connect to it.

  14. After making your key pair selection, choose Launch Instances.

  15. A confirmation page lets you know that your instance is launching. To close the confirmation page and return to the console, choose View Instances.

  16. Under Instances, you can view the status of the launch. It takes a short time for an instance to launch. When you launch an instance, its initial state is pending. After the instance starts, its state changes to running.

  17. It can take a few minutes for the instance to be ready so that you can connect to it. Check that your instance has passed its status checks. You can view this information in the Status Checks column.

To launch an instance with Elastic Inference (AWS CLI)

To launch an instance with Elastic Inference at the command line, you need your key pair name, subnet ID, security group ID, AMI ID, and the name of the instance profile that you created in the section Configuring an Instance Role with an Elastic Inference Policy. For the security group ID, use the one you created for your instance that contains the AWS PrivateLink endpoint. For more information, see Configuring Your Security Groups for Elastic Inference). For more information about the AMI ID, see Finding a Linux AMI.

  1. Use the run-instances command to launch your instance and accelerator:

    aws ec2 run-instances --image-id ami-image ID --instance-type m5.large --subnet-id subnet-subnet ID --elastic-inference-accelerator Type=eia1.large --key-name key pair name --security-group-ids sg-security group ID --iam-instance-profile Name="accelerator profile name"
  2. When the run-instances operation succeeds, your output is similar to the following. The ElasticInferenceAcceleratorArn identifies the Elastic Inference accelerator.

    "ElasticInferenceAcceleratorAssociations": [ { "ElasticInferenceAcceleratorArn": "arn:aws:elastic-inference:us-west-2:204044812891:elastic-inference-accelerator/eia-3e1de7c2f64a4de8b970c205e838af6b", "ElasticInferenceAcceleratorAssociationId": "eia-assoc-031f6f53ddcd5f260", "ElasticInferenceAcceleratorAssociationState": "associating", "ElasticInferenceAcceleratorAssociationTime": "2018-10-05T17:22:20.000Z" } ],

You are now ready to run your models using either TensorFlow or MXNet on the provided AMI.