Amazon Elastic Compute Cloud
User Guide for Linux Instances

Setting Up to Launch Amazon EC2 with Amazon EI

To launch an instance and associate it with an Amazon EI accelerator, you must first configure your security groups and AWS PrivateLink endpoint services. Then, you must configure an instance role with the Amazon EI policy.

Amazon EI uses VPC endpoints to privately connect the instance in your VPC with their associated Amazon EI accelerator. You must create a VPC endpoint for Amazon EI before you launch instances with accelerators. This needs to be done just one time per VPC. For more information, see Interface VPC Endpoints (AWS PrivateLink).

To configure an AWS PrivateLink endpoint service (console)

  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the left navigation pane, choose Endpoints, Create Endpoint.

  3. For Service category, choose Find service by name.

  4. For Service Name, select com.amazonaws.<your-region>.elastic-inference.runtime.

    For example, for the us-west-2 region, select com.amazonaws.us-west-2.elastic-inference.runtime.

  5. For Subnets, select one or more Availability Zones where the endpoint should be created. Where you plan to launch instances with accelerators, you must select subnets for the Availability Zone.

  6. Enable the private DNS name and enter the security group for your endpoint. Choose Create endpoint. Note the VPC endpoint ID for later.

  7. The security group for the endpoint must allow inbound traffic to port 443.

To configure a AWS PrivateLink endpoint service (AWS CLI)

  • Use the create-vpc-endpoint command and specify the VPC ID, type of VPC endpoint (interface), service name, subnets to use the endpoint, and security groups to associate with the endpoint network interfaces. For information about how to set up a security group for your VPC endpoint, see Configuring Your Security Groups for Amazon EI.

    aws ec2 create-vpc-endpoint --vpc-id vpc-insert VPC ID --vpc-endpoint-type Interface --service-name com.amazonaws.us-west-2.elastic-inference.runtime --subnet-id subnet-insert subnet --security-group-id sg-insert security group ID

Configuring Your Security Groups for Amazon EI

You need two security groups: one for inbound and outbound traffic for the new Amazon EI VPC endpoint and another for outbound traffic for the associated EC2 instances that you launch.

Configure Your Security Groups for Amazon EI

To configure a security group for an Amazon EI accelerator (console)

  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the left navigation pane, choose Security, Security Groups, Create a Security Group.

  3. Under Create Security Group, enter field values and choose Create.

  4. Choose Close.

  5. Select the box next to your security group and choose Inbound Rules.

  6. Choose Edit rules.

  7. Choose Add rule.

  8. To allow traffic from only port 443 from any source, or the security group to which you plan to associate your instance, for Type, select HTTPS.

  9. Choose Add rule.

  10. Choose Save rules.

  11. Choose Outbound Rules. To allow traffic for port 443 to any destination, for Type, select HTTPS.

    Choose Add rule.

    To allow traffic for port 22 to the EC2 instance, for Type, select SSH.

    Choose Add rule.

  12. Choose Save rules.

  13. Add an outbound rule that either restricts traffic to the endpoint security group that you created in the previous step or that allows traffic to HTTPS (TCP port 443) to any destination.

  14. Choose Save.

To configure a security group for an Amazon EI accelerator (AWS CLI)

  1. Create a security group using the create-security-group command:

    aws ec2 create-security-group --description description for the security group --group-name name for the security group [--vpc-id VPC ID]
  2. Create an inbound rule using the authorize-security-group-ingress command:

    aws ec2 authorize-security-group-ingress --group-id security group ID --group-name security group name --protocol tcp --port 443
  3. Use the authorize-security-group-egress command to create an outbound rule:

    aws ec2 authorize-security-group-egress --group-id security group ID --protocol tcp --port 443 --port 22 --cidr 0.0.0.0/0

Configuring an Instance Role with an Amazon EI Policy

To launch an instance with an Amazon EI accelerator, you must provide an IAM role that allows actions on Amazon EI accelerators.

To configure an instance role with an Amazon EI policy (console)

  1. Open the IAM console at https://console.aws.amazon.com/iam/.

  2. In the left navigation pane, choose Policies, Create Policy.

  3. Choose JSON and paste the following policy:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elastic-inference:Connect", "iam:List*", "iam:Get*", "ec2:Describe*", "ec2:Get*" ], "Resource": "*" } ] }

    You may get a warning message about the elastic-inference service not being recognizable. This is a known issue and does not block creation of the policy.

  4. Choose Review policy and enter a name for the policy, such as ec2-role-trust-policy.json, and a description.

  5. Choose Create policy.

  6. In the left navigation pane, choose Roles, Create role.

  7. Choose AWS service, EC2, Next: Permissions.

  8. Select the name of the policy that you just created (ec2-role-trust-policy.json). Choose Next: Tags.

  9. Provide a role name and choose Create Role.

When you create your instance, select the role under Configure Instance Details in the launch wizard.

To configure an instance role with an Amazon EI policy (AWS CLI)

  • To configure an instance role with an Amazon EI policy, follow the steps in Creating an IAM Role. Add the following policy to your instance:

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elastic-inference:Connect", "iam:List*", "iam:Get*", "ec2:Describe*", "ec2:Get*" ], "Resource": "*" } ] }

    You may get a warning message about the elastic-inference service not being recognizable. This is a known issue and does not block creation of the policy.

Launching an Instance with Amazon EI

You can now configure EC2 instances with accelerators to launch within your subnet. You can choose any supported Amazon EC2 instance type and Amazon EI accelerator size. Amazon EI accelerators are available to all current generation instance types. There are three Amazon EI accelerator sizes to choose from:

  • eia1.medium with 1 GB of accelerator memory

  • eia1.large with 2 GB of accelerator memory

  • eia1.xlarge with 4 GB of accelerator memory

To launch an instance with Amazon EI (console)

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. Choose Launch Instance.

  3. Under Choose an Amazon Machine Image, select an Amazon Linux or Ubuntu AMI. We recommend one of the Deep Learning AMIs.

  4. Under Choose an Instance Type, select the hardware configuration of your instance.

  5. Choose Next: Configure Instance Details.

  6. Under Configure Instance Details, check the configuration settings. Ensure that you are using the VPC with the security groups for the instance and the Amazon EI accelerator that you set up earlier. For more information, see Configuring Your Security Groups for Amazon EI.

  7. For IAM role, select the role that you created in the Configuring an Instance Role with an Amazon EI Policy procedure.

  8. Select Add an Amazon EI accelerator.

  9. Select the size of the Amazon EI accelerator. Your options are: eia1.medium, eia1.large, and eia1.xlarge.

  10. (Optional) You can choose to add storage and tags by choosing Next at the bottom of the page. Or, you can let the instance wizard complete the remaining configuration steps for you.

  11. Review the configuration of your instance and choose Launch.

  12. You are prompted to choose an existing key pair for your instance or to create a new key pair. For more information, see Amazon EC2 Key Pairs..

    Warning

    Don’t select the Proceed without a key pair option. If you launch your instance without a key pair, then you can’t connect to it.

  13. After making your key pair selection, choose Launch Instances.

  14. A confirmation page lets you know that your instance is launching. To close the confirmation page and return to the console, choose View Instances.

  15. Under Instances, you can view the status of the launch. It takes a short time for an instance to launch. When you launch an instance, its initial state is pending. After the instance starts, its state changes to running.

  16. It can take a few minutes for the instance to be ready so that you can connect to it. Check that your instance has passed its status checks. You can view this information in the Status Checks column.

To launch an instance with Amazon EI (AWS CLI)

To launch an instance with Amazon EI at the command line, you need your key pair name, subnet ID, security group ID, AMI ID, and the name of the instance profile that you created in the section Configuring an Instance Role with an Amazon EI Policy. For the security group ID, use the one you created for your instance that contains the AWS PrivateLink endpoint. For more information, see Configuring Your Security Groups for Amazon EI). For more information about the AMI ID, see Finding a Linux AMI.

  1. Use the run-instances command to launch your instance and accelerator:

    aws ec2 run-instances --image-id ami-image ID --instance-type m5.large --subnet-id subnet-subnet ID --elastic-inference-accelerator Type=eia1.large --key-name key pair name --security-group-ids sg-security group ID --iam-instance-profile Name="accelerator profile name"
  2. When the run-instances operation succeeds, your output is similar to the following. The ElasticInferenceAcceleratorArn identifies the Amazon EI accelerator.

    "ElasticInferenceAcceleratorAssociations": [ { "ElasticInferenceAcceleratorArn": "arn:aws:elastic-inference:us-west-2:204044812891:elastic-inference-accelerator/eia-3e1de7c2f64a4de8b970c205e838af6b", "ElasticInferenceAcceleratorAssociationId": "eia-assoc-031f6f53ddcd5f260", "ElasticInferenceAcceleratorAssociationState": "associating", "ElasticInferenceAcceleratorAssociationTime": "2018-10-05T17:22:20.000Z" } ],

You are now ready to run your models using either TensorFlow or MXNet on the provided AMI.