Task networking with the awsvpc network mode - Amazon Elastic Container Service

Task networking with the awsvpc network mode

The task networking features provided by the awsvpc network mode give Amazon ECS tasks the same networking properties as Amazon EC2 instances. Using the awsvpc network mode simplifies container networking and gives you more control over how containerized applications communicate with each other and other services within your VPCs. The awsvpc network mode also provides greater security for your containers by enabling you to use security groups and network monitoring tools at a more granular level within your tasks. Because each task gets its own elastic network interface (ENI), you can also take advantage of other Amazon EC2 networking features like VPC Flow Logs so that you can monitor traffic to and from your tasks. Additionally, containers that belong to the same task can communicate over the localhost interface.

The task ENI is fully managed by Amazon ECS. Amazon ECS creates the ENI and attaches it to the host Amazon EC2 instance with the specified security group. The task sends and receives network traffic over the ENI in the same way that Amazon EC2 instances do with their primary network interfaces. Each task ENI is assigned a private IPv4 address by default. If your VPC is enabled for dual-stack mode and you use a subnet with an IPv6 CIDR block, the task ENI will also receive an IPv6 address. Each task can only have one ENI.

These ENIs are visible in the Amazon EC2 console for your account, but they cannot be detached manually or modified by your account. This is to prevent accidental deletion of an ENI that is associated with a running task. You can view the ENI attachment information for tasks in the Amazon ECS console or with the DescribeTasks API operation. When the task stops or if the service is scaled down, the task ENI is detached and deleted.

If your account, IAM user, or role has opted in to the awsvpcTrunking account setting and you have launched a container instance with the increased ENI density, Amazon ECS also creates and attaches a "trunk" network interface for your container instance. The trunk network is fully managed by Amazon ECS. The trunk ENI is deleted when you either terminate or deregister your container instance from the Amazon ECS cluster. For more information on opting in to the awsvpcTrunking account setting, see Working with container instances with increased ENI limits.

Considerations

There are several things to consider when using the awsvpc network mode.

The following are considerations when you use the Linux operating system:

  • Tasks and services that use the awsvpc network mode require the Amazon ECS service-linked role to provide Amazon ECS with the permissions to make calls to other AWS services on your behalf. This role is created for you automatically when you create a cluster, or if you create or update a service, in the AWS Management Console. For more information, see Service-linked role for Amazon ECS. You can also create the service-linked role with the following AWS CLI command:

    aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com
  • Your Amazon EC2 Linux instance requires version 1.15.0 or later of the container agent to run tasks that use the awsvpc network mode. If you are using an Amazon ECS-optimized AMI, your instance needs at least version 1.15.0-4 of the ecs-init package as well.

  • Amazon ECS populates the hostname of the task with an Amazon-provided (internal) DNS hostname when both the enableDnsHostnames and enableDnsSupport options are enabled on your VPC. If these options are not enabled, the DNS hostname of the task will be a random hostname. For more information on the DNS settings for a VPC, see Using DNS with Your VPC in the Amazon VPC User Guide.

  • Each Amazon ECS task that uses the awsvpc network mode receives its own elastic network interface (ENI), which is attached to the Amazon EC2 instance that hosts it. There is a default limit to the number of network interfaces that can be attached to an Amazon EC2 Linux instance, and the primary network interface counts as one. For example, by default a c5.large instance may have up to three ENIs attached to it. The primary network interface for the instance counts as one, so you can attach an additional two ENIs to the instance. Because each task using the awsvpc network mode requires an ENI, you can typically only run two such tasks on this instance type. For more information on the default ENI limits for each instance type, see IP addresses per network interface per instance type in the Amazon EC2 User Guide for Linux Instances.

  • Amazon ECS supports the launch of Amazon EC2 Linux instances using supported instance types with increased ENI density. When you opt in to the awsvpcTrunking account setting and register Amazon EC2 Linux instances using these instance types to your cluster, these instances have higher ENI limits. This enables you to place more tasks on each Amazon EC2 Linux instance. To take advantage of the increased ENI density with the trunking feature, your Amazon EC2 instance requires at least version 1.28.1 of the container agent. If you are using an Amazon ECS-optimized AMI, your instance also requires at least version 1.28.1-2 of the ecs-init package. For more information on opting in to the awsvpcTrunking account setting, see Account settings. For more information on ENI trunking, see Elastic network interface trunking.

  • When hosting tasks that use the awsvpc network mode on Amazon EC2 Linux instances, your task ENIs are not given public IP addresses. To access the internet, tasks should be launched in a private subnet that is configured to use a NAT gateway. For more information, see NAT gateways in the Amazon VPC User Guide. Inbound network access must be from within the VPC using the private IP address or routed through a load balancer from within the VPC. Tasks launched within public subnets do not have access to the internet.

  • Amazon ECS only accounts for the ENIs that it attaches to your Amazon EC2 Linux instances for you. If you have attached ENIs to your instances manually, then Amazon ECS could try to place a task on an instance with too few available network adapter attachments. In this case, the task would time out, move from PROVISIONING to DEPROVISIONING, and then to STOPPED. We recommend that you do not attach ENIs to your instances manually.

  • Amazon EC2 Linux instances must be registered with the ecs.capability.task-eni capability to be considered for placement of tasks with the awsvpc network mode. Instances running version 1.15.0-4 or later of ecs-init are registered with this attribute automatically.

  • The ENIs that are created and attached to your Amazon EC2 Linux instances cannot be detached manually or modified by your account. This is to prevent the accidental deletion of an ENI that is associated with a running task. To release the ENIs for a task, stop the task.

  • There is a limit of 16 subnets and 5 security groups that are able to be specified in the awsvpcConfiguration when running a task or creating a service that uses the awsvpc network mode. For more information, see AwsVpcConfiguration in the Amazon Elastic Container Service API Reference.

  • When a task is started with the awsvpc network mode, the Amazon ECS container agent creates an additional pause container for each task before starting the containers in the task definition. It then configures the network namespace of the pause container by running the amazon-ecs-cni-plugins CNI plugins. The agent then starts the rest of the containers in the task so that they share the network stack of the pause container. This means that all containers in a task are addressable by the IP addresses of the ENI, and they can communicate with each other over the localhost interface.

  • Services with tasks that use the awsvpc network mode only support Application Load Balancers and Network Load Balancers; Classic Load Balancers are not supported. Also, when you create any target groups for these services, you must choose ip as the target type, not instance. This is because tasks that use the awsvpc network mode are associated with an ENI, not with an Amazon EC2 Linux instance. For more information, see Service load balancing.

  • If a VPC is updated, for example to change the DHCP options set it uses, and you want tasks using the VPC to pick up the changes, those tasks must be stopped and new tasks started.

The following are considerations when you use the Windows operating system:

  • Container instances using the Amazon ECS-optimized Windows Server 2016 AMI can't host tasks that use the awsvpc network mode. If you have a cluster that contains Amazon ECS-optimized Windows Server 2016 AMIs and Windows AMIs that do support awsvpc network mode, tasks that use awsvpc network mode are not launched on the Windows 2016 Server instances, but will be launched on instances that support awsvpc network mode.

  • Tasks and services that use the awsvpc network mode require the Amazon ECS service-linked role to provide Amazon ECS with the permissions to make calls to other AWS services on your behalf. This role is created for you automatically when you create a cluster, or if you create or update a service, in the AWS Management Console. For more information, see Service-linked role for Amazon ECS. You can also create the service-linked role with the following AWS CLI command:

    aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com
  • Your Amazon EC2 Windows instance requires version 1.54.0 or later of the container agent to run tasks that use the awsvpc network mode. When you boostrap the instance, you must configure the options that are required for awsvpc network mode. For more information, see Bootstrapping Windows container instances with Amazon EC2 user data.

  • Amazon ECS populates the hostname of the task with an Amazon-provided (internal) DNS hostname when both the enableDnsHostnames and enableDnsSupport options are enabled on your VPC. If these options are not enabled, the DNS hostname of the task will be a random hostname. For more information on the DNS settings for a VPC, see Using DNS with Your VPC in the Amazon VPC User Guide.

  • Each Amazon ECS task that uses the awsvpc network mode receives its own elastic network interface (ENI), which is attached to the Amazon EC2 Windows instance that hosts it. There is a default limit to the number of network interfaces that can be attached to an Amazon EC2 Windows instance, and the primary network interface counts as one. For example, by default a c5.large instance may have up to three ENIs attached to it. The primary network interface for the instance counts as one, so you can attach an additional two ENIs to the instance. Because each task using the awsvpc network mode requires an ENI, you can typically only run two such tasks on this instance type. For more information on the default ENI limits for each instance type, see IP addresses per network interface per instance type in the Amazon EC2 User Guide for Windows Instances.

  • When hosting tasks that use the awsvpc network mode on Amazon EC2 Windows instances, your task ENIs are not given public IP addresses. To access the internet, tasks should be launched in a private subnet that is configured to use a NAT gateway. For more information, see NAT gateways in the Amazon VPC User Guide. Inbound network access must be from within the VPC using the private IP address or routed through a load balancer from within the VPC. Tasks launched within public subnets do not have access to the internet.

  • Amazon ECS only accounts for the ENIs that it attaches to your Amazon EC2 Windows instances for you. If you have attached ENIs to your instances manually, then Amazon ECS could try to place a task on an instance with too few available network adapter attachments. In this case, the task would time out, move from PROVISIONING to DEPROVISIONING, and then to STOPPED. We recommend that you do not attach ENIs to your instances manually.

  • Amazon EC2 Windows instances must be registered with the ecs.capability.task-eni capability to be considered for placement of tasks with the awsvpc network mode.

  • The ENIs that are created and attached to your Amazon EC2 Windows instances cannot be detached manually or modified by your account. This is to prevent the accidental deletion of an ENI that is associated with a running task. To release the ENIs for a task, stop the task.

  • There is a limit of 16 subnets and 5 security groups that are able to be specified in the awsvpcConfiguration when running a task or creating a service that uses the awsvpc network mode. For more information, see AwsVpcConfiguration in the Amazon Elastic Container Service API Reference.

  • When a task is started with the awsvpc network mode, the Amazon ECS container agent creates an additional pause container for each task before starting the containers in the task definition. It then configures the network namespace of the pause container by running the amazon-ecs-cni-plugins CNI plugins. The agent then starts the rest of the containers in the task so that they share the network stack of the pause container. This means that all containers in a task are addressable by the IP addresses of the ENI, and they can communicate with each other over the localhost interface.

  • Services with tasks that use the awsvpc network mode only support Application Load Balancers and Network Load Balancers; Classic Load Balancers are not supported. Also, when you create any target groups for these services, you must choose ip as the target type, not instance. This is because tasks that use the awsvpc network mode are associated with an ENI, not with an Amazon EC2 Windows instance. For more information, see Service load balancing.

  • If a VPC is updated, for example to change the DHCP options set it uses, and you want tasks using the VPC to pick up the changes, those tasks must be stopped and new tasks started.

  • The following are not supported when you use awsvpc network mode in a Windows configuration:

    • Dual-stack configuration

    • IPv6

    • ENI trunking

Enabling task networking

In order for tasks to use the awsvpc network mode, it must be specified in the task definition. For more information, see Network mode. Then, when you run a task or create a service, specify a network configuration that includes one or more subnets in which to place your tasks and one or more security groups to attach to its associated ENI. The tasks are placed on compatible Amazon EC2 instances in the same Availability Zones as those subnets, and the specified security groups are associated with the ENI that is provisioned for the task.

Using a VPC in dual-stack mode

When using a VPC in dual-stack mode, your tasks can communicate over IPv4 or IPv6, or both. IPv4 and IPv6 addresses are independent of each other and you must configure routing and security in your VPC separately for IPv4 and IPv6. For more information about configuring your VPC for dual-stack mode, see Migrating to IPv6 in the Amazon VPC User Guide.

One of the benefits of using a VPC in dual-stack mode is that tasks that are assigned an IPv6 address are able to access the internet as long as the VPC is configured with either an internet gateway or an egress-only internet gateway. NAT gateways are not needed. For more information, see Internet gateways and Egress-only internet gateways in the Amazon VPC User Guide.

Amazon ECS tasks are assigned an IPv6 address if the following conditions are met:

  • The Amazon EC2 Linux instance hosting the task is using version 1.45.0 or later of the container agent. For information on checking the agent version your instance is using, and updating it if needed, see Updating the Amazon ECS container agent.

  • The dualStackIPv6 account setting is enabled. For more information, see Account settings.

  • Your task is using the awsvpc network mode.

  • Your VPC and subnet are configured for IPv6 and that network interfaces created in the specified subnet should be assigned an IPv6 address. For more information about configuring your VPC for dual-stack mode, see Migrating to IPv6 and Modify the IPv6 addressing attribute for your subnet in the Amazon VPC User Guide.