Amazon Virtual Private Cloud
User Guide

Troubleshooting NAT Gateways

The following topics help you to troubleshoot common issues that you might encounter when creating or using a NAT gateway.

NAT Gateway Goes to a Status of Failed

Problem

You create a NAT gateway and it goes to a status of Failed.

Cause

There was an error when the NAT gateway was created. The returned error message provides the reason for the error.

Solution

To view the error message, go to the Amazon VPC console, and then choose NAT Gateways. Select your NAT gateway, and then view the error message in the Status box in the details pane.

The following table lists the possible causes of the failure as indicated in the Amazon VPC console. After you've applied any of the remedial steps indicated, you can try to create a NAT gateway again.

Note

A failed NAT gateway is automatically deleted after a short period; usually about an hour.

Displayed error Cause Solution
Subnet has insufficient free addresses to create this NAT gateway The subnet that you specified does not have any free private IP addresses. The NAT gateway requires a network interface with a private IP address allocated from the subnet's range. Check how many IP addresses are available in your subnet by going to the Subnets page in the Amazon VPC console. You can view the Available IPs in the details pane for your subnet. To create free IP addresses in your subnet, you can delete unused network interfaces, or terminate instances that you do not require.
Network vpc-xxxxxxxx has no internet gateway attached A NAT gateway must be created in a VPC with an internet gateway. Create and attach an internet gateway to your VPC. For more information, see Creating and Attaching an Internet Gateway.
Elastic IP address eipalloc-xxxxxxxx could not be associated with this NAT gateway The Elastic IP address that you specified does not exist or could not be found. Check the allocation ID of the Elastic IP address to ensure that you entered it correctly. Ensure that you have specified an Elastic IP address that's in the same AWS Region in which you're creating the NAT gateway.
Elastic IP address eipalloc-xxxxxxxx is already associated The Elastic IP address that you specified is already associated with another resource, and cannot be associated with the NAT gateway. Check which resource is associated with the Elastic IP address. Go to the Elastic IPs page in the Amazon VPC console, and view the values specified for the instance ID or network interface ID. If you do not require the Elastic IP address for that resource, you can disassociate it. Alternatively, allocate a new Elastic IP address to your account. For more information, see Working with Elastic IP Addresses.
Network interface eni-xxxxxxxx, created and used internally by this NAT gateway is in an invalid state. Please try again. There was a problem creating or using the network interface for the NAT gateway. You cannot resolve this error. Try creating a NAT gateway again.

Elastic IP Address and NAT Gateway Limits

Problem

When you try to allocate an Elastic IP address, you get the following error:

The maximum number of addresses has been reached.

When you try to create a NAT gateway, you get the following error:

Performing this operation would exceed the limit of 5 NAT gateways

Cause

There are 2 possible causes:

  • You've reached the limit for the number of Elastic IP addresses for your account for that Region.

  • You've reached the limit for the number of NAT gateways for your account for that Availability Zone.

Solution

If you've reached your Elastic IP address limit, you can disassociate an Elastic IP address from another resource. Alternatively, you can request a limit increase using the Amazon VPC Limits form.

If you've reached your NAT gateway limit, you can do one of the following:

  • Request a limit increase using the Amazon VPC Limits form. The NAT gateway limit is enforced per Availability Zone.

  • Check the status of your NAT gateway. A status of Pending, Available, or Deleting counts against your limit. If you've recently deleted a NAT gateway, wait a few minutes for the status to go from Deleting to Deleted. Then try creating a new NAT gateway.

  • If you do not need your NAT gateway in a specific Availability Zone, try creating a NAT gateway in an Availability Zone where you haven't reached your limit.

For more information, see Amazon VPC Limits.

Availability Zone Is Unsupported

Problem

When you try to create a NAT gateway, you get the following error: NotAvailableInZone.

Cause

You might be trying to create the NAT gateway in a constrained Availability Zone — a zone in which our ability to expand is constrained.

Solution

We cannot support NAT gateways in these Availability Zones. You can create a NAT gateway in another Availability Zone and use it for private subnets in the constrained zone. You can also move your resources to an unconstrained Availability Zone so that your resources and your NAT gateway are in the same zone.

NAT Gateway Is No Longer Visible

Problem

You created a NAT gateway but it's no longer visible in the Amazon VPC console.

Cause

There may have been an error when your NAT gateway was being created, and it failed. A NAT gateway with a status of Failed is visible in the Amazon VPC console for a short time (usually an hour). After an hour, it's automatically deleted.

Solution

Review the information in NAT Gateway Goes to a Status of Failed, and try creating a new NAT gateway.

NAT Gateway Doesn't Respond to a Ping Command

Problem

When you try to ping a NAT gateway's Elastic IP address or private IP address from the internet (for example, from your home computer) or from an instance in your VPC, you do not get a response.

Cause

A NAT gateway only passes traffic from an instance in a private subnet to the internet.

Solution

To test that your NAT gateway is working, see Testing a NAT Gateway.

Instances Cannot Access the Internet

Problem

You created a NAT gateway and followed the steps to test it, but the ping command fails, or your instances in the private subnet cannot access the internet.

Causes

The cause of this problem might be one of the following:

  • The NAT gateway is not ready to serve traffic.

  • Your route tables are not configured correctly.

  • Your security groups or network ACLs are blocking inbound or outbound traffic.

  • You're using an unsupported protocol.

Solution

Check the following information:

  • Check that the NAT gateway is in the Available state. In the Amazon VPC console, go to the NAT Gateways page and view the status information in the details pane. If the NAT gateway is in a failed state, there may have been an error when it was created. For more information, see NAT Gateway Goes to a Status of Failed.

  • Check that you've configured your route tables correctly:

    • The NAT gateway must be in a public subnet with a route table that routes internet traffic to an internet gateway. For more information, see Creating a Custom Route Table.

    • Your instance must be in a private subnet with a route table that routes internet traffic to the NAT gateway. For more information, see Updating Your Route Table.

    • Check that there are no other route table entries that route all or part of the internet traffic to another device instead of the NAT gateway.

  • Ensure that your security group rules for your private instance allow outbound internet traffic. For the ping command to work, the rules must also allow outbound ICMP traffic.

    Note

    The NAT gateway itself allows all outbound traffic and traffic received in response to an outbound request (it is therefore stateful).

  • Ensure that the network ACLs that are associated with the private subnet and public subnets do not have rules that block inbound or outbound internet traffic. For the ping command to work, the rules must also allow inbound and outbound ICMP traffic.

    Note

    You can enable flow logs to help you diagnose dropped connections because of network ACL or security group rules. For more information, see VPC Flow Logs.

  • If you are using the ping command, ensure that you are pinging a website that has ICMP enabled. If ICMP is not enabled, you will not receive reply packets. To test this, perform the same ping command from the command line terminal on your own computer.

  • Check that your instance is able to ping other resources, for example, other instances in the private subnet (assuming that security group rules allow this).

  • Ensure that your connection is using a TCP, UDP, or ICMP protocol only.

TCP Connection to a Destination Fails

Problem

Some of your TCP connections from instances in a private subnet to a specific destination through a NAT gateway are successful, but some are failing or timing out.

Causes

The cause of this problem might be one of the following:

  • The destination endpoint is responding with fragmented TCP packets. A NAT gateway currently does not support IP fragmentation for TCP or ICMP. For more information, see Comparison of NAT Instances and NAT Gateways.

  • The tcp_tw_recycle option is enabled on the remote server, which is known to cause issues when there are multiple connections from behind a NAT device.

Solutions

Verify whether the endpoint to which you're trying to connect is responding with fragmented TCP packets by doing the following:

  1. Use an instance in a public subnet with a public IP address to trigger a response large enough to cause fragmentation from the specific endpoint.

  2. Use the tcpdump utility to verify that the endpoint is sending fragmented packets.

    Important

    You must use an instance in a public subnet to perform these checks. You cannot use the instance from which the original connection was failing, or an instance in a private subnet behind a NAT gateway or a NAT instance.

    Note

    Diagnostic tools that send or receive large ICMP packets will report packet loss. For example, the command ping -s 10000 example.com does not work behind a NAT gateway.

  3. If the endpoint is sending fragmented TCP packets, you can use a NAT instance instead of a NAT gateway.

If you have access to the remote server, you can verify whether the tcp_tw_recycle option is enabled by doing the following:

  1. From the server, run the following command.

    cat /proc/sys/net/ipv4/tcp_tw_recycle

    If the output is 1, then the tcp_tw_recycle option is enabled.

  2. If tcp_tw_recycle is enabled, we recommend disabling it. If you need to reuse connections, tcp_tw_reuse is a safer option.

If you don't have access to the remote server, you can test by temporarily disabling the tcp_timestamps option on an instance in the private subnet. Then connect to the remote server again. If the connection is successful, the cause of the previous failure is likely because tcp_tw_recycle is enabled on the remote server. If possible, contact the owner of the remote server to verify if this option is enabled and request for it to be disabled.

Traceroute Output Does Not Display NAT Gateway Private IP Address

Problem

Your instance can access the internet, but when you perform the traceroute command, the output does not display the private IP address of the NAT gateway.

Cause

Your instance is accessing the internet using a different gateway, such as an internet gateway.

Solution

In the route table of the subnet in which your instance is located, check the following information:

  • Ensure that there is a route that sends internet traffic to the NAT gateway.

  • Ensure that there isn't a more specific route that's sending internet traffic to other devices, such as a virtual private gateway or an internet gateway.

Internet Connection Drops After 350 Seconds

Problem

Your instances can access the internet, but the connection drops after 350 seconds.

Cause

If a connection that's using a NAT gateway is idle for 350 seconds or more, the connection times out.

Solution

To prevent the connection from being dropped, you can initiate more traffic over the connection. Alternatively, you can enable TCP keepalive on the instance with a value less than 350 seconds.

IPsec Connection Cannot Be Established

Problem

You cannot establish an IPsec connection to a destination.

Cause

NAT gateways currently do not support the IPsec protocol.

Solution

You can use NAT-Traversal (NAT-T) to encapsulate IPsec traffic in UDP, which is a supported protocol for NAT gateways. Ensure that you test your NAT-T and IPsec configuration to verify that your IPsec traffic is not dropped.

Cannot Initiate More Connections

Problem

You have existing connections to a destination through a NAT gateway, but cannot establish more connections.

Cause

You might have reached the limit for simultaneous connections for a single NAT gateway. For more information, see NAT Gateway Rules and Limitations. If your instances in the private subnet create a large number of connections, you might reach this limit.

Solution

Do one of the following:

  • Create a NAT gateway per Availability Zone and spread your clients across those zones.

  • Create additional NAT gateways in the public subnet and split your clients into multiple private subnets, each with a route to a different NAT gateway.

  • Limit the number of connections your clients can create to the destination.

  • Close idle connections to release capacity.