Troubleshooting API Request Errors
In the Amazon EC2 Query API, errors codes are indicated as being either client or server. Client errors usually occur because there is a problem with the structure, content, or validity of the request. Server errors usually indicate a server-side issue.
For more information about API error codes, see Error Codes.
Query API Request Rate
We throttle Amazon EC2 API requests for each AWS account on a per-region basis to help the performance of the service. We ensure that all calls to the Amazon EC2 API (whether they originate from an application, calls to a command line interface, or the Amazon EC2 console) don't exceed the maximum allowed API request rate. The maximum API request rate may vary across regions. Note that API requests made by IAM users are attributed to the underlying AWS account.
The Amazon EC2 API actions are divided into the following categories:
Describe actions, such as
DescribeVolumes. These requests simply retrieve cached data, so they have the highest request limit.
Modify actions, such as
CreateVolumes. These requests create or modify resources, so they have a lower request limit than describe calls.
RevokeSecurityGroupIngressactions. These requests take the most time and resource to complete, so they have the lowest request limit.
If an API request exceeds the API request rate for its category, the request
RequestLimitExceeded error code. To prevent this error,
ensure that your application doesn't retry API requests at a high rate. You can do
this by using care when polling and by using exponential backoff retries.
Your application might need to call an API repeatedly to check for an update in status. Before you start polling, give the request time to potentially complete. When you begin polling, use an appropriate sleep interval between successive requests. For best results, use an increasing sleep interval.
Retries or batch processing
Your application might need to retry an API request after it fails, or to process multiple resources (for example, all your volumes). To lower the rate of API requests, use an appropriate sleep interval between successive requests. For best results, use an increasing or variable sleep interval.
Calculating the sleep interval
When you have to poll or retry an API request, we recommend using an exponential backoff algorithm to calculate the sleep interval between API calls. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses. For more information, and implementation examples of this algorithm, see Error Retries and Exponential Backoff in AWS.
The Amazon EC2 API follows an eventual consistency model, due to the distributed nature of the system supporting the API. This means that the result of an API command you run that affects your Amazon EC2 resources might not be immediately visible to all subsequent commands you run. You should keep this in mind when you carry out an API command that immediately follows a previous API command.
Eventual consistency can affect the way you manage your resources. For example, if you run a command to create a resource, it will eventually be visible to other commands. This means that if you run a command to modify or describe the resource that you just created, its ID might not have propagated throughout the system, and you will get an error responding that the resource does not exist.
To manage eventual consistency, you can do the following:
Confirm the state of the resource before you run a command to modify it. Run the appropriate
Describecommand using an exponential backoff algorithm to ensure that you allow enough time for the previous command to propagate through the system. To do this, run the
Describecommand repeatedly, starting with a couple of seconds of wait time, and increasing gradually up to five minutes of wait time.
Add wait time between subsequent commands, even if a
Describecommand returns an accurate response. Apply an exponential backoff algorithm starting with a couple of seconds of wait time, and increase gradually up to about five minutes of wait time.
Eventual Consistency Error Examples
The following are examples of error codes you may encounter as a result of eventual consistency.
If you successfully run the
RunInstancescommand, and then immediately run another command using the instance ID that was provided in the response of
RunInstances, it may return an
InvalidInstanceID.NotFounderror. This does not mean the instance does not exist.
Some specific commands that may be affected are:
DescribeInstances: To confirm the actual state of the instance, run this command using an exponential backoff algorithm.
TerminateInstances: To confirm the state of the instance, first run the
DescribeInstancescommand using an exponential backoff algorithm.
If you get an
InvalidInstanceID.NotFounderror after running
TerminateInstances, this does not mean that the instance is or will be terminated. Your instance could still be running. This is why it is important to first confirm the instance’s state using
If you successfully run the
CreateSecurityGroupcommand, and then immediately run another command using the instance ID that was provided in the response of
CreateSecurityGroup, it may return an
InvalidGroup.NotFounderror. To confirm the state of the security group, run the
DescribeSecurityGroupscommand using an exponential backoff algorithm.
You have requested more instances than your current instance limit allows for the specified instance type. You could reach this limit unexpectedly if you are launching and terminating instances rapidly, as terminated instances count toward your instance limit for a while after they've been terminated.
By default, AWS Identity and Access Management (IAM) users don't have permission to create or modify Amazon EC2
resources, or perform tasks using the Amazon EC2 API, unless they've been explicitly
granted permission through IAM policies. If an IAM user attempts to perform an
action for which permission has not been granted, the request returns the following
This error may occur when a policy is unintentionally restrictive. For example, to allow an IAM user to launch instances into a specific subnet, you need to grant permissions for the following resources by specifying their ARNs in your IAM policy: instances, volumes, AMIs, the specific subnet, network interfaces, key pairs, and security groups. If you omit the permission for volumes, for example, the user is only able to launch an instance from an instance store-backed AMI, as they do not have permission to create the root EBS volume for an EBS-backed instance.
For more information about creating IAM policies for Amazon EC2, see IAM Policies for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances.
Currently, not all API actions support resource-level permissions; we'll add support for more in the future. For more information about which ARNs you can use with which Amazon EC2 API actions, see Granting IAM Users Required Permissions for Amazon EC2 Resources.