When a consumer receives and processes a message from a queue, the message remains in the queue. Amazon SQS doesn't automatically delete the message: Because it's a distributed system, there is no guarantee that the component will actually receive the message (the connection can break or a component can fail to receive the message). Thus, the consumer must delete the message from the queue after receiving and processing it.
Immediately after the message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consuming components from receiving and processing the message.
For standard queues, the visibility timeout isn't a guarantee against receiving a message twice. For more information, see At-Least-Once Delivery.
FIFO queues allow the producer or consumer to attempt multiple retries:
If the producer detects a failed
SendMessageaction, it can retry sending as many times as necessary, using the same message deduplication ID. Assuming that the producer receives at least one acknowledgement before the deduplication interval expires, multiple retries neither affect the ordering of messages nor introduce duplicates.
If the consumer detects a failed
ReceiveMessageaction, it can retry as many times as necessary, using the same receive request attempt ID. Assuming that the consumer receives at least one acknowledgement before the visibility timeout expires, multiple retries do not affect the ordering of messages.
When you receive a message with a message group ID, no more messages for the same message group ID are returned unless you delete the message or it becomes visible.
A message is considered to be in flight after it's received from a queue by a consumer, but not yet deleted from the queue.
For standard queues, there can be a maximum of 120,000 inflight messages per queue. If you reach this limit, Amazon SQS returns the
OverLimit error message.
To avoid reaching the limit, you should delete messages from the queue after they're processed. You can also increase the number of queues you use to process your messages.
For FIFO queues, there can be a maximum of 20,000 inflight messages per queue. If you reach this limit, Amazon SQS returns no error messages.
The following figure illustrates the visibility timeout.
Configuring the Visibility Timeout
The visibility timeout clock starts ticking once Amazon SQS returns the message. During that time, the component processes and deletes the message. But what happens if the component fails before deleting the message? If your system doesn't call DeleteMessage for that message before the visibility timeout expires, the message again becomes visible to the ReceiveMessage calls placed by the components in your system and it will be received again. If a message should only be received once, your system should delete it within the duration of the visibility timeout.
Each queue starts with a default setting of 30 seconds for the visibility timeout. You can change that setting for the entire queue. Typically, you'll set the visibility timeout to the average time it takes to process and delete a message from the queue. When receiving messages, you can also set a special visibility timeout for the returned messages without changing the overall queue timeout.
If you don't know how long it takes to process a message, specify the initial visibility timeout (for example, 2 minutes) and the period of time after which you can check whether the message is processed (for example, 1 minute). If the message isn't processed, extend the visibility timeout (for example, to 3 minutes).
Changing a Message's Visibility Timeout
When you receive a message for a queue and begin to process it, the visibility timeout for
the queue may be insufficient (for example, you might need to process and delete a
message). You can shorten or extend a message's visibility by specifying a new
timeout value using the
For example, if the timeout for a queue is 60 seconds, 15 seconds have elapsed, and you send a
ChangeMessageVisibility call with
VisibilityTimeout set to 10 seconds,
the total timeout value will be the elapsed time (15 seconds) plus the new timeout value (10 seconds),
a total of 25 seconds. Sending a call after 25 seconds will result in an error.
The new timeout period will take effect from the time you call the
action. In addition, the new timeout period will apply only to the particular receipt of the message.
ChangeMessageVisibility action does not affect the timeout of later receipts of the
message or later queues.
Terminating a Message's Visibility Timeout
When you receive a message from a queue, you might find that you actually don't want to process and delete that message. Amazon SQS allows you to terminate the visibility timeout for a specific message. This makes the message immediately visible to other components in the system and available for processing.
To terminate a message's visibility timeout after calling
VisibilityTimeout set to 0 seconds.
API Actions Related to Visibility Timeout
The following table lists the API actions to use to manipulate the visibility timeout.
Use each action's
VisibilityTimeout parameter to set or get the value.
|To do this...||Use this action|
Set the visibility timeout for a queue
Get the visibility timeout for a queue
Set the visibility timeout for the received messages without affecting the queue's visibility timeout
Extending or terminating a message's visibility timeout
Extending or terminating the visibility timeout for up to ten messages.