When a consuming component in your system receives and processes a message from the queue, the message remains in the queue. Why doesn't Amazon SQS automatically delete it?
Because your system is distributed, there's no guarantee that the component will actually receive the message (it's possible the connection could break or the component could fail before receiving the message). Therefore, Amazon SQS does not delete the message, and instead, your consuming component must delete the message from the queue after receiving and processing it.
Immediately after the component receives the message, the message is still in the queue. However, you don't want other components in the system receiving and processing the message again. Therefore, Amazon SQS blocks them with a visibility timeout, which is a period of time during which Amazon SQS prevents other consuming components from receiving and processing that message. The following figure and discussion illustrate the concept.
There is a 120,000 limit for the number of inflight messages per queue. Messages are inflight after they have been received from the queue by a consuming component, but have not yet been deleted from the queue. If you reach the 120,000 limit, you will receive an OverLimit error message from Amazon SQS. To help avoid reaching the limit, you should delete the messages from the queue after they have been processed. You can also increase the number of queues you use to process the messages.
General Recommendations for Visibility Timeout
The visibility timeout clock starts ticking once Amazon SQS returns the message. During that time, the component processes and deletes the message. But what happens if the component fails before deleting the message? If your system doesn't call DeleteMessage for that message before the visibility timeout expires, the message again becomes visible to the ReceiveMessage calls placed by the components in your system and it will be received again. If a message should only be received once, your system should delete it within the duration of the visibility timeout.
Each queue starts with a default setting of 30 seconds for the visibility timeout. You can change that setting for the entire queue. Typically, you'll set the visibility timeout to the average time it takes to process and delete a message from the queue. When receiving messages, you can also set a special visibility timeout for the returned messages without changing the overall queue timeout.
We recommend that if you have a system that produces messages that require varying amounts of time to process and delete, you create multiple queues, each with a different visibility timeout setting. Your system can then send all messages to a single queue that forwards each message to another queue with the appropriate visibility timeout based on the expected processing and deletion time for that message.
Changing a Message's Visibility Timeout
When you receive a message for a queue and begin to process it, the visibility timeout for the queue
may be insufficient (for example, more time might be needed to process and delete the message). You
can shorten or extend a message's visibility by specifying a new timeout value using the
For example, if the timeout for a queue is 60 seconds, 15 seconds have elapsed, and you send a
ChangeMessageVisibility call with
VisibilityTimeout set to 10 seconds,
the total timeout value will be the elapsed time (15 seconds) plus the new timeout value (10 seconds),
a total of 25 seconds. Sending a call after 25 seconds will result in an error.
The new timeout period will take effect from the time you call the
action. In addition, the new timeout period will apply only to the particular receipt of the message.
ChangeMessageVisibility action does not affect the timeout of later receipts of the
message or later queues.
Terminating a Message's Visibility Timeout
When you receive a message from the queue, you might find that you actually don't want to process and delete that message. Amazon SQS allows you to terminate the visibility timeout for a specific message, which immediately makes the message visible to other components in the system to process. To do this, you call ChangeMessageVisibility with
API Actions Related to Visibility Timeout
The following table lists the API actions to use to manipulate the visibility timeout.
Use each action's
VisibilityTimeout parameter to set or get the value.
|To do this...||Use this action|
Set the visibility timeout for a queue
Get the visibility timeout for a queue
Set the visibility timeout for the received messages without affecting the queue's visibility timeout
Extending or terminating a message's visibility timeout
Extending or terminating the visibility timeout for up to ten messages.