Amazon Simple Queue Service
Developer Guide

What is Amazon Simple Queue Service?

Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly-scalable hosted queue for storing messages as they travel between applications or microservices. It moves data between distributed application components and helps you decouple these components. Amazon SQS provides familiar middleware constructs such as dead-letter queues and poison-pill management. It also provides a generic web services API and can be accessed by any programming language that the AWS SDK supports. Amazon SQS supports both standard and FIFO queues.

What Can I Use Amazon SQS For?

Use Amazon SQS for cases such as the following:

  • Decoupling the components of an application – You have a queue of work items and want to track the successful completion of each item independently. Amazon SQS tracks the ACK/FAIL results, so the application doesn't have to maintain a persistent checkpoint or cursor. After a configured visibility timeout, Amazon SQS deletes acknowledged messages and redelivers failed messages.

  • Configuring individual message delay – You have a job queue and you need to schedule individual jobs with a delay. With standard queues, you can configure individual messages to have a delay of up to 15 minutes.

  • Dynamically increasing concurrency or throughput at read time – You have a work queue and want to add more consumers until the backlog is cleared. Amazon SQS requires no pre-provisioning.

  • Scaling transparently – You buffer requests and the load changes as a result of occasional load spikes or the natural growth of your business. Because Amazon SQS can process each buffered request independently, Amazon SQS can scale transparently to handle the load without any provisioning instructions from you.

What Type of Queue Do I Need?

Standard Queue FIFO Queue

Available in all regions.

Unlimited Throughput – Standard queues support a nearly unlimited number of transactions per second (TPS) per API action.

At-Least-Once Delivery – A message is delivered at least once, but occasionally more than one copy of a message is delivered.

Best-Effort Ordering – Occasionally, messages might be delivered in an order different from which they were sent.

Available in the US East (N. Virginia), US East (Ohio), US West (Oregon), and EU (Ireland) regions.

High Throughput – FIFO queues support up to 300 messages per second (300 send, receive, or delete operations per second). When you batch 10 messages per operation (maximum), FIFO queues can support up to 3,000 messages per second. To request a limit increase, file a support request.

Exactly-Once Processing – A message is delivered once and remains available until a consumer processes and deletes it. Duplicates aren't introduced into the queue.

First-In-First-Out Delivery – The order in which messages are sent and received is strictly preserved.

Send data between applications when the throughput is important, for example:

  • Decouple live user requests from intensive background work: let users upload media while resizing or encoding it.

  • Allocate tasks to multiple worker nodes: process a high number of credit card validation requests.

  • Batch messages for future processing: schedule multiple entries to be added to a database.

Send data between applications when the order of events is important, for example:

  • Ensure that user-entered commands are executed in the right order.

  • Display the correct product price by sending price modifications in the right order.

  • Prevent a student from enrolling in a course before registering for an account.

What Are the Main Features of Amazon SQS?

Amazon SQS provides the following major features:

  • Redundant infrastructure – Standard queues support at-least-once message delivery, while FIFO queues support exactly-once message processing. Amazon SQS provides highly-concurrent access to messages and high availability for producing and consuming messages.

  • Multiple producers and consumers – Multiple parts of your system can send or receive messages at the same time. Amazon SQS locks the message during processing, keeping other parts of your system from processing the message simultaneously.

  • Configurable settings per queue – All of your queues don't have to be exactly alike. For example, you can optimize one queue for messages that require a longer processing time than others.

  • Variable message size – Your messages can be up to 262,144 bytes (256 KB) in size. You can store the contents of larger messages using the Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB, with Amazon SQS holding a pointer to the Amazon S3 object. For more information, see Managing Amazon SQS Messages with Amazon S3. You can also split a large message into smaller ones.

  • Access control – You control who can send messages to a queue, and who can receive messages from a queue.

  • Delay queues – You can set a default delay on a queue, so that delivery of all enqueued messages is postponed for the specified duration. You can set the delay value when you create a queue with CreateQueue, and you can update the value with SetQueueAttributes. If you update the value, the new value affects only messages enqueued after the update.

What Is the Basic Architecture of Amazon SQS?

There are three main actors in the overall system:

  • The components of your distributed system

  • Queues

  • Messages in the queues

In the following diagram, your system has several components that send messages to the queue and receive messages from the queue. The diagram shows that a single queue, which has its messages (A-E), is redundantly saved across multiple Amazon SQS servers.

We Want to Hear from You

We welcome your feedback. To contact us, visit the Amazon SQS Discussion Forum.