Troubleshooting queue processing by Lambda functions

This section uses SQS as an example but the principles apply to other services that queue messages for Lambda functions. The Lambda service processes messages from SQS in batches determined by the function’s batch size property. If there are still messages on the queue, Lambda then scales up the processing function up to an additional 60 instances per minute, to retrieve and process batches more quickly. In normal operation, this allows Lambda to maintain efficient processing of the queue, while scaling up and down in response to traffic.

This continues as long as the function is not producing errors, there is sufficient unreserved concurrency in your account in the current Region (or there is remaining reserved concurrency for the function), and there are messages available in the queue.

Service	Default batch size	Maximum batch size
Amazon Kinesis	100	10,000
Amazon DynamoDB Streams	100	1,000
Amazon SQS	10	10,000
Amazon MSK	100	10,000

To send batches of messages to SQS, this example uses the following code to generate messages from a Node.js script:

const AWS = require('aws-sdk')
AWS.config.update({region: 'us-east-1'})
const sqs = new AWS.SQS()
const BATCHES = 10

const sendMessage = async () => {
  const params = {
    Entries: [],
    QueueUrl: 'https://sqs.us-east-1.amazonaws.com/1234123412/testQ'
  }

  for (let i = 0; i < 10; i++ ) {
    const Id = parseInt(Math.random()*100000).toString()
    params.Entries.push ({
        Id, 
        MessageBody: `Message payload: ${Id}`
    })
  }

  await sqs.sendMessageBatch(params).promise() 
}

const main = async() => {
  for (let i = 0; i < BATCHES; i++ ) {
    await sendMessage()
  }
}

main().catch(error => console.error(error))

In this example:

• An event producer is adding 10 messages to an SQS queue every second.

• The AWS account only has unreserved concurrency of 100.