Tutorial: Listening for AWS Batch CloudWatch Events - AWS Batch

Tutorial: Listening for AWS Batch CloudWatch Events

In this tutorial, you set up a simple AWS Lambda function that listens for AWS Batch job events and writes them out to a CloudWatch Logs log stream.


This tutorial assumes that you have a working compute environment and job queue that are ready to accept jobs. If you don't have a running compute environment and job queue to capture events from, follow the steps in Getting Started with AWS Batch to create one. At the end of this tutorial, you can optionally submit a job to this job queue to test that you have configured your Lambda function correctly.

Step 1: Create the Lambda Function

In this procedure, you create a simple Lambda function to serve as a target for AWS Batch event stream messages.

To create a target Lambda function

  1. Open the AWS Lambda console at https://console.aws.amazon.com/lambda/.

  2. Choose Create a Lambda function, Author from scratch.

  3. For Function name, enter batch-event-stream-handler.

  4. For Runtime, choose Python 3.8.

  5. Choose Create function.

  6. In the Function code section, edit the sample code to match the following example:

    import json def lambda_handler(event, _context): # _context is not used del _context if event["source"] != "aws.batch": raise ValueError("Function only supports input from events with a source type of: aws.batch") print(json.dumps(event))

    This is a simple Python 3.8 function that prints the events sent by AWS Batch. If everything is configured correctly, at the end of this tutorial, you will see that the event details appear in the CloudWatch Logs log stream that's associated with this Lambda function.

  7. Choose Deploy.

Step 2: Register Event Rule

In this section, you create a CloudWatch Events event rule that captures job events that are coming from your AWS Batch resources. This rule captures all events coming from AWS Batch within the account where it's defined. The job messages themselves contain information about the event source, including the job queue where it was submitted. You can use this information to filter and sort events programmatically.


If you use the AWS Management Console to create an event rule, the console automatically adds the IAM permissions for CloudWatch Events to call your Lambda function. However, if you're creating an event rule using the AWS CLI, you must grant permissions explicitly. For more information, see Events and Event Patterns in the Amazon CloudWatch User Guide.

To create your CloudWatch Events rule

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. On the navigation pane, choose Events, Create rule.

  3. For Event source, select Event Pattern as the event source, and then select Build custom event pattern.

  4. Paste the following event pattern into the text area.

    { "source": [ "aws.batch" ] }

    This rule applies across all of your AWS Batch groups and to every AWS Batch event. Alternatively, you can create a more specific rule to filter out some results.

  5. For Targets, choose Add target. For Target type, choose Lambda function, and select your Lambda function.

  6. Choose Configure details.

  7. For Rule definition, enter a name and description for your rule and choose Create rule.

Step 3: Test Your Configuration

You can now test your CloudWatch Events configuration by submitting a job to your job queue. If everything is configured properly, your Lambda function is triggered and it writes the event data to a CloudWatch Logs log stream for the function.

To test your configuration

  1. Open the AWS Batch console at https://console.aws.amazon.com/batch/.

  2. Submit a new AWS Batch job. For more information, see Submitting a Job.

  3. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  4. On the navigation pane, choose Logs and select the log group for your Lambda function (for example, /aws/lambda/my-function).

  5. Select a log stream to view the event data.