Managing revisions in AWS Batch job definitions

This example assumes that you're deploying batch jobs by using the AWS Batch service and using Terraform as an IaC tool.

Challenge

AWS Batch creates a new revision of a job definition with each update, which leads to an accumulation of revisions over time. This can complicate resource management and create confusion about which revisions are current, especially in enterprise environments where job definitions are frequently updated and downstream resources require accurate references to the latest versions.

Solution

The following Terraform code uses several key components to address AWS Batch revision management.

resource "terraform_data" "batch_job_definition_cleanup" {
  triggers_replace = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "..."  # AWS CLI commands to list and de-register older revisions
  }
}

In this code:

• terraform_data is combined with a local-exec provisioner to run AWS Command Line Interface (AWS CLI) commands during Terraform operations.

• Timestamp triggers ensure that every terraform apply command runs the cleanup script.

• AWS CLI integration queries AWS Batch for all active job definition revisions and deregisters older revisions.

• You can add a custom retention configuration to the command section to specify how many recent revisions to preserve.

Benefits

• Simplified management: Prevents accumulation of outdated revisions to job definitions.

• Clear references: Maintains accurate pointers to current versions of job definitions.

• Resource optimization: Reduces clutter in the AWS Batch environment.

• Automation: Integrates cleanly with existing Terraform workflows.

This approach demonstrates how Terraform's extensibility features can overcome provider limitations while maintaining IaC best practices. You can use this solution as a template for similar scenarios where dynamic resource discovery and management are needed.