AWS Batch Construct Library
This module is part of the AWS Cloud Development Kit project.
AWS Batch is a batch processing tool for efficiently running hundreds of thousands computing jobs in AWS. Batch can dynamically provision Amazon EC2 Instances to meet the resource requirements of submitted jobs and simplifies the planning, scheduling, and executions of your batch workloads. Batch achieves this through four different resources:
- ComputeEnvironments: Contain the resources used to execute Jobs
- JobDefinitions: Define a type of Job that can be submitted
- JobQueues: Route waiting Jobs to ComputeEnvironments
- SchedulingPolicies: Applied to Queues to control how and when Jobs exit the JobQueue and enter the ComputeEnvironment
ComputeEnvironments can be managed or unmanaged. Batch will automatically provision EC2 Instances in a managed ComputeEnvironment and will not provision any Instances in an unmanaged ComputeEnvironment. Managed ComputeEnvironments can use ECS, Fargate, or EKS resources to spin up EC2 Instances in (ensure your EKS Cluster has been configured to support a Batch ComputeEnvironment before linking it). You can use Launch Templates and Placement Groups to configure exactly how these resources will be provisioned.
JobDefinitions can use either ECS resources or EKS resources. ECS JobDefinitions can use multiple containers to execute distributed workloads. EKS JobDefinitions can only execute a single container. Submitted Jobs use JobDefinitions as templates.
JobQueues must link at least one ComputeEnvironment. Jobs exit the Queue in FIFO order unless a SchedulingPolicy is specified.
SchedulingPolicys tell the Scheduler how to choose which Jobs should be executed next by the ComputeEnvironment.
Use Cases & Examples
Cost Optimization
Spot Instances
Spot instances are significantly discounted EC2 instances that can be reclaimed at any time by AWS. Workloads that are fault-tolerant or stateless can take advantage of spot pricing. To use spot spot instances, set spot to true on a managed Ec2 or Fargate Compute Environment:
const vpc = new ec2.Vpc(this, 'VPC');
new batch.FargateComputeEnvironment(this, 'myFargateComputeEnv', {
vpc,
spot: true,
});
vpc = ec2.Vpc(self, "VPC")
batch.FargateComputeEnvironment(self, "myFargateComputeEnv",
vpc=vpc,
spot=True
)
Vpc vpc = new Vpc(this, "VPC");
FargateComputeEnvironment.Builder.create(this, "myFargateComputeEnv")
.vpc(vpc)
.spot(true)
.build();
var vpc = new Vpc(this, "VPC");
new FargateComputeEnvironment(this, "myFargateComputeEnv", new FargateComputeEnvironmentProps {
Vpc = vpc,
Spot = true
});
vpc := ec2.NewVpc(this, jsii.String("VPC"))
batch.NewFargateComputeEnvironment(this, jsii.String("myFargateComputeEnv"), &FargateComputeEnvironmentProps{
Vpc: Vpc,
Spot: jsii.Boolean(true),
})
Batch allows you to specify the percentage of the on-demand instance that the current spot price must be to provision the instance using the spotBidPercentage. This defaults to 100%, which is the recommended value. This value cannot be specified for FargateComputeEnvironments and only applies to ManagedEc2EcsComputeEnvironments. The following code configures a Compute Environment to only use spot instances that are at most 20% the price of the on-demand instance price:
Note: For FargateComputeEnvironment, while the FargateComputeEnvironmentProps interface includes properties like replaceComputeEnvironment, terminateOnUpdate, updateTimeout, and updateToLatestImageVersion, these specific properties are not applicable when configuring AWS Batch Fargate compute environments. They primarily apply to EC2-based compute environments. Please refer to the official AWS Batch UpdateComputeEnvironment API documentation and User Guide for details on updating Fargate compute environments.
const vpc = new ec2.Vpc(this, 'VPC');
new batch.ManagedEc2EcsComputeEnvironment(this, 'myEc2ComputeEnv', {
vpc,
spot: true,
spotBidPercentage: 20,
});
vpc = ec2.Vpc(self, "VPC")
batch.ManagedEc2EcsComputeEnvironment(self, "myEc2ComputeEnv",
vpc=vpc,
spot=True,
spot_bid_percentage=20
)
Vpc vpc = new Vpc(this, "VPC");
ManagedEc2EcsComputeEnvironment.Builder.create(this, "myEc2ComputeEnv")
.vpc(vpc)
.spot(true)
.spotBidPercentage(20)
.build();
var vpc = new Vpc(this, "VPC");
new ManagedEc2EcsComputeEnvironment(this, "myEc2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
Spot = true,
SpotBidPercentage = 20
});
vpc := ec2.NewVpc(this, jsii.String("VPC"))
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("myEc2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
Spot: jsii.Boolean(true),
SpotBidPercentage: jsii.Number(20),
})
For stateful or otherwise non-interruption-tolerant workflows, omit spot or set it to false to only provision on-demand instances.
Choosing Your Instance Types
There are several ways to configure instance types for your compute environment:
Using Default Instance Classes (Recommended)
AWS Batch provides default instance classes that automatically select cost-effective, up-to-date instances based on your region. This is the recommended approach for new projects:
const vpc = new ec2.Vpc(this, 'Vpc');
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
new batch.ManagedEc2EcsComputeEnvironment(this, 'Arm64Ec2ComputeEnv', {
vpc,
defaultInstanceClasses: [batch.DefaultInstanceClass.ARM64],
});
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
new batch.ManagedEc2EcsComputeEnvironment(this, 'X86_64Ec2ComputeEnv', {
vpc,
defaultInstanceClasses: [batch.DefaultInstanceClass.X86_64],
});
vpc = ec2.Vpc(self, "Vpc")
# Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
batch.ManagedEc2EcsComputeEnvironment(self, "Arm64Ec2ComputeEnv",
vpc=vpc,
default_instance_classes=[batch.DefaultInstanceClass.ARM64]
)
# Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
batch.ManagedEc2EcsComputeEnvironment(self, "X86_64Ec2ComputeEnv",
vpc=vpc,
default_instance_classes=[batch.DefaultInstanceClass.X86_64]
)
Vpc vpc = new Vpc(this, "Vpc");
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
ManagedEc2EcsComputeEnvironment.Builder.create(this, "Arm64Ec2ComputeEnv")
.vpc(vpc)
.defaultInstanceClasses(List.of(DefaultInstanceClass.ARM64))
.build();
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
ManagedEc2EcsComputeEnvironment.Builder.create(this, "X86_64Ec2ComputeEnv")
.vpc(vpc)
.defaultInstanceClasses(List.of(DefaultInstanceClass.X86_64))
.build();
var vpc = new Vpc(this, "Vpc");
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
new ManagedEc2EcsComputeEnvironment(this, "Arm64Ec2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
DefaultInstanceClasses = new [] { DefaultInstanceClass.ARM64 }
});
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
new ManagedEc2EcsComputeEnvironment(this, "X86_64Ec2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
DefaultInstanceClasses = new [] { DefaultInstanceClass.X86_64 }
});
vpc := ec2.NewVpc(this, jsii.String("Vpc"))
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
// Use ARM64 instances (e.g., m6g, c6g, r6g, c7g families)
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("Arm64Ec2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
DefaultInstanceClasses: []DefaultInstanceClass{
batch.DefaultInstanceClass_ARM64,
},
})
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
// Use x86_64 instances (e.g., m6i, c6i, r6i, c7i families)
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("X86_64Ec2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
DefaultInstanceClasses: []DefaultInstanceClass{
batch.DefaultInstanceClass_X86_64,
},
})
The default_x86_64 and default_arm64 categories are dynamically updated by AWS as new instance families become available in your region.
Using Specific Instance Types Only
To use only specific instance types without any automatic defaults, set useOptimalInstanceClasses: false:
const vpc = new ec2.Vpc(this, 'Vpc');
// Use only R4 instance class (Batch chooses the size)
new batch.ManagedEc2EcsComputeEnvironment(this, 'R4Ec2ComputeEnv', {
vpc,
useOptimalInstanceClasses: false,
instanceClasses: [ec2.InstanceClass.R4],
});
// Use only a specific instance type
new batch.ManagedEc2EcsComputeEnvironment(this, 'M5AdLargeEc2ComputeEnv', {
vpc,
useOptimalInstanceClasses: false,
instanceTypes: [ec2.InstanceType.of(ec2.InstanceClass.M5AD, ec2.InstanceSize.LARGE)],
});
vpc = ec2.Vpc(self, "Vpc")
# Use only R4 instance class (Batch chooses the size)
batch.ManagedEc2EcsComputeEnvironment(self, "R4Ec2ComputeEnv",
vpc=vpc,
use_optimal_instance_classes=False,
instance_classes=[ec2.InstanceClass.R4]
)
# Use only a specific instance type
batch.ManagedEc2EcsComputeEnvironment(self, "M5AdLargeEc2ComputeEnv",
vpc=vpc,
use_optimal_instance_classes=False,
instance_types=[ec2.InstanceType.of(ec2.InstanceClass.M5AD, ec2.InstanceSize.LARGE)]
)
Vpc vpc = new Vpc(this, "Vpc");
// Use only R4 instance class (Batch chooses the size)
// Use only R4 instance class (Batch chooses the size)
ManagedEc2EcsComputeEnvironment.Builder.create(this, "R4Ec2ComputeEnv")
.vpc(vpc)
.useOptimalInstanceClasses(false)
.instanceClasses(List.of(InstanceClass.R4))
.build();
// Use only a specific instance type
// Use only a specific instance type
ManagedEc2EcsComputeEnvironment.Builder.create(this, "M5AdLargeEc2ComputeEnv")
.vpc(vpc)
.useOptimalInstanceClasses(false)
.instanceTypes(List.of(InstanceType.of(InstanceClass.M5AD, InstanceSize.LARGE)))
.build();
var vpc = new Vpc(this, "Vpc");
// Use only R4 instance class (Batch chooses the size)
// Use only R4 instance class (Batch chooses the size)
new ManagedEc2EcsComputeEnvironment(this, "R4Ec2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
UseOptimalInstanceClasses = false,
InstanceClasses = new [] { InstanceClass.R4 }
});
// Use only a specific instance type
// Use only a specific instance type
new ManagedEc2EcsComputeEnvironment(this, "M5AdLargeEc2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
UseOptimalInstanceClasses = false,
InstanceTypes = new [] { InstanceType.Of(InstanceClass.M5AD, InstanceSize.LARGE) }
});
vpc := ec2.NewVpc(this, jsii.String("Vpc"))
// Use only R4 instance class (Batch chooses the size)
// Use only R4 instance class (Batch chooses the size)
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("R4Ec2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
UseOptimalInstanceClasses: jsii.Boolean(false),
InstanceClasses: []InstanceClass{
ec2.InstanceClass_R4,
},
})
// Use only a specific instance type
// Use only a specific instance type
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("M5AdLargeEc2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
UseOptimalInstanceClasses: jsii.Boolean(false),
InstanceTypes: []InstanceType{
ec2.InstanceType_Of(ec2.InstanceClass_M5AD, ec2.InstanceSize_LARGE),
},
})
Using Optimal Instance Classes
By default, useOptimalInstanceClasses is true, which adds the optimal instance type.
Note: Since November 2025, optimal behaves the same as default_x86_64 and is dynamically updated as AWS introduces new instance families. Both options automatically select cost-effective x86_64 instance types (from the m6i, c6i, r6i, and c7i families) based on your region.
You can combine this with additional instance types:
declare const vpc: ec2.IVpc;
const computeEnv = new batch.ManagedEc2EcsComputeEnvironment(this, 'Ec2ComputeEnv', {
vpc,
instanceTypes: [ec2.InstanceType.of(ec2.InstanceClass.M5AD, ec2.InstanceSize.LARGE)],
// useOptimalInstanceClasses: true (default)
});
// Result: ['m5ad.large', 'optimal']
# vpc: ec2.IVpc
compute_env = batch.ManagedEc2EcsComputeEnvironment(self, "Ec2ComputeEnv",
vpc=vpc,
instance_types=[ec2.InstanceType.of(ec2.InstanceClass.M5AD, ec2.InstanceSize.LARGE)]
)
IVpc vpc;
ManagedEc2EcsComputeEnvironment computeEnv = ManagedEc2EcsComputeEnvironment.Builder.create(this, "Ec2ComputeEnv")
.vpc(vpc)
.instanceTypes(List.of(InstanceType.of(InstanceClass.M5AD, InstanceSize.LARGE)))
.build();
IVpc vpc;
var computeEnv = new ManagedEc2EcsComputeEnvironment(this, "Ec2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
InstanceTypes = new [] { InstanceType.Of(InstanceClass.M5AD, InstanceSize.LARGE) }
});
var vpc IVpc
computeEnv := batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("Ec2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
InstanceTypes: []InstanceType{
ec2.InstanceType_Of(ec2.InstanceClass_M5AD, ec2.InstanceSize_LARGE),
},
})
Instance Type Configuration Reference
| Goal | Configuration |
|---|---|
| Use latest x86_64 instances | defaultInstanceClasses: [DefaultInstanceClass.X86_64] or no configuration (default) |
| Use latest ARM64 instances | defaultInstanceClasses: [DefaultInstanceClass.ARM64] |
| Use only specific instance classes | useOptimalInstanceClasses: false + instanceClasses: [...] |
| Use only specific instance types | useOptimalInstanceClasses: false + instanceTypes: [...] |
| Use optimal + additional instances | instanceClasses: [...] or instanceTypes: [...] |
Note: Batch does not allow specifying instance types or classes with different architectures. For example, InstanceClass.A1 (ARM) cannot be specified alongside optimal (x86_64). When using ARM-based instances (e.g., Graviton), use defaultInstanceClasses: [DefaultInstanceClass.ARM64], or set useOptimalInstanceClasses: false and explicitly specify ARM instance classes/types.
Note: useOptimalInstanceClasses and defaultInstanceClasses cannot be used together.
Configure AMIs
You can configure Amazon Machine Images (AMIs). This example configures your ComputeEnvironment to use Amazon Linux 2023.
declare const vpc: ec2.IVpc;
new batch.ManagedEc2EcsComputeEnvironment(this, 'myEc2ComputeEnv', {
vpc,
images: [
{
imageType: batch.EcsMachineImageType.ECS_AL2023,
},
],
});
# vpc: ec2.IVpc
batch.ManagedEc2EcsComputeEnvironment(self, "myEc2ComputeEnv",
vpc=vpc,
images=[batch.EcsMachineImage(
image_type=batch.EcsMachineImageType.ECS_AL2023
)
]
)
IVpc vpc;
ManagedEc2EcsComputeEnvironment.Builder.create(this, "myEc2ComputeEnv")
.vpc(vpc)
.images(List.of(EcsMachineImage.builder()
.imageType(EcsMachineImageType.ECS_AL2023)
.build()))
.build();
IVpc vpc;
new ManagedEc2EcsComputeEnvironment(this, "myEc2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
Images = new [] { new EcsMachineImage {
ImageType = EcsMachineImageType.ECS_AL2023
} }
});
var vpc IVpc
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("myEc2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
Images: []EcsMachineImage{
&EcsMachineImage{
ImageType: batch.EcsMachineImageType_ECS_AL2023,
},
},
})
If your image needs GPU resources, specify ECS_AL2023_NVIDIA:
declare const vpc: ec2.IVpc;
new batch.ManagedEc2EcsComputeEnvironment(this, 'myGpuComputeEnv', {
vpc,
images: [
{
imageType: batch.EcsMachineImageType.ECS_AL2023_NVIDIA,
},
],
});
# vpc: ec2.IVpc
batch.ManagedEc2EcsComputeEnvironment(self, "myGpuComputeEnv",
vpc=vpc,
images=[batch.EcsMachineImage(
image_type=batch.EcsMachineImageType.ECS_AL2023_NVIDIA
)
]
)
IVpc vpc;
ManagedEc2EcsComputeEnvironment.Builder.create(this, "myGpuComputeEnv")
.vpc(vpc)
.images(List.of(EcsMachineImage.builder()
.imageType(EcsMachineImageType.ECS_AL2023_NVIDIA)
.build()))
.build();
IVpc vpc;
new ManagedEc2EcsComputeEnvironment(this, "myGpuComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
Images = new [] { new EcsMachineImage {
ImageType = EcsMachineImageType.ECS_AL2023_NVIDIA
} }
});
var vpc IVpc
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("myGpuComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
Images: []EcsMachineImage{
&EcsMachineImage{
ImageType: batch.EcsMachineImageType_ECS_AL2023_NVIDIA,
},
},
})
Allocation Strategies
| Allocation Strategy | Optimized for | Downsides |
|---|---|---|
| BEST_FIT | Cost | May limit throughput |
| BEST_FIT_PROGRESSIVE | Throughput | May increase cost |
| SPOT_CAPACITY_OPTIMIZED | Least interruption | Only useful on Spot instances |
| SPOT_PRICE_CAPACITY_OPTIMIZED | Least interruption + Price | Only useful on Spot instances |
Batch provides different Allocation Strategies to help it choose which instances to provision. If your workflow tolerates interruptions, you should enable spot on your ComputeEnvironment and use SPOT_PRICE_CAPACITY_OPTIMIZED (this is the default if spot is enabled). This will tell Batch to choose the instance types from the ones you’ve specified that have the most spot capacity available to minimize the chance of interruption and have the lowest price. To get the most benefit from your spot instances, you should allow Batch to choose from as many different instance types as possible. If you only care about minimal interruptions and not want Batch to optimize for cost, use SPOT_CAPACITY_OPTIMIZED. SPOT_PRICE_CAPACITY_OPTIMIZED is recommended over SPOT_CAPACITY_OPTIMIZED for most use cases.
If your workflow does not tolerate interruptions and you want to minimize your costs at the expense of potentially longer waiting times, use AllocationStrategy.BEST_FIT. This will choose the lowest-cost instance type that fits all the jobs in the queue. If instances of that type are not available, the queue will not choose a new type; instead, it will wait for the instance to become available. This can stall your Queue, with your compute environment only using part of its max capacity (or none at all) until the BEST_FIT instance becomes available.
If you are running a workflow that does not tolerate interruptions and you want to maximize throughput, you can use AllocationStrategy.BEST_FIT_PROGRESSIVE. This is the default Allocation Strategy if spot is false or unspecified. This strategy will examine the Jobs in the queue and choose whichever instance type meets the requirements of the jobs in the queue and with the lowest cost per vCPU, just as BEST_FIT. However, if not all of the capacity can be filled with this instance type, it will choose a new next-best instance type to run any jobs that couldn’t fit into the BEST_FIT capacity. To make the most use of this allocation strategy, it is recommended to use as many instance classes as is feasible for your workload. This example shows a ComputeEnvironment that uses BEST_FIT_PROGRESSIVE with 'optimal' and InstanceClass.M5 instance types:
declare const vpc: ec2.IVpc;
const computeEnv = new batch.ManagedEc2EcsComputeEnvironment(this, 'myEc2ComputeEnv', {
vpc,
instanceClasses: [ec2.InstanceClass.M5],
});
# vpc: ec2.IVpc
compute_env = batch.ManagedEc2EcsComputeEnvironment(self, "myEc2ComputeEnv",
vpc=vpc,
instance_classes=[ec2.InstanceClass.M5]
)
IVpc vpc;
ManagedEc2EcsComputeEnvironment computeEnv = ManagedEc2EcsComputeEnvironment.Builder.create(this, "myEc2ComputeEnv")
.vpc(vpc)
.instanceClasses(List.of(InstanceClass.M5))
.build();
IVpc vpc;
var computeEnv = new ManagedEc2EcsComputeEnvironment(this, "myEc2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
InstanceClasses = new [] { InstanceClass.M5 }
});
var vpc IVpc
computeEnv := batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("myEc2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
InstanceClasses: []InstanceClass{
ec2.InstanceClass_M5,
},
})
This example shows a ComputeEnvironment that uses BEST_FIT with 'optimal' instances:
declare const vpc: ec2.IVpc;
const computeEnv = new batch.ManagedEc2EcsComputeEnvironment(this, 'myEc2ComputeEnv', {
vpc,
allocationStrategy: batch.AllocationStrategy.BEST_FIT,
});
# vpc: ec2.IVpc
compute_env = batch.ManagedEc2EcsComputeEnvironment(self, "myEc2ComputeEnv",
vpc=vpc,
allocation_strategy=batch.AllocationStrategy.BEST_FIT
)
IVpc vpc;
ManagedEc2EcsComputeEnvironment computeEnv = ManagedEc2EcsComputeEnvironment.Builder.create(this, "myEc2ComputeEnv")
.vpc(vpc)
.allocationStrategy(AllocationStrategy.BEST_FIT)
.build();
IVpc vpc;
var computeEnv = new ManagedEc2EcsComputeEnvironment(this, "myEc2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
AllocationStrategy = AllocationStrategy.BEST_FIT
});
var vpc IVpc
computeEnv := batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("myEc2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
AllocationStrategy: batch.AllocationStrategy_BEST_FIT,
})
Note: allocationStrategy cannot be specified on Fargate Compute Environments.
Controlling vCPU allocation
You can specify the maximum and minimum vCPUs a managed ComputeEnvironment can have at any given time. Batch will always maintain minvCpus worth of instances in your ComputeEnvironment, even if it is not executing any jobs, and even if it is disabled. Batch will scale the instances up to maxvCpus worth of instances as jobs exit the JobQueue and enter the ComputeEnvironment. If you use AllocationStrategy.BEST_FIT_PROGRESSIVE, AllocationStrategy.SPOT_PRICE_CAPACITY_OPTIMIZED, or AllocationStrategy.SPOT_CAPACITY_OPTIMIZED, batch may exceed maxvCpus; it will never exceed maxvCpus by more than a single instance type. This example configures a minvCpus of 10 and a maxvCpus of 100:
declare const vpc: ec2.IVpc;
new batch.ManagedEc2EcsComputeEnvironment(this, 'myEc2ComputeEnv', {
vpc,
instanceClasses: [ec2.InstanceClass.R4],
minvCpus: 10,
maxvCpus: 100,
});
# vpc: ec2.IVpc
batch.ManagedEc2EcsComputeEnvironment(self, "myEc2ComputeEnv",
vpc=vpc,
instance_classes=[ec2.InstanceClass.R4],
minv_cpus=10,
maxv_cpus=100
)
IVpc vpc;
ManagedEc2EcsComputeEnvironment.Builder.create(this, "myEc2ComputeEnv")
.vpc(vpc)
.instanceClasses(List.of(InstanceClass.R4))
.minvCpus(10)
.maxvCpus(100)
.build();
IVpc vpc;
new ManagedEc2EcsComputeEnvironment(this, "myEc2ComputeEnv", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc,
InstanceClasses = new [] { InstanceClass.R4 },
MinvCpus = 10,
MaxvCpus = 100
});
var vpc IVpc
batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("myEc2ComputeEnv"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
InstanceClasses: []InstanceClass{
ec2.InstanceClass_R4,
},
MinvCpus: jsii.Number(10),
MaxvCpus: jsii.Number(100),
})
Tagging Instances
You can tag any instances launched by your managed EC2 ComputeEnvironments by using the CDK Tags API:
declare const vpc: ec2.IVpc;
const tagCE = new batch.ManagedEc2EcsComputeEnvironment(this, 'CEThatMakesTaggedInstnaces', {
vpc,
});
Tags.of(tagCE).add('super', 'salamander');
# vpc: ec2.IVpc
tag_ce = batch.ManagedEc2EcsComputeEnvironment(self, "CEThatMakesTaggedInstnaces",
vpc=vpc
)
Tags.of(tag_ce).add("super", "salamander")
IVpc vpc;
ManagedEc2EcsComputeEnvironment tagCE = ManagedEc2EcsComputeEnvironment.Builder.create(this, "CEThatMakesTaggedInstnaces")
.vpc(vpc)
.build();
Tags.of(tagCE).add("super", "salamander");
IVpc vpc;
var tagCE = new ManagedEc2EcsComputeEnvironment(this, "CEThatMakesTaggedInstnaces", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc
});
Tags.Of(tagCE).Add("super", "salamander");
var vpc IVpc
tagCE := batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("CEThatMakesTaggedInstnaces"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: Vpc,
})
awscdk.Tags_Of(tagCE).Add(jsii.String("super"), jsii.String("salamander"))
Unmanaged ComputeEnvironments do not support maxvCpus or minvCpus because you must provision and manage the instances yourself; that is, Batch will not scale them up and down as needed.
Sharing a ComputeEnvironment between multiple JobQueues
Multiple JobQueues can share the same ComputeEnvironment. If multiple Queues are attempting to submit Jobs to the same ComputeEnvironment, Batch will pick the Job from the Queue with the highest priority. This example creates two JobQueues that share a ComputeEnvironment:
declare const vpc: ec2.IVpc;
const sharedComputeEnv = new batch.FargateComputeEnvironment(this, 'spotEnv', {
vpc,
spot: true,
});
const lowPriorityQueue = new batch.JobQueue(this, 'JobQueue', {
priority: 1,
});
const highPriorityQueue = new batch.JobQueue(this, 'JobQueue', {
priority: 10,
});
lowPriorityQueue.addComputeEnvironment(sharedComputeEnv, 1);
highPriorityQueue.addComputeEnvironment(sharedComputeEnv, 1);
# vpc: ec2.IVpc
shared_compute_env = batch.FargateComputeEnvironment(self, "spotEnv",
vpc=vpc,
spot=True
)
low_priority_queue = batch.JobQueue(self, "JobQueue",
priority=1
)
high_priority_queue = batch.JobQueue(self, "JobQueue",
priority=10
)
low_priority_queue.add_compute_environment(shared_compute_env, 1)
high_priority_queue.add_compute_environment(shared_compute_env, 1)
IVpc vpc;
FargateComputeEnvironment sharedComputeEnv = FargateComputeEnvironment.Builder.create(this, "spotEnv")
.vpc(vpc)
.spot(true)
.build();
JobQueue lowPriorityQueue = JobQueue.Builder.create(this, "JobQueue")
.priority(1)
.build();
JobQueue highPriorityQueue = JobQueue.Builder.create(this, "JobQueue")
.priority(10)
.build();
lowPriorityQueue.addComputeEnvironment(sharedComputeEnv, 1);
highPriorityQueue.addComputeEnvironment(sharedComputeEnv, 1);
IVpc vpc;
var sharedComputeEnv = new FargateComputeEnvironment(this, "spotEnv", new FargateComputeEnvironmentProps {
Vpc = vpc,
Spot = true
});
var lowPriorityQueue = new JobQueue(this, "JobQueue", new JobQueueProps {
Priority = 1
});
var highPriorityQueue = new JobQueue(this, "JobQueue", new JobQueueProps {
Priority = 10
});
lowPriorityQueue.AddComputeEnvironment(sharedComputeEnv, 1);
highPriorityQueue.AddComputeEnvironment(sharedComputeEnv, 1);
var vpc IVpc
sharedComputeEnv := batch.NewFargateComputeEnvironment(this, jsii.String("spotEnv"), &FargateComputeEnvironmentProps{
Vpc: Vpc,
Spot: jsii.Boolean(true),
})
lowPriorityQueue := batch.NewJobQueue(this, jsii.String("JobQueue"), &JobQueueProps{
Priority: jsii.Number(1),
})
highPriorityQueue := batch.NewJobQueue(this, jsii.String("JobQueue"), &JobQueueProps{
Priority: jsii.Number(10),
})
lowPriorityQueue.AddComputeEnvironment(sharedComputeEnv, jsii.Number(1))
highPriorityQueue.AddComputeEnvironment(sharedComputeEnv, jsii.Number(1))
React to jobs stuck in RUNNABLE state
You can react to jobs stuck in RUNNABLE state by setting a jobStateTimeLimitActions in JobQueue. Specifies actions that AWS Batch will take after the job has remained at the head of the queue in the specified state for longer than the specified time.
new batch.JobQueue(this, 'JobQueue', {
jobStateTimeLimitActions: [
{
action: batch.JobStateTimeLimitActionsAction.CANCEL,
maxTime: cdk.Duration.minutes(10),
reason: batch.JobStateTimeLimitActionsReason.INSUFFICIENT_INSTANCE_CAPACITY,
state: batch.JobStateTimeLimitActionsState.RUNNABLE,
},
]
});
batch.JobQueue(self, "JobQueue",
job_state_time_limit_actions=[batch.JobStateTimeLimitAction(
action=batch.JobStateTimeLimitActionsAction.CANCEL,
max_time=cdk.Duration.minutes(10),
reason=batch.JobStateTimeLimitActionsReason.INSUFFICIENT_INSTANCE_CAPACITY,
state=batch.JobStateTimeLimitActionsState.RUNNABLE
)
]
)
JobQueue.Builder.create(this, "JobQueue")
.jobStateTimeLimitActions(List.of(JobStateTimeLimitAction.builder()
.action(JobStateTimeLimitActionsAction.CANCEL)
.maxTime(Duration.minutes(10))
.reason(JobStateTimeLimitActionsReason.INSUFFICIENT_INSTANCE_CAPACITY)
.state(JobStateTimeLimitActionsState.RUNNABLE)
.build()))
.build();
new JobQueue(this, "JobQueue", new JobQueueProps {
JobStateTimeLimitActions = new [] { new JobStateTimeLimitAction {
Action = JobStateTimeLimitActionsAction.CANCEL,
MaxTime = Duration.Minutes(10),
Reason = JobStateTimeLimitActionsReason.INSUFFICIENT_INSTANCE_CAPACITY,
State = JobStateTimeLimitActionsState.RUNNABLE
} }
});
batch.NewJobQueue(this, jsii.String("JobQueue"), &JobQueueProps{
JobStateTimeLimitActions: []JobStateTimeLimitAction{
&JobStateTimeLimitAction{
Action: batch.JobStateTimeLimitActionsAction_CANCEL,
MaxTime: cdk.Duration_Minutes(jsii.Number(10)),
Reason: batch.JobStateTimeLimitActionsReason_INSUFFICIENT_INSTANCE_CAPACITY,
State: batch.JobStateTimeLimitActionsState_RUNNABLE,
},
},
})
Fairshare Scheduling
Batch JobQueues execute Jobs submitted to them in FIFO order unless you specify a SchedulingPolicy. FIFO queuing can cause short-running jobs to be starved while long-running jobs fill the compute environment. To solve this, Jobs can be associated with a share.
Shares consist of a shareIdentifier and a weightFactor, which is inversely correlated with the vCPU allocated to that share identifier. When submitting a Job, you can specify its shareIdentifier to associate that particular job with that share. Let's see how the scheduler uses this information to schedule jobs.
For example, if there are two shares defined as follows:
| Share Identifier | Weight Factor |
|---|---|
| A | 1 |
| B | 1 |
The weight factors share the following relationship:
where BvCpus is the number of vCPUs allocated to jobs with share identifier 'B', and B_weight is the weight factor of B.
The total number of vCpus allocated to a share is equal to the amount of jobs in that share times the number of vCpus necessary for every job. Let's say that each A job needs 32 VCpus (A_requirement = 32) and each B job needs 64 vCpus (B_requirement = 64):
We have:
Thus the scheduler will schedule two 'A' jobs for each 'B' job.
You can control the weight factors to change these ratios, but note that weight factors are inversely correlated with the vCpus allocated to the corresponding share.
This example would be configured like this:
const fairsharePolicy = new batch.FairshareSchedulingPolicy(this, 'myFairsharePolicy');
fairsharePolicy.addShare({
shareIdentifier: 'A',
weightFactor: 1,
});
fairsharePolicy.addShare({
shareIdentifier: 'B',
weightFactor: 1,
});
new batch.JobQueue(this, 'JobQueue', {
schedulingPolicy: fairsharePolicy,
});
fairshare_policy = batch.FairshareSchedulingPolicy(self, "myFairsharePolicy")
fairshare_policy.add_share(
share_identifier="A",
weight_factor=1
)
fairshare_policy.add_share(
share_identifier="B",
weight_factor=1
)
batch.JobQueue(self, "JobQueue",
scheduling_policy=fairshare_policy
)
FairshareSchedulingPolicy fairsharePolicy = new FairshareSchedulingPolicy(this, "myFairsharePolicy");
fairsharePolicy.addShare(Share.builder()
.shareIdentifier("A")
.weightFactor(1)
.build());
fairsharePolicy.addShare(Share.builder()
.shareIdentifier("B")
.weightFactor(1)
.build());
JobQueue.Builder.create(this, "JobQueue")
.schedulingPolicy(fairsharePolicy)
.build();
var fairsharePolicy = new FairshareSchedulingPolicy(this, "myFairsharePolicy");
fairsharePolicy.AddShare(new Share {
ShareIdentifier = "A",
WeightFactor = 1
});
fairsharePolicy.AddShare(new Share {
ShareIdentifier = "B",
WeightFactor = 1
});
new JobQueue(this, "JobQueue", new JobQueueProps {
SchedulingPolicy = fairsharePolicy
});
fairsharePolicy := batch.NewFairshareSchedulingPolicy(this, jsii.String("myFairsharePolicy"))
fairsharePolicy.AddShare(&Share{
ShareIdentifier: jsii.String("A"),
WeightFactor: jsii.Number(1),
})
fairsharePolicy.AddShare(&Share{
ShareIdentifier: jsii.String("B"),
WeightFactor: jsii.Number(1),
})
batch.NewJobQueue(this, jsii.String("JobQueue"), &JobQueueProps{
SchedulingPolicy: fairsharePolicy,
})
Note: The scheduler will only consider the current usage of the compute environment unless you specify shareDecay. For example, a shareDecay of 5 minutes in the above example means that at any given point in time, twice as many 'A' jobs will be scheduled for each 'B' job, but only for the past 5 minutes. If 'B' jobs run longer than 5 minutes, then the scheduler is allowed to put more than two 'A' jobs for each 'B' job, because the usage of those long-running 'B' jobs will no longer be considered after 5 minutes. shareDecay linearly decreases the usage of long running jobs for calculation purposes. For example if share decay is 60 seconds, then jobs that run for 30 seconds have their usage considered to be only 50% of what it actually is, but after a whole minute the scheduler pretends they don't exist for fairness calculations.
The following code specifies a shareDecay of 5 minutes:
const fairsharePolicy = new batch.FairshareSchedulingPolicy(this, 'myFairsharePolicy', {
shareDecay: cdk.Duration.minutes(5),
});
fairshare_policy = batch.FairshareSchedulingPolicy(self, "myFairsharePolicy",
share_decay=cdk.Duration.minutes(5)
)
FairshareSchedulingPolicy fairsharePolicy = FairshareSchedulingPolicy.Builder.create(this, "myFairsharePolicy")
.shareDecay(Duration.minutes(5))
.build();
var fairsharePolicy = new FairshareSchedulingPolicy(this, "myFairsharePolicy", new FairshareSchedulingPolicyProps {
ShareDecay = Duration.Minutes(5)
});
fairsharePolicy := batch.NewFairshareSchedulingPolicy(this, jsii.String("myFairsharePolicy"), &FairshareSchedulingPolicyProps{
ShareDecay: cdk.Duration_Minutes(jsii.Number(5)),
})
If you have high priority jobs that should always be executed as soon as they arrive, you can define a computeReservation to specify the percentage of the maximum vCPU capacity that should be reserved for shares that are not in the queue. The actual reserved percentage is defined by Batch as:
where ActiveFairShares is the number of shares for which there exists at least one job in the queue with a unique share identifier.
This is best illustrated with an example. Suppose there are three shares with share identifiers A, B and C respectively and we specify the computeReservation to be 75%. The queue is currently empty, and no other shares exist.
There are no active fair shares, since the queue is empty. Thus (75/100)^0 = 1 = 100% of the maximum vCpus are reserved for all shares.
A job with identifier A enters the queue.
The number of active fair shares is now 1, hence (75/100)^1 = .75 = 75% of the maximum vCpus are reserved for all shares that do not have the identifier A; for this example, this is B and C, (but if jobs are submitted with a share identifier not covered by this fairshare policy, those would be considered just as B and C are).
Now a B job enters the queue. The number of active fair shares is now 2, so (75/100)^2 = .5625 = 56.25% of the maximum vCpus are reserved for all shares that do not have the identifier A or B.
Now a second A job enters the queue. The number of active fair shares is still 2, so the percentage reserved is still 56.25%
Now a C job enters the queue. The number of active fair shares is now 3, so (75/100)^3 = .421875 = 42.1875% of the maximum vCpus are reserved for all shares that do not have the identifier A, B, or C.
If there are no other shares that your jobs can specify, this means that 42.1875% of your capacity will never be used!
Now, A, B, and C can only consume 100% - 42.1875% = 57.8125% of the maximum vCpus. Note that the this percentage is not split between A, B, and C. Instead, the scheduler will use their weightFactors to decide which jobs to schedule; the only difference is that instead of competing for 100% of the max capacity, jobs compete for 57.8125% of the max capacity.
This example specifies a computeReservation of 75% that will behave as explained in the example above:
new batch.FairshareSchedulingPolicy(this, 'myFairsharePolicy', {
computeReservation: 75,
shares: [
{ weightFactor: 1, shareIdentifier: 'A' },
{ weightFactor: 0.5, shareIdentifier: 'B' },
{ weightFactor: 2, shareIdentifier: 'C' },
],
});
batch.FairshareSchedulingPolicy(self, "myFairsharePolicy",
compute_reservation=75,
shares=[batch.Share(weight_factor=1, share_identifier="A"), batch.Share(weight_factor=0.5, share_identifier="B"), batch.Share(weight_factor=2, share_identifier="C")
]
)
FairshareSchedulingPolicy.Builder.create(this, "myFairsharePolicy")
.computeReservation(75)
.shares(List.of(Share.builder().weightFactor(1).shareIdentifier("A").build(), Share.builder().weightFactor(0.5).shareIdentifier("B").build(), Share.builder().weightFactor(2).shareIdentifier("C").build()))
.build();
new FairshareSchedulingPolicy(this, "myFairsharePolicy", new FairshareSchedulingPolicyProps {
ComputeReservation = 75,
Shares = new [] { new Share { WeightFactor = 1, ShareIdentifier = "A" }, new Share { WeightFactor = 0.5, ShareIdentifier = "B" }, new Share { WeightFactor = 2, ShareIdentifier = "C" } }
});
batch.NewFairshareSchedulingPolicy(this, jsii.String("myFairsharePolicy"), &FairshareSchedulingPolicyProps{
ComputeReservation: jsii.Number(75),
Shares: []Share{
&Share{
WeightFactor: jsii.Number(1),
ShareIdentifier: jsii.String("A"),
},
&Share{
WeightFactor: jsii.Number(0.5),
ShareIdentifier: jsii.String("B"),
},
&Share{
WeightFactor: jsii.Number(2),
ShareIdentifier: jsii.String("C"),
},
},
})
You can specify a priority on your JobDefinitions to tell the scheduler to prioritize certain jobs that share the same share identifier.
Configuring Job Retry Policies
Certain workflows may result in Jobs failing due to intermittent issues. Jobs can specify retry policies to respond to different failures with different actions. There are three different ways information about the way a Job exited can be conveyed;
exitCode: the exit code returned from the process executed by the container. Will only match non-zero exit codes.reason: any middleware errors, like your Docker registry being down.statusReason: infrastructure errors, most commonly your spot instance being reclaimed.
For most use cases, only one of these will be associated with a particular action at a time. To specify common exitCodes, reasons, or statusReasons, use the corresponding value from the Reason class. This example shows some common failure reasons:
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
}),
retryAttempts: 5,
retryStrategies: [
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.CANNOT_PULL_CONTAINER),
],
});
jobDefn.addRetryStrategy(
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.SPOT_INSTANCE_RECLAIMED),
);
jobDefn.addRetryStrategy(
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.CANNOT_PULL_CONTAINER),
);
jobDefn.addRetryStrategy(
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.custom({
onExitCode: '40*',
onReason: 'some reason',
})),
);
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsEc2ContainerDefinition(self, "containerDefn",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256
),
retry_attempts=5,
retry_strategies=[
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.CANNOT_PULL_CONTAINER)
]
)
job_defn.add_retry_strategy(
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.SPOT_INSTANCE_RECLAIMED))
job_defn.add_retry_strategy(
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.CANNOT_PULL_CONTAINER))
job_defn.add_retry_strategy(
batch.RetryStrategy.of(batch.Action.EXIT, batch.Reason.custom(
on_exit_code="40*",
on_reason="some reason"
)))
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsEc2ContainerDefinition.Builder.create(this, "containerDefn")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.build())
.retryAttempts(5)
.retryStrategies(List.of(RetryStrategy.of(Action.EXIT, Reason.CANNOT_PULL_CONTAINER)))
.build();
jobDefn.addRetryStrategy(RetryStrategy.of(Action.EXIT, Reason.SPOT_INSTANCE_RECLAIMED));
jobDefn.addRetryStrategy(RetryStrategy.of(Action.EXIT, Reason.CANNOT_PULL_CONTAINER));
jobDefn.addRetryStrategy(RetryStrategy.of(Action.EXIT, Reason.custom(CustomReason.builder()
.onExitCode("40*")
.onReason("some reason")
.build())));
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsEc2ContainerDefinition(this, "containerDefn", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256
}),
RetryAttempts = 5,
RetryStrategies = new [] { RetryStrategy.Of(Action.EXIT, Reason.CANNOT_PULL_CONTAINER) }
});
jobDefn.AddRetryStrategy(RetryStrategy.Of(Action.EXIT, Reason.SPOT_INSTANCE_RECLAIMED));
jobDefn.AddRetryStrategy(RetryStrategy.Of(Action.EXIT, Reason.CANNOT_PULL_CONTAINER));
jobDefn.AddRetryStrategy(RetryStrategy.Of(Action.EXIT, Reason.Custom(new CustomReason {
OnExitCode = "40*",
OnReason = "some reason"
})));
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("containerDefn"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
}),
RetryAttempts: jsii.Number(5),
RetryStrategies: []RetryStrategy{
batch.RetryStrategy_Of(batch.Action_EXIT, batch.Reason_CANNOT_PULL_CONTAINER()),
},
})
jobDefn.addRetryStrategy(batch.RetryStrategy_Of(batch.Action_EXIT, batch.Reason_SPOT_INSTANCE_RECLAIMED()))
jobDefn.addRetryStrategy(batch.RetryStrategy_Of(batch.Action_EXIT, batch.Reason_CANNOT_PULL_CONTAINER()))
jobDefn.addRetryStrategy(batch.RetryStrategy_Of(batch.Action_EXIT, batch.Reason_Custom(&CustomReason{
OnExitCode: jsii.String("40*"),
OnReason: jsii.String("some reason"),
})))
When specifying a custom reason, you can specify a glob string to match each of these and react to different failures accordingly. Up to five different retry strategies can be configured for each Job, and each strategy can match against some or all of exitCode, reason, and statusReason. You can optionally configure the number of times a job will be retried, but you cannot configure different retry counts for different strategies; they all share the same count. If multiple conditions are specified in a given retry strategy, they must all match for the action to be taken; the conditions are ANDed together, not ORed.
Running single-container ECS workflows
Batch can run jobs on ECS or EKS. ECS jobs can be defined as single container or multinode. This example creates a JobDefinition that runs a single container with ECS:
declare const myFileSystem: efs.IFileSystem;
declare const myJobRole: iam.Role;
myFileSystem.grantRead(myJobRole);
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
volumes: [batch.EcsVolume.efs({
name: 'myVolume',
fileSystem: myFileSystem,
containerPath: '/Volumes/myVolume',
useJobRole: true,
})],
jobRole: myJobRole,
}),
});
# my_file_system: efs.IFileSystem
# my_job_role: iam.Role
my_file_system.grant_read(my_job_role)
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsEc2ContainerDefinition(self, "containerDefn",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256,
volumes=[batch.EcsVolume.efs(
name="myVolume",
file_system=my_file_system,
container_path="/Volumes/myVolume",
use_job_role=True
)],
job_role=my_job_role
)
)
IFileSystem myFileSystem;
Role myJobRole;
myFileSystem.grantRead(myJobRole);
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsEc2ContainerDefinition.Builder.create(this, "containerDefn")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.volumes(List.of(EcsVolume.efs(EfsVolumeOptions.builder()
.name("myVolume")
.fileSystem(myFileSystem)
.containerPath("/Volumes/myVolume")
.useJobRole(true)
.build())))
.jobRole(myJobRole)
.build())
.build();
IFileSystem myFileSystem;
Role myJobRole;
myFileSystem.GrantRead(myJobRole);
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsEc2ContainerDefinition(this, "containerDefn", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256,
Volumes = new [] { EcsVolume.Efs(new EfsVolumeOptions {
Name = "myVolume",
FileSystem = myFileSystem,
ContainerPath = "/Volumes/myVolume",
UseJobRole = true
}) },
JobRole = myJobRole
})
});
var myFileSystem IFileSystem
var myJobRole Role
myFileSystem.GrantRead(myJobRole)
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("containerDefn"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
Volumes: []EcsVolume{
batch.EcsVolume_Efs(&EfsVolumeOptions{
Name: jsii.String("myVolume"),
FileSystem: myFileSystem,
ContainerPath: jsii.String("/Volumes/myVolume"),
UseJobRole: jsii.Boolean(true),
}),
},
JobRole: myJobRole,
}),
})
For workflows that need persistent storage, batch supports mounting Volumes to the container. You can both provision the volume and mount it to the container in a single operation:
declare const myFileSystem: efs.IFileSystem;
declare const jobDefn: batch.EcsJobDefinition;
jobDefn.container.addVolume(batch.EcsVolume.efs({
name: 'myVolume',
fileSystem: myFileSystem,
containerPath: '/Volumes/myVolume',
}));
# my_file_system: efs.IFileSystem
# job_defn: batch.EcsJobDefinition
job_defn.container.add_volume(batch.EcsVolume.efs(
name="myVolume",
file_system=my_file_system,
container_path="/Volumes/myVolume"
))
IFileSystem myFileSystem;
EcsJobDefinition jobDefn;
jobDefn.container.addVolume(EcsVolume.efs(EfsVolumeOptions.builder()
.name("myVolume")
.fileSystem(myFileSystem)
.containerPath("/Volumes/myVolume")
.build()));
IFileSystem myFileSystem;
EcsJobDefinition jobDefn;
jobDefn.Container.AddVolume(EcsVolume.Efs(new EfsVolumeOptions {
Name = "myVolume",
FileSystem = myFileSystem,
ContainerPath = "/Volumes/myVolume"
}));
var myFileSystem IFileSystem
var jobDefn EcsJobDefinition
jobDefn.Container.AddVolume(batch.EcsVolume_Efs(&EfsVolumeOptions{
Name: jsii.String("myVolume"),
FileSystem: myFileSystem,
ContainerPath: jsii.String("/Volumes/myVolume"),
}))
Running an ECS workflow with Fargate container
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsFargateContainerDefinition(this, 'myFargateContainer', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
ephemeralStorageSize: cdk.Size.gibibytes(100),
fargateCpuArchitecture: ecs.CpuArchitecture.ARM64,
fargateOperatingSystemFamily: ecs.OperatingSystemFamily.LINUX,
}),
});
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsFargateContainerDefinition(self, "myFargateContainer",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256,
ephemeral_storage_size=cdk.Size.gibibytes(100),
fargate_cpu_architecture=ecs.CpuArchitecture.ARM64,
fargate_operating_system_family=ecs.OperatingSystemFamily.LINUX
)
)
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsFargateContainerDefinition.Builder.create(this, "myFargateContainer")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.ephemeralStorageSize(Size.gibibytes(100))
.fargateCpuArchitecture(CpuArchitecture.ARM64)
.fargateOperatingSystemFamily(OperatingSystemFamily.LINUX)
.build())
.build();
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsFargateContainerDefinition(this, "myFargateContainer", new EcsFargateContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256,
EphemeralStorageSize = Size.Gibibytes(100),
FargateCpuArchitecture = CpuArchitecture.ARM64,
FargateOperatingSystemFamily = OperatingSystemFamily.LINUX
})
});
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsFargateContainerDefinition(this, jsii.String("myFargateContainer"), &EcsFargateContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
EphemeralStorageSize: cdk.Size_Gibibytes(jsii.Number(100)),
FargateCpuArchitecture: ecs.CpuArchitecture_ARM64(),
FargateOperatingSystemFamily: ecs.OperatingSystemFamily_LINUX(),
}),
})
Enable Execute Command (ECS Exec)
You can enable ECS Exec for interactive debugging and troubleshooting by setting enableExecuteCommand to true. When enabled, you'll be able to execute commands interactively in running containers.
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'Ec2Container', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
enableExecuteCommand: true, // Enable ECS Exec
}),
});
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsEc2ContainerDefinition(self, "Ec2Container",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256,
enable_execute_command=True
)
)
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsEc2ContainerDefinition.Builder.create(this, "Ec2Container")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.enableExecuteCommand(true)
.build())
.build();
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsEc2ContainerDefinition(this, "Ec2Container", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256,
EnableExecuteCommand = true
})
});
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("Ec2Container"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
EnableExecuteCommand: jsii.Boolean(true),
}),
})
The same functionality is available for Fargate containers:
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsFargateContainerDefinition(this, 'FargateContainer', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
enableExecuteCommand: true, // Enable ECS Exec for Fargate
}),
});
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsFargateContainerDefinition(self, "FargateContainer",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256,
enable_execute_command=True
)
)
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsFargateContainerDefinition.Builder.create(this, "FargateContainer")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.enableExecuteCommand(true)
.build())
.build();
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsFargateContainerDefinition(this, "FargateContainer", new EcsFargateContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256,
EnableExecuteCommand = true
})
});
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsFargateContainerDefinition(this, jsii.String("FargateContainer"), &EcsFargateContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
EnableExecuteCommand: jsii.Boolean(true),
}),
})
When enableExecuteCommand is set to true:
- If no
jobRoleis provided, a new IAM role will be automatically created with the required SSM permissions - If a
jobRoleis already provided, the necessary SSM permissions will be added to the existing role
Secrets
You can expose SecretsManager Secret ARNs or SSM Parameters to your container as environment variables. The following example defines the MY_SECRET_ENV_VAR environment variable that contains the ARN of the Secret defined by mySecret:
declare const mySecret: secretsmanager.ISecret;
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
secrets: {
MY_SECRET_ENV_VAR: batch.Secret.fromSecretsManager(mySecret),
}
}),
});
# my_secret: secretsmanager.ISecret
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsEc2ContainerDefinition(self, "containerDefn",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256,
secrets={
"MY_SECRET_ENV_VAR": batch.Secret.from_secrets_manager(my_secret)
}
)
)
ISecret mySecret;
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsEc2ContainerDefinition.Builder.create(this, "containerDefn")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.secrets(Map.of(
"MY_SECRET_ENV_VAR", Secret.fromSecretsManager(mySecret)))
.build())
.build();
ISecret mySecret;
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsEc2ContainerDefinition(this, "containerDefn", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256,
Secrets = new Dictionary<string, Secret> {
{ "MY_SECRET_ENV_VAR", Secret.FromSecretsManager(mySecret) }
}
})
});
var mySecret ISecret
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("containerDefn"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
Secrets: map[string]Secret{
"MY_SECRET_ENV_VAR": batch.Secret_fromSecretsManager(mySecret),
},
}),
})
Running Kubernetes Workflows
Batch also supports running workflows on EKS. The following example creates a JobDefinition that runs on EKS:
const jobDefn = new batch.EksJobDefinition(this, 'eksf2', {
container: new batch.EksContainerDefinition(this, 'container', {
image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'),
volumes: [batch.EksVolume.emptyDir({
name: 'myEmptyDirVolume',
mountPath: '/mount/path',
medium: batch.EmptyDirMediumType.MEMORY,
readonly: true,
sizeLimit: cdk.Size.mebibytes(2048),
})],
}),
});
job_defn = batch.EksJobDefinition(self, "eksf2",
container=batch.EksContainerDefinition(self, "container",
image=ecs.ContainerImage.from_registry("amazon/amazon-ecs-sample"),
volumes=[batch.EksVolume.empty_dir(
name="myEmptyDirVolume",
mount_path="/mount/path",
medium=batch.EmptyDirMediumType.MEMORY,
readonly=True,
size_limit=cdk.Size.mebibytes(2048)
)]
)
)
EksJobDefinition jobDefn = EksJobDefinition.Builder.create(this, "eksf2")
.container(EksContainerDefinition.Builder.create(this, "container")
.image(ContainerImage.fromRegistry("amazon/amazon-ecs-sample"))
.volumes(List.of(EksVolume.emptyDir(EmptyDirVolumeOptions.builder()
.name("myEmptyDirVolume")
.mountPath("/mount/path")
.medium(EmptyDirMediumType.MEMORY)
.readonly(true)
.sizeLimit(Size.mebibytes(2048))
.build())))
.build())
.build();
var jobDefn = new EksJobDefinition(this, "eksf2", new EksJobDefinitionProps {
Container = new EksContainerDefinition(this, "container", new EksContainerDefinitionProps {
Image = ContainerImage.FromRegistry("amazon/amazon-ecs-sample"),
Volumes = new [] { EksVolume.EmptyDir(new EmptyDirVolumeOptions {
Name = "myEmptyDirVolume",
MountPath = "/mount/path",
Medium = EmptyDirMediumType.MEMORY,
Readonly = true,
SizeLimit = Size.Mebibytes(2048)
}) }
})
});
jobDefn := batch.NewEksJobDefinition(this, jsii.String("eksf2"), &EksJobDefinitionProps{
Container: batch.NewEksContainerDefinition(this, jsii.String("container"), &EksContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("amazon/amazon-ecs-sample")),
Volumes: []EksVolume{
batch.EksVolume_EmptyDir(&EmptyDirVolumeOptions{
Name: jsii.String("myEmptyDirVolume"),
MountPath: jsii.String("/mount/path"),
Medium: batch.EmptyDirMediumType_MEMORY,
Readonly: jsii.Boolean(true),
SizeLimit: cdk.Size_Mebibytes(jsii.Number(2048)),
}),
},
}),
})
You can mount Volumes to these containers in a single operation:
declare const jobDefn: batch.EksJobDefinition;
jobDefn.container.addVolume(batch.EksVolume.emptyDir({
name: 'emptyDir',
mountPath: '/Volumes/emptyDir',
}));
jobDefn.container.addVolume(batch.EksVolume.hostPath({
name: 'hostPath',
hostPath: '/sys',
mountPath: '/Volumes/hostPath',
}));
jobDefn.container.addVolume(batch.EksVolume.secret({
name: 'secret',
optional: true,
mountPath: '/Volumes/secret',
secretName: 'mySecret',
}));
# job_defn: batch.EksJobDefinition
job_defn.container.add_volume(batch.EksVolume.empty_dir(
name="emptyDir",
mount_path="/Volumes/emptyDir"
))
job_defn.container.add_volume(batch.EksVolume.host_path(
name="hostPath",
host_path="/sys",
mount_path="/Volumes/hostPath"
))
job_defn.container.add_volume(batch.EksVolume.secret(
name="secret",
optional=True,
mount_path="/Volumes/secret",
secret_name="mySecret"
))
EksJobDefinition jobDefn;
jobDefn.container.addVolume(EksVolume.emptyDir(EmptyDirVolumeOptions.builder()
.name("emptyDir")
.mountPath("/Volumes/emptyDir")
.build()));
jobDefn.container.addVolume(EksVolume.hostPath(HostPathVolumeOptions.builder()
.name("hostPath")
.hostPath("/sys")
.mountPath("/Volumes/hostPath")
.build()));
jobDefn.container.addVolume(EksVolume.secret(SecretPathVolumeOptions.builder()
.name("secret")
.optional(true)
.mountPath("/Volumes/secret")
.secretName("mySecret")
.build()));
EksJobDefinition jobDefn;
jobDefn.Container.AddVolume(EksVolume.EmptyDir(new EmptyDirVolumeOptions {
Name = "emptyDir",
MountPath = "/Volumes/emptyDir"
}));
jobDefn.Container.AddVolume(EksVolume.HostPath(new HostPathVolumeOptions {
Name = "hostPath",
HostPath = "/sys",
MountPath = "/Volumes/hostPath"
}));
jobDefn.Container.AddVolume(EksVolume.Secret(new SecretPathVolumeOptions {
Name = "secret",
Optional = true,
MountPath = "/Volumes/secret",
SecretName = "mySecret"
}));
var jobDefn EksJobDefinition
jobDefn.Container.AddVolume(batch.EksVolume_EmptyDir(&EmptyDirVolumeOptions{
Name: jsii.String("emptyDir"),
MountPath: jsii.String("/Volumes/emptyDir"),
}))
jobDefn.Container.AddVolume(batch.EksVolume_HostPath(&HostPathVolumeOptions{
Name: jsii.String("hostPath"),
HostPath: jsii.String("/sys"),
MountPath: jsii.String("/Volumes/hostPath"),
}))
jobDefn.Container.AddVolume(batch.EksVolume_Secret(&SecretPathVolumeOptions{
Name: jsii.String("secret"),
Optional: jsii.Boolean(true),
MountPath: jsii.String("/Volumes/secret"),
SecretName: jsii.String("mySecret"),
}))
Running Distributed Workflows
Some workflows benefit from parallellization and are most powerful when run in a distributed environment, such as certain numerical calculations or simulations. Batch offers MultiNodeJobDefinitions, which allow a single job to run on multiple instances in parallel, for this purpose. Message Passing Interface (MPI) is often used with these workflows. You must configure your containers to use MPI properly, but Batch allows different nodes running different containers to communicate easily with one another. You must configure your containers to use certain environment variables that Batch will provide them, which lets them know which one is the main node, among other information. For an in-depth example on using MPI to perform numerical computations on Batch, see this blog post In particular, the environment variable that tells the containers which one is the main node can be configured on your MultiNodeJobDefinition as follows:
const multiNodeJob = new batch.MultiNodeJobDefinition(this, 'JobDefinition', {
instanceType: ec2.InstanceType.of(ec2.InstanceClass.R4, ec2.InstanceSize.LARGE), // optional, omit to let Batch choose the type for you
containers: [{
container: new batch.EcsEc2ContainerDefinition(this, 'mainMPIContainer', {
image: ecs.ContainerImage.fromRegistry('yourregsitry.com/yourMPIImage:latest'),
cpu: 256,
memory: cdk.Size.mebibytes(2048),
}),
startNode: 0,
endNode: 5,
}],
});
// convenience method
multiNodeJob.addContainer({
startNode: 6,
endNode: 10,
container: new batch.EcsEc2ContainerDefinition(this, 'multiContainer', {
image: ecs.ContainerImage.fromRegistry('amazon/amazon-ecs-sample'),
cpu: 256,
memory: cdk.Size.mebibytes(2048),
}),
});
multi_node_job = batch.MultiNodeJobDefinition(self, "JobDefinition",
instance_type=ec2.InstanceType.of(ec2.InstanceClass.R4, ec2.InstanceSize.LARGE), # optional, omit to let Batch choose the type for you
containers=[batch.MultiNodeContainer(
container=batch.EcsEc2ContainerDefinition(self, "mainMPIContainer",
image=ecs.ContainerImage.from_registry("yourregsitry.com/yourMPIImage:latest"),
cpu=256,
memory=cdk.Size.mebibytes(2048)
),
start_node=0,
end_node=5
)]
)
# convenience method
multi_node_job.add_container(
start_node=6,
end_node=10,
container=batch.EcsEc2ContainerDefinition(self, "multiContainer",
image=ecs.ContainerImage.from_registry("amazon/amazon-ecs-sample"),
cpu=256,
memory=cdk.Size.mebibytes(2048)
)
)
MultiNodeJobDefinition multiNodeJob = MultiNodeJobDefinition.Builder.create(this, "JobDefinition")
.instanceType(InstanceType.of(InstanceClass.R4, InstanceSize.LARGE)) // optional, omit to let Batch choose the type for you
.containers(List.of(MultiNodeContainer.builder()
.container(EcsEc2ContainerDefinition.Builder.create(this, "mainMPIContainer")
.image(ContainerImage.fromRegistry("yourregsitry.com/yourMPIImage:latest"))
.cpu(256)
.memory(Size.mebibytes(2048))
.build())
.startNode(0)
.endNode(5)
.build()))
.build();
// convenience method
multiNodeJob.addContainer(MultiNodeContainer.builder()
.startNode(6)
.endNode(10)
.container(EcsEc2ContainerDefinition.Builder.create(this, "multiContainer")
.image(ContainerImage.fromRegistry("amazon/amazon-ecs-sample"))
.cpu(256)
.memory(Size.mebibytes(2048))
.build())
.build());
var multiNodeJob = new MultiNodeJobDefinition(this, "JobDefinition", new MultiNodeJobDefinitionProps {
InstanceType = InstanceType.Of(InstanceClass.R4, InstanceSize.LARGE), // optional, omit to let Batch choose the type for you
Containers = new [] { new MultiNodeContainer {
Container = new EcsEc2ContainerDefinition(this, "mainMPIContainer", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("yourregsitry.com/yourMPIImage:latest"),
Cpu = 256,
Memory = Size.Mebibytes(2048)
}),
StartNode = 0,
EndNode = 5
} }
});
// convenience method
multiNodeJob.AddContainer(new MultiNodeContainer {
StartNode = 6,
EndNode = 10,
Container = new EcsEc2ContainerDefinition(this, "multiContainer", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("amazon/amazon-ecs-sample"),
Cpu = 256,
Memory = Size.Mebibytes(2048)
})
});
multiNodeJob := batch.NewMultiNodeJobDefinition(this, jsii.String("JobDefinition"), &MultiNodeJobDefinitionProps{
InstanceType: ec2.InstanceType_Of(ec2.InstanceClass_R4, ec2.InstanceSize_LARGE),
// optional, omit to let Batch choose the type for you
Containers: []MultiNodeContainer{
&MultiNodeContainer{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("mainMPIContainer"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("yourregsitry.com/yourMPIImage:latest")),
Cpu: jsii.Number(256),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
}),
StartNode: jsii.Number(0),
EndNode: jsii.Number(5),
},
},
})
// convenience method
multiNodeJob.AddContainer(&MultiNodeContainer{
StartNode: jsii.Number(6),
EndNode: jsii.Number(10),
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("multiContainer"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_*FromRegistry(jsii.String("amazon/amazon-ecs-sample")),
Cpu: jsii.Number(256),
Memory: cdk.Size_*Mebibytes(jsii.Number(2048)),
}),
})
If you need to set the control node to an index other than 0, specify it in directly:
const multiNodeJob = new batch.MultiNodeJobDefinition(this, 'JobDefinition', {
mainNode: 5,
instanceType: ec2.InstanceType.of(ec2.InstanceClass.R4, ec2.InstanceSize.LARGE),
});
multi_node_job = batch.MultiNodeJobDefinition(self, "JobDefinition",
main_node=5,
instance_type=ec2.InstanceType.of(ec2.InstanceClass.R4, ec2.InstanceSize.LARGE)
)
MultiNodeJobDefinition multiNodeJob = MultiNodeJobDefinition.Builder.create(this, "JobDefinition")
.mainNode(5)
.instanceType(InstanceType.of(InstanceClass.R4, InstanceSize.LARGE))
.build();
var multiNodeJob = new MultiNodeJobDefinition(this, "JobDefinition", new MultiNodeJobDefinitionProps {
MainNode = 5,
InstanceType = InstanceType.Of(InstanceClass.R4, InstanceSize.LARGE)
});
multiNodeJob := batch.NewMultiNodeJobDefinition(this, jsii.String("JobDefinition"), &MultiNodeJobDefinitionProps{
MainNode: jsii.Number(5),
InstanceType: ec2.InstanceType_Of(ec2.InstanceClass_R4, ec2.InstanceSize_LARGE),
})
Pass Parameters to a Job
Batch allows you define parameters in your JobDefinition, which can be referenced in the container command. For example:
new batch.EcsJobDefinition(this, 'JobDefn', {
parameters: { echoParam: 'foobar' },
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
command: [
'echo',
'Ref::echoParam',
],
}),
});
batch.EcsJobDefinition(self, "JobDefn",
parameters={"echo_param": "foobar"},
container=batch.EcsEc2ContainerDefinition(self, "containerDefn",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256,
command=["echo", "Ref::echoParam"
]
)
)
EcsJobDefinition.Builder.create(this, "JobDefn")
.parameters(Map.of("echoParam", "foobar"))
.container(EcsEc2ContainerDefinition.Builder.create(this, "containerDefn")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.command(List.of("echo", "Ref::echoParam"))
.build())
.build();
new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Parameters = new Dictionary<string, object> { { "echoParam", "foobar" } },
Container = new EcsEc2ContainerDefinition(this, "containerDefn", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256,
Command = new [] { "echo", "Ref::echoParam" }
})
});
batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Parameters: map[string]interface{}{
"echoParam": jsii.String("foobar"),
},
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("containerDefn"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
Command: []*string{
jsii.String("echo"),
jsii.String("Ref::echoParam"),
},
}),
})
Job Definition Version Management
By default, when you update a Job Definition, AWS Batch automatically deregisters the previous revision. This means any jobs that were submitted using the old revision may fail if they haven't started yet.
You can preserve previous revisions by setting skipDeregisterOnUpdate to true:
const jobDefn = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
}),
skipDeregisterOnUpdate: true, // Keep previous revisions active
});
job_defn = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsEc2ContainerDefinition(self, "containerDefn",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256
),
skip_deregister_on_update=True
)
EcsJobDefinition jobDefn = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsEc2ContainerDefinition.Builder.create(this, "containerDefn")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.build())
.skipDeregisterOnUpdate(true)
.build();
var jobDefn = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsEc2ContainerDefinition(this, "containerDefn", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256
}),
SkipDeregisterOnUpdate = true
});
jobDefn := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("containerDefn"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
}),
SkipDeregisterOnUpdate: jsii.Boolean(true),
})
- This applies to all Job Definition types: ECS (EC2 and Fargate), EKS, and MultiNode
- Default behavior (when not specified) follows AWS Batch defaults: previous revisions are deregistered
Understanding Progressive Allocation Strategies
AWS Batch uses an allocation strategy to determine what compute resource will efficiently handle incoming job requests. By default, BEST_FIT will pick an available compute instance based on vCPU requirements. If none exist, the job will wait until resources become available. However, with this strategy, you may have jobs waiting in the queue unnecessarily despite having more powerful instances available. Below is an example of how that situation might look like:
Compute Environment:
1. m5.xlarge => 4 vCPU
2. m5.2xlarge => 8 vCPU
Job Queue:
---------
| A | B |
---------
Job Requirements:
A => 4 vCPU - ALLOCATED TO m5.xlarge
B => 2 vCPU - WAITING
In this situation, Batch will allocate Job A to compute resource #1 because it is the most cost efficient resource that matches the vCPU requirement. However, with this BEST_FIT strategy, Job B will not be allocated to our other available compute resource even though it is strong enough to handle it. Instead, it will wait until the first job is finished processing or wait a similar m5.xlarge resource to be provisioned.
The alternative would be to use the BEST_FIT_PROGRESSIVE strategy in order for the remaining job to be handled in larger containers regardless of vCPU requirement and costs.
Permissions
You can grant any Principal the batch:submitJob permission on both a job definition and a job queue like this:
declare const vpc: ec2.IVpc;
const ecsJob = new batch.EcsJobDefinition(this, 'JobDefn', {
container: new batch.EcsEc2ContainerDefinition(this, 'containerDefn', {
image: ecs.ContainerImage.fromRegistry('public.ecr.aws/amazonlinux/amazonlinux:latest'),
memory: cdk.Size.mebibytes(2048),
cpu: 256,
}),
});
const queue = new batch.JobQueue(this, 'JobQueue', {
computeEnvironments: [{
computeEnvironment: new batch.ManagedEc2EcsComputeEnvironment(this, 'managedEc2CE', {
vpc,
}),
order: 1,
}],
priority: 10,
});
const user = new iam.User(this, 'MyUser');
ecsJob.grantSubmitJob(user, queue);
# vpc: ec2.IVpc
ecs_job = batch.EcsJobDefinition(self, "JobDefn",
container=batch.EcsEc2ContainerDefinition(self, "containerDefn",
image=ecs.ContainerImage.from_registry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
memory=cdk.Size.mebibytes(2048),
cpu=256
)
)
queue = batch.JobQueue(self, "JobQueue",
compute_environments=[batch.OrderedComputeEnvironment(
compute_environment=batch.ManagedEc2EcsComputeEnvironment(self, "managedEc2CE",
vpc=vpc
),
order=1
)],
priority=10
)
user = iam.User(self, "MyUser")
ecs_job.grant_submit_job(user, queue)
IVpc vpc;
EcsJobDefinition ecsJob = EcsJobDefinition.Builder.create(this, "JobDefn")
.container(EcsEc2ContainerDefinition.Builder.create(this, "containerDefn")
.image(ContainerImage.fromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"))
.memory(Size.mebibytes(2048))
.cpu(256)
.build())
.build();
JobQueue queue = JobQueue.Builder.create(this, "JobQueue")
.computeEnvironments(List.of(OrderedComputeEnvironment.builder()
.computeEnvironment(ManagedEc2EcsComputeEnvironment.Builder.create(this, "managedEc2CE")
.vpc(vpc)
.build())
.order(1)
.build()))
.priority(10)
.build();
User user = new User(this, "MyUser");
ecsJob.grantSubmitJob(user, queue);
IVpc vpc;
var ecsJob = new EcsJobDefinition(this, "JobDefn", new EcsJobDefinitionProps {
Container = new EcsEc2ContainerDefinition(this, "containerDefn", new EcsEc2ContainerDefinitionProps {
Image = ContainerImage.FromRegistry("public.ecr.aws/amazonlinux/amazonlinux:latest"),
Memory = Size.Mebibytes(2048),
Cpu = 256
})
});
var queue = new JobQueue(this, "JobQueue", new JobQueueProps {
ComputeEnvironments = new [] { new OrderedComputeEnvironment {
ComputeEnvironment = new ManagedEc2EcsComputeEnvironment(this, "managedEc2CE", new ManagedEc2EcsComputeEnvironmentProps {
Vpc = vpc
}),
Order = 1
} },
Priority = 10
});
var user = new User(this, "MyUser");
ecsJob.GrantSubmitJob(user, queue);
var vpc IVpc
ecsJob := batch.NewEcsJobDefinition(this, jsii.String("JobDefn"), &EcsJobDefinitionProps{
Container: batch.NewEcsEc2ContainerDefinition(this, jsii.String("containerDefn"), &EcsEc2ContainerDefinitionProps{
Image: ecs.ContainerImage_FromRegistry(jsii.String("public.ecr.aws/amazonlinux/amazonlinux:latest")),
Memory: cdk.Size_Mebibytes(jsii.Number(2048)),
Cpu: jsii.Number(256),
}),
})
queue := batch.NewJobQueue(this, jsii.String("JobQueue"), &JobQueueProps{
ComputeEnvironments: []OrderedComputeEnvironment{
&OrderedComputeEnvironment{
ComputeEnvironment: batch.NewManagedEc2EcsComputeEnvironment(this, jsii.String("managedEc2CE"), &ManagedEc2EcsComputeEnvironmentProps{
Vpc: *Vpc,
}),
Order: jsii.Number(1),
},
},
Priority: jsii.Number(10),
})
user := iam.NewUser(this, jsii.String("MyUser"))
ecsJob.GrantSubmitJob(user, queue)
Managing Consumable Resources
Consumable resources are finite resources that are consumed by jobs, such as third-party software licenses or API rate limits. AWS Batch tracks the usage of these resources and ensures that jobs only run when the required resources are available.
There are two types of consumable resources:
- REPLENISHABLE: Resources that can be re-used after a job completes (e.g., software licenses)
- NON_REPLENISHABLE: Resources that cannot be re-used after a job completes (e.g., API rate limits)
Creating Consumable Resources
You can create consumable resources to track and manage finite resources in your Batch workflows:
// Create a replenishable consumable resource (e.g., software licenses)
const licenseResource = new batch.ConsumableResource(this, 'LicenseResource', {
consumableResourceName: 'my-software-license',
resourceType: batch.ConsumableResourceType.REPLENISHABLE,
totalQuantity: 100,
});
// Create a non-replenishable consumable resource (e.g., API rate limits)
const apiRateLimitResource = new batch.ConsumableResource(this, 'ApiRateLimitResource', {
consumableResourceName: 'api-rate-limit',
resourceType: batch.ConsumableResourceType.NON_REPLENISHABLE,
totalQuantity: 1000,
});
# Create a replenishable consumable resource (e.g., software licenses)
license_resource = batch.ConsumableResource(self, "LicenseResource",
consumable_resource_name="my-software-license",
resource_type=batch.ConsumableResourceType.REPLENISHABLE,
total_quantity=100
)
# Create a non-replenishable consumable resource (e.g., API rate limits)
api_rate_limit_resource = batch.ConsumableResource(self, "ApiRateLimitResource",
consumable_resource_name="api-rate-limit",
resource_type=batch.ConsumableResourceType.NON_REPLENISHABLE,
total_quantity=1000
)
// Create a replenishable consumable resource (e.g., software licenses)
ConsumableResource licenseResource = ConsumableResource.Builder.create(this, "LicenseResource")
.consumableResourceName("my-software-license")
.resourceType(ConsumableResourceType.REPLENISHABLE)
.totalQuantity(100)
.build();
// Create a non-replenishable consumable resource (e.g., API rate limits)
ConsumableResource apiRateLimitResource = ConsumableResource.Builder.create(this, "ApiRateLimitResource")
.consumableResourceName("api-rate-limit")
.resourceType(ConsumableResourceType.NON_REPLENISHABLE)
.totalQuantity(1000)
.build();
// Create a replenishable consumable resource (e.g., software licenses)
var licenseResource = new ConsumableResource(this, "LicenseResource", new ConsumableResourceProps {
ConsumableResourceName = "my-software-license",
ResourceType = ConsumableResourceType.REPLENISHABLE,
TotalQuantity = 100
});
// Create a non-replenishable consumable resource (e.g., API rate limits)
var apiRateLimitResource = new ConsumableResource(this, "ApiRateLimitResource", new ConsumableResourceProps {
ConsumableResourceName = "api-rate-limit",
ResourceType = ConsumableResourceType.NON_REPLENISHABLE,
TotalQuantity = 1000
});
// Create a replenishable consumable resource (e.g., software licenses)
licenseResource := batch.NewConsumableResource(this, jsii.String("LicenseResource"), &ConsumableResourceProps{
ConsumableResourceName: jsii.String("my-software-license"),
ResourceType: batch.ConsumableResourceType_REPLENISHABLE,
TotalQuantity: jsii.Number(100),
})
// Create a non-replenishable consumable resource (e.g., API rate limits)
apiRateLimitResource := batch.NewConsumableResource(this, jsii.String("ApiRateLimitResource"), &ConsumableResourceProps{
ConsumableResourceName: jsii.String("api-rate-limit"),
ResourceType: batch.ConsumableResourceType_NON_REPLENISHABLE,
TotalQuantity: jsii.Number(1000),
})
resourceType defaults to REPLENISHABLE, matching the AWS Batch API, and totalQuantity must be a non-negative integer. Both consumableResourceName and resourceType replace the resource when changed, so changing the type of a resource that has an explicit name also requires a new name.
Tags can be passed directly to the resource:
const taggedResource = new batch.ConsumableResource(this, 'TaggedResource', {
totalQuantity: 100,
tags: {
Team: 'batch',
},
});
tagged_resource = batch.ConsumableResource(self, "TaggedResource",
total_quantity=100,
tags={
"Team": "batch"
}
)
ConsumableResource taggedResource = ConsumableResource.Builder.create(this, "TaggedResource")
.totalQuantity(100)
.tags(Map.of(
"Team", "batch"))
.build();
var taggedResource = new ConsumableResource(this, "TaggedResource", new ConsumableResourceProps {
TotalQuantity = 100,
Tags = new Dictionary<string, string> {
{ "Team", "batch" }
}
});
taggedResource := batch.NewConsumableResource(this, jsii.String("TaggedResource"), &ConsumableResourceProps{
TotalQuantity: jsii.Number(100),
Tags: map[string]*string{
"Team": jsii.String("batch"),
},
})
Granting Access to Consumable Resources
Use the grants property to give an identity permission to inspect or update a consumable resource:
const licenseResource = new batch.ConsumableResource(this, 'LicenseResource', {
totalQuantity: 100,
});
const user = new iam.User(this, 'MyUser');
// Describe the resource and list the jobs that use it
licenseResource.grants.read(user);
// Change the total quantity, e.g. to replenish a NON_REPLENISHABLE pool
licenseResource.grants.update(user);
license_resource = batch.ConsumableResource(self, "LicenseResource",
total_quantity=100
)
user = iam.User(self, "MyUser")
# Describe the resource and list the jobs that use it
license_resource.grants.read(user)
# Change the total quantity, e.g. to replenish a NON_REPLENISHABLE pool
license_resource.grants.update(user)
ConsumableResource licenseResource = ConsumableResource.Builder.create(this, "LicenseResource")
.totalQuantity(100)
.build();
User user = new User(this, "MyUser");
// Describe the resource and list the jobs that use it
licenseResource.grants.read(user);
// Change the total quantity, e.g. to replenish a NON_REPLENISHABLE pool
licenseResource.grants.update(user);
var licenseResource = new ConsumableResource(this, "LicenseResource", new ConsumableResourceProps {
TotalQuantity = 100
});
var user = new User(this, "MyUser");
// Describe the resource and list the jobs that use it
licenseResource.Grants.Read(user);
// Change the total quantity, e.g. to replenish a NON_REPLENISHABLE pool
licenseResource.Grants.Update(user);
licenseResource := batch.NewConsumableResource(this, jsii.String("LicenseResource"), &ConsumableResourceProps{
TotalQuantity: jsii.Number(100),
})
user := iam.NewUser(this, jsii.String("MyUser"))
// Describe the resource and list the jobs that use it
licenseResource.grants.Read(user)
// Change the total quantity, e.g. to replenish a NON_REPLENISHABLE pool
licenseResource.grants.Update(user)
Importing Existing Consumable Resources
You can import existing consumable resources by their ARN:
const importedResource = batch.ConsumableResource.fromConsumableResourceArn(
this,
'ImportedResource',
'arn:aws:batch:us-east-1:123456789012:consumable-resource/my-resource'
);
imported_resource = batch.ConsumableResource.from_consumable_resource_arn(self, "ImportedResource", "arn:aws:batch:us-east-1:123456789012:consumable-resource/my-resource")
IConsumableResource importedResource = ConsumableResource.fromConsumableResourceArn(this, "ImportedResource", "arn:aws:batch:us-east-1:123456789012:consumable-resource/my-resource");
var importedResource = ConsumableResource.FromConsumableResourceArn(this, "ImportedResource", "arn:aws:batch:us-east-1:123456789012:consumable-resource/my-resource");
importedResource := batch.ConsumableResource_FromConsumableResourceArn(this, jsii.String("ImportedResource"), jsii.String("arn:aws:batch:us-east-1:123456789012:consumable-resource/my-resource"))
Types
| Name | Kind | Summary |
|---|---|---|
ConsumableResource | Construct (L2) | A Batch Consumable Resource. |
EcsJobDefinition | Construct (L2) | A JobDefinition that uses ECS orchestration. |
EksJobDefinition | Construct (L2) | A JobDefinition that uses Eks orchestration. |
FairshareSchedulingPolicy | Construct (L2) | Represents a Fairshare Scheduling Policy. Instructs the scheduler to allocate ComputeEnvironment vCPUs based on Job shareIdentifiers. |
FargateComputeEnvironment | Construct (L2) | A ManagedComputeEnvironment that uses ECS orchestration on Fargate instances. |
JobQueue | Construct (L2) | JobQueues can receive Jobs, which are removed from the queue when sent to the linked ComputeEnvironment(s) to be executed. |
ManagedEc2EcsComputeEnvironment | Construct (L2) | A ManagedComputeEnvironment that uses ECS orchestration on EC2 instances. |
ManagedEc2EksComputeEnvironment | Construct (L2) | A ManagedComputeEnvironment that uses ECS orchestration on EC2 instances. |
MultiNodeJobDefinition | Construct (L2) | A JobDefinition that uses Ecs orchestration to run multiple containers. |
UnmanagedComputeEnvironment | Construct (L2) | Unmanaged ComputeEnvironments do not provision or manage EC2 instances on your behalf. |
CfnComputeEnvironment | Construct (L1) | The AWS::Batch::ComputeEnvironment resource defines your AWS Batch compute environment. |
CfnConsumableResource | Construct (L1) | The AWS::Batch::ConsumableResource resource specifies the parameters for an AWS Batch consumable resource. |
CfnJobDefinition | Construct (L1) | The AWS::Batch::JobDefinition resource specifies the parameters for an AWS Batch job definition. |
CfnJobQueue | Construct (L1) | The AWS::Batch::JobQueue resource specifies the parameters for an AWS Batch job queue definition. |
CfnQuotaShare | Construct (L1) | Creates an AWS Batch quota share. |
CfnSchedulingPolicy | Construct (L1) | The AWS::Batch::SchedulingPolicy resource specifies the parameters for an AWS Batch scheduling policy. |
CfnServiceEnvironment | Construct (L1) | Creates a service environment for running service jobs. |
Action | Enum | The Action to take when all specified conditions in a RetryStrategy are met. |
AllocationStrategy | Enum | Determines how this compute environment chooses instances to spawn. |
CfnComputeEnvironment.CapacityReservationsProperty | Struct | |
CfnComputeEnvironment.ComputeResourcesProperty | Struct | Details about the compute resources managed by the compute environment. |
CfnComputeEnvironment.ComputeScalingPolicyProperty | Struct | |
CfnComputeEnvironment.Ec2ConfigurationObjectProperty | Struct | Provides information used to select Amazon Machine Images (AMIs) for instances in the compute environment. |
CfnComputeEnvironment.EcsSettingsProperty | Struct | |
CfnComputeEnvironment.EksAccessEntryProperty | Struct | The EKS access entry configuration for the compute environment. |
CfnComputeEnvironment.EksConfigurationProperty | Struct | Configuration for the Amazon EKS cluster that supports the AWS Batch compute environment. |
CfnComputeEnvironment.InfrastructureOptimizationProperty | Struct | |
CfnComputeEnvironment.InstanceLaunchTemplateProperty | Struct | |
CfnComputeEnvironment.InstanceRequirementsProperty | Struct | |
CfnComputeEnvironment.LaunchTemplateSpecificationOverrideProperty | Struct | An object that represents a launch template to use in place of the default launch template. |
CfnComputeEnvironment.LaunchTemplateSpecificationProperty | Struct | An object that represents a launch template that's associated with a compute resource. |
CfnComputeEnvironment.ManagedInstancesLocalStorageConfigurationProperty | Struct | |
CfnComputeEnvironment.ManagedInstancesNetworkConfigurationProperty | Struct | |
CfnComputeEnvironment.ManagedInstancesProviderProperty | Struct | |
CfnComputeEnvironment.ManagedInstancesStorageConfigurationProperty | Struct | |
CfnComputeEnvironment.UpdatePolicyProperty | Struct | Specifies the infrastructure update policy for the Amazon EC2 compute environment. |
CfnComputeEnvironmentProps | Struct | Properties for defining a CfnComputeEnvironment. |
CfnConsumableResourceProps | Struct | Properties for defining a CfnConsumableResource. |
CfnJobDefinition.AuthorizationConfigProperty | Struct | |
CfnJobDefinition.ConsumableResourcePropertiesProperty | Struct | Contains a list of consumable resources required by a job. |
CfnJobDefinition.ConsumableResourceRequirementProperty | Struct | Information about a consumable resource required to run a job. |
CfnJobDefinition.ContainerPropertiesProperty | Struct | Container properties are used for Amazon ECS based job definitions. |
CfnJobDefinition.DeviceProperty | Struct | An object that represents a container instance host device. |
CfnJobDefinition.EcsPropertiesProperty | Struct | An object that contains the properties for the Amazon ECS resources of a job. |
CfnJobDefinition.EcsTaskPropertiesProperty | Struct | The properties for a task definition that describes the container and volume definitions of an Amazon ECS task. |
CfnJobDefinition.EfsVolumeConfigurationProperty | Struct | |
CfnJobDefinition.EksContainerEnvironmentVariableProperty | Struct | An environment variable. |
CfnJobDefinition.EksContainerProperty | Struct | EKS container properties are used in job definitions for Amazon EKS based job definitions to describe the properties for a container node in the pod that's launched as part of a job. |
CfnJobDefinition.EksContainerVolumeMountProperty | Struct | The volume mounts for a container for an Amazon EKS job. |
CfnJobDefinition.EksPersistentVolumeClaimProperty | Struct | A persistentVolumeClaim volume is used to mount a PersistentVolume into a Pod. PersistentVolumeClaims are a way for users to "claim" durable storage without knowing the details of the particular cloud environment. See the information about PersistentVolumes in the Kubernetes documentation . |
CfnJobDefinition.EksPropertiesProperty | Struct | An object that contains the properties for the Kubernetes resources of a job. |
CfnJobDefinition.EksSecretProperty | Struct | Specifies the configuration of a Kubernetes secret volume. |
CfnJobDefinition.EksVolumeProperty | Struct | Specifies an Amazon EKS volume for a job definition. |
CfnJobDefinition.EmptyDirProperty | Struct | |
CfnJobDefinition.EnvironmentProperty | Struct | The Environment property type specifies environment variables to use in a job definition. |
CfnJobDefinition.EphemeralStorageProperty | Struct | The amount of ephemeral storage to allocate for the task. |
CfnJobDefinition.EvaluateOnExitProperty | Struct | Specifies an array of up to 5 conditions to be met, and an action to take ( RETRY or EXIT ) if all conditions are met. |
CfnJobDefinition.FargatePlatformConfigurationProperty | Struct | The platform configuration for jobs that are running on Fargate resources. |
CfnJobDefinition.FirelensConfigurationProperty | Struct | The FireLens configuration for the container. |
CfnJobDefinition.HostPathProperty | Struct | |
CfnJobDefinition.ImagePullSecretProperty | Struct | References a Kubernetes secret resource. |
CfnJobDefinition.JobTimeoutProperty | Struct | An object that represents a job timeout configuration. |
CfnJobDefinition.LinuxParametersProperty | Struct | Linux-specific modifications that are applied to the container, such as details for device mappings. |
CfnJobDefinition.LogConfigurationProperty | Struct | Log configuration options to send to a custom log driver for the container. |
CfnJobDefinition.MetadataProperty | Struct | |
CfnJobDefinition.MountPointProperty | Struct | Details for a Docker volume mount point that's used in a job's container properties. |
CfnJobDefinition.MountPointsProperty | Struct | |
CfnJobDefinition.MultiNodeEcsPropertiesProperty | Struct | An object that contains the properties for the Amazon ECS resources of a job. |
CfnJobDefinition.MultiNodeEcsTaskPropertiesProperty | Struct | The properties for a task definition that describes the container and volume definitions of an Amazon ECS task. |
CfnJobDefinition.NetworkConfigurationProperty | Struct | The network configuration for jobs that are running on Fargate resources. |
CfnJobDefinition.NodePropertiesProperty | Struct | An object that represents the node properties of a multi-node parallel job. |
CfnJobDefinition.NodeRangePropertyProperty | Struct | This is an object that represents the properties of the node range for a multi-node parallel job. |
CfnJobDefinition.PodPropertiesProperty | Struct | |
CfnJobDefinition.RepositoryCredentialsProperty | Struct | The repository credentials for private registry authentication. |
CfnJobDefinition.ResourceRequirementProperty | Struct | The type and amount of a resource to assign to a container. |
CfnJobDefinition.ResourceRetentionPolicyProperty | Struct | Specifies the resource retention policy settings for a job definition. |
CfnJobDefinition.ResourcesProperty | Struct | |
CfnJobDefinition.RetryStrategyProperty | Struct | The retry strategy that's associated with a job. |
CfnJobDefinition.RuntimePlatformProperty | Struct | An object that represents the compute environment architecture for AWS Batch jobs on Fargate. |
CfnJobDefinition.S3FilesVolumeConfigurationProperty | Struct | |
CfnJobDefinition.SecretProperty | Struct | An object that represents the secret to expose to your container. |
CfnJobDefinition.SecurityContextProperty | Struct | |
CfnJobDefinition.TaskContainerDependencyProperty | Struct | A list of containers that this task depends on. |
CfnJobDefinition.TaskContainerPropertiesProperty | Struct | Container properties are used for Amazon ECS-based job definitions. |
CfnJobDefinition.TimeoutProperty | Struct | |
CfnJobDefinition.TmpfsProperty | Struct | The container path, mount options, and size of the tmpfs mount. |
CfnJobDefinition.UlimitProperty | Struct | The ulimit settings to pass to the container. For more information, see Ulimit . |
CfnJobDefinition.VolumesHostProperty | Struct | |
CfnJobDefinition.VolumesProperty | Struct | |
CfnJobDefinitionProps | Struct | Properties for defining a CfnJobDefinition. |
CfnJobQueue.ComputeEnvironmentOrderProperty | Struct | The order that compute environments are tried in for job placement within a queue. |
CfnJobQueue.JobStateTimeLimitActionProperty | Struct | Specifies an action that AWS Batch will take after the job has remained at the head of the queue in the specified state for longer than the specified time. |
CfnJobQueue.ServiceEnvironmentOrderProperty | Struct | Specifies the order of a service environment for a job queue. |
CfnJobQueueProps | Struct | Properties for defining a CfnJobQueue. |
CfnQuotaShare.QuotaShareCapacityLimitProperty | Struct | |
CfnQuotaShare.QuotaSharePreemptionConfigurationProperty | Struct | |
CfnQuotaShare.QuotaShareResourceSharingConfigurationProperty | Struct | |
CfnQuotaShareProps | Struct | Properties for defining a CfnQuotaShare. |
CfnSchedulingPolicy.FairsharePolicyProperty | Struct | The fair-share scheduling policy details. |
CfnSchedulingPolicy.QuotaSharePolicyProperty | Struct | Quota Share Policy for the Job Queue. |
CfnSchedulingPolicy.ShareAttributesProperty | Struct | Specifies the weights for the share identifiers for the fair-share policy. |
CfnSchedulingPolicyProps | Struct | Properties for defining a CfnSchedulingPolicy. |
CfnServiceEnvironment.CapacityLimitProperty | Struct | Defines the capacity limit for a service environment. |
CfnServiceEnvironmentProps | Struct | Properties for defining a CfnServiceEnvironment. |
ComputeEnvironmentProps | Struct | Props common to all ComputeEnvironments. |
ConsumableResourceGrants | Facade | Collection of grant methods for a IConsumableResourceRef. |
ConsumableResourceProps | Struct | Properties for defining a Batch Consumable Resource. |
ConsumableResourceType | Enum | The type of consumable resource. |
CustomReason | Struct | The corresponding Action will only be taken if all of the conditions specified here are met. |
DefaultInstanceClass | Enum | Batch default instances types. |
Device | Struct | A container instance host device. |
DevicePermission | Enum | Permissions for device access. |
DnsPolicy | Enum | The DNS Policy for the pod used by the Job Definition. |
EcsContainerDefinitionProps | Struct | Props to configure an EcsContainerDefinition. |
EcsEc2ContainerDefinition | Construct | A container orchestrated by ECS that uses EC2 resources. |
EcsEc2ContainerDefinitionProps | Struct | Props to configure an EcsEc2ContainerDefinition. |
EcsFargateContainerDefinition | Construct | A container orchestrated by ECS that uses Fargate resources. |
EcsFargateContainerDefinitionProps | Struct | Props to configure an EcsFargateContainerDefinition. |
EcsJobDefinitionProps | Struct | Props for EcsJobDefinition. |
EcsMachineImage | Struct | A Batch MachineImage that is compatible with ECS. |
EcsMachineImageType | Enum | Maps the image to instance types. |
EcsVolume | Class | Represents a Volume that can be mounted to a container that uses ECS. |
EcsVolumeOptions | Struct | Options to configure an EcsVolume. |
EfsVolume | Class | A Volume that uses an AWS Elastic File System (EFS); |
EfsVolumeOptions | Struct | Options for configuring an EfsVolume. |
EksContainerDefinition | Construct | A container that can be run with EKS orchestration on EC2 resources. |
EksContainerDefinitionProps | Struct | Props to configure an EksContainerDefinition. |
EksJobDefinitionProps | Struct | Props for EksJobDefinition. |
EksMachineImage | Struct | A Batch MachineImage that is compatible with EKS. |
EksMachineImageType | Enum | Maps the image to instance types. |
EksVolume | Class | A Volume that can be mounted to a container supported by EKS. |
EksVolumeOptions | Struct | Options to configure an EksVolume. |
EmptyDirMediumType | Enum | What medium the volume will live in. |
EmptyDirVolume | Class | A Kubernetes EmptyDir volume. |
EmptyDirVolumeOptions | Struct | Options for a Kubernetes EmptyDir volume. |
FairshareSchedulingPolicyProps | Struct | Fairshare SchedulingPolicy configuration. |
FargateComputeEnvironmentProps | Struct | Props for a FargateComputeEnvironment. |
HostPathVolume | Class | A Kubernetes HostPath volume. |
HostPathVolumeOptions | Struct | Options for a kubernetes HostPath volume. |
HostVolume | Class | Creates a Host volume. |
HostVolumeOptions | Struct | Options for configuring an ECS HostVolume. |
IComputeEnvironment | Interface | Represents a ComputeEnvironment. |
IConsumableResource | Interface | Represents a Batch Consumable Resource. |
IEcsContainerDefinition | Interface | A container that can be run with ECS orchestration. |
IEcsEc2ContainerDefinition | Interface | A container orchestrated by ECS that uses EC2 resources. |
IEcsFargateContainerDefinition | Interface | A container orchestrated by ECS that uses Fargate resources and is orchestrated by ECS. |
IEksContainerDefinition | Interface | A container that can be run with EKS orchestration on EC2 resources. |
IEksJobDefinition | Interface | A JobDefinition that uses Eks orchestration. |
IFairshareSchedulingPolicy | Interface | Represents a Fairshare Scheduling Policy. Instructs the scheduler to allocate ComputeEnvironment vCPUs based on Job shareIdentifiers. |
IFargateComputeEnvironment | Interface | A ManagedComputeEnvironment that uses ECS orchestration on Fargate instances. |
IJobDefinition | Interface | Represents a JobDefinition. |
IJobQueue | Interface | Represents a JobQueue. |
ImagePullPolicy | Enum | Determines when the image is pulled from the registry to launch a container. |
IManagedComputeEnvironment | Interface | Represents a Managed ComputeEnvironment. |
IManagedEc2EcsComputeEnvironment | Interface | A ManagedComputeEnvironment that uses ECS orchestration on EC2 instances. |
ISchedulingPolicy | Interface | Represents a Scheduling Policy. |
IUnmanagedComputeEnvironment | Interface | Represents an UnmanagedComputeEnvironment. |
JobDefinitionProps | Struct | Props common to all JobDefinitions. |
JobQueueProps | Struct | Props to configure a JobQueue. |
JobStateTimeLimitAction | Struct | Specifies an action that AWS Batch will take after the job has remained at the head of the queue in the specified state for longer than the specified time. |
JobStateTimeLimitActionsAction | Enum | The action to take when a job is at the head of the job queue in the specified state for the specified period of time. |
JobStateTimeLimitActionsReason | Enum | The reason to log for the action being taken. |
JobStateTimeLimitActionsState | Enum | The state of the job needed to trigger the action. |
LinuxParameters | Construct | Linux-specific options that are applied to the container. |
LinuxParametersProps | Struct | The properties for defining Linux-specific options that are applied to the container. |
ManagedComputeEnvironmentProps | Struct | Props for a ManagedComputeEnvironment. |
ManagedEc2ComputeEnvironmentProps | Struct | Props for a ManagedEc2ComputeEnvironment. |
ManagedEc2EcsComputeEnvironmentProps | Struct | Props for a ManagedEc2EcsComputeEnvironment. |
ManagedEc2EksComputeEnvironmentProps | Struct | Props for a ManagedEc2EksComputeEnvironment. |
MultiNodeContainer | Struct | Runs the container on nodes [startNode, endNode]. |
MultiNodeJobDefinitionProps | Struct | Props to configure a MultiNodeJobDefinition. |
OptimalInstanceType | Class | Not a real instance type! |
OrderedComputeEnvironment | Struct | Assigns an order to a ComputeEnvironment. |
Reason | Class | Common job exit reasons. |
RetryStrategy | Class | Define how Jobs using this JobDefinition respond to different exit conditions. |
Secret | Class | A secret environment variable. |
SecretPathVolume | Class | Specifies the configuration of a Kubernetes secret volume. |
SecretPathVolumeOptions | Struct | Options for a Kubernetes SecretPath Volume. |
SecretVersionInfo | Struct | Specify the secret's version id or version stage. |
Share | Struct | Represents a group of Job Definitions. |
Tmpfs | Struct | The details of a tmpfs mount for a container. |
TmpfsMountOption | Enum | The supported options for a tmpfs mount for a container. |
Ulimit | Struct | Sets limits for a resource with ulimit on linux systems. |
UlimitName | Enum | The resources to be limited. |
UnmanagedComputeEnvironmentProps | Struct | Represents an UnmanagedComputeEnvironment. |