Poll for Job Status (Lambda, AWS Batch)
This sample project creates an AWS Batch job poller. It implements an AWS Step Functions state
machine that uses AWS Lambda to create a Wait
state loop that checks on an AWS Batch
job.
This sample project creates and configures all resources so that your Step Functions workflow will submit an AWS Batch job, and will wait for that job to complete before ending successfully.
You can also implement this pattern without using a Lambda function. For information about controlling AWS Batch directly, see Using AWS Step Functions with other services.
This sample project creates the state machine, two Lambda functions, and an AWS Batch queue, and configures the related IAM permissions.
For more information about how AWS Step Functions can control other AWS services, see Using AWS Step Functions with other services.
Create the State Machine and Provision Resources
-
Open the Step Functions console
and choose Create state machine. -
Choose Run a sample project, and then choose Job Poller.
The state machine Definition and Visual Workflow are displayed.
Note The Definition section in this state machine references the AWS resources that will be created for this sample project.
-
Choose Next.
The Deploy resources page is displayed, listing the resources that will be created. For this sample project, the resources include:
-
A
SubmitJob
Lambda function -
A
CheckJob
Lambda function -
A
SampleJobQueue
Batch Job Queue
-
-
Choose Deploy resources.
Note It can take up to 10 minutes for these resources and related IAM permissions to be created. While the Deploy resources page is displayed, you can open the Stack ID link to see which resources are being provisioned.
Starting an Execution
After all the resources are provisioned and deployed, the Start execution dialog box is displayed with example input similar to the following.
{ "jobName": "my-job", "jobDefinition": "arn:aws:batch:us-east-1:123456789012:job-definition/SampleJobDefinition-343f54b445d5312:1", "jobQueue": "arn:aws:batch:us-east-1:123456789012:job-queue/SampleJobQueue-4d9d696031e1449", "wait_time": 60 }
wait_time
instructs the Wait
state to loop every 60
seconds.
To start a new execution
(Optional) To identify your execution, you can specify a name for it in the Name box. By default, Step Functions generates a unique execution name automatically.
Note Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.
-
Choose Start Execution.
-
(Optional) After the execution is complete, choose individual states on the Graph inspector, and then choose the Step input and Step output tabs to view each state's input and output respectively.
For example, to view the changing status of your AWS Batch job and the looping results of your execution, choose Step output.
Example State Machine Code
The state machine in this sample project integrates with AWS Lambda to submit an AWS Batch job. Browse through this example state machine to see how Step Functions controls Lambda and AWS Batch.
For more information about how AWS Step Functions can control other AWS services, see Using AWS Step Functions with other services.
{
"Comment": "An example of the Amazon States Language that runs an AWS Batch job and monitors the job until it completes.",
"StartAt": "Submit Job",
"States": {
"Submit Job": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:111122223333:function:StepFunctionsSample-JobStatusPol-SubmitJobFunction-jDaYcl4cx55r",
"ResultPath": "$.guid",
"Next": "Wait X Seconds"
},
"Wait X Seconds": {
"Type": "Wait",
"SecondsPath": "$.wait_time",
"Next": "Get Job Status"
},
"Get Job Status": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:111122223333:function:StepFunctionsSample-JobStatusPoll-CheckJobFunction-1JkJwY10vonI",
"Next": "Job Complete?",
"InputPath": "$.guid",
"ResultPath": "$.status"
},
"Job Complete?": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.status",
"StringEquals": "FAILED",
"Next": "Job Failed"
},
{
"Variable": "$.status",
"StringEquals": "SUCCEEDED",
"Next": "Get Final Job Status"
}
],
"Default": "Wait X Seconds"
},
"Job Failed": {
"Type": "Fail",
"Cause": "AWS Batch Job Failed",
"Error": "DescribeJob returned FAILED"
},
"Get Final Job Status": {
"Type": "Task",
"Resource": "arn:aws::lambda:us-east-1:111122223333:function:StepFunctionsSample-JobStatusPoll-CheckJobFunction-1JkJwY10vonI",
"InputPath": "$.guid",
"End": true
}
}
}