Manage AWS Glue Jobs with Step Functions - AWS Step Functions

Manage AWS Glue Jobs with Step Functions

Step Functions can control certain AWS services directly from the Amazon States Language (ASL). For more information about working with AWS Step Functions and its integrations, see the following:

How the Optimized AWS Glue integration is different than the AWS GlueAWS SDK integration
  • The Run a Job (.sync) integration pattern is available.

  • The JobName field is extracted from the request and inserted into the response, which normally only contains JobRunID.

Supported AWS Glue API:


Parameters in Step Functions are expressed in PascalCase, even if the native service API is in camelCase. For example, you could use the Step Functions API action startSyncExecution and specify its parameter as StateMachineArn.

The following includes a Task state that starts an AWS Glue job.

"Glue StartJobRun": { "Type": "Task", "Resource": "arn:aws:states:::glue:startJobRun.sync", "Parameters": { "JobName": "GlueJob-JTrRO5l98qMG" }, "Next": "ValidateOutput" },

For information about how to configure IAM permissions when using Step Functions with other AWS services, see IAM Policies for integrated services.