Tasks for AWS Step Functions¶
---AWS Step Functions is a web service that enables you to coordinate the components of distributed applications and microservices using visual workflows. You build applications from individual components that each perform a discrete function, or task, allowing you to scale and change applications quickly.
A Task state represents a single unit of work performed by a state machine. All work in your state machine is performed by tasks.
This module is part of the AWS Cloud Development Kit project.
Table Of Contents¶
Task¶
A Task state represents a single unit of work performed by a state machine. In the
CDK, the exact work to be done is determined by a class that implements IStepFunctionsTask
.
AWS Step Functions integrates with some AWS services so that you can call API actions, and coordinate executions directly from the Amazon States Language in Step Functions. You can directly call and pass parameters to the APIs of those services.
Paths¶
In the Amazon States Language, a path is a string beginning with $
that you
can use to identify components within JSON text.
Learn more about input and output processing in Step Functions here
InputPath¶
Both InputPath
and Parameters
fields provide a way to manipulate JSON as it
moves through your workflow. AWS Step Functions applies the InputPath
field first,
and then the Parameters
field. You can first filter your raw input to a selection
you want using InputPath, and then apply Parameters to manipulate that input
further, or add new values. If you don’t specify an InputPath
, a default value
of $
will be used.
The following example provides the field named input
as the input to the Task
state that runs a Lambda function.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
submit_job = tasks.LambdaInvoke(stack, "Invoke Handler",
lambda_function=submit_job_lambda,
input_path="$.input"
)
OutputPath¶
Tasks also allow you to select a portion of the state output to pass to the next
state. This enables you to filter out unwanted information, and pass only the
portion of the JSON that you care about. If you don’t specify an OutputPath
,
a default value of $
will be used. This passes the entire JSON node to the next
state.
The response from a Lambda function includes the response from the function as well as other metadata.
The following example assigns the output from the Task to a field named result
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
submit_job = tasks.LambdaInvoke(stack, "Invoke Handler",
lambda_function=submit_job_lambda,
output_path="$.Payload.result"
)
ResultPath¶
The output of a state can be a copy of its input, the result it produces (for
example, output from a Task state’s Lambda function), or a combination of its
input and result. Use ResultPath
to control which combination of these is
passed to the state output. If you don’t specify an ResultPath
, a default
value of $
will be used.
The following example adds the item from calling DynamoDB’s getItem
API to the state
input and passes it to the next state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.DynamoGetItem(self, "PutItem",
item={"MessageId": {"s": "12345"}},
table_name="my-table",
result_path="$.Item"
)
⚠️ The OutputPath
is computed after applying ResultPath
. All service integrations
return metadata as part of their response. When using ResultPath
, it’s not possible to
merge a subset of the task output to the input.
Task parameters from the state JSON¶
Most tasks take parameters. Parameter values can either be static, supplied directly
in the workflow definition (by specifying their values), or a value available at runtime
in the state machine’s execution (either as its input or an output of a prior state).
Parameter values available at runtime can be specified via the JsonPath
class,
using methods such as JsonPath.stringAt()
.
The following example provides the field named input
as the input to the Lambda function
and invokes it asynchronously.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
submit_job = tasks.LambdaInvoke(stack, "Invoke Handler",
lambda_function=submit_job_lambda,
payload=sfn.JsonPath.StringAt("$.input"),
invocation_type=tasks.InvocationType.EVENT
)
Each service integration has its own set of parameters that can be supplied.
Evaluate Expression¶
Use the EvaluateExpression
to perform simple operations referencing state paths. The
expression
referenced in the task will be evaluated in a Lambda function
(eval()
). This allows you to not have to write Lambda code for simple operations.
Example: convert a wait time from milliseconds to seconds, concat this in a message and wait:
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
convert_to_seconds = tasks.EvaluateExpression(self, "Convert to seconds",
expression="$.waitMilliseconds / 1000",
result_path="$.waitSeconds"
)
create_message = tasks.EvaluateExpression(self, "Create message",
# Note: this is a string inside a string.
expression="`Now waiting ${$.waitSeconds} seconds...`",
runtime=lambda_.Runtime.NODEJS_10_X,
result_path="$.message"
)
publish_message = tasks.SnsPublish(self, "Publish message",
topic=topic,
message=sfn.TaskInput.from_data_at("$.message"),
result_path="$.sns"
)
wait = sfn.Wait(self, "Wait",
time=sfn.WaitTime.seconds_path("$.waitSeconds")
)
sfn.StateMachine(self, "StateMachine",
definition=convert_to_seconds.next(create_message).next(publish_message).next(wait)
)
The EvaluateExpression
supports a runtime
prop to specify the Lambda
runtime to use to evaluate the expression. Currently, the only runtime
supported is lambda.Runtime.NODEJS_10_X
.
Athena¶
Step Functions supports Athena through the service integration pattern.
StartQueryExecution¶
The StartQueryExecution API runs the SQL query statement.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
import ??? as tasks
start_query_execution_job = tasks.AthenaStartQueryExecution(stack, "Start Athena Query",
query_string=sfn.JsonPath.string_at("$.queryString"),
query_execution_context={
"database": "mydatabase"
},
result_configuration={
"encryption_configuration": {
"encryption_option": tasks.EncryptionOption.S3_MANAGED
},
"output_location": sfn.JsonPath.string_at("$.outputLocation")
}
)
GetQueryExecution¶
The GetQueryExecution API gets information about a single execution of a query.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
import ??? as tasks
get_query_execution_job = tasks.AthenaGetQueryExecution(stack, "Get Query Execution",
query_execution_id=sfn.JsonPath.string_at("$.QueryExecutionId")
)
GetQueryResults¶
The GetQueryResults API that streams the results of a single query execution specified by QueryExecutionId from S3.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
import ??? as tasks
get_query_results_job = tasks.AthenaGetQueryResults(stack, "Get Query Results",
query_execution_id=sfn.JsonPath.string_at("$.QueryExecutionId")
)
StopQueryExecution¶
The StopQueryExecution API that stops a query execution.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
import ??? as tasks
stop_query_execution_job = tasks.AthenaStopQueryExecution(stack, "Stop Query Execution",
query_execution_id=sfn.JsonPath.string_at("$.QueryExecutionId")
)
Batch¶
Step Functions supports Batch through the service integration pattern.
SubmitJob¶
The SubmitJob API submits an AWS Batch job from a job definition.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_batch as batch
import aws_cdk.aws_stepfunctions_tasks as tasks
batch_queue = batch.JobQueue(self, "JobQueue",
compute_environments=[JobQueueComputeEnvironment(
order=1,
compute_environment=batch.ComputeEnvironment(self, "ComputeEnv",
compute_resources=ComputeResources(vpc=vpc)
)
)
]
)
batch_job_definition = batch.JobDefinition(self, "JobDefinition",
container=JobDefinitionContainer(
image=ecs.ContainerImage.from_asset(path.resolve(__dirname, "batchjob-image"))
)
)
task = tasks.BatchSubmitJob(self, "Submit Job",
job_definition=batch_job_definition,
job_name="MyJob",
job_queue=batch_queue
)
CodeBuild¶
Step Functions supports CodeBuild through the service integration pattern.
StartBuild¶
StartBuild starts a CodeBuild Project by Project Name.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_codebuild as codebuild
import aws_cdk.aws_stepfunctions_tasks as tasks
import aws_cdk.aws_stepfunctions as sfn
codebuild_project = codebuild.Project(stack, "Project",
project_name="MyTestProject",
build_spec=codebuild.BuildSpec.from_object({
"version": "0.2",
"phases": {
"build": {
"commands": ["echo "Hello, CodeBuild!""
]
}
}
})
)
task = tasks.CodeBuildStartBuild(stack, "Task",
project=codebuild_project,
integration_pattern=sfn.IntegrationPattern.RUN_JOB,
environment_variables_override={
"ZONE": BuildEnvironmentVariable(
type=codebuild.BuildEnvironmentVariableType.PLAINTEXT,
value=sfn.JsonPath.string_at("$.envVariables.zone")
)
}
)
DynamoDB¶
You can call DynamoDB APIs from a Task
state.
Read more about calling DynamoDB APIs here
GetItem¶
The GetItem operation returns a set of attributes for the item with the given primary key.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.DynamoGetItem(self, "Get Item",
key={"message_id": tasks.DynamoAttributeValue.from_string("message-007")},
table=table
)
PutItem¶
The PutItem operation creates a new item, or replaces an old item with a new item.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.DynamoPutItem(self, "PutItem",
item={
"MessageId": tasks.DynamoAttributeValue.from_string("message-007"),
"Text": tasks.DynamoAttributeValue.from_string(sfn.JsonPath.string_at("$.bar")),
"TotalCount": tasks.DynamoAttributeValue.from_number(10)
},
table=table
)
DeleteItem¶
The DeleteItem operation deletes a single item in a table by primary key.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
import aws_cdk.aws_stepfunctions_tasks as tasks
tasks.DynamoDeleteItem(self, "DeleteItem",
key={"MessageId": tasks.DynamoAttributeValue.from_string("message-007")},
table=table,
result_path=sfn.JsonPath.DISCARD
)
UpdateItem¶
The UpdateItem operation edits an existing item’s attributes, or adds a new item to the table if it does not already exist.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.DynamoUpdateItem(self, "UpdateItem",
key={"MessageId": tasks.DynamoAttributeValue.from_string("message-007")},
table=table,
expression_attribute_values={
":val": tasks.DynamoAttributeValue.number_from_string(sfn.JsonPath.string_at("$.Item.TotalCount.N")),
":rand": tasks.DynamoAttributeValue.from_number(20)
},
update_expression="SET TotalCount = :val + :rand"
)
ECS¶
Step Functions supports ECS/Fargate through the service integration pattern.
RunTask¶
RunTask starts a new task using the specified task definition.
EC2¶
The EC2 launch type allows you to run your containerized applications on a cluster of Amazon EC2 instances that you manage.
When a task that uses the EC2 launch type is launched, Amazon ECS must determine where to place the task based on the requirements specified in the task definition, such as CPU and memory. Similarly, when you scale down the task count, Amazon ECS must determine which tasks to terminate. You can apply task placement strategies and constraints to customize how Amazon ECS places and terminates tasks. Learn more about task placement
The latest ACTIVE revision of the passed task definition is used for running the task.
The following example runs a job from a task definition on EC2
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_ecs as ecs
import aws_cdk.aws_stepfunctions_tasks as tasks
import aws_cdk.aws_stepfunctions as sfn
vpc = ec2.Vpc.from_lookup(stack, "Vpc",
is_default=True
)
cluster = ecs.Cluster(stack, "Ec2Cluster", vpc=vpc)
cluster.add_capacity("DefaultAutoScalingGroup",
instance_type=ec2.InstanceType("t2.micro"),
vpc_subnets=SubnetSelection(subnet_type=ec2.SubnetType.PUBLIC)
)
task_definition = ecs.TaskDefinition(stack, "TD",
compatibility=ecs.Compatibility.EC2
)
task_definition.add_container("TheContainer",
image=ecs.ContainerImage.from_registry("foo/bar"),
memory_limit_mi_b=256
)
run_task = tasks.EcsRunTask(stack, "Run",
integration_pattern=sfn.IntegrationPattern.RUN_JOB,
cluster=cluster,
task_definition=task_definition,
launch_target=tasks.EcsEc2LaunchTarget(
placement_strategies=[
ecs.PlacementStrategy.spread_across_instances(),
ecs.PlacementStrategy.packed_by_cpu(),
ecs.PlacementStrategy.randomly()
],
placement_constraints=[
ecs.PlacementConstraint.member_of("blieptuut")
]
)
)
Fargate¶
AWS Fargate is a serverless compute engine for containers that works with Amazon Elastic Container Service (ECS). Fargate makes it easy for you to focus on building your applications. Fargate removes the need to provision and manage servers, lets you specify and pay for resources per application, and improves security through application isolation by design. Learn more about Fargate
The Fargate launch type allows you to run your containerized applications without the need to provision and manage the backend infrastructure. Just register your task definition and Fargate launches the container for you. The latest ACTIVE revision of the passed task definition is used for running the task. Learn more about Fargate Versioning
The following example runs a job from a task definition on Fargate
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_ecs as ecs
import aws_cdk.aws_stepfunctions_tasks as tasks
import aws_cdk.aws_stepfunctions as sfn
vpc = ec2.Vpc.from_lookup(stack, "Vpc",
is_default=True
)
cluster = ecs.Cluster(stack, "FargateCluster", vpc=vpc)
task_definition = ecs.TaskDefinition(stack, "TD",
memory_mi_b="512",
cpu="256",
compatibility=ecs.Compatibility.FARGATE
)
container_definition = task_definition.add_container("TheContainer",
image=ecs.ContainerImage.from_registry("foo/bar"),
memory_limit_mi_b=256
)
run_task = tasks.EcsRunTask(stack, "RunFargate",
integration_pattern=sfn.IntegrationPattern.RUN_JOB,
cluster=cluster,
task_definition=task_definition,
assign_public_ip=True,
container_overrides=[ContainerOverride(
container_definition=container_definition,
environment=[TaskEnvironmentVariable(name="SOME_KEY", value=sfn.JsonPath.string_at("$.SomeKey"))]
)],
launch_target=tasks.EcsFargateLaunchTarget()
)
EMR¶
Step Functions supports Amazon EMR through the service integration pattern. The service integration APIs correspond to Amazon EMR APIs but differ in the parameters that are used.
Read more about the differences when using these service integrations.
Create Cluster¶
Creates and starts running a cluster (job flow).
Corresponds to the runJobFlow
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
cluster_role = iam.Role(stack, "ClusterRole",
assumed_by=iam.ServicePrincipal("ec2.amazonaws.com")
)
service_role = iam.Role(stack, "ServiceRole",
assumed_by=iam.ServicePrincipal("elasticmapreduce.amazonaws.com")
)
auto_scaling_role = iam.Role(stack, "AutoScalingRole",
assumed_by=iam.ServicePrincipal("elasticmapreduce.amazonaws.com")
)
auto_scaling_role.assume_role_policy.add_statements(
iam.PolicyStatement(
effect=iam.Effect.ALLOW,
principals=[
iam.ServicePrincipal("application-autoscaling.amazonaws.com")
],
actions=["sts:AssumeRole"
]
))
tasks.EmrCreateCluster(stack, "Create Cluster",
instances={},
cluster_role=cluster_role,
name=sfn.TaskInput.from_data_at("$.ClusterName").value,
service_role=service_role,
auto_scaling_role=auto_scaling_role,
integration_pattern=sfn.ServiceIntegrationPattern.FIRE_AND_FORGET
)
Termination Protection¶
Locks a cluster (job flow) so the EC2 instances in the cluster cannot be terminated by user intervention, an API call, or a job-flow error.
Corresponds to the setTerminationProtection
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.EmrSetClusterTerminationProtection(stack, "Task",
cluster_id="ClusterId",
termination_protected=False
)
Terminate Cluster¶
Shuts down a cluster (job flow).
Corresponds to the terminateJobFlows
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.EmrTerminateCluster(stack, "Task",
cluster_id="ClusterId"
)
Add Step¶
Adds a new step to a running cluster.
Corresponds to the addJobFlowSteps
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.EmrAddStep(stack, "Task",
cluster_id="ClusterId",
name="StepName",
jar="Jar",
action_on_failure=tasks.ActionOnFailure.CONTINUE
)
Cancel Step¶
Cancels a pending step in a running cluster.
Corresponds to the cancelSteps
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.EmrCancelStep(stack, "Task",
cluster_id="ClusterId",
step_id="StepId"
)
Modify Instance Fleet¶
Modifies the target On-Demand and target Spot capacities for the instance fleet with the specified InstanceFleetName.
Corresponds to the modifyInstanceFleet
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.EmrModifyInstanceFleetByName(stack, "Task",
cluster_id="ClusterId",
instance_fleet_name="InstanceFleetName",
target_on_demand_capacity=2,
target_spot_capacity=0
)
Modify Instance Group¶
Modifies the number of nodes and configuration settings of an instance group.
Corresponds to the modifyInstanceGroups
API in EMR.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.EmrModifyInstanceGroupByName(stack, "Task",
cluster_id="ClusterId",
instance_group_name=sfn.JsonPath.string_at("$.InstanceGroupName"),
instance_group={
"instance_count": 1
}
)
EKS¶
Step Functions supports Amazon EKS through the service integration pattern. The service integration APIs correspond to Amazon EKS APIs.
Read more about the differences when using these service integrations.
Call¶
Read and write Kubernetes resource objects via a Kubernetes API endpoint.
Corresponds to the call
API in Step Functions Connector.
The following code snippet includes a Task state that uses eks:call to list the pods.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_eks as eks
import aws_cdk.aws_stepfunctions as sfn
import aws_cdk.aws_stepfunctions_tasks as tasks
my_eks_cluster = eks.Cluster(self, "my sample cluster",
version=eks.KubernetesVersion.V1_18,
cluster_name="myEksCluster"
)
tasks.EksCall(stack, "Call a EKS Endpoint",
cluster=my_eks_cluster,
http_method=MethodType.GET,
http_path="/api/v1/namespaces/default/pods"
)
Glue¶
Step Functions supports AWS Glue through the service integration pattern.
You can call the StartJobRun
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
GlueStartJobRun(stack, "Task",
glue_job_name="my-glue-job",
arguments={
"key": "value"
},
timeout=cdk.Duration.minutes(30),
notify_delay_after=cdk.Duration.minutes(5)
)
Glue DataBrew¶
Step Functions supports AWS Glue DataBrew through the service integration pattern.
You can call the StartJobRun
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
GlueDataBrewStartJobRun(stack, "Task",
Name="databrew-job"
)
Lambda¶
Invoke a Lambda function.
You can specify the input to your Lambda function through the payload
attribute.
By default, Step Functions invokes Lambda function with the state input (JSON path ‘$’)
as the input.
The following snippet invokes a Lambda Function with the state input as the payload
by referencing the $
path.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_lambda as lambda_
import aws_cdk.aws_stepfunctions as sfn
import aws_cdk.aws_stepfunctions_tasks as tasks
my_lambda = lambda_.Function(self, "my sample lambda",
code=Code.from_inline("""exports.handler = async () => {
return {
statusCode: '200',
body: 'hello, world!'
};
};"""),
runtime=Runtime.NODEJS_12_X,
handler="index.handler"
)
tasks.LambdaInvoke(self, "Invoke with state input",
lambda_function=my_lambda
)
When a function is invoked, the Lambda service sends these response elements back.
⚠️ The response from the Lambda function is in an attribute called Payload
The following snippet invokes a Lambda Function by referencing the $.Payload
path
to reference the output of a Lambda executed before it.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.LambdaInvoke(self, "Invoke with empty object as payload",
lambda_function=my_lambda,
payload=sfn.TaskInput.from_object()
)
# use the output of myLambda as input
tasks.LambdaInvoke(self, "Invoke with payload field in the state input",
lambda_function=my_other_lambda,
payload=sfn.TaskInput.from_data_at("$.Payload")
)
The following snippet invokes a Lambda and sets the task output to only include the Lambda function response.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.LambdaInvoke(self, "Invoke and set function response as task output",
lambda_function=my_lambda,
payload=sfn.TaskInput.from_data_at("$"),
output_path="$.Payload"
)
If you want to combine the input and the Lambda function response you can use
the payloadResponseOnly
property and specify the resultPath
. This will put the
Lambda function ARN directly in the “Resource” string, but it conflicts with the
integrationPattern, invocationType, clientContext, and qualifier properties.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.LambdaInvoke(self, "Invoke and combine function response with task input",
lambda_function=my_lambda,
payload_response_only=True,
result_path="$.myLambda"
)
You can have Step Functions pause a task, and wait for an external process to return a task token. Read more about the callback pattern
To use the callback pattern, set the token
property on the task. Call the Step
Functions SendTaskSuccess
or SendTaskFailure
APIs with the token to
indicate that the task has completed and the state machine should resume execution.
The following snippet invokes a Lambda with the task token as part of the input to the Lambda.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
tasks.LambdaInvoke(stack, "Invoke with callback",
lambda_function=my_lambda,
integration_pattern=sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN,
payload=sfn.TaskInput.from_object(
token=sfn.JsonPath.task_token,
input=sfn.JsonPath.string_at("$.someField")
)
)
⚠️ The task will pause until it receives that task token back with a SendTaskSuccess
or SendTaskFailure
call. Learn more about Callback with the Task
Token.
AWS Lambda can occasionally experience transient service errors. In this case, invoking Lambda
results in a 500 error, such as ServiceException
, AWSLambdaException
, or SdkClientException
.
As a best practive, the LambdaInvoke
task will retry on those errors with an interval of 2 seconds,
a back-off rate of 2 and 6 maximum attempts. Set the retryOnServiceExceptions
prop to false
to
disable this behavior.
SageMaker¶
Step Functions supports AWS SageMaker through the service integration pattern.
Create Training Job¶
You can call the CreateTrainingJob
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.SageMakerCreateTrainingJob(self, "TrainSagemaker",
training_job_name=sfn.JsonPath.string_at("$.JobName"),
role=role,
algorithm_specification={
"algorithm_name": "BlazingText",
"training_input_mode": tasks.InputMode.FILE
},
input_data_config=[{
"channel_name": "train",
"data_source": {
"s3_data_source": {
"s3_data_type": tasks.S3DataType.S3_PREFIX,
"s3_location": tasks.S3Location.from_json_expression("$.S3Bucket")
}
}
}],
output_data_config={
"s3_output_location": tasks.S3Location.from_bucket(s3.Bucket.from_bucket_name(stack, "Bucket", "mybucket"), "myoutputpath")
},
resource_config={
"instance_count": 1,
"instance_type": ec2.InstanceType.of(ec2.InstanceClass.P3, ec2.InstanceSize.XLARGE2),
"volume_size": cdk.Size.gibibytes(50)
},
stopping_condition={
"max_runtime": cdk.Duration.hours(1)
}
)
Create Transform Job¶
You can call the CreateTransformJob
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.SageMakerCreateTransformJob(self, "Batch Inference",
transform_job_name="MyTransformJob",
model_name="MyModelName",
model_client_options={
"invocation_max_retries": 3, # default is 0
"invocation_timeout": cdk.Duration.minutes(5)
},
role=role,
transform_input={
"transform_data_source": {
"s3_data_source": {
"s3_uri": "s3://inputbucket/train",
"s3_data_type": S3DataType.S3Prefix
}
}
},
transform_output={
"s3_output_path": "s3://outputbucket/TransformJobOutputPath"
},
transform_resources={
"instance_count": 1,
"instance_type": ec2.InstanceType.of(ec2.InstanceClass.M4, ec2.InstanceSize.XLarge)
}
)
Create Endpoint¶
You can call the CreateEndpoint
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.SageMakerCreateEndpoint(self, "SagemakerEndpoint",
endpoint_name=sfn.JsonPath.string_at("$.EndpointName"),
endpoint_config_name=sfn.JsonPath.string_at("$.EndpointConfigName")
)
Create Endpoint Config¶
You can call the CreateEndpointConfig
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.SageMakerCreateEndpointConfig(self, "SagemakerEndpointConfig",
endpoint_config_name="MyEndpointConfig",
production_variants=[{
"initial_instance_count": 2,
"instance_type": ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.XLARGE),
"model_name": "MyModel",
"variant_name": "awesome-variant"
}]
)
Create Model¶
You can call the CreateModel
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.SageMakerCreateModel(self, "Sagemaker",
model_name="MyModel",
primary_container=tasks.ContainerDefinition(
image=tasks.DockerImage.from_json_expression(sfn.JsonPath.string_at("$.Model.imageName")),
mode=tasks.Mode.SINGLE_MODEL,
model_s3_location=tasks.S3Location.from_json_expression("$.TrainingJob.ModelArtifacts.S3ModelArtifacts")
)
)
Update Endpoint¶
You can call the UpdateEndpoint
API from a Task
state.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
sfn.SageMakerUpdateEndpoint(self, "SagemakerEndpoint",
endpoint_name=sfn.JsonPath.string_at("$.Endpoint.Name"),
endpoint_config_name=sfn.JsonPath.string_at("$.Endpoint.EndpointConfig")
)
SNS¶
Step Functions supports Amazon SNS through the service integration pattern.
You can call the Publish
API from a Task
state to publish to an SNS topic.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_sns as sns
import aws_cdk.aws_stepfunctions as sfn
import aws_cdk.aws_stepfunctions_tasks as tasks
# ...
topic = sns.Topic(self, "Topic")
# Use a field from the execution data as message.
task1 = tasks.SnsPublish(self, "Publish1",
topic=topic,
integration_pattern=sfn.IntegrationPattern.REQUEST_RESPONSE,
message=sfn.TaskInput.from_data_at("$.state.message")
)
# Combine a field from the execution data with
# a literal object.
task2 = tasks.SnsPublish(self, "Publish2",
topic=topic,
message=sfn.TaskInput.from_object({
"field1": "somedata",
"field2": sfn.JsonPath.string_at("$.field2")
})
)
Step Functions¶
Start Execution¶
You can manage AWS Step Functions executions.
AWS Step Functions supports it’s own StartExecution
API as a service integration.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
# Define a state machine with one Pass state
child = sfn.StateMachine(stack, "ChildStateMachine",
definition=sfn.Chain.start(sfn.Pass(stack, "PassState"))
)
# Include the state machine in a Task state with callback pattern
task = StepFunctionsStartExecution(stack, "ChildTask",
state_machine=child,
integration_pattern=sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN,
input=sfn.TaskInput.from_object(
token=sfn.JsonPath.task_token,
foo="bar"
),
name="MyExecutionName"
)
# Define a second state machine with the Task state above
sfn.StateMachine(stack, "ParentStateMachine",
definition=task
)
Invoke Activity¶
You can invoke a Step Functions Activity which enables you to have a task in your state machine where the work is performed by a worker that can be hosted on Amazon EC2, Amazon ECS, AWS Lambda, basically anywhere. Activities are a way to associate code running somewhere (known as an activity worker) with a specific task in a state machine.
When Step Functions reaches an activity task state, the workflow waits for an activity worker to poll for a task. An activity worker polls Step Functions by using GetActivityTask, and sending the ARN for the related activity.
After the activity worker completes its work, it can provide a report of its
success or failure by using SendTaskSuccess
or SendTaskFailure
. These two
calls use the taskToken provided by GetActivityTask to associate the result
with that task.
The following example creates an activity and creates a task that invokes the activity.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
submit_job_activity = sfn.Activity(self, "SubmitJob")
tasks.StepFunctionsInvokeActivity(self, "Submit Job",
activity=submit_job_activity
)
SQS¶
Step Functions supports Amazon SQS
You can call the SendMessage
API from a Task
state
to send a message to an SQS queue.
# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import aws_cdk.aws_stepfunctions as sfn
import aws_cdk.aws_stepfunctions_tasks as tasks
import aws_cdk.aws_sqs as sqs
# ...
queue = sqs.Queue(self, "Queue")
# Use a field from the execution data as message.
task1 = tasks.SqsSendMessage(self, "Send1",
queue=queue,
message_body=sfn.TaskInput.from_data_at("$.message")
)
# Combine a field from the execution data with
# a literal object.
task2 = tasks.SqsSendMessage(self, "Send2",
queue=queue,
message_body=sfn.TaskInput.from_object({
"field1": "somedata",
"field2": sfn.JsonPath.string_at("$.field2")
})
)