Developing workflows with Step Functions
We recommend starting to build workflows in the Step Functions console and Workflow Studio visual editor. You can start from a blank canvas or choose starter templates for common scenarios.
Building your workflows require the following tasks:
-
Defining your workflow
-
Running and debugging your workflow
-
Deploying your workflow
You define a state machine in Amazon States Language. You can manually create your Amazon States Language definitions, but Workflow Studio will be featured in tutorials. With Workflow Studio, you can define, your machine definition, visualize and edit the steps, run and debug your workflow, and view the results all from within the Step Functions console.
You can also use many Step Functions features from the AWS Command Line Interface (AWS CLI). For example, you can create a state machine and list your existing state machines. You can use Step Functions commands in the AWS CLI to start and manage executions, poll for activities, record task heartbeats, and more. For a complete list of Step Functions commands, descriptions of the available arguments, and examples showing their use, see the AWS CLI Command Reference. AWS CLI Command Reference
AWS CLI commands follow the Amazon States Language closely, so you can use the AWS CLI to learn about the Step Functions API actions. You can also use your existing API knowledge to prototype code or perform Step Functions actions from the command line.
Validating state machine definitions
You can use the API to validate state machines and find potential problems before creating your workflow.
To learn more about validating workflows, see ValidateStateMachineDefinition in the Step Functions API Reference.
To get started with minimal setup, you can follow the Creating a Lambda State Machine tutorial, which shows you how to define a workflow with a single step that calls a Lambda function, then run the workflow, and view the results.
Defining your workflow
The first step in developing your workflow is defining the steps in Amazon States Language. Depending on your preference and tool, you can define your Step Functions state machines in JSON, YAML, or as a stringified Amazon States Language (ASL) definition.
The following table shows ASL-based definition format support by tool.
AWS Tool | Supported format(s) |
---|---|
Step Functions Console | JSON |
HTTPS Service API | Stringified ASL |
AWS CLI | Stringified ASL |
Step Functions Local | Stringified ASL |
AWS Toolkit for Visual Studio Code | JSON, YAML |
AWS SAM | JSON, YAML |
AWS CloudFormation | JSON, YAML, Stringified ASL |
YAML single line comments in the state machine definition of a template will not
be carried forward into the created resource’s definition. If you need to persist a
comment, you should use the Comment
property within the state machine
definition. For information, see State machine structure.
With AWS CloudFormation and AWS SAM, you can upload your state machine definitions to Amazon S3 (JSON or YAML format) and provide the definition's Amazon S3 location in the template. For information see the AWS::StepFunctions::StateMachine S3Location page.
The following example AWS CloudFormation templates show how you can provide the same state machine definition using different input formats.
Develop workflows with AWS SDKs
Step Functions is supported by the AWS SDKs for Java, .NET, Ruby, PHP, Python (Boto 3),
JavaScript, Go, and C++. These SDKs provide a convenient way to use the Step Functions HTTPS API
actions in multiple programming languages. You can develop state machines, activities, or state machine starters using the API
actions exposed by these SDK libraries. You can also access visibility operations using
these libraries to develop your own Step Functions monitoring and reporting tools. See the reference documentation for the current
AWS SDKs and Tools for Amazon Web
Services
Develop workflows through HTTPS requests
Step Functions provides service operations that are accessible through HTTPS requests. You can use these operations to communicate directly with Step Functions from your own libraries. You can develop state machines, workers, or state machine starters using the service API actions. You can also access visibility operations through the API actions to develop your own monitoring and reporting tools. For details see the AWS Step Functions API Reference.
Develop workflows with the AWS Step Functions Data Science SDK
Data scientists can create workflows that
process and publish machine learning models using SageMaker and Step Functions. You can also create multi-step machine learning workflows in Python that orchestrate
AWS infrastructure at scale. The AWS Step Functions Data Science SDK provides a Python API that can create and invoke Step Functions workflows. You can manage and execute these workflows directly in Python, as well as Jupyter notebooks. For more information, see: AWS Step Functions Data Science Project
on Github
Running and debugging your workflows
You can start workflows in a number of ways, including from the console, an API call (for example, from a Lambda function), from Amazon EventBridge and EventBridge Scheduler, from another Step Functions state machine. Running workflows can connect to third party services, use AWS SDKs, and manipulate data while running. Various tools exist to both run and debug the execution steps and data flowing through your state machine. The following sections provide additional resources for running and debugging your workflows.
To learn more about the ways to start state machine executions, see Starting state machine executions in Step Functions.
Choose an endpoint to run your workflows
To reduce latency and store data in a location that meets your requirements, Step Functions
provides endpoints in different AWS Regions. Each endpoint in Step Functions is completely independent. A state machine or activity exists
only within the Region where it was created. Any state machines and activities that you
create in one Region do not share any data or attributes with those created in another
Region. For example, you can register a state machine named STATES-Flows-1
in
two different Regions. The STATES-Flows-1
state machine in one region won't
share data or attributes with the STATES-Flow-1
state machine in the other
region. For a list of Step Functions endpoints, see AWS Step Functions Regions and Endpoints in the
AWS General Reference.
Control the flow of data in your workflows
You can use the data flow
simulatorInputPath
,
Parameters
, ResultSelector
, OutputPath
, and
ResultPath
. For information, see Using data flow simulator to test data flow in Step Functions.
Local development version of Step Functions
For testing and development purposes, you have the option to install and run Step Functions on your local machine. The local version of Step Functions can invoke AWS Lambda functions, both in AWS and when running locally. You can also coordinate other supported AWS services. For more information, see Testing state machines locally in Step Functions.
Local and remote development with VS Code
You can also use VS Code to interact with remote state machines and develop state machines locally. You can create or update state machines, list existing state machines, and execute or download a state machine. VS Code also lets you create new state machines from templates, see a visualization of your state machine, and provides code snippets, code completion, and code validation. For more information, see the AWS Toolkit for Visual Studio Code User Guide
Deploying your workflows
After you have defined and debugged your workflows, you'll probably want to deploy using Infrastructure as Code frameworks. You can choose to deploy your state machines using a variety of IaC options, including: AWS Serverless Application Model, AWS CloudFormation, AWS CDK, and Terraform.
- AWS Serverless Application Model
You can use AWS Serverless Application Model with Step Functions to build workflows and deploy the infrastructure you need, including Lambda functions, APIs and events, to create serverless applications. You can also use the AWS SAM CLI in conjunction with the AWS Toolkit for Visual Studio Code as part of an integrated experience.
For more information, see Using AWS SAM to build Step Functions workflows.
- AWS CloudFormation
You can use your state machine definitions directly in AWS CloudFormation templates.
For more information, see Using AWS CloudFormation to create a workflow in Step Functions.
- AWS CDK
You can build Standard and Express state machines with AWS CDK.
To build a Standard workflow, see Using CDK to create a Standard workflow.
To build an Express workflow, see Using CDK to create an Express workflow.
- Terraform
Terraform
by HashiCorp is a framework for building applications using infrastructure as code (IaC). With Terraform, you can create state machines and use features, such as previewing infrastructure deployments and creating reusable templates. Terraform templates help you maintain and reuse the code by breaking it down into smaller chunks. For more information, see Using Terraform to deploy state machines in Step Functions.