There are more AWS SDK examples available in the AWS Doc SDK Examples
Step Functions examples using AWS CLI with Bash script
The following code examples show you how to perform actions and implement common scenarios by using the AWS Command Line Interface with Bash script with Step Functions.
Scenarios are code examples that show you how to accomplish specific tasks by calling multiple functions within a service or combined with other AWS services.
Each example includes a link to the complete source code, where you can find instructions on how to set up and run the code in context.
Topics
Scenarios
The following code example shows how to:
Create an IAM role for Step Functions
Create your first state machine
Start your state machine execution
Clean up resources
- AWS CLI with Bash script
-
Note
There's more on GitHub. Find the complete example and learn how to set up and run in the Sample developer tutorials
repository. #!/bin/bash # AWS Step Functions Getting Started Tutorial Script # This script creates and runs a Step Functions state machine based on the AWS Step Functions Getting Started tutorial set -euo pipefail # Security: Restrict umask to prevent unintended file permissions umask 077 # Parse command line arguments AUTO_CLEANUP=true while [[ $# -gt 0 ]]; do case $1 in --auto-cleanup) AUTO_CLEANUP=true shift ;; -h|--help) echo "Usage: $0 [--auto-cleanup] [--help]" echo " --auto-cleanup: Automatically clean up resources without prompting" echo " --help: Show this help message" exit 0 ;; *) echo "Unknown option: $1" echo "Use --help for usage information" exit 1 ;; esac done # Set up logging with secure permissions LOG_FILE="step-functions-tutorial.log" touch "$LOG_FILE" chmod 600 "$LOG_FILE" # Security: Use process substitution with explicit FD cleanup exec 3>&1 4>&2 exec > >(tee -a "$LOG_FILE") 2>&1 trap 'exec 1>&3 2>&4 3>&- 4>&-' EXIT echo "Starting AWS Step Functions Getting Started Tutorial..." echo "Logging to $LOG_FILE" # Verify AWS CLI is installed and configured if ! command -v aws &> /dev/null; then echo "ERROR: AWS CLI is not installed" exit 1 fi # Verify AWS credentials are configured if ! aws sts get-caller-identity &> /dev/null; then echo "ERROR: AWS credentials are not configured or invalid" exit 1 fi # Check if jq is available for better JSON parsing if ! command -v jq &> /dev/null; then echo "WARNING: jq is not installed. Using basic JSON parsing which may be less reliable." echo "Consider installing jq for better error handling: brew install jq (macOS) or apt-get install jq (Ubuntu)" USE_JQ=false else USE_JQ=true fi # Use fixed region that supports Amazon Comprehend CURRENT_REGION="us-west-2" echo "Using fixed AWS region: $CURRENT_REGION (supports Amazon Comprehend)" # Set AWS CLI to use the fixed region for all commands export AWS_DEFAULT_REGION="$CURRENT_REGION" export AWS_REGION="$CURRENT_REGION" # Amazon Comprehend is available in us-west-2, so we can always enable it echo "Amazon Comprehend is available in region $CURRENT_REGION" SKIP_COMPREHEND=false # Security: Initialize all resource variables STATE_MACHINE_ARN="" ROLE_NAME="" ROLE_ARN="" POLICY_ARN="" STEPFUNCTIONS_POLICY_ARN="" EXECUTION_ARN="" EXECUTION2_ARN="" EXECUTION3_ARN="" # Performance: Cache for AWS API calls to reduce redundant requests declare -A API_CACHE # Function to make cached AWS CLI calls aws_call_cached() { local cache_key="$1" shift if [[ -v API_CACHE["$cache_key"] ]]; then echo "${API_CACHE[$cache_key]}" return 0 fi local result result=$(aws "$@" 2>&1) || return $? API_CACHE["$cache_key"]="$result" echo "$result" } # Function to check for API errors in JSON response with optimized jq usage check_api_error() { local response="$1" local operation="$2" if [[ "$USE_JQ" == "true" ]]; then # Use jq for more reliable JSON parsing with efficient error detection if echo "$response" | jq -e '.Error // .error // empty' > /dev/null 2>&1; then local error_message=$(echo "$response" | jq -r '.Error.Message // .Error.Code // .error // "Unknown error"' 2>/dev/null) handle_error "$operation failed: $error_message" fi else # Fallback to grep-based detection with optimized pattern if echo "$response" | grep -qE '"[Ee]rror":|"error":'; then handle_error "$operation failed: $response" fi fi } # Function to extract JSON field efficiently extract_json_field() { local json="$1" local field="$2" if [[ "$USE_JQ" == "true" ]]; then echo "$json" | jq -r "$field" 2>/dev/null else echo "$json" | grep -oP "\"${field}\":\s*\"\K[^\"]+|\"${field}\":\s*\K[^,}]+" | head -1 fi } # Function to securely wait for resource propagation with exponential backoff wait_for_propagation() { local resource_type="$1" local wait_time="${2:-10}" # Validate wait_time is a positive integer if ! [[ "$wait_time" =~ ^[0-9]+$ ]] || [ "$wait_time" -lt 1 ] || [ "$wait_time" -gt 300 ]; then echo "WARNING: Invalid wait time $wait_time, using default 10 seconds" wait_time=10 fi echo "Waiting for $resource_type to propagate ($wait_time seconds)..." sleep "$wait_time" } # Function to validate JSON file efficiently validate_json_file() { local file="$1" if [[ "$USE_JQ" == "true" ]]; then if ! jq empty "$file" 2>/dev/null; then handle_error "Invalid JSON in $file" fi fi } # Function to handle errors handle_error() { echo "ERROR: $1" echo "Resources created:" if [ -n "${STATE_MACHINE_ARN:-}" ]; then echo "- State Machine: $STATE_MACHINE_ARN" fi if [ -n "${ROLE_NAME:-}" ]; then echo "- IAM Role: $ROLE_NAME" fi if [ -n "${POLICY_ARN:-}" ]; then echo "- IAM Policy: $POLICY_ARN" fi if [ -n "${STEPFUNCTIONS_POLICY_ARN:-}" ]; then echo "- Step Functions Policy: $STEPFUNCTIONS_POLICY_ARN" fi echo "Attempting to clean up resources..." cleanup exit 1 } # Function to securely clean up resources with parallel deletion cleanup() { echo "Cleaning up resources..." # Delete state machine if it exists if [ -n "${STATE_MACHINE_ARN:-}" ]; then echo "Deleting state machine: $STATE_MACHINE_ARN" aws stepfunctions delete-state-machine --state-machine-arn "$STATE_MACHINE_ARN" 2>/dev/null & fi # Detach and delete policies if they exist if [ -n "${POLICY_ARN:-}" ] && [ -n "${ROLE_NAME:-}" ]; then echo "Detaching Comprehend policy $POLICY_ARN from role $ROLE_NAME" aws iam detach-role-policy --role-name "$ROLE_NAME" --policy-arn "$POLICY_ARN" 2>/dev/null & fi if [ -n "${STEPFUNCTIONS_POLICY_ARN:-}" ] && [ -n "${ROLE_NAME:-}" ]; then echo "Detaching Step Functions policy $STEPFUNCTIONS_POLICY_ARN from role $ROLE_NAME" aws iam detach-role-policy --role-name "$ROLE_NAME" --policy-arn "$STEPFUNCTIONS_POLICY_ARN" 2>/dev/null & fi # Wait for detach operations to complete wait 2>/dev/null || true # Delete custom policies if they exist if [ -n "${POLICY_ARN:-}" ]; then echo "Deleting Comprehend policy: $POLICY_ARN" aws iam delete-policy --policy-arn "$POLICY_ARN" 2>/dev/null & fi if [ -n "${STEPFUNCTIONS_POLICY_ARN:-}" ]; then echo "Deleting Step Functions policy: $STEPFUNCTIONS_POLICY_ARN" aws iam delete-policy --policy-arn "$STEPFUNCTIONS_POLICY_ARN" 2>/dev/null & fi # Wait for policy deletion to complete wait 2>/dev/null || true # Delete role if it exists if [ -n "${ROLE_NAME:-}" ]; then echo "Deleting role: $ROLE_NAME" aws iam delete-role --role-name "$ROLE_NAME" 2>/dev/null || echo "Failed to delete role" fi # Remove temporary files securely echo "Removing temporary files" local temp_files=( "hello-world.json" "updated-hello-world.json" "sentiment-hello-world.json" "step-functions-trust-policy.json" "comprehend-policy.json" "stepfunctions-policy.json" "input.json" "sentiment-input.json" ) for file in "${temp_files[@]}"; do if [ -f "$file" ]; then if command -v shred &> /dev/null; then shred -vfz -n 3 "$file" 2>/dev/null || rm -f "$file" else rm -f "$file" fi fi done } # Security: Set trap to cleanup on script exit trap cleanup EXIT # Generate a secure random identifier for resource names RANDOM_ID=$(openssl rand -hex 4) ROLE_NAME="StepFunctionsHelloWorldRole-${RANDOM_ID}" POLICY_NAME="DetectSentimentPolicy-${RANDOM_ID}" STATE_MACHINE_NAME="MyFirstStateMachine-${RANDOM_ID}" echo "Using random identifier: $RANDOM_ID" echo "Role name: $ROLE_NAME" echo "Policy name: $POLICY_NAME" echo "State machine name: $STATE_MACHINE_NAME" # Step 1: Create the state machine definition echo "Creating state machine definition..." cat > hello-world.json << 'EOF' { "Comment": "A Hello World example of the Amazon States Language using a Pass state", "StartAt": "SetVariables", "States": { "SetVariables": { "Type": "Pass", "Result": { "IsHelloWorldExample": true, "ExecutionWaitTimeInSeconds": 10 }, "Next": "IsHelloWorldExample" }, "IsHelloWorldExample": { "Type": "Choice", "Choices": [ { "Variable": "$.IsHelloWorldExample", "BooleanEquals": true, "Next": "WaitState" } ], "Default": "FailState" }, "WaitState": { "Type": "Wait", "SecondsPath": "$.ExecutionWaitTimeInSeconds", "Next": "ParallelProcessing" }, "ParallelProcessing": { "Type": "Parallel", "Branches": [ { "StartAt": "Process1", "States": { "Process1": { "Type": "Pass", "Result": { "message": "Processing task 1" }, "End": true } } }, { "StartAt": "Process2", "States": { "Process2": { "Type": "Pass", "Result": { "message": "Processing task 2" }, "End": true } } } ], "Next": "CheckpointState" }, "CheckpointState": { "Type": "Pass", "Result": { "CheckpointMessage": "Workflow completed successfully!" }, "Next": "SuccessState" }, "SuccessState": { "Type": "Succeed" }, "FailState": { "Type": "Fail", "Error": "NotHelloWorldExample", "Cause": "The IsHelloWorldExample value was false" } } } EOF # Create IAM role trust policy echo "Creating IAM role trust policy..." cat > step-functions-trust-policy.json << 'EOF' { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "states.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } EOF # Create IAM role echo "Creating IAM role: $ROLE_NAME" ROLE_RESULT=$(aws iam create-role \ --role-name "$ROLE_NAME" \ --assume-role-policy-document file://step-functions-trust-policy.json 2>&1) check_api_error "$ROLE_RESULT" "Create IAM role" echo "Role created successfully" # Get the role ARN ROLE_ARN=$(extract_json_field "$ROLE_RESULT" ".Role.Arn") if [ -z "$ROLE_ARN" ]; then handle_error "Failed to extract role ARN" fi echo "Role ARN: $ROLE_ARN" # Create a custom policy for Step Functions with least privilege echo "Creating custom policy for Step Functions..." cat > stepfunctions-policy.json << 'EOF' { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "states:StartExecution", "states:DescribeExecution", "states:StopExecution" ], "Resource": "arn:aws:states:*:*:stateMachine:*" } ] } EOF # Create the policy echo "Creating Step Functions policy..." STEPFUNCTIONS_POLICY_RESULT=$(aws iam create-policy \ --policy-name "StepFunctionsPolicy-${RANDOM_ID}" \ --policy-document file://stepfunctions-policy.json 2>&1) check_api_error "$STEPFUNCTIONS_POLICY_RESULT" "Create Step Functions policy" echo "Step Functions policy created successfully" # Get the policy ARN STEPFUNCTIONS_POLICY_ARN=$(extract_json_field "$STEPFUNCTIONS_POLICY_RESULT" ".Policy.Arn") if [ -z "$STEPFUNCTIONS_POLICY_ARN" ]; then handle_error "Failed to extract Step Functions policy ARN" fi echo "Step Functions policy ARN: $STEPFUNCTIONS_POLICY_ARN" # Attach policy to the role echo "Attaching Step Functions policy to role..." ATTACH_RESULT=$(aws iam attach-role-policy \ --role-name "$ROLE_NAME" \ --policy-arn "$STEPFUNCTIONS_POLICY_ARN" 2>&1) if [ $? -ne 0 ]; then handle_error "Failed to attach Step Functions policy to role: $ATTACH_RESULT" fi # Wait for role to propagate (IAM changes can take time to propagate) wait_for_propagation "IAM role" 8 # Create state machine echo "Creating state machine: $STATE_MACHINE_NAME" SM_RESULT=$(aws stepfunctions create-state-machine \ --name "$STATE_MACHINE_NAME" \ --definition file://hello-world.json \ --role-arn "$ROLE_ARN" \ --type STANDARD 2>&1) check_api_error "$SM_RESULT" "Create state machine" echo "State machine created successfully" # Get the state machine ARN STATE_MACHINE_ARN=$(extract_json_field "$SM_RESULT" ".stateMachineArn") if [ -z "$STATE_MACHINE_ARN" ]; then handle_error "Failed to extract state machine ARN" fi echo "State machine ARN: $STATE_MACHINE_ARN" # Step 2: Start the state machine execution echo "Starting state machine execution..." EXEC_RESULT=$(aws stepfunctions start-execution \ --state-machine-arn "$STATE_MACHINE_ARN" \ --name "hello001-${RANDOM_ID}" 2>&1) check_api_error "$EXEC_RESULT" "Start execution" echo "Execution started successfully" # Get the execution ARN EXECUTION_ARN=$(extract_json_field "$EXEC_RESULT" ".executionArn") if [ -z "$EXECUTION_ARN" ]; then handle_error "Failed to extract execution ARN" fi echo "Execution ARN: $EXECUTION_ARN" # Wait for execution to complete (the workflow has a 10-second wait state) echo "Waiting for execution to complete (12 seconds)..." sleep 12 # Check execution status echo "Checking execution status..." EXEC_STATUS=$(aws stepfunctions describe-execution \ --execution-arn "$EXECUTION_ARN" 2>&1) echo "Execution status: $EXEC_STATUS" # Step 3: Update state machine to process external input echo "Updating state machine to process external input..." cat > updated-hello-world.json << 'EOF' { "Comment": "A Hello World example of the Amazon States Language using a Pass state", "StartAt": "SetVariables", "States": { "SetVariables": { "Type": "Pass", "Parameters": { "IsHelloWorldExample.$": "$.hello_world", "ExecutionWaitTimeInSeconds.$": "$.wait" }, "Next": "IsHelloWorldExample" }, "IsHelloWorldExample": { "Type": "Choice", "Choices": [ { "Variable": "$.IsHelloWorldExample", "BooleanEquals": true, "Next": "WaitState" } ], "Default": "FailState" }, "WaitState": { "Type": "Wait", "SecondsPath": "$.ExecutionWaitTimeInSeconds", "Next": "ParallelProcessing" }, "ParallelProcessing": { "Type": "Parallel", "Branches": [ { "StartAt": "Process1", "States": { "Process1": { "Type": "Pass", "Result": { "message": "Processing task 1" }, "End": true } } }, { "StartAt": "Process2", "States": { "Process2": { "Type": "Pass", "Result": { "message": "Processing task 2" }, "End": true } } } ], "Next": "CheckpointState" }, "CheckpointState": { "Type": "Pass", "Result": { "CheckpointMessage": "Workflow completed successfully!" }, "Next": "SuccessState" }, "SuccessState": { "Type": "Succeed" }, "FailState": { "Type": "Fail", "Error": "NotHelloWorldExample", "Cause": "The IsHelloWorldExample value was false" } } } EOF # Update state machine echo "Updating state machine..." UPDATE_RESULT=$(aws stepfunctions update-state-machine \ --state-machine-arn "$STATE_MACHINE_ARN" \ --definition file://updated-hello-world.json \ --role-arn "$ROLE_ARN" 2>&1) check_api_error "$UPDATE_RESULT" "Update state machine" echo "State machine updated successfully" # Create input file with strict validation echo "Creating input file..." cat > input.json << 'EOF' { "wait": 5, "hello_world": true } EOF # Validate input JSON validate_json_file "input.json" # Start execution with input echo "Starting execution with input..." EXEC2_RESULT=$(aws stepfunctions start-execution \ --state-machine-arn "$STATE_MACHINE_ARN" \ --name "hello002-${RANDOM_ID}" \ --input file://input.json 2>&1) check_api_error "$EXEC2_RESULT" "Start execution with input" echo "Execution with input started successfully" # Get the execution ARN EXECUTION2_ARN=$(extract_json_field "$EXEC2_RESULT" ".executionArn") if [ -z "$EXECUTION2_ARN" ]; then handle_error "Failed to extract execution ARN" fi echo "Execution ARN: $EXECUTION2_ARN" # Wait for execution to complete (the workflow has a 5-second wait state) echo "Waiting for execution to complete (8 seconds)..." sleep 8 # Check execution status echo "Checking execution status..." EXEC2_STATUS=$(aws stepfunctions describe-execution \ --execution-arn "$EXECUTION2_ARN" 2>&1) echo "Execution status: $EXEC2_STATUS" # Step 4: Integrate Amazon Comprehend for sentiment analysis (if available) if [[ "$SKIP_COMPREHEND" == "false" ]]; then echo "Creating policy for Amazon Comprehend access with least privilege..." cat > comprehend-policy.json << 'EOF' { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "comprehend:DetectSentiment" ], "Resource": "*" } ] } EOF # Create policy echo "Creating IAM policy: $POLICY_NAME" POLICY_RESULT=$(aws iam create-policy \ --policy-name "$POLICY_NAME" \ --policy-document file://comprehend-policy.json 2>&1) check_api_error "$POLICY_RESULT" "Create Comprehend policy" echo "Comprehend policy created successfully" # Get policy ARN POLICY_ARN=$(extract_json_field "$POLICY_RESULT" ".Policy.Arn") if [ -z "$POLICY_ARN" ]; then handle_error "Failed to extract policy ARN" fi echo "Policy ARN: $POLICY_ARN" # Attach policy to role echo "Attaching policy to role..." ATTACH2_RESULT=$(aws iam attach-role-policy \ --role-name "$ROLE_NAME" \ --policy-arn "$POLICY_ARN" 2>&1) if [ $? -ne 0 ]; then handle_error "Failed to attach policy to role: $ATTACH2_RESULT" fi # Create updated state machine definition with sentiment analysis echo "Creating updated state machine definition with sentiment analysis..." cat > sentiment-hello-world.json << 'EOF' { "Comment": "A Hello World example with sentiment analysis", "StartAt": "SetVariables", "States": { "SetVariables": { "Type": "Pass", "Parameters": { "IsHelloWorldExample.$": "$.hello_world", "ExecutionWaitTimeInSeconds.$": "$.wait", "FeedbackComment.$": "$.feedback_comment" }, "Next": "IsHelloWorldExample" }, "IsHelloWorldExample": { "Type": "Choice", "Choices": [ { "Variable": "$.IsHelloWorldExample", "BooleanEquals": true, "Next": "WaitState" } ], "Default": "DetectSentiment" }, "WaitState": { "Type": "Wait", "SecondsPath": "$.ExecutionWaitTimeInSeconds", "Next": "ParallelProcessing" }, "ParallelProcessing": { "Type": "Parallel", "Branches": [ { "StartAt": "Process1", "States": { "Process1": { "Type": "Pass", "Result": { "message": "Processing task 1" }, "End": true } } }, { "StartAt": "Process2", "States": { "Process2": { "Type": "Pass", "Result": { "message": "Processing task 2" }, "End": true } } } ], "Next": "CheckpointState" }, "CheckpointState": { "Type": "Pass", "Result": { "CheckpointMessage": "Workflow completed successfully!" }, "Next": "SuccessState" }, "DetectSentiment": { "Type": "Task", "Resource": "arn:aws:states:::aws-sdk:comprehend:detectSentiment", "Parameters": { "LanguageCode": "en", "Text.$": "$.FeedbackComment" }, "Next": "SuccessState" }, "SuccessState": { "Type": "Succeed" } } } EOF # Validate sentiment state machine JSON validate_json_file "sentiment-hello-world.json" # Wait for IAM changes to propagate wait_for_propagation "IAM changes" 8 # Update state machine echo "Updating state machine with sentiment analysis..." UPDATE2_RESULT=$(aws stepfunctions update-state-machine \ --state-machine-arn "$STATE_MACHINE_ARN" \ --definition file://sentiment-hello-world.json \ --role-arn "$ROLE_ARN" 2>&1) check_api_error "$UPDATE2_RESULT" "Update state machine with sentiment analysis" echo "State machine updated with sentiment analysis successfully" # Create input file with feedback comment echo "Creating input file with feedback comment..." cat > sentiment-input.json << 'EOF' { "hello_world": false, "wait": 5, "feedback_comment": "This getting started with Step Functions workshop is a challenge!" } EOF # Validate sentiment input JSON validate_json_file "sentiment-input.json" # Start execution with sentiment analysis input echo "Starting execution with sentiment analysis input..." EXEC3_RESULT=$(aws stepfunctions start-execution \ --state-machine-arn "$STATE_MACHINE_ARN" \ --name "hello003-${RANDOM_ID}" \ --input file://sentiment-input.json 2>&1) check_api_error "$EXEC3_RESULT" "Start execution with sentiment analysis" echo "Execution with sentiment analysis started successfully" # Get the execution ARN EXECUTION3_ARN=$(extract_json_field "$EXEC3_RESULT" ".executionArn") if [ -z "$EXECUTION3_ARN" ]; then handle_error "Failed to extract execution ARN" fi echo "Execution ARN: $EXECUTION3_ARN" # Wait for execution to complete echo "Waiting for execution to complete (3 seconds)..." sleep 3 # Check execution status echo "Checking execution status..." EXEC3_STATUS=$(aws stepfunctions describe-execution \ --execution-arn "$EXECUTION3_ARN" 2>&1) echo "Execution status: $EXEC3_STATUS" else echo "Skipping Amazon Comprehend integration (not available in $CURRENT_REGION)" EXECUTION3_ARN="" fi # Display summary of resources created echo "" echo "===========================================" echo "RESOURCES CREATED" echo "===========================================" echo "State Machine: $STATE_MACHINE_ARN" echo "IAM Role: $ROLE_NAME" echo "Step Functions Policy: StepFunctionsPolicy-${RANDOM_ID} ($STEPFUNCTIONS_POLICY_ARN)" if [[ "$SKIP_COMPREHEND" == "false" ]]; then echo "Comprehend Policy: $POLICY_NAME ($POLICY_ARN)" fi echo "Executions:" echo " - hello001-${RANDOM_ID}: $EXECUTION_ARN" echo " - hello002-${RANDOM_ID}: $EXECUTION2_ARN" if [[ "$SKIP_COMPREHEND" == "false" ]]; then echo " - hello003-${RANDOM_ID}: $EXECUTION3_ARN" fi echo "===========================================" # Cleanup echo "" echo "===========================================" echo "CLEANUP" echo "===========================================" echo "Auto-cleanup enabled. Cleaning up resources..." echo "All resources have been cleaned up." echo "Script completed successfully!"-
For API details, see the following topics in AWS CLI Command Reference.
-