Menu
AWS Step Functions
Developer Guide

Continue as a New Execution

AWS Step Functions is designed to run workflows that have a finite duration and number of steps. Executions are limited to a duration of one year, and a maximum of 25,000 events (see Limits). However, you can create a state machine that uses a Lambda function to start a new execution, before allowing the current execution to terminate. This enables you to have a state machine that can break large jobs into smaller workflows, or to have a state machine that runs indefinitely.

This tutorial builds on the concept of using an external Lambda function to modify your workflow, which was demonstrated in the Iterating a Loop Using Lambda tutorial. You'll use the same Lambda function (Iterator) to iterate a loop for a specific number of times. In addition, you'll create another Lambda function to start a new execution of your workflow, and to decrement a count each time it starts a new execution. By setting the number of executions in the input, this state machine will end and restart an execution a specified number of times.

This tutorial shows you how to create a state machine with a Lambda function that can start a new execution, continuing your ongoing work in that new execution.


      Workflow overview

The state machine you'll create implements the following states.

State Purpose
ConfigureCount

A Pass state that configures the count, index, and step values that are used by the Iterator Lambda function to step through iterations of work.

Iterator

A Task state that references the Iterator Lambda function.

IsCountReached A Choice state that uses a Boolean value from the Iterator function to decide if the state machine should continue the example work, or move to the ShouldRestart choice state.
ExampleWork In this example, ExampleWork is a Pass state that represents the Task state that would perform work in an actual implementation.
ShouldRestart A Choice state that uses the executionCount value to decide if it should end one execution and start another, or simply end.
Restart A Task state that uses a Lambda function to start a new execution of your state machine. Like the Iterator function, this function also decrements a count. It passes that value to the input of the new execution.

Prerequisites

Before you begin, go through the Creating a Lambda State Machine tutorial to ensure you have created an initial IAM role, and that you are familiar with using Lambda and Step Functions together.

Step 1: Create an Iterate Lambda Function to Iterate a Count

Note

If you have completed the Iterating a Loop Using Lambda tutorial, you can skip this step and use that Lambda function.

This section, and the Iterating a Loop Using Lambda tutorial, shows how you can use a Lambda function to track a count so that you can track the number of iterations of a loop in your state machine.

The following Lambda function receives input values for count, index, and step. It returns these values with an updated index and a Boolean named continue. The Lambda function sets continue to true if the index is less than count.

Your state machine then implements a Choice state that executes some application logic if continue is true, or moves on to ShouldRestart if continue is false.

To create the Iterate Lambda function

  1. Sign in to the Lambda console, and then choose Create function.

  2. In the Create function section, choose Author from scratch.

  3. In the Author from scratch section, configure your Lambda function, as follows:

    1. For Name, type Iterator.

    2. For Runtime, select Node.js 6.10.

    3. For Role, select Choose an existing role.

    4. For Existing role, select the Lambda role that you created in the Creating a Lambda State Machine tutorial.

      Note

      If the IAM role that you created doesn't appear in the list, the role might still need a few minutes to propagate to Lambda.

    5. Choose Create function.

      When your Lambda function is created, make a note of its Amazon Resource Name (ARN) in the upper-right corner of the page. For example:

      arn:aws:lambda:us-east-1:123456789012:function:Iterator
  4. Copy the following code for the Lambda function into the Configuration section of the Iterator page in the Lambda console.

    exports.iterator = function iterator (event, context, callback) { let index = event.iterator.index let step = event.iterator.step let count = event.iterator.count index += step callback(null, { index, step, count, continue: index < count }) }

    This code accepts input values for count, index, and step. It increments the index by the value of step and returns these values, and the Boolean continue. The value of continue is true if index is less than count.

  5. Choose Save.

Test the Iterate Lambda Function

To see your Iterate function working, run it with numeric values. You can provide input values for your Lambda function that mimic an iteration to see what output you get with specific input values.

To test your Lambda function

  1. In the Configure test event dialog box, choose Create new test event, and then type TestIterator for Event name.

  2. Replace the example data with the following.

    { "Comment": "Test my Iterator function", "iterator": { "count": 10, "index": 5, "step": 1 } }

    These values mimic what would come from your state machine during an iteration. The Lambda function increments the index and returns continue as true. Once the index is not less than the count, it returns continue as false. For this test, the index has already incremented to 5. The results should increment the index to 6 and set continue to true.

  3. Choose Create.

  4. On the Iterator page in your Lambda console, be sure TestIterator is listed, and then choose Test.

    The results of the test are displayed at the top of the page. Choose Details and review the result.

    { "index": 6, "step": 1, "count": 10, "continue": true }

    Note

    If you set index to 9 for this test, the index increments to 10, and continue is false.

Step 2: Create a Restart Lambda Function to Start a New Step Functions Execution

  1. Sign in to the Lambda console, and then choose Create function.

  2. In the Author from scratch section, configure your Lambda function, as follows:

    1. For Name, type Restart.

    2. For Runtime, select Node.js 6.10.

    3. For Role, select Choose an existing role.

    4. Under Existing role, select the role that includes the IAM policy you created previously.

    5. Choose Create function.

      When your Lambda function is created, make a note of its Amazon Resource Name (ARN) in the upper-right corner of the page. For example:

      arn:aws:lambda:us-east-1:123456789012:function:Restart
  3. Copy the following code for the Lambda function into the Configuration section of the Restart page in the Lambda console.

    The following code decrements a count of the number of executions, and starts a new execution of your state machine, including the decremented value.

    var aws = require('aws-sdk'); var sfn = new aws.StepFunctions(); exports.restart = function(event, context, callback) { let StateMachineArn = event.restart.StateMachineArn; event.restart.executionCount -= 1; event = JSON.stringify(event); let params = { input: event, stateMachineArn: StateMachineArn }; sfn.startExecution(params, function(err, data) { if (err) callback(err); else callback(null,event); }); }
  4. Choose Save.

Step 3: Create a State Machine

Now that you've created your two Lambda functions, create a state machine. In this state machine, the ShouldRestart and Restart states are how you break your work across multiple executions.

Example ShouldRestart Choice state

This excerpt of your state machine shows the ShouldRestart Choice state. This state decides if you should restart the execution.

"ShouldRestart": { "Type": "Choice", "Choices": [ { "Variable": "$.restart.executionCount", "NumericGreaterThan": 1, "Next": "Restart" } ],

The $.restart.executionCount value is included in the input of the initial execution. It's decremented by one each time the Restart function is called, and then placed into the input for each subsequent execution.

Example Restart Task state

This excerpt of your state machine shows the Restart Task state. This state uses the Lambda function you created earlier to restart the execution, and to decrement the count to track the remaining number of executions to start.

"Restart": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Restart", "Next": "Done" },
  1. In the Step Functions console, choose Create a state machine.

  2. Select Author from scratch, and enter ContinueAsNew as your state machine name.

  3. Under IAM role for your state machine executions, select the IAM role that you use for Lambda functions.

  4. Paste the following into the Code pane.

    Example ContinueAsNew state machine

    { "Comment": "Continue-as-new State Machine Example", "StartAt": "ConfigureCount", "States": { "ConfigureCount": { "Type": "Pass", "Result": { "count": 100, "index": -1, "step": 1 }, "ResultPath": "$.iterator", "Next": "Iterator" }, "Iterator": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Iterator", "ResultPath": "$.iterator", "Next": "IsCountReached" }, "IsCountReached": { "Type": "Choice", "Choices": [ { "Variable": "$.iterator.continue", "BooleanEquals": true, "Next": "ExampleWork" } ], "Default": "ShouldRestart" }, "ExampleWork": { "Comment": "Your application logic, to run a specific number of times", "Type": "Pass", "Result": { "success": true }, "ResultPath": "$.result", "Next": "Iterator" }, "ShouldRestart": { "Type": "Choice", "Choices": [ { "Variable": "$.restart.count", "NumericGreaterThan": 0, "Next": "Restart" } ], "Default": "Done" }, "Restart": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Restart", "Next": "Done" }, "Done": { "Type": "Pass", "End": true } } }
  5. Update the Resource string in the Restart and Iterator states to reference the respective Lambda functions you created earlier.

  6. Select Create State Machine.

Note

Save the Amazon Resource Name of this state machine.

Step 4: Update the IAM Policy

To ensure your Lambda function has permissions to start a new Step Functions execution, attach an inline policy to the IAM role you use for your Restart Lambda function. For more information, see Embedding Inline Policies in the IAM User Guide.

{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "states:StartExecution" ], "Resource": "*" } ] }

Note

You can update the "Resource": "*" line in the previous example to reference the ARN of your ContinueAsNew state machine. This restricts the policy so that it can only start an execution of that specific state machine.

Step 5: Run an Execution

To start an execution, provide input that includes the ARN of the state machine and an executionCount for how many times it should start a new execution.

  1. On the ContinueAsNew page, choose New execution.

  2. In the Input section, on the New execution page, enter Test1 for the execution name. Then enter the following in the Input.

    { "restart": { "StateMachineArn": "arn:aws:lambda:us-east-1:123456789012:stateMachine:ContinueAsNew", "executionCount": 4 } }
  3. Update the StateMachineArn field with the Amazon Resource Name for your ContinueAsNew state machine.

  4. Choose Start Execution.

The Visual Workflow graph will display the first of the four executions. Before it completes, it will pass through the Restart state and start a new execution.


        First execution of four.

With this execution complete, you can go look at the next execution that's running. Select the ContinueAsNew link at the top to see the list of executions. You should see both the recently closed execution, and an ongoing execution that the Restart Lambda function kicked off.


        One execution complete, the next one running.

Once all the executions are complete, you should see four successful executions in the list. The first execution started displays the name you chose, and subsequent executions have a generated name.


        All executions complete.