AWS Step Functions
Developer Guide

Iterating a Loop Using Lambda

In this tutorial, you implement a design pattern that uses a state machine and an AWS Lambda function to iterate a loop a specific number of times.

Use this design pattern any time you need to keep track of the number of loops in a state machine. This implementation can help you break up large tasks or long-running executions into smaller chunks, or to end an execution after a specific number of events. You can use a similar implementation to periodically end and restart a long-running execution to avoid exceeding service limits for AWS Step Functions, AWS Lambda, or other AWS services.

Before you begin, go through the Creating a Lambda State Machine tutorial to ensure you have created the necessary IAM role, and are familiar with using Lambda and Step Functions together.

Step 1: Create a Lambda Function to Iterate a Count

By using a Lambda function you can track the number of iterations of a loop in your state machine. The following Lambda function receives input values for count, index, and step. It returns these values with an updated index and a Boolean named continue. The Lambda function sets continue to true if the index is less than count.

Your state machine then implements a Choice state that executes some application logic if continue is true, or exits if it is false.

To create the Lambda function

  1. Sign in to the Lambda console, and then choose Create function.

  2. In the Create function section, choose Author from scratch.

  3. In the Author from scratch section, configure your Lambda function, as follows:

    1. For Name, type Iterator.

    2. For Runtime, select Node.js 6.10.

    3. For Role, select Choose an existing role.

    4. For Existing role, select the Lambda role that you created in the Creating a Lambda State Machine tutorial.

      Note

      If the IAM role that you created doesn't appear in the list, the role might still need a few minutes to propagate to Lambda.

    5. Choose Create function.

      When your Lambda function is created, make a note of its Amazon Resource Name (ARN) in the upper-right corner of the page. For example:

      arn:aws:lambda:us-east-1:123456789012:function:Iterator
  4. Copy the following code for the Lambda function into the Configuration section of the Iterator page in the Lambda console.

    exports.iterator = function iterator (event, context, callback) { let index = event.iterator.index let step = event.iterator.step let count = event.iterator.count index += step callback(null, { index, step, count, continue: index < count }) }

    This code accepts input values for count, index, and step. It increments the index by the value of step and returns these values, and the Boolean continue. The value of continue is true if index is less than count.

  5. Choose Save.

Step 2: Test the Lambda Function

Run your Lambda function with numeric values to see it in operation. You can provide input values for your Lambda function that mimic an iteration, to see what output you get with specific input values.

To test your Lambda function

  1. In the Configure test event dialog box, choose Create new test event, and then type TestIterator for Event name.

  2. Replace the example data with the following.

    { "Comment": "Test my Iterator function", "iterator": { "count": 10, "index": 5, "step": 1 } }

    These values mimic what would come from your state machine during an iteration. The Lambda function will increment the index and return continue as true. Once the index is not less than the count, it will return continue as false. For this test, the index has already incremented to 5. The results should increment the index to 6 and set continue to true.

  3. Choose Create.

  4. On the Iterator page in your Lambda console, be sure TestIterator is listed, and then choose Test.

    The results of the test are displayed at the top of the page. Choose Details and review the result.

    { "index": 6, "step": 1, "count": 10, "continue": true }

    Note

    If you set index to 9 for this test, the index will increment to 10, and continue will be false.

Step 3: Create a State Machine

To create the state machine

  1. Sign in to the Step Functions console, and then choose Create a state machine.

    Important

    Ensure that your state machine is under the same AWS account and region as the Lambda function you created earlier.

  2. On the Create a state machine page, choose Author from scratch. For Give a name to your state machine, enter IterateCount.

    Note

    State machine, execution, and activity names must be 1–80 characters in length, must be unique for your account and region, and must not contain any of the following:

    • Whitespace

    • Wildcard characters (? *)

    • Bracket characters (< > { } [ ])

    • Special characters (: ; , \ | ^ ~ $ # % & ` ")

    • Control characters (\\u0000 - \\u001f or \\u007f - \\u009f).

    Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.

  3. Create or enter an IAM role.

    • To create a new IAM role for Step Functions, choose Create a role for me, and then choose I acknowledge that Step Functions will create an IAM role which allows access to my Lambda functions.

    • If you have previously created an IAM role for Step Functions, choose I will provide an IAM role ARN and enter your existing IAM role ARN.

    Note

    If you delete the IAM role that Step Functions creates, Step Functions can't recreate it later. Similarly, if you modify the role (for example, by removing Step Functions from the principals in the IAM policy), Step Functions can't restore its original settings later.

  4. The following code describes a state machine with the following states:

    • ConfigureCount: Sets the default values for count, index, and step.

      "ConfigureCount": { "Type": "Pass", "Result": { "count": 10, "index": 0, "step": 1 }, "ResultPath": "$.iterator", "Next": "Iterator" },
    • Iterator: References your Lambda function you created earlier, passing in the values configured in ConfigureCount.

      "Iterator": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Iterate", "ResultPath": "$.iterator", "Next": "IsCountReached" },
    • IsCountReached: A choice state that will either run your sample work again or will go to Done based on a boolean returned from your Iterator Lambda function.

      "IsCountReached": { "Type": "Choice", "Choices": [ { "Variable": "$.iterator.continue", "BooleanEquals": true, "Next": "ExampleWork" } ], "Default": "Done" },
    • ExampleWork: A stub for the work you want to accomplish in your execution. In this example it is a pass state. In an actual implementation this would be a task state. See Tasks.

    • Done: The end state of your execution.

    In the Code pane, add the following state machine definition using the Amazon Resource Name of the Lambda function that you created earlier.

    { "Comment": "Iterator State Machine Example", "StartAt": "ConfigureCount", "States": { "ConfigureCount": { "Type": "Pass", "Result": { "count": 10, "index": 0, "step": 1 }, "ResultPath": "$.iterator", "Next": "Iterator" }, "Iterator": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Iterate", "ResultPath": "$.iterator", "Next": "IsCountReached" }, "IsCountReached": { "Type": "Choice", "Choices": [ { "Variable": "$.iterator.continue", "BooleanEquals": true, "Next": "ExampleWork" } ], "Default": "Done" }, "ExampleWork": { "Comment": "Your application logic, to run a specific number of times", "Type": "Pass", "Result": { "success": true }, "ResultPath": "$.result", "Next": "Iterator" }, "Done": { "Type": "Pass", "End": true } } }

    Be sure to update the Amazon Resource Name in the Iterator state above so that it references the Lambda you created earlier. For more information about the Amazon States Language, see State Machine Structure.

  5. Use the graph in the Visual Workflow pane to check that your Amazon States Language code describes your state machine correctly.

    This graph shows the logic expressed in the above state machine code.

    
                            State machine workflow

    If you don't see the graph, choose 
       refresh
    in the Visual Workflow pane.

  6. Choose Create State Machine.

  7. Choose OK.

    The state machine is created and an acknowledgement page is displayed.

Step 4: Start a New Execution

After you create your state machine, you can start an execution.

To start a new execution

  1. On the IterateCount page, choose New execution.

  2. (Optional) To help identify your execution, you can specify an ID for it in the Enter an execution name box. If you don't enter an ID, Step Functions generates a unique ID automatically.

    Note

    Step Functions allows you to create state machine, execution, and activity names that contain non-ASCII characters. These non-ASCII names don't work with Amazon CloudWatch. To ensure that you can track CloudWatch metrics, choose a name that uses only ASCII characters.

  3. Choose Start Execution.

    A new execution of your state machine starts, showing your running execution.

    
                            State machine execution

    The execution increments in steps, tracking the count using your Lambda function. On each iteration, it performs the example work referenced in the ExampleWork state in your state machine.

  4. (Optional) In the Execution Details section, choose the Info tab to view the Execution Status and the Started and Closed time stamps.

  5. Once the count reaches the number configured in the ConfigureCount state in your state machine, the execution quits iterating and ends.

    
                            State machine execution complete