Iterate a loop with a Lambda function in Step Functions
In this tutorial, you implement a design pattern that uses a state machine and an AWS Lambda function to iterate a loop a specific number of times.
Use this design pattern any time you need to keep track of the number of loops in a state machine. This implementation can help you break up large tasks or long-running executions into smaller chunks, or to end an execution after a specific number of events. You can use a similar implementation to periodically end and restart a long-running execution to avoid exceeding service quotas for AWS Step Functions, AWS Lambda, or other AWS services.
Before you begin, go through the Creating a Step Functions state machine that uses Lambda tutorial to ensure you are familiar with using Lambda and Step Functions together.
Step 1: Create a Lambda function to iterate a count
By using a Lambda function you can track the number of iterations of a loop in your
state machine. The following Lambda function receives input values for
count
, index
, and step
. It returns these values
with an updated index
and a Boolean value named continue
. The
Lambda function sets continue
to true
if the index
is less than count
.
Your state machine then implements a Choice
state that executes some
application logic if continue
is true
, or exits if it is
false
.
To create the Lambda function
-
Sign in to the Lambda console
, and then choose Create function. -
On the Create function page, choose Author from scratch.
-
In the Basic information section, configure your Lambda function, as follows:
-
For Function name, enter
Iterator
. -
For Runtime, choose Node.js.
-
In Change default execution role, choose Create a new role with basic Lambda permissions.
-
Choose Create function.
-
-
Copy the following code for the Lambda function into the Code source.
export const handler = function (event, context, callback) { let index = event.iterator.index let step = event.iterator.step let count = event.iterator.count index = index + step callback(null, { index, step, count, continue: index < count }) }
This code accepts input values for
count
,index
, andstep
. It increments theindex
by the value ofstep
and returns these values, and the Booleancontinue
. The value ofcontinue
istrue
ifindex
is less thancount
. -
Choose Deploy.
Step 2: Test the Lambda Function
Run your Lambda function with numeric values to see it in operation. You can provide input values for your Lambda function that mimic an iteration.
To test your Lambda function
-
Choose Test.
-
In the Configure test event dialog box, enter
TestIterator
in the Event name box. -
Replace the example data with the following.
{ "Comment": "Test my Iterator function", "iterator": { "count": 10, "index": 5, "step": 1 } }
These values mimic what would come from your state machine during an iteration. The Lambda function will increment the index and return
true
forcontinue
when the index is less thancount
. For this test, the index has already incremented to5
. The test will incrementindex
to6
and setcontinue
totrue
. -
Choose Create.
-
Choose Test to test your Lambda function.
The results of the test are displayed in the Execution results tab.
-
Choose the Execution results tab to see the output.
{ "index": 6, "step": 1, "count": 10, "continue": true }
Note
If you set
index
to9
and test again, theindex
increments to10
, andcontinue
will befalse
.
Step 3: Create a State Machine
Before you leave the Lambda console…
Copy the Lambda function ARN. Paste it into a note. You'll need it in the next step.
Next, you will create a state machine with the following states:
-
ConfigureCount
– Sets default values forcount
,index
, andstep
. -
Iterator
– Refers to the Lambda function you created earlier, passing in the values configured inConfigureCount
. -
IsCountReached
– A choice state that continues the loop or proceeds toDone
state, based on the value returned from yourIterator
function. -
ExampleWork
– A stub for work that needs to be done. In this example, the workflow has aPass
state, but in a real solution, you would likely use aTask
. -
Done
– End state of your workflow.
To create the state machine in the console:
-
Open the Step Functions console
, and then choose Create a state machine. Important
Your state machine must be in the same AWS account and Region as your Lambda function.
-
Select the Blank template.
-
In the Code pane, paste the following JSON which defines the state machine.
For more information about the Amazon States Language, see State Machine Structure.
{ "Comment": "Iterator State Machine Example", "StartAt": "ConfigureCount", "States": { "ConfigureCount": { "Type": "Pass", "Result": { "count": 10, "index": 0, "step": 1 }, "ResultPath": "$.iterator", "Next": "Iterator" }, "Iterator": { "Type": "Task", "Resource": "
arn:aws:lambda:us-east-1:123456789012:function:Iterate
", "ResultPath": "$.iterator", "Next": "IsCountReached" }, "IsCountReached": { "Type": "Choice", "Choices": [ { "Variable": "$.iterator.continue", "BooleanEquals": true, "Next": "ExampleWork" } ], "Default": "Done" }, "ExampleWork": { "Comment": "Your application logic, to run a specific number of times", "Type": "Pass", "Result": { "success": true }, "ResultPath": "$.result", "Next": "Iterator" }, "Done": { "Type": "Pass", "End": true } } } -
Replace the
Iterator Resource
field with the ARN for yourIterator
Lambda function that you created earlier. -
Select Config, and enter a Name for your state machine, such as
.IterateCount
Note
Names of state machines, executions, and activity tasks must not exceed 80 characters in length. These names must be unique for your account and AWS Region, and must not contain any of the following:
-
Whitespace
-
Wildcard characters (
? *
) -
Bracket characters (
< > { } [ ]
) -
Special characters (
" # % \ ^ | ~ ` $ & , ; : /
) -
Control characters (
\\u0000
-\\u001f
or\\u007f
-\\u009f
).
Step Functions accepts names for state machines, executions, activities, and labels that contain non-ASCII characters. Because names with non-ASCII characters will not work with Amazon CloudWatch, we recommend using only ASCII charcters so you can track metrics in CloudWatch.
-
-
For Type, accept default value of Standard. For Permissions, choose Create new role.
-
Choose Create, and then Confirm the role creations.
Step 4: Start a New Execution
After you create your state machine, you can start an execution.
-
On the IterateCount page, choose Start execution.
-
(Optional) To identify your execution, you can specify a name, or use the default generated execution name .
Note
Step Functions accepts names for state machines, executions, activities, and labels that contain non-ASCII characters. Because names with non-ASCII characters will not work with Amazon CloudWatch, we recommend using only ASCII charcters so you can track metrics in CloudWatch.
-
Choose Start Execution.
A new execution of your state machine starts, showing your running execution.
The execution increments in steps, tracking the count using your Lambda function. On each iteration, it performs the example work referenced in the
ExampleWork
state in your state machine.When the count reaches the number specified in the
ConfigureCount
state in your state machine, the execution quits iterating and ends.