Using data flow simulator to test data flow in Step Functions

You can design, implement, and debug workflows in the Step Functions console. You can also control the flow of data in your workflows with JsonPath input and output data processing. With the Data flow simulator, you can simulate the order that the Task workflow state states in your workflow process data at runtime. Using the simulator, you can understand how to filter and manipulate data as it flows from one state to another. It simulates each of the following fields that Step Functions uses to process and control the flow of JSON data:

InputPath

Selects WHAT portion of the entire input payload to be used as a task’s input. If you specify this field, Step Functions first applies this field.

Parameters

Specifies HOW the input should look like before invoking the task. With the Parameters field, you can create a collection of key-value pairs that are passed as input to an AWS service integration, such as an AWS Lambda function. These values can be static, or dynamically selected from either the state input or the workflow context object.

ResultSelector

Determines WHAT to choose from a task's output. With the ResultSelector field, you can create a collection of key-value pairs that replace a state’s result and pass that collection to ResultPath.

Specifying state output using ResultPath in Step Functions workflows

Determines WHERE to put a task's output. Use the ResultPath to determine whether the output of a state is a copy of its input, the result it produces, or a combination of both.

Filtering state output using OutputPath in Step Functions workflows

Determines WHAT to send to the next state. With OutputPath, you can filter out unwanted information, and pass only the portion of JSON data that you care about.

Using Data flow simulator

The simulator provides a real-time, side-by-side comparison of your data before and after you apply an input and output data processing field. To use the simulator, specify a JSON input. Then, evaluate it through each of the input and output processing fields. The simulator automatically validates your JSON input and highlights any syntax errors.

To use the data flow simulator

In the following steps, you provide JSON input and apply the InputPath and Parameters fields. You can also apply the other available fields and view their outputs.

1. Open the Step Functions console.

2. In the navigation pane, choose Data flow simulator.

3. In the State input area, replace the prepopulated example JSON data with the following JSON data. Then, choose Next.

   {
     "data": {
       "firstname": "Jane",
       "lastname": "Doe",
       "identity": {
         "email": "[e-mail]",
         "ssn": "123-45-6789"
       },
       "address": {
         "street": "123 Main St",
         "city": "Columbus",
         "state": "OH",
         "zip": "43219"
       }
     }
   }

4. For InputPath, enter $.data.address to select the address node of the input JSON data.

   The State input after InputPath box displays the following results.

   {
     "street": "123 Main St",
     "city": "Columbus",
     "state": "OH",
     "zip": "43219"
   }

5. Choose Next.

6. Apply the Parameters field to convert the resulting JSON data to a string. To convert the data, do the following:

   1. In the Parameters box, enter the following code to create a string called addressString.

      {
          "addressString.$": "States.Format('{}. {}, {} - {}', $.street, $.city, $.state, $.zip)"
        }

7. View the result of the Parameters field application in the Filtered input after Parameters box.

Considerations about using the Data flow simulator

Before you use the Data flow simulator, consider its limitations, including, but not limited to:

• Unsupported filter expressions

  Filter expressions in the simulator behave differently than in the Step Functions service. This includes expressions that use the following syntax: [?(expression)]. The following is a list of unsupported expressions. If used, these expressions may not return the expected outcome after their evaluation.

  • $..book[?(@.isInStock==true)]

  • $..book[?(@.price > 10.0)].title

• Incorrect JsonPath evaluation for single item arrays

  If you filter your data with a JsonPath expression that'd return a single array item, the simulator returns the item without the array. For example, consider the following array of data, called items:

  {
    "items": [
      {
        "name": "shoe",
        "color": "blue",
        "comment": "nice shoe"
      },
      {
        "name": "hat",
        "color": "red"
      },
      {
        "name": "shirt",
        "color": "yellow"
      }
    ]
  }

  Given this items array, if you enter $..comment in the InputPath field, you'd expect the following output:

  [
    "nice shoe"
  ]

  However, the Data flow simulator returns the following output instead:

  "nice shoe"

  For JsonPath evaluation of an array that contains multiple items, the simulator returns the expected output.