Create a Human Review Workflow
Use an Amazon Augmented AI (Amazon A2I) human review workflow, or flow definition, to specify the following:
-
For the Amazon Textract and Amazon Rekognition built-in task types, the conditions under which your human loop is called
-
The workforce to which your tasks are sent
-
The set of instructions that your workforce receives, which is called a worker task template
-
The configuration of your worker tasks, including the number of workers that receive a task and time limits to complete tasks
-
Where your output data is stored
You can create a human review workflow in the SageMaker AI console or using the SageMaker AI CreateFlowDefinition
operation. You can build a worker task template
using the console for Amazon Textract and Amazon Rekognition task types while creating your flow
definition.
Important
Human loop activation conditions, which initiate the human loop—for example,
confidence thresholds—aren't available for Amazon A2I custom task types. When
using the console to create a flow definition for a custom task type, you can't specify
activation conditions. When using the Amazon A2I API to create a flow definition for a
custom task type, you can't set the HumanLoopActivationConditions
attribute
of the HumanLoopActivationConditionsConfig
parameter. To control when human
reviews are initiated, specify conditions under which StartHumanLoop
is
called in your custom application. In this case, every StartHumanLoop
invocation results in a human review. For more information, see Use Amazon Augmented AI with Custom Task Types.
Prerequisites
To create a human review workflow definition, you must have completed the prerequisites described in Prerequisites to Using Augmented AI.
If you use the API to create a flow definition for any task type, or if you use a custom task type when creating a flow definition in the console, first create a worker task template. For more information, see Create and Manage Worker Task Templates.
If you want to preview your worker task template while creating a flow definition for a built-in task type in the console, ensure that you grant the role that you use to create the flow definition permission to access the Amazon S3 bucket that contains your template artifacts using a policy like the one described in Enable Worker Task Template Previews .
Topics
Create a Human Review Workflow (Console)
Use this procedure to create a Amazon Augmented AI (Amazon A2I) human review workflow using the SageMaker AI console. If you are new to Amazon A2I, we recommend that you create a private work team using people in your organization, and use this work team's ARN when creating your flow definition. To learn how to set up a private workforce and create a work team, see Create a Private Workforce (Amazon SageMaker AI Console). If you have already set up a private workforce, see Create a Work Team Using the SageMaker AI Console to learn how to add a work team to that workforce.
If you are using Amazon A2I with one of the built-in task types, you can create worker instructions using a default worker task template provided by Augmented AI while creating a human review workflow in the console. To see samples of the default templates provided by Augmented AI, see the built-in task types in Use Cases and Examples Using Amazon A2I.
To create flow definition (console)
-
Open the SageMaker AI console at https://console.aws.amazon.com/sagemaker/
. -
In the navigation pane, under the Augmented AI section, choose Human review workflows and then choose Create human review workflow.
-
In Overview, do the following:
-
For Name, enter a unique workflow name. The name must be lowercase, unique within the AWS Region in your account, and can have up to 63 characters. Valid characters include: a-z, 0-9, and - (hyphen).
-
For S3 location for output, enter the S3 bucket where you want to store the human review results. The bucket must be located in the same AWS Region as the workflow.
-
For IAM role, choose the role that has the required permissions. If you choose a built-in task type and want to preview your worker template in the console, provide a role with the type of policy described in Enable Worker Task Template Previews attached.
-
-
For Task type, choose the task type that you want the human worker to perform.
-
If you chose the Amazon Rekognition or Amazon Textract task type, specify the conditions that invoke human review.
-
For Amazon Rekognition image moderation tasks, choose an inference confidence score threshold interval that initiates human review.
-
For Amazon Textract tasks, you can initiate a human review when specific form keys are missing or when form key detection confidence is low. You can also initiate a human review if, after evaluating all of the form keys in the text, confidence is lower than your required threshold for any form key. Two variables specify your confidence thresholds: Identification confidence and Qualification confidence. To learn more about these variables, see Use Amazon Augmented AI with Amazon Textract.
-
For both task types, you can randomly send a percentage of data objects (images or forms) and their labels to humans for review.
-
-
Configure and specify your worker task template:
If you are using the Amazon Rekognition or Amazon Textract task type:
-
In the Create template section:
-
To create instructions for your workers using the Amazon A2I default template for Amazon Rekognition and Amazon Textract task types, choose Build from a default template.
-
If you choose Build from a default template, create your instructions under Worker task design:
-
Provide a Template name that is unique in the AWS Region you are in.
-
In the Instructions section, provide detailed instructions on how to complete your task. To help workers achieve greater accuracy, provide good and bad examples.
-
(Optional) In Additional instructions, provide your workers with additional information and instructions.
For information on creating effective instructions, see Creating Good Worker Instructions.
-
-
-
To select a custom template that you've created, choose it from the Template menu and provide a Task description to briefly describe the task for your workers. To learn how to create a custom template, see Create a Worker Task Template.
-
-
-
If you are using the custom task type:
-
In the Worker task template section, select your template from the list. All of the templates that you have created in the SageMaker AI console appear in this list. To learn how to create a template for custom task types, see Create and Manage Worker Task Templates.
-
-
(Optional) Preview your worker template:
For Amazon Rekognition and Amazon Textract task types, you have the option to choose See a sample worker task to preview your worker task UI.
If you are creating a flow definition for a custom task type, you can preview your worker task UI using the
RenderUiTemplate
operation. For more information, see Preview a Worker Task Template. -
For Workers, choose a workforce type.
-
Choose Create.
Next Steps
After you've created a human review workflow, it appears in the console under Human review workflows. To see your flow definition's Amazon Resource Name (ARN) and configuration details, choose the workflow by selecting its name.
If you are using a built-in task type, you can use the flow definition ARN to start a human loop using that AWS service's API (for example, the Amazon Textract API). For custom task types, you can use the ARN to start a human loop using the Amazon Augmented AI Runtime API. To learn more about both options, see Create and Start a Human Loop.
Create a Human Review Workflow (API)
To create a flow definition using the SageMaker API, you use the
CreateFlowDefinition
operation. After you complete the Prerequisites to Using Augmented AI, use the following procedure to
learn how to use this API operation.
For an overview of the CreateFlowDefinition
operation, and details about each
parameter, see CreateFlowDefinition
.
To create a flow definition (API)
-
For
FlowDefinitionName
, enter a unique name. The name must be unique within the AWS Region in your account, and can have up to 63 characters. Valid characters include: a-z, 0-9, and - (hyphen). -
For
RoleArn
, enter the ARN of the role that you configured to grant access to your data sources. -
For
HumanLoopConfig
, enter information about the workers and what they should see. For information about each parameter inHumanLoopConfig
, see HumanLoopConfig. -
(Optional) If you are using a built-in task type, provide conditions that initiate a human loop in
HumanLoopActivationConfig
. To learn how to create the input required for theHumanLoopActivationConfig
parameter, see JSON Schema for Human Loop Activation Conditions in Amazon Augmented AI. If you do not specify conditions here, when you provide a flow definition to the AWS service associated with a built-in task type (for example, Amazon Textract or Amazon Rekognition), that service sends every task to a human worker for review.If you are using a custom task type,
HumanLoopActivationConfig
is disabled. To learn how to control when tasks are sent to human workers using a custom task type, see Use Amazon Augmented AI with Custom Task Types. -
(Optional) If you are using a built-in task type, specify the integration source (for example, Amazon Rekognition or Amazon Textract) in the HumanLoopRequestSource parameter.
-
For
OutputConfig
, indicate where in Amazon Simple Storage Service (Amazon S3) to store the output of the human loop. -
(Optional) Use
Tags
to enter key-value pairs to help you categorize and organize a flow definition. Each tag consists of a key and a value, both of which you define.
Next Steps
The return value of a successful call of the CreateFlowDefinition
API
operation is a flow definition Amazon Resource Name (ARN).
If you are using a built-in task type, you can use the flow definition ARN to start a human loop using that AWS service's API (i.e. the Amazon Textract API). For custom task types, you can use the ARN to start a human loop using the Amazon Augmented AI Runtime API. To learn more about both of these options, see Create and Start a Human Loop.