HIT Review Policy

Description

HIT Review Policy data structures represent HIT review policies, which you specify when you create a Human Intelligence Task (HIT). For more information about Review Policies, see Review Policies.

The following two types of HIT Review Policy data structures are used when calling the CreateHIT operation.

• AssignmentReviewPolicy data structures set the review results and actions at the Assignment level. For more information, see Assignment Review Policies.

• HITReviewPolicy data structures set the review results and actions at the HIT-level. For more information, see HIT Review Policies.

The HIT Review Policy data structure is used in the following operation:

• CreateHIT

HIT Review Policy Elements

The HIT Review Policy data structures can contain the following elements.

Name	Description	Required
PolicyName	Name of a Review Policy. For policy names and descriptions, see Assignment Review Policies and HIT Review Policies. Type: String Constraints: SimplePlurality/2011-09-01 or ScoreMyKnownAnswers/2011-09-01	Yes
Parameter	Name of the parameter from the Review policy. Type: Parameter data structure, for a description see the next section Parameter Elements.

Parameter Elements

The Parameter data structure contains the elements described in the following table.

Name	Description	Required
Key	Name of the parameter from the list of Review Polices. For a list of valid parameter names, see Assignment Review Policies and HIT Review Policies. Type: String Constraints: none Default: none
Value	Value of the parameter. Type: Varies.
MapEntry	This data structure is the data type for the AnswerKey parameter of the ScoreMyKnownAnswers/2011-09-01 Review Policy, see Assignment Review Policies. Type: MapEntry data structure, for a description, see the next section MapEntry Elements.

MapEntry Elements

The MapEntry element contains the elements described in the following table.

Name	Description	Required
Key	The QuestionID from the HIT that is used to identify which question requires Mechanical Turk to score as part of the ScoreMyKnownAnswers/2011-09-01 Review Policy. Type: String none
Value	The answer to the question specified in the MapEntry Key element. Note: You can provide multiple Value elements, but the Worker must match all values in order for the answer to be scored correctly. Type: String

Examples

The following example request shows the structure of a CreateHIT request using HIT Review Policy data structures. The example request is followed by an example response.

Sample Request

The following is an example request.

<CreateHITRequest>
    <HITTypeId>T100CN9P324W00EXAMPLE</HITTypeId>
    <Question>URL-encoded question data</Question>
    <LifetimeInSeconds>604800</LifetimeInSeconds>
    <AssignmentReviewPolicy>
        <PolicyName>ScoreMyKnownAnswers/2011-09-01</PolicyName>
        <Parameter>
            <Key>AnswerKey</Key>
            <MapEntry>
                <Key>QuestionId1</Key>   <!—correct answer is “B” -->
                <Value>B</Value>
            </MapEntry
            <MapEntry>
                <Key>QuestionId2</Key>  <!—correct answer is “A” -->
                <Value>A</Value>
            </MapEntry>
            <MapEntry>
                <Key>QuestionId1</Key>   <!—correct answer is “F” -->
                <Value>F</Value>
            </MapEntry>
            <MapEntry>
                <Key>QuestionId1</Key>   <!—correct answer is “C” -->
                <Value>C</Value>
            </MapEntry>
            <MapEntry>
                <Key>QuestionId1</Key>   <!—correct answer is “A” -->
                <Value>A</Value>
            </MapEntry
        <Parameter>
           <Key>ApproveIfKnownAnswerScoreIsAtLeast</Key>
           <Value>79</Value>
        </Parameter>
        <Parameter>
           <Key>ExtendIfKnownAnswerScoreIsLessThan</Key>
           <Value>79</Value>
        </Parameter>
        <Parameter>
           <Key>ExtendMaximumAssignments</Key>
           <Value>3</Value>
        </Parameter>
    </AssignmentReviewPolicy>
    <HITReviewPolicy>
        <PolicyName>SimplePlurality/2011-09-01</PolicyName>
        <Parameter>
           <Key>QuestionIDs</Key>
           <Value>questionid1</Value>
           <Value>questionid2</Value>
           <Value>questionid3</Value>
           <Value>questionid4</Value>
           <Value>questionid5</Value>
                 ..... <! Add your additional 10 questionIDs for a total of 15 questions>
        </Parameter>
        <Parameter>
           <Key>QuestionAgreementThreshold</Key>
           <Value>100</Value>
        </Parameter>
    </HITReviewPolicy>
</CreateHITRequest>           

Sample Response

The following is an example response.

<CreateHITResponse>
  <OperationRequest>
    <RequestId>ece2785b-6292-4b12-a60e-4c34847a7916</RequestId>
  </OperationRequest>
  <HIT>
    <Request>
      <IsValid>True</IsValid>
    </Request>
    <HITId>GBHZVQX3EHXZ2AYDY2T0</HITId>
    <HITTypeId>NYVZTQ1QVKJZXCYZCZVZ</HITTypeId>
  </HIT>
</CreateHITResponse>