Amazon Mechanical Turk
Command Line Reference (API Version 2014-08-15)

Files Used by the Command Line Tools

This section describes the files that the Command Line Tools use.

File Description Used by

The Amazon Mechanical Turk properties file

This file is named mturk.properties and it must exist in the directory that you run the Command Line Tools from.

This is a text file that contains specific information about your AWS identifiers as well as the Amazon Mechanical Turk service endpoint.

System

The approve file

A text file that contains a list of assignment IDs and optional approval comments.

The first row must contain a column header named assignmentIDToApprove.

The file can have an optional column named assignmentIdToApproveComment.

Separate fields with a tab and enclose comments in quotes.

The -approvefile argument of approveWork

The failure file

A tab delimited text file that contains the rows from the input file that failed to load into Amazon Mechanical Turk. The system generates this file if failures occur when you use loadHITs. You can fix the entries, rename the file, and then use it to load the HITs that failed.

The system creates this file with the name [your input filename].failure.

Not used in command arguments.

The HIT properties file

A text file that defines the properties of the HITs you are creating. Use the format [property]:[value] to list the properties in the file.

Example: reward:$0.06

For a list of properties, see HIT Properties.

The -properties argument of the following commands:

loadHITs

updateHITs

The input file

A tab delimited text file that contains the dynamic fields of the HIT. This file can contain any information that you need for your HIT. Amazon Mechanical Turk merges this file into your Question template file and your HIT properties file.

The first row of the file contains the field headings. Subsequent rows represent the custom field values for the HITs to be loaded. Each row represents one HIT (e.g. 1000 data rows = 1000 HITs).

The -input argument of the loadHITs command.

The output file

A tab delimited text file that contains the results of submitted HITs that are retrieved from Amazon Mechanical Turk. This file contains all data related to the submitted HITs and assignments. Each HIT in this file has a link to your Manage HITs page on the Requester web site. The last column contains a tab delimited set of question/answer pairs for each question field defined in your QuestionForm.

The -outputfile argument of getResults

The Qualification answer file

A text file that contains the AnswerKey XML that defines the answer key for your Qualification test. The Amazon Mechanical Turk system automatically scores the test.

The -answer argument of the following commands:

createQualificationType

updateQualificationType

The Qualification answer key file

A text file that contains name/value pairs of your questions and expected answers. This answer key provides another method to evaluate Qualification requests by giving you control over when requests are granted. This format also provides more flexibility by allowing you to evaluate FreeTextAnswer answers which cannot be autograded by Amazon Mechanical Turk.

The format is questionid=expected answer value.

The -answers argument of evaluateQualificationRequests

The Qualification approve file

A tab delimited text file that contains the list of Qualification requests to approve.

The file must contain a column called qualificationRequestToApprove that contains the Qualification request IDs.

The file can have an optional column called qualificationRequestToApproveValue that lists the Qualification score to assign for each Qualification request.

The -approvefile argument of approveQualificationRequests

The Qualification ID file

A text file that contains the single Qualification ID you want to assign to workers. The file must have a column called qualtypeid.

The -input argument of the following commands:

assignQualification

evaluateQualificationRequests

The Qualification properties file

A text file that defines the properties of the Qualification you are creating. Use the format [property]=[value] to list the properties in the file.

Example: autogranted=false

The -properties argument of the following commands:

createQualificationType

updateQualificationType

The Qualification question file

A text file that contains the QuestionForm XML that defines your Qualification test. This file is similar to the question file except that templating is not applicable.

The -question argument of the following commands:

createQualificationType

updateQualificationType

The Qualification reject file

A tab delimited text file that contains the list of Qualification requests to reject and optional comments.

The file must contain a column called qualificationRequestToReject that contains the Qualification request IDs.

The file can include an optional column called qualificationRequestToRejectComment that contains comments you want to provide to the Worker regarding the rejection.

The -rejectfile argument of rejectQualificationRequests

The Qualification request file

A tab delimited text file that contains the details of the Qualification requests made by Workers.

The -outputfile argument of getQualificationRequests

The question file

A text file that contains the QuestionForm XML that defines your HIT.

Tip

To make your QuestionForm a template and merge it with values from your input file use the syntax ${the field name}, where the field name is the column name from the input file. Amazon Mechanical Turk uses Apache Velocity to perform the merge. For more information about merge syntax, go to http://velocity.apache.org.

The -question argument of loadHITs

The reject file

A text file that contains a list of assignment ID and optional rejection comments.

The first row must contain a column called assignmentIdToReject.

The file can have an optional column called assignmentIdToRejectComment.

Separate fields with a tab and enclose comments in quotes.

The -rejectfile argument of the rejectWork command

The score file

A tab delimited file that contains the Worker IDs and the Qualification scores to assign.

The file must contain a column called workerid that contains the Worker IDs.

The file can optionally contain a column called score that specifies the scores to assign to Workers.

The -scorefile argument of the assignQualification command.

The success file

A tab delimited text file that contains the HIT IDs and the HIT Type IDs that were successfully loaded into Amazon Mechanical Turk. This file is generated when you load HITs using loadHITs or updateHITs.

The file must contain a column called hitid.

The system creates this file with the name [your input filename].success.

The -successfile argument of the following commands:

approveWork

deleteHITs

extendHITs

getResults

updateHITs

The Worker ID file

A text file that contains a tab delimited list of Worker IDs to assign the Qualification to, and optional Qualification scores.

The file must contain a column called workerid.

The file can optionally contain a column called score that lists the Qualification scores to assign to each Worker.

The -input argument of the updateQualificationScore command

HIT Properties

The following table describes the HIT properties.

Property Description
title Title of the HIT
description Description of the HIT
keywords Keywords to associate with the HIT
reward Reward for the HIT formatted as $0.00 USD
assignments Maximum number of assignments available for this HIT
annotation

Value you use to identify this HIT

Tip

You can merge values from the input file into this property. To do this, use the syntax: ${the field name}, where the field name is the column name from the input file. Amazon Mechanical Turk uses Apache Velocity to perform the merge. For more information about merge syntax, go to http://velocity.apache.org.

assignmentduration Amount of time, in seconds, a Worker has to complete the assignment
hitlifetime Amount of time, in seconds, the HIT is active before it expires
autoapprovaldelay Amount of time, in seconds, before a submitted assignment is automatically approved
qualification

Qualification requirements for your HIT. To specify the requirements, use the following syntax.

  • qualification.1—Qualification Type ID

  • qualification.comparator.1—String that specifies the kind of comparison to make

  • qualification.value.1—Integer value for the comparator to use

  • qualification.locale.1—Locale value to use

  • qualification.private.1—Boolean value that specifies whether Workers who do not meet the qualification requirements can preview the HIT

To specify additional qualification requirements, increment the .1 suffix (e.g. qualification.value.2).

For more information about the qualification requirements, see QualificationRequirement Data Structure. For more information about the properties see HIT Data Structure.

Qualification Properties

The following table describes the qualification properties.

Property Description
name Name of the Qualification
description Description of the Qualification
keywords Keywords to associate with the Qualification
retrydelayinseconds Minimum amount of time, in seconds, required before a Worker can re-request the Qualification
testdurationinseconds Amount of time, in seconds, allowed for the Worker to complete the Qualification test (if a test exists)
autogranted Specifies whether the Qualification should be autogranted upon request. This parameter is only valid if a Qualification does not have a test associated with it. Valid Values: true | false