Amazon Mechanical Turk
Getting Started Guide (API Version 2014-08-15)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Creating a HIT

In the previous section you set up your AWS account and installed the programming tools you need to use Amazon Mechanical Turk. The following topics describe how to write, test, publish, and manage a Human Intelligence Task (HIT).

Workflow

The following procedure gives you an overview of creating, testing, publishing, and managing a HIT.

Workflow for Requesters

1

Write your HIT.

Write the question or task for Workers to answer or perform.

2

Test your HIT.

Publish your HIT on the Amazon Mechanical Turk Developer Sandbox. The Amazon Mechanical Turk Developer Sandbox is a simulated environment that allows you to view your HIT as it would appear to Workers.

For more information about the Amazon Mechanical Turk Developer Sandbox and how to use it, go to the Amazon Mechanical Turk Developer Sandbox.

3

Publish your HIT on the Amazon Mechanical Turk production system.

This step makes your HIT available to Workers.

4

Workers accept your HIT and complete the assignment.

You can view the status of your HITs. For more information, go to the Requester website.

5

Process the assignment results.

When a Worker completes an assignment, you can view the results, output the results to a file, and accept or reject the work. Accepting the work means that you agree to pay the Worker.

6

Manage your HIT.

You can extend the completion time for your HIT, expire the HIT early, add additional assignments, modify the HIT properties, or block Workers whose work does not meet your standards.


The examples in this section provide instructions for performing tasks 1, 2, and 3. For information about tasks 5 and 6, see Implementing Amazon Mechanical Turk.

Working with the Examples

The following examples use SDK operations or the command line tools to show you how to create a survey that asks the question "How many movies have you seen this month?" The samples create a HIT and print the HIT ID to the console. These examples demonstrate how to publish the HIT to the Amazon Mechanical Turk Developer Sandbox for testing and then how to publish the HIT on the Amazon Mechanical Turk production system.

The following sections show you how to create and publish a HIT using the command line interface or the API and a programming language:

Showing Your Preferred Programming Language

If you are viewing the HTML version of this guide, you can hide the text in this section that does not pertain to the programming language you are using. There is a Filter View: drop-down menu in the upper-right corner of pages with language-specific text. Select a computer language to show the examples in only that computer language, or select All to show the examples in all available computer languages.

You can also select Command Line Tools, which hides all code samples and leaves only information related to the command line tools.

Creating a HIT Using the Command Line Interface

This example uses the commands in the Amazon Mechanical Turk command line tools to create a survey. You must have the command line tools installed and configured correctly to run this example.

How to Create a HIT

The following procedure describes how to create the HIT, "MovieSurvey."

To create the HIT

  1. Create an input file, moviesurvey.input, a tab-delimited text file, and populate it with the variable data for the HIT.

    question
    How many movies have you seen this month?
    		    
  2. Create a question template file, moviesurvey.question, and add the text of your HIT template. This XML file contains placeholders for the fields defined in the input file and conforms to the QuestionForm schema.

    <?xml version="1.0" encoding="UTF-8"?>
    <QuestionForm xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">
      <Question>
       <QuestionIdentifier>1</QuestionIdentifier>
       <QuestionContent>
         <Text>${question}</Text>
       </QuestionContent>
       <AnswerSpecification>
         <FreeTextAnswer/>
       </AnswerSpecification>
      </Question>
    </QuestionForm>
    			
  3. Create a HIT properties file, moviesurvey.properties, and populate it with the properties of the HIT (title, description, etc.)

    ######################################
    ##  HIT Properties
    ######################################
    
    title:Movie Survey
    description:This is a survey to find out how many movies you have watched recently.
    keywords:movie, survey
    reward:0.05
    assignments:100
    annotation:sample#command
    
    ######################################
    ## HIT Timing Properties
    ######################################
    
    # this Assignment Duration value is 60 * 60 = 1 hour
    assignmentduration:3600
    
    # this HIT Lifetime value is 60*60*24*3 = 3 days
    hitlifetime:259200
    
    # this Auto Approval period is 60*60*24*15 = 15 days
    autoapprovaldelay:1296000
    		    
  4. Navigate to the [Command Line Tools Installation Directory]\bin directory to run the commands.

How to Test Your HIT

To test your HIT, publish it in the sandbox by including the -sandbox argument with the loadHITs command.

To test your HIT

  1. Open a command prompt, navigate to the [Command Line Tools Installation Directory]\bin directory, and run the following command:

    loadHITs -input c:\moviesurvey\moviesurvey.input -question  c:\moviesurvey\moviesurvey.question -properties c:\moviesurvey\moviesurvey.properties -sandbox
  2. Go to the Amazon Mechanical Turk Developer Sandbox and view your HIT.

    You can rewrite and rerun your HIT as often as you like.

How to Publish Your HIT

When you are satisfied with your HIT, publish it to the production system.

To publish your HIT to the production system

  • Open a command prompt, navigate to the [Command Line Tools Installation Directory]\bin directory and run the following command:

    loadHITs -input c:\moviesurvey\moviesurvey.input -question  c:\moviesurvey\moviesurvey.question -properties c:\moviesurvey\moviesurvey.properties 
    			  

    Your HIT is now on the production site.

Creating a HIT Using the API

The following sections explain how to create a HIT using the Amazon Mechanical Turk API.

C#

The following example uses the methods of the Amazon Mechanical Turk SDK for .Net to create the movie survey. You must have the SDK installed and configured correctly to run this example. To download the SDK go to Amazon Mechanical Turk SDK for .NET.

How to Create a HIT

This section shows how to create a HIT.

To create the HIT

  1. Add a project reference to [SDK Installation Directory]\lib\Amazon.WebServices.MechanicalTurk.dll.

  2. Add the following using statements to your source code:

    • using Amazon.WebServices.MechanicalTurk

    • using Amazon.WebServices.MechanicalTurk.Domain

  3. Copy the app.config file from the [SDK Installation Directory]\src directory to your project directory and add the file to your project. This file contains the configuration and the AWS identifier information for your application.

  4. Create a SimpleClient object. This class provides a client used to invoke single operations for Mechanical Turk. Typically, you create this object once and use it throughout your application.

    static SimpleClient client = new SimpleClient();
    				
  5. Use one of the CreateHIT() methods in the SimpleClient class to create a HIT. The following example uses the CreateHIT() method that requires values for the title, description, reward, question, and maxAssignments parameters of the method.

    HIT hit = client.CreateHIT(
        "Movie Survey", 
        "This is a survey to find out how many movies you have watched recently.",
        new decimal(0.05), "How many movies have you seen this month?", 100);    

The following example pulls together all the code described in the preceding procedure.

using System;
using System.Collections.Generic;
using System.Text;
using Amazon.WebServices.MechanicalTurk;
using Amazon.WebServices.MechanicalTurk.Domain;

namespace CreateHITExample
{
    class Program
    {
        // Create a client as a class member.
        static SimpleClient client = new SimpleClient();
            
        static void Main(string[] args)
        {
            CreateNewHIT();
        }
            
        static void CreateNewHIT()
        {
            
            // Create a HIT with the specified parameters.
            HIT hit = client.CreateHIT(
                "Movie Survey",                                                               // title
                "This is a survey to find out how many movies you have watched recently.",    // description
                new decimal(0.05),                                                            // reward  
                "How many movies have you seen this month?",                                  // question
                100);                                                                         // number of assignments

            // Write the new HIT Id to the console.
            Console.WriteLine("Created HIT: {0}", hit.HITId);
            Console.WriteLine("Hit Location: {0}", client.GetPreviewURL(hit.HITTypeId));
            
        }
    }
}
			
How to Test Your HIT

When you compile and run the code example, your HIT is published on the Amazon Mechanical Turk Developer Sandbox by default. This is a simulated environment that enables you to view your HIT as it would appear to Workers. The sandbox is a free test environment for all Requesters.

To test your HIT

  1. Compile and run your project.

    If no errors occur, you see output similar to the following:

    Created HIT: 2X6T66XZQARRM98X5030
    HIT Location: http://workersandbox.mturk.com/mturk/preview?groupId=2X6T66XZQARRM98X5030		     
  2. Go to the Amazon Mechanical Turk Developer Sandbox and view your HIT. You can rewrite and recompile your HIT as often as necessary.

How to Publish Your HIT

When you are satisfied with your HIT, publish it on the Amazon Mechanical Turk production system. This makes your HIT available for Workers to complete.

To publish your HIT

  1. Open the App.config file for your project.

  2. Under element <appSettings>, find the following entry:

    <add key="MechanicalTurk.ServiceEndpoint" value="https://mechanicalturk.sandbox.amazonaws.com?Service=AWSMechanicalTurkRequester"/>			
  3. Replace the entry with the following string:

    <add key="MechanicalTurk.ServiceEndpoint" value="https://mechanicalturk.amazonaws.com/?Service=AWSMechanicalTurkRequester"/>
  4. Save and close the file.

  5. Run your application.

    Your HIT is now on the production site.

Java

The following example uses the APIs of the Amazon Mechanical Turk SDK for Java to create the movie survey. You must have the SDK installed and configured correctly before you run this example. To download the SDK go to Amazon Mechanical Turk SDK for Java.

How to Create a HIT

This section shows how to create a HIT.

To create the HIT

  1. Specify all the *.jar files in the [SDK Installation Directory]\lib directory and in the [SDK Installation Directory]\lib\third-party directory in your classpath.

  2. Specify all the *.jar files in the [SDK Installation Directory]\build\lib directory in your classpath.

  3. Copy the mturk.properties file to your project. This file contains the configuration information needed for your project.

  4. Import the following classes to your application.

    • com.amazonaws.mturk.service.axis.RequesterService

    • com.amazonaws.mturk.service.exception.ServiceException

    • com.amazonaws.mturk.util.ClientConfig

    • com.amazonaws.mturk.requester.HIT

  5. Create a RequesterService object.This class provides the configuration information for your application.

    service = new RequesterService(new PropertiesClientConfig());
  6. Use one of the CreateHIT() methods in the RequesterService class to create a HIT. The following example uses the CreateHIT() method with parameter values for the title, description, reward, and maxAssignments properties of the HIT.

    HIT hit = service.createHIT
    (
         title,
         description,
         reward,
         RequesterService.getBasicFreeTextQuestion(
         "How many movies have you seen this month?"),
         numAssignments);

The following example pulls together all the code described in the preceding procedure.

package createnewhit;

import com.amazonaws.mturk.service.axis.RequesterService;
import com.amazonaws.mturk.service.exception.ServiceException;
import com.amazonaws.mturk.util.PropertiesClientConfig;
import com.amazonaws.mturk.requester.HIT;

/**
 * The MovieSurvey sample application creates a simple HIT using the
 * Amazon Mechanical Turk SDK for Java. The file mturk.properties must be found in the current file path.
 */
public class MovieSurvey{

   private RequesterService service;

   // Define the properties of the HIT to be created.
   private String title = "Movie Survey";
   private String description = 
    "This is a survey to find out how many movies you have watched recently.";
   private int numAssignments = 100;
   private double reward = 0.05;

   /**
    * Constructor
    */
   public MovieSurvey()
   {
      service = new RequesterService(new PropertiesClientConfig());
   }

   /**
    * Create a simple survey.
    * 
    */
   public void createMovieSurvey()
   {
     try 
     {
       // The createHIT method is called using a convenience static method 
       // RequesterService.getBasicFreeTextQuestion() that generates the question format
       // for the HIT.
       HIT hit = service.createHIT
       (
          title,
          description,
          reward,
          RequesterService.getBasicFreeTextQuestion(
               "How many movies have you seen this month?"),
          numAssignments);
              
       // Print out the HITId and the URL to view the HIT.
       System.out.println("Created HIT: " + hit.getHITId());
       System.out.println("HIT location: ");
       System.out.println(service.getWebsiteURL() + "/mturk/preview?groupId=" + hit.getHITTypeId());
      
    }
    catch (ServiceException e) 
    {
       System.err.println(e.getLocalizedMessage());
    }
  }

   /**
    * Main method
    * 
    * @param args
    */
   public static void main(String[] args)
   {
     // Create an instance of this class.  
     MovieSurvey app = new MovieSurvey();
     
     // Create the new HIT.
     app.createMovieSurvey();
   }
}
How to Test Your HIT

This section shows how to test your HIT.

When you compile and run the code example, your HIT is published on the Amazon Mechanical Turk Developer Sandbox. This is a simulated environment that enables you to view your HIT as it would appear to Workers. The sandbox is a free test environment for all Requesters.

To test your HIT

  1. Compile and run your project.

    If no errors occur, you see output similar to the following:

    Created HIT: 2X6T66XZQARRM98X5030 
    HIT Location: 
    http://workersandbox.mturk.com/mturk/preview?groupId=2X6T66XZQARRM98X5030
  2. Go to the Amazon Mechanical Turk Developer Sandbox and view your HIT. You can rewrite and recompile your HIT as often as necessary.

How to Publish Your HIT

When you are satisfied with your HIT, publish it on the Amazon Mechanical Turk production system. This makes your HIT available for Workers to complete.

To publish your HIT

  1. Open the mturk.properties file for your project.

  2. Comment out the Developer Sandbox service_url with a "#".

    #service_url=https://mechanicalturk.sandbox.amazonaws.com/?Service=AWSMechanicalTurkRequester
    			
  3. Remove the comment mark # from the Production site service_url.

    service_url=https://mechanicalturk.amazonaws.com/?Service=AWSMechanicalTurkRequester
  4. Save and close the file.

  5. Compile and run your application.

    Your HIT is now on the production site.

Perl

This example creates a movie survey using the methods of the Amazon Mechanical Turk SDK for Perl. You must have the SDK installed and configured correctly before you can run this example. To download the SDK go to Amazon Mechanical Turk SDK for Perl.

How to Create a HIT

You can use several different methods to create a HIT. You can define the format for the question and answer in a file. The following example writes the question format and the question in the script file. The example prints out the ID and the location of the new HIT to the console.

To create the HIT

  1. Add the following line to your module to use the MechanicalTurk class.

    use NET::Amazon::MechanicalTurk;
  2. Define the question format. In this example, this is a string that defines the XML format of the question.

    my $questionXml = <<END_XML;
    <?xml version="1.0" encoding="UTF-8"?>
    <QuestionForm xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">
      <Question>
        <QuestionIdentifier>1</QuestionIdentifier>
        <QuestionContent>
          <Text>$question</Text>
        </QuestionContent>
        <AnswerSpecification>
          <FreeTextAnswer/>
        </AnswerSpecification>
      </Question>
    </QuestionForm>
    END_XML
    				
  3. Create a string that specifies the question for the HIT.

    my $question = "How many movies have you seen this month?";				
  4. Create a client by creating an instance of the MechanicalTurk class.

    my $mturk = Net::Amazon::MechanicalTurk->new;
  5. Use the CreateHIT() method of the MechanicalTurk class and specify the properties of the HIT.

    my $result = $mturk->CreateHIT(
        Title       => 'Movie Survey',
        Description => 'This is a survey to find out how many movies you have watched recently',
        Keywords    => 'movie, survey',
        Reward => {
            CurrencyCode => 'USD',
            Amount       => 0.05
        },
        RequesterAnnotation         => 'Movie Survey',
        AssignmentDurationInSeconds => 60 * 60 * 24,
        AutoApprovalDelayInSeconds  => 60 * 60 * 10,
        MaxAssignments              => 100,
        LifetimeInSeconds           => 60 * 60,
        Question                    => $questionXml
    );

The following example pulls together all the code described in the preceding procedure.

#!/usr/bin/perl
use strict;
use warnings;
use Net::Amazon::MechanicalTurk;
use Net::Amazon::MechanicalTurk::IOUtil;

# Create the question to ask in the HIT.
my $question = "How many movies have you seen this month?";

# Create the format for the question.
my $questionXml = <<END_XML;
<?xml version="1.0" encoding="UTF-8"?>
<QuestionForm xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">
  <Question>
    <QuestionIdentifier>1</QuestionIdentifier>
    <QuestionContent>
      <Text>$question</Text>
    </QuestionContent>
    <AnswerSpecification>
      <FreeTextAnswer/>
    </AnswerSpecification>
  </Question>
</QuestionForm>
END_XML

# Create a MechanicalTurk object.
my $mturk = Net::Amazon::MechanicalTurk->new;

# Create the HIT with the properties specified.
my $result = $mturk->CreateHIT(
    Title       => 'Movie Survey',
    Description => 'This is a survey to find out how many movies you have watched recently',
    Keywords    => 'movie, survey',
    Reward => {
        CurrencyCode => 'USD',
        Amount       => 0.05
    },
    RequesterAnnotation         => 'Movie Survey',
    AssignmentDurationInSeconds => 60 * 60 * 24,
    AutoApprovalDelayInSeconds  => 60 * 60 * 10,
    MaxAssignments              => 100,
    LifetimeInSeconds           => 60 * 60,
    Question                    => $questionXml
);

# Print the HIT Id to the console.
printf "Created HIT:\n";
printf "HITId:     %s\n", $result->{HITId}[0];
printf "\nHIT Location: %s\n", $mturk->getHITTypeURL($result->{HITTypeId}[0]);
How to Test Your HIT

When you run this code example, your HIT is published on the Amazon Mechanical Turk Developer Sandbox by default. This is a simulated environment that enables you to view your HIT as it would appear to Workers. The sandbox is a free test environment for all Requesters.

To test your HIT

  1. Run the sample to publish the HIT on the Amazon Mechanical Turk Developer Sandbox.

    If no errors occur, you see output similar to the following:

    Created HIT: 2X6T66XZQARRM98X5030 
    HIT Location: http://workersandbox.mturk.com/mturk/preview?groupId=2X6T66XZQARRM98X5030
  2. Go to the Amazon Mechanical Turk Developer Sandbox and view your HIT. You can rewrite and recompile your HIT as often as necessary.

How to Publish Your HIT

When you are satisfied with your HIT, publish it on the Amazon Mechanical Turk production system. This makes your HIT available for Workers to complete.

To publish your HIT

  1. Specify the production system in the constructor of the MechanicalTurk object.

    Your code should look like the following:

    # Create a MechanicalTurk object.
    my $mturk = Net::Amazon::MechanicalTurk->new(serviceUrl=>"prod");
  2. Save and close the file.

  3. Compile and run your application.

    Your HIT is now on the production site.

Note

To go back to using the Developer Sandbox, change the constructor to "sandbox":

# Create a MechanicalTurk object.
my $mturk = Net::Amazon::MechanicalTurk->new(serviceUrl=>"sandbox");

Ruby

This example uses the methods in the Ruby Libraries for Amazon Web Services to create the movie survey. You can use several different methods to create a HIT. You can write the question format and the question in the script file. In this example we create the question and the format in a file, and then print the ID of the new HIT to the console.

You must have the Ruby Libraries for Amazon Web Services installed and configured correctly before you run this example. To download the Ruby Librairies go to Ruby Libraries for Amazon Web Services. The first time you run this example, or one of the examples in the Ruby Libraries for Mechanical Turk, you must provide your Access Key ID and Secret Access Key.

How to Create a HIT

This section shows how to create a HIT.

To write the HIT

  1. Create a question file, moviesurvey.question, that contains the question for your HIT and the template for the question and answer.This is an XML file that conforms to the QuestionForm schema.

    <?xml version="1.0" encoding="UTF-8"?>
    <QuestionForm xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionForm.xsd">
      <Question>
       <QuestionIdentifier>1</QuestionIdentifier>
       <QuestionContent>
         <Text>How many movies have you seen this month?</Text>
       </QuestionContent>
       <AnswerSpecification>
         <FreeTextAnswer/>
       </AnswerSpecification>
      </Question>
    </QuestionForm>
  2. Require the ruby-aws library.

    require 'ruby-aws'
  3. Use the createHIT() method to create a new HIT.

    def createNewHIT
       title = "Movie Survey"
       desc = "This is a survey to find out how many movies you have watched recently."
       keywords = "movie, survey"
       numAssignments = 100
       rewardAmount = 0.05

The following example pulls together all the code described in the preceding procedure.

#!/usr/bin/env ruby

# This sample application creates a simple HIT using Libraries for Amazon Web Services.

require 'ruby-aws'
@mturk = Amazon::WebServices::MechanicalTurkRequester.new :Host => :Sandbox

# Use this line instead if you want the production website.
#@mturk = Amazon::WebServices::MechanicalTurkRequester.new :Host => :Production


def createNewHIT
  title = "Movie Survey"
  desc = "This is a survey to find out how many movies you have watched recently."
  keywords = "movie, survey"
  numAssignments = 100
  rewardAmount = 0.05 # 5 cents
  
  # Define the location of the externalized question (QuestionForm) file.
  rootDir = File.dirname $0
  questionFile = rootDir + "/moviesurvey.question"

  # Load the question (QuestionForm) file
  question = File.read( questionFile )
  
  result = @mturk.createHIT( :Title => title,
    :Description => desc,
    :MaxAssignments => numAssignments,
    :Reward => { :Amount => rewardAmount, :CurrencyCode => 'USD' },
    :Question => question,
    :Keywords => keywords )

  puts "Created HIT: #{result[:HITId]}"
  puts "HIT Location: #{getHITUrl( result[:HITTypeId] )}"
  return result
end

def getHITUrl( hitTypeId )
  if @mturk.host =~ /sandbox/
    "http://workersandbox.mturk.com/mturk/preview?groupId=#{hitTypeId}"   # Sandbox Url
  else
    "http://mturk.com/mturk/preview?groupId=#{hitTypeId}"   # Production Url
  end
end

createNewHIT
How to Test Your HIT

When you run this code example, your HIT is published on the Amazon Mechanical Turk Developer Sandbox by default. This is a simulated environment that enables you to view your HIT as it would appear to Workers. The sandbox is a free test environment for all Requesters.

To test your HIT

  1. Run the sample to publish the HIT on the Amazon Mechanical Turk Developer Sandbox.

    If no errors occur, you see output similar to the following:

    Created HIT: 2X6T66XZQARRM98X5030
    HIT Location: http://workersandbox.mturk.com/mturk/preview?groupId=2X6T66XZQARRM98X5030       
  2. Go to the Amazon Mechanical Turk Developer Sandbox and view your HIT. You can rewrite and recompile your HIT as often as necessary.

How to Publish Your HIT

When you are satisfied with your HIT, publish it on the Amazon Mechanical Turk production system. This makes your HIT available for Workers to complete.

To publish your HIT

  1. In the sample code, find the following line and comment it out:

    @mturk = Amazon::WebServices::MechanicalTurkRequester.new :Host => :Sandbox
  2. In the sample code, remove the comment mark # from the following line:

    #@mturk = Amazon::WebServices::MechanicalTurkRequester.new :Host => :Production
  3. Save and close the file.

  4. Run your application.

    Your HIT is now on the production site.

What's Next?

You now have a HIT on the Amazon Mechanical Turk product site. You've become familiar with the architecture of the system, some of its basic functionality, and the kind of responses you can expect. The following section explains how to learn more about Amazon Mechanical Turk and how to implement advanced Amazon Mechanical Turk functionality in your applications.