Implementing Amazon Mechanical Turk
The following topics describe information and additional resources you can use to implement a project using Amazon Mechanical Turk.
In the Creating a HIT tutorial you learned how to complete basic Amazon Mechanical Turk tasks. If you didn't use the tutorial, you can learn how to complete basic and advanced Amazon Mechanical Turk tasks using the Amazon Mechanical Turk Developer Guide and by looking at code samples. For more information, go to Amazon Mechanical Turk Developer Guide and Amazon Mechanical Turk Sample Code and Libraries, respectively.
Amazon Mechanical Turk offers the following interfaces:
Requester user interface
The Amazon Mechanical Turk command line interface (CLI) makes it easy to use most of the Amazon Mechanical Turk functionality. When you wish to have a hands-on approach to using Amazon Mechanical Turk and have a relatively small number of assignments and results, the CLI is a good choice. For more information, go to Amazon Mechanical Turk Command Line Tool Reference.
When the number of assignments you have or the number of results you have is large, the Amazon Mechanical Turk API is a good choice. The API exercises all of Amazon Mechanical Turk's functionality and enables you to integrate Amazon Mechanical Turk functions programmatically. For more information, go to Amazon Mechanical Turk Developer Guide.
If you have a very large number of similar HITs, consider using the Requester user interface. It merges one question template with lots of question data to create many similar HITs. For more information, go to Amazon Mechanical Turk Requester User Interface.
Creating a successful HIT involves more than programming. There is a certain art involved, for example, in pricing a HIT correctly, laying out the question correctly, breaking down the task into HITs, and minimizing the Worker's time spent with the HIT. For that reason, we created a best practices guide that gives detailed instructions about creating an effective HIT. For more information, go to Amazon Mechanical Turk Best Practices Guide.
Some of the HITs you create require the Workers to have special skills. You might, for
example, publish a HIT that asks for a translation. Whenever you have a HIT that
requires specialized skills, we recommend that you qualify the Workers. Only those who
pass the test receive the opportunity to work on your HITs. For more information, go to
CreateHIT and read about the
Once a Worker becomes qualified, you can grant them qualification status so that they no longer need to complete a qualification test before working on your HITs. For more information, go to AssignQualification.
Considerations for Writing a HIT
This guide presented a HIT for you to use. When you have to create your own, there are several things you should consider:
What is the problem you are trying to solve?
What questions do you want the Workers to answer? What is the best way to present the task to Workers? You need this information when you write the description and question for your HIT. You should be familiar with the format to create your question or task. For more information, go to the Amazon Mechanical Turk Developer Guide.
How much do you want to pay Workers?
You need to specify a reward for your HIT. Setting too low a reward discourages Workers from working on your HITs. You need to determine what a fair price is for the work you're asking the Workers to do. The best way is to look at similar tasks advertised on the Amazon Mechanical Turk website. For more information, go to the Amazon Mechanical Turk website.
How many responses do you want?
This is the number of assignments for your HIT. Sometimes you want only one answer per HIT. When the answer is controversial, however, you might like to get multiple answers (and thus use multiple assignments) per HIT and reach an answer by consensus.
How much time do you want to allow to complete the task?
Giving a Worker too long a time potentially delays getting results. Making the duration too short frustrates Workers.
Common Use Scenarios
This section describes some of the ways you can use Amazon Mechanical Turk.
Photo and Video Processing
Amazon Mechanical Turk is well-suited for processing images. In the past, companies have used Mechanical Turk to:
Tag objects found in an image for easier searching and advertising targeting
Select from a set of images the best picture to represent a product
Audit user-uploaded images for inappropriate content
Classify objects found in satellite imagery
Data Verification and Clean-up
Companies with large online catalogs use Amazon Mechanical Turk to identify duplicate entries and verify item details. Examples include:
De-duplication of yellow pages directory listings
Identification of duplicate products in an online product catalog
Verification of restaurant details, such as the phone number or hours of operation
The Amazon Mechanical Turk workforce enables you to gather information, such as:
Allowing people to ask questions from a computer or mobile device about any topic and have Workers return results to those questions
Filling out survey data on a variety of topics
Writing reviews, descriptions and blog entries for websites
Finding specific fields or data elements in large legal and government documents
Use Amazon Mechanical Turk to process data, for example:
Podcast editing and transcription
Human powered translation services
Search engine rating
To help you code your applications, we provide the following resources:
Developer Resources page— Click the Developer tab on the Requester website located at https://requester.mturk.com/ to get to the Developer Resources page, which has links to sample code, documentation, the sandbox, and other helpful information.
Sample Code and Libraries— You can use code samples as a means of understanding how to implement the Amazon Mechanical Turk API. For more information, go to Amazon Mechanical Turk Sample Code and Libraries.
Customer Forum—We recommend you look at the Amazon Mechanical Turk Forum to get an idea of what other users are doing and to benefit from the questions they've asked.
The forum can help you understand what you can and can't do with Amazon Mechanical Turk. The forum also serves as a place for you to ask questions that other users or Amazon representatives might answer. You can use the forum to report issues with the service, the API, or the documentation. For more information, go to Amazon Mechanical Turk Discussion Forums.
The tutorial in this guide showed how to accomplish the basic tasks of creating, testing, and publishing your HITs. The Amazon Mechanical Turk API, however, offers advanced functionality, which the following table summarizes. For more information, go to the Amazon Mechanical Turk Developer Guide.
|Review the work submitted by Workers||You reward work well done by paying the Worker. If work is not done well, you can elect not to pay the Worker and block them from future assignments. For more information, go to GetReviewableHITs, RejectAssignment, BlockWorker, and ApproveAssignment.|
|Give bonuses||You might like to give a bonus to a Worker that does especially good work. This gives the Worker incentive to work on one of your other HITs. By rewarding Workers, you can create a small group of qualified people who you know do good work. They, in turn, will prioritize your HITs over others so that Workers complete your HITs in a timely manner. For more information about awarding bonuses, go to GrantBonus.|
|Review results programmatically||Reviewing the results manually is an option when the number of assignments is small. As the number of assignments grows, reviewing results programmatically becomes more practical. The Amazon Mechanical Turk API enables you to do that. For more information, go to GetReviewableHITs|
|Cancel a HIT||There are times when the results, although accurate, are unexpected and unusable by you. In that case, you have to expire the HIT, revise, and recreate it. For more information, go to ForceExpireHIT.|
|Update HIT properties||You can use API operations and the command line tools to update HIT properties, such as Title, Description, Reward, and Keywords. For more information, go to ChangeHITTypeOfHIT.|
|Extend or eliminate HITs||You can also use the API operations to extend the completion time for a HIT, expire a HIT early, or add additional assignments. For more information, go to ExtendHIT and DisposeHIT.|
|Communicate with Workers||You can use NotifyWorkers to send emails to specified Workers.|
|Return the amount of money in your account that you use to pay Workers||Before you can publish Amazon Mechanical Turk HITs, your account must contain the money required to pay for all of the assignments. You can programmatically access the amount of money in your account using GetAccountBalance.|
|Create your own qualifications for Workers||We recommend that you use qualification tests when completing your HITs requires specialized skills. You can create your own qualification tests using CreateQualificationType.|
The following list shows additional resources you can use to further your understanding of Amazon Mechanical Turk.
Learn the pricing for Amazon Mechanical Turk.
For more information, go to Amazon Mechanical Turk website.
Use libraries for Amazon Mechanical Turk written in Java and PHP.
For more information, go to Sample Code and Libraries.
The Quick Reference Card provides quick access to important Amazon Mechanical Turk operations and functionality.
For more information, go to Quick Reference Card.
When you have questions about developing with Amazon Mechanical Turk visit the customer forum to get your questions answered.
A wide variety of questions have already been answered in the forum. If your question has not been answered, ask it and wait for a fellow developer or Amazon representative to help. For more information, go to Amazon Mechanical Turk Developer Forums.
The Service Health Dashboard shows you the status of the Amazon Mechanical Turk web service.
The dashboard shows you whether Amazon Mechanical Turk (and all other AWS web services) are functioning properly. For more information, go to Service Health Dashboard.
The Amazon Mechanical Turk Developer Guide provides a detailed discussion of the service.
For more information, go to Amazon Mechanical Turk Developer Guide.
The HIT page shows the Amazon Mechanical Turk HITs available to work on.
For more information, go to the HIT page.