Amazon Elastic MapReduce
Developer Guide (API Version 2009-03-31)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Supported Hadoop Versions

This documentation is for AMI versions 2.x and 3.x of Amazon EMR. See the Amazon EMR Release Guide for information about Amazon EMR releases 4.0.0 and above. For information about managing the Amazon EMR service in 4.x releases, see the Amazon EMR Management Guide.

Amazon EMR allows you to choose which version of Hadoop to run. You do this using the CLI and setting the --ami-version as shown in the following table. We recommend using the latest version of Hadoop to take advantage of performance enhancements and new functionality.

Note

The AMI version determines the Hadoop version and the --hadoop-version parameter is no longer supported.

Hadoop VersionAMI Versions
2.4.0 --ami-version 3.1.X and higher (all 3.1 and 3.2 versions)
2.2.0 --ami-version 3.0.X (all 3.0 versions)
1.0.3 --ami-version 2.2.X, 2.3.X, 2.4.X (all 2.2, 2.3, and 2.4 versions)
0.20.205 --ami-version 2.1.4
0.20 --ami-version 1.0

For details about the default configuration and software available on AMIs used by Amazon Elastic MapReduce (Amazon EMR) see Choose an Amazon Machine Image (AMI).

Note

The Asia Pacific (Sydney) region and AWS GovCloud (US) regions support only Hadoop 1.0.3 and later. AWS GovCloud (US) additionally requires AMI 2.3.0 or later.

To specify the Hadoop version using the AWS CLI

To specify the Hadoop version using the AWS CLI, type the create-cluster subcommand with the --ami-version parameter. The AMI version determines the version of Hadoop for Amazon EMR to use. For details about the version of Hadoop available on an AMI, see AMI Versions Supported in Amazon EMR.

  • To launch a cluster running Hadoop 2.4.0 using AMI version 3.3, type the following command and replace myKey with the name of your EC2 key pair.

    • Linux, UNIX, and Mac OS X users:

      aws emr create-cluster --name "Test cluster" --ami-version 3.3 --applications Name=Hue Name=Hive Name=Pig \
      --use-default-roles --ec2-attributes KeyName=myKey \
      --instance-count 5 --instance-type m3.xlarge
    • Windows users:

      aws emr create-cluster --name "Test cluster" --ami-version 3.3 --applications Name=Hue Name=Hive Name=Pig --use-default-roles --ec2-attributes KeyName=myKey --instance-count 5 --instance-type m3.xlarge

    When you specify the instance count without using the --instance-groups parameter, a single Master node is launched, and the remaining instances are launched as core nodes. All nodes will use the instance type specified in the command.

    Note

    If you have not previously created the default EMR service role and EC2 instance profile, type aws emr create-default-roles to create them before typing the create-cluster subcommand.

    For more information on using Amazon EMR commands in the AWS CLI, see http://docs.aws.amazon.com/cli/latest/reference/emr.