Amazon Elastic MapReduce
Developer Guide (API Version 2009-03-31)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Hadoop Patches Applied in Amazon EMR

The following sections detail the patches the Amazon Elastic MapReduce (Amazon EMR) team has applied to the Hadoop versions loaded on Amazon EMR AMIs.

Hadoop 1.0.3 Patches

The Amazon EMR team has applied the following patches to Hadoop 1.0.3 on the Amazon EMR AMI version 2.2.

PatchDescription
All of the patches applied to the Amazon EMR version of Hadoop 0.20.205.

See Hadoop 0.20.205 Patches for details.

HADOOP-5861

Files stored on the native Amazon S3 file system, those with URLs of the form s3n://, now report a block size determined by fs.s3n.block.size. For more information, go to https://issues.apache.org/jira/browse/HADOOP-5861.

Status: Fixed

Fixed in AWS Hadoop Version: 1.0.3

Fixed in Apache Hadoop Version: 0.21.0

HADOOP-6346

Supports specifying a pattern to RunJar.unJar that determines which files are unpacked. For more information, go to https://issues.apache.org/jira/browse/HADOOP-6346.

Status: Fixed

Fixed in AWS Hadoop Version: 1.0.3

Fixed in Apache Hadoop Version: 0.21.0

MAPREDUCE-967

Changes the TaskTracker node so it does not fully unjar job jars into the job cache directory. For more information, go to https://issues.apache.org/jira/browse/MAPREDUCE-967.

Status: Fixed

Fixed in AWS Hadoop Version: 1.0.3

Fixed in Apache Hadoop Version: 0.21.0

MAPREDUCE-2219

Changes the JobTracker service to remove the contents of mapred.system.dir during startup instead of removing the directory itself. For more information, go to https://issues.apache.org/jira/browse/MAPREDUCE-2219.

Status: Fixed

Fixed in AWS Hadoop Version: 1.0.3

Fixed in Apache Hadoop Version: 0.22.0

Hadoop 0.20.205 Patches

The Amazon EMR team has applied the following patches to Hadoop 0.20.205 on the Amazon EMR AMI version 2.0.

PatchDescription
Add hadoop-lzo

Install the hadoop-lzo third-party package. For more information about hadoop-lzo, go to https://github.com/kevinweil/hadoop-lzo

Status: Third-party Package

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: n/a

Install the hadoop-snappy library

Add the hadoop-snappy library to provide access to the snappy compression. For more information about this library, go to http://code.google.com/p/hadoop-snappy/.

Status: Third-party Library

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: n/a

MAPREDUCE-1597/2021/2046

Fixes to how CombineFileInputFormat handles split locations and files that can be split. For more information about these patches, go to https://issues.apache.org/jira/browse/MAPREDUCE-1597, https://issues.apache.org/jira/browse/MAPREDUCE-2021, and https://issues.apache.org/jira/browse/MAPREDUCE-2046.

Status: Resolved, Fixed

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: 0.22.0

HADOOP-6436

Remove the files generated by automake and autoconf of the native build and use the host's automake and autoconf to generate the files instead. For more information about this patch, go to https://issues.apache.org/jira/browse/HADOOP-6436.

Status: Closed, Fixed

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: 0.22.0,0.23.0

MAPREDUCE-2185

Prevent an infinite loop from occurring when creating splits using CombineFileInputFormat. For more information about this patch, go to https://issues.apache.org/jira/browse/MAPREDUCE-2185.

Status: Closed, Fixed

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: 0.23.0

HADOOP-7082

Change Configuration.writeXML to not hold a lock while outputting. For more information about this patch, go to https://issues.apache.org/jira/browse/HADOOP-7082.

Status: Resolved, Fixed

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: 0.22.0

HADOOP-7015

Update RawLocalFileSystem#listStatus to deal with a directory that has changing entries, as in a multi-threaded or multi-process environment. For more information about this patch, go to https://issues.apache.org/jira/browse/HADOOP-7015.

Status: Closed, Fixed

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: 0.23.0

HADOOP-4675

Update the Ganglia metrics to be compatible with Ganglia 3.1. For more information about this patch go to https://issues.apache.org/jira/browse/HADOOP-4675.

Status: Resolved, Fixed

Fixed in AWS Hadoop Version: 0.20.205

Fixed in Apache Hadoop Version: 0.22.0