| « PreviousNext » | |
   | Did this page help you? Yes | No | Tell us about it... |
Hadoop Configuration (AMI 2.0 and 2.1)
The following Amazon Elastic MapReduce (Amazon EMR) default configuration settings for clusters launched with Amazon EMR AMI 2.0 or 2.1 are appropriate for most workloads.
If your cluster tasks are memory-intensive, you can enhance performance by using fewer tasks per core node and reducing your job tracker heap size.
The following tables list the default configuration settings for each EC2 instance type in clusters launched with the Amazon EMR AMI version 2.0 or 2.1. For more information about the AMI versions supported by Amazon EMR, see Choose a Machine Image .
m1.small
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 768 |
| HADOOP_NAMENODE_HEAPSIZE | 256 |
| HADOOP_TASKTRACKER_HEAPSIZE | 256 |
| HADOOP_DATANODE_HEAPSIZE | 128 |
| mapred.child.java.opts | -Xmx384m |
| mapred.tasktracker.map.tasks.maximum | 2 |
| mapred.tasktracker.reduce.tasks.maximum | 1 |
m1.medium
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 1536 |
| HADOOP_NAMENODE_HEAPSIZE | 512 |
| HADOOP_TASKTRACKER_HEAPSIZE | 256 |
| HADOOP_DATANODE_HEAPSIZE | 256 |
| mapred.child.java.opts | -Xmx768m |
| mapred.tasktracker.map.tasks.maximum | 2 |
| mapred.tasktracker.reduce.tasks.maximum | 1 |
m1.large
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 3072 |
| HADOOP_NAMENODE_HEAPSIZE | 1024 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx1152m |
| mapred.tasktracker.map.tasks.maximum | 3 |
| mapred.tasktracker.reduce.tasks.maximum | 1 |
m1.xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 9216 |
| HADOOP_NAMENODE_HEAPSIZE | 3072 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx1024m |
| mapred.tasktracker.map.tasks.maximum | 8 |
| mapred.tasktracker.reduce.tasks.maximum | 3 |
c1.medium
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 768 |
| HADOOP_NAMENODE_HEAPSIZE | 256 |
| HADOOP_TASKTRACKER_HEAPSIZE | 256 |
| HADOOP_DATANODE_HEAPSIZE | 128 |
| mapred.child.java.opts | -Xmx384m |
| mapred.tasktracker.map.tasks.maximum | 2 |
| mapred.tasktracker.reduce.tasks.maximum | 1 |
c1.xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 3072 |
| HADOOP_NAMENODE_HEAPSIZE | 1024 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx512m |
| mapred.tasktracker.map.tasks.maximum | 7 |
| mapred.tasktracker.reduce.tasks.maximum | 2 |
m2.xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 12288 |
| HADOOP_NAMENODE_HEAPSIZE | 4096 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx3072m |
| mapred.tasktracker.map.tasks.maximum | 3 |
| mapred.tasktracker.reduce.tasks.maximum | 1 |
m2.2xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 24576 |
| HADOOP_NAMENODE_HEAPSIZE | 8192 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx3584m |
| mapred.tasktracker.map.tasks.maximum | 6 |
| mapred.tasktracker.reduce.tasks.maximum | 2 |
m2.4xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 49152 |
| HADOOP_NAMENODE_HEAPSIZE | 16384 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx3072m |
| mapred.tasktracker.map.tasks.maximum | 14 |
| mapred.tasktracker.reduce.tasks.maximum | 4 |
cc1.4xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 10240 |
| HADOOP_NAMENODE_HEAPSIZE | 5120 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx1216m |
| mapred.tasktracker.map.tasks.maximum | 12 |
| mapred.tasktracker.reduce.tasks.maximum | 3 |
cc2.8xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 40152 |
| HADOOP_NAMENODE_HEAPSIZE | 16384 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx2048m |
| mapred.tasktracker.map.tasks.maximum | 24 |
| mapred.tasktracker.reduce.tasks.maximum | 6 |
cg1.4xlarge
| Parameter | Value |
|---|---|
| HADOOP_JOBTRACKER_HEAPSIZE | 10240 |
| HADOOP_NAMENODE_HEAPSIZE | 5120 |
| HADOOP_TASKTRACKER_HEAPSIZE | 512 |
| HADOOP_DATANODE_HEAPSIZE | 512 |
| mapred.child.java.opts | -Xmx1152m |
| mapred.tasktracker.map.tasks.maximum | 12 |
| mapred.tasktracker.reduce.tasks.maximum | 3 |
