Command runner - Amazon EMR

Command runner

Many scripts or programs are placed on the shell login path environment so you do not need to specify the full path when executing them when using command-runner.jar. You also do not have to know the full path to command-runner.jar. command-runner.jar is located on the AMI so there is no need to know a full URI as was the case with script-runner.jar. For more information, see Run a script in a cluster.

The following is a list of scripts that can be executed with command-runner.jar:


Submit a Hadoop streaming program. In the console and some SDKs, this is a streaming step.


Run a Hive script. In the console and SDKs, this is a Hive step.


Run a Pig script. In the console and SDKs, this is a Pig step.


Run a Spark application. In the console, this is a Spark step.


Distributed copy large amounts of data from Amazon S3 into HDFS.


Run the Hadoop LZO indexer on a directory.

The following is an example usage of command-runner.jar using the AWS CLI:

aws emr add-steps --cluster-id j-2AXXXXXXGAPLF --steps Name="Command runner jar step",Jar=command-runner.jar,Args=[spark-submit,<additional-args>]