Menu
Amazon EMR
Amazon EMR Release Guide

Enable Consistent View

You can enable Amazon S3 server-side encryption or consistent view for EMRFS using the AWS Management Console, AWS CLI, or the emrfs-site configuration classification.

To configure consistent view using the console

  1. Choose Create Cluster.

  2. Navigate to the File System Configuration section.

  3. To enable Consistent view, choose Enabled.

  4. For EMRFS Metadata store, type the name of your metadata store. The default value is EmrFSMetadata. If the EmrFSMetadata table does not exist, it is created for you in DynamoDB.

    Note

    Amazon EMR does not automatically remove the EMRFS metadata from DynamoDB when the cluster is terminated.

  5. For Number of retries, type an integer value. This value represents the number of times EMRFS retries calling Amazon S3 if an inconsistency is detected. The default value is 5.

  6. For Retry period (in seconds), type an integer value. This value represents the amount of time that lapses before EMRFS retries calling Amazon S3. The default value is 10.

    Note

    Subsequent retries use an exponential backoff.

To launch a cluster with consistent view enabled using the AWS CLI

We recommend you install the current version of AWS CLI. To download the latest release, see https://aws.amazon.com//cli/.

  • Note

    Linux line continuation characters (\) are included for readability. They can be removed or used in Linux commands. For Windows, remove them or replace with a caret (^).

    Copy
    aws emr create-cluster --instance-type m1.large --instance-count 3 --emrfs Consistent=true \ --release-label emr-5.6.0 --ec2-attributes KeyName=myKey

To check if consistent view is enabled using the AWS Management Console

  • To check whether consistent view is enabled in the console, navigate to the Cluster List and select your cluster name to view Cluster Details. The "EMRFS consistent view" field has a value of Enabled or Disabled.

To check if consistent view is enabled by examining the emrfs-site.xml file

  • You can check if consistency is enabled by inspecting the emrfs-site.xml configuration file on the master node of the cluster. If the Boolean value for fs.s3.consistent is set to true then consistent view is enabled for file system operations involving Amazon S3.