Menu
Amazon EMR
Amazon EMR Release Guide

Specifying Amazon EMR Encryption Options Using a Security Configuration

Using a security configuration to specify cluster encryption settings is a two-step process. First, you create a security configuration, which you can use for any number of clusters. Then you specify the security configuration to use when you create a cluster. Before you create a security configuration, decide on the key and certificate management systems you want to use and create the keys and certificates. For more information, see Providing Keys for At-Rest Data Encryption with Amazon EMR and Providing Certificates for In-Transit Data Encryption with Amazon EMR Encryption.

Creating a Security Configuration

When you create a security configuration, you specify two sets of encryption options: at-rest data encryption and in-transit data encryption. Options for at-rest data encryption include both Amazon S3 with EMRFS and local-disk encryption. In-transit encryption options enable the open-source encryption features for certain applications that support Transport Layer Security (TLS). At-rest options and in-transit options can be enabled together or separately. You can use the AWS Management Console, the AWS CLI, or the AWS SDKs to create a security configuration.

Creating a Security Configuration Using the Console

To create a security configuration:

  1. Sign in to the AWS Management Console and open the Amazon EMR console at https://console.aws.amazon.com/elasticmapreduce/.

  2. In the navigation pane, choose Security Configurations, Create security configuration.

  3. Type a Name for the security configuration.

  4. Choose At rest encryption to encrypt data stored within the file system. This also enables Hadoop Distributed File System (HDFS) block-transfer encryption and RPC encryption, which need no further configuration.

  5. Under S3 data encryption, for Encryption mode, choose a value to determines how Amazon EMR encrypts Amazon S3 data with EMRFS.

    What you do next depends on the encryption mode you chose:

  6. Under Local disk encryption, choose a value for Key provider type. Amazon EMR uses this key for Linux Unified Key System (LUKS) encryption for the local volumes (except boot volumes) attached to your cluster nodes.

    • AWS KMS

      Select this option to specify an AWS KMS customer master key (CMK). For AWS KMS Key, select a key. The key must exist in the same region as your Amazon EMR cluster. For more information about key requirements, see Using AWS KMS Customer Master Keys (CMKs) for Encryption.

    • Custom

      Select this option to specify a custom key provider. For S3 object, enter the location in Amazon S3, or the Amazon S3 ARN, of your custom key-provider JAR file. For Key provider class, enter the full class name of a class declared in your application that implements the EncryptionMaterialsProvider interface. The class name you provide here must be different from the class name provided for CSE-Custom.

  7. Choose In-transit encryption to enable the open-source TLS encryption features for in-transit data. Choose a Certificate provider type according to the following guidelines:

    • PEM

      Select this option to use PEM files that you provide within a zip file. Two artifacts are required within the zip file: privateKey.pem and certificateChain.pem. A third file, trustedCertificates.pem, is optional. See Providing Certificates for In-Transit Data Encryption with Amazon EMR Encryption for details. For S3 object, specify the location in Amazon S3, or the Amazon S3 ARN, of the zip filefield.

    • Custom

      Select this option to specify a custom certificate provider and then, for S3 object, enter the location in Amazon S3, or the Amazon S3 ARN, of your custom certificate-provider JAR file. For Key provider class, enter the full class name of a class declared in your application that implements the TLSArtifactsProvider interface.

  8. Click Create.

Creating a Security Configuration Using the AWS CLI

To create a security configuration with the AWS CLI, use the following command:

Copy
aws emr create-security-configuration --name "SecConfigName" --security-configuration SecConfigDef
  • --name SecConfigName specifies the name of the security configuration, which you specify when you create a cluster.

  • --security-configuration 'SecConfigDef' specifies a JSON blob (examples below) or the path to a JSON file in Amazon S3 (such as file://./MySecConfig.json) that defines encryption parameters.

The sections that follow use sample scenarios to illustrate well-formed --security-configuration JSON for different configurations and key providers, as well as a reference for JSON parameters.

Example In-Transit Data Encryption Options

The example below illustrates the following scenario:

Copy
aws emr create-security-configuration --name "MySecConfig" --security-configuration '{ "EncryptionConfiguration": { "EnableInTransitEncryption" : true, "EnableAtRestEncryption" : false, "InTransitEncryptionConfiguration" : { "TLSCertificateConfiguration" : { "CertificateProviderType" : "PEM", "S3Object" : "s3://MyConfigStore/artifacts/MyCerts.zip" } } } }'

The example below illustrates the following scenario:

Copy
aws emr create-security-configuration --name "MySecConfig" --security-configuration '{ "EncryptionConfiguration": { "EnableInTransitEncryption" : true, "EnableAtRestEncryption" : false, "InTransitEncryptionConfiguration" : { "TLSCertificateConfiguration" : { "CertificateProviderType" : "Custom", "S3Object" : "s3://MyConfig/artifacts/MyCerts.jar", "CertificateProviderClass" : "com.mycompany.MyCertProvider" } } } }'

Example At-Rest Data Encryption Options

The example below illustrates the following scenario:

  • In-transit data encryption is disabled and at-rest data encryption is enabled

  • SSE-S3 is used for Amazon S3 encryption

  • Local disk encryption uses AWS KMS as the key provider

Copy
aws emr create-security-configuration --name "MySecConfig" --security-configuration '{ "EncryptionConfiguration": { "EnableInTransitEncryption" : false, "EnableAtRestEncryption" : true, "AtRestEncryptionConfiguration" : { "S3EncryptionConfiguration" : { "EncryptionMode" : "SSE-S3" }, "LocalDiskEncryptionConfiguration" : { "EncryptionKeyProviderType" : "AwsKms", "AwsKmsKey" : "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012" } } } }'

The example below illustrates the following scenario:

  • In-transit data encryption is enabled and references a zip file with PEM certificates in Amazon S3, using the ARN

  • SSE-KMS is used for Amazon S3 encryption

  • Local disk encryption uses AWS KMS as the key provider

Copy
aws emr create-security-configuration --name "MySecConfig" --security-configuration '{ "EncryptionConfiguration": { "EnableInTransitEncryption" : true, "EnableAtRestEncryption" : true, "InTransitEncryptionConfiguration" : { "TLSCertificateConfiguration" : { "CertificateProviderType" : "PEM", "S3Object" : "arn:aws:s3:::MyConfigStore/artifacts/MyCerts.zip" } }, "AtRestEncryptionConfiguration" : { "S3EncryptionConfiguration" : { "EncryptionMode" : "SSE-KMS", "AwsKmsKey" : "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012" }, "LocalDiskEncryptionConfiguration" : { "EncryptionKeyProviderType" : "AwsKms", "AwsKmsKey" : "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012" } } } }'

The example below illustrates the following scenario:

  • In-transit data encryption is enabled and references a zip file with PEM certificates in Amazon S3

  • CSE-KMS is used for Amazon S3 encryption

  • Local disk encryption uses a custom key provider referenced by its ARN

Copy
aws emr create-security-configuration --name "MySecConfig" --security-configuration '{ "EncryptionConfiguration": { "EnableInTransitEncryption" : true, "EnableAtRestEncryption" : true, "InTransitEncryptionConfiguration" : { "TLSCertificateConfiguration" : { "CertificateProviderType" : "PEM", "S3Object" : "s3://MyConfigStore/artifacts/MyCerts.zip" } }, "AtRestEncryptionConfiguration" : { "S3EncryptionConfiguration" : { "EncryptionMode" : "CSE-KMS", "AwsKmsKey" : "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012" }, "LocalDiskEncryptionConfiguration" : { "EncryptionKeyProviderType" : "Custom", "S3Object" : "arn:aws:s3:::artifacts/MyKeyProvider.jar", "EncryptionKeyProviderClass" : "com.mycompany.MyKeyProvider.jar" } } } }'

The example below illustrates the following scenario:

  • In-transit data encryption is enabled with a custom key provider

  • CSE-Custom is used for Amazon S3 data

  • Local disk encryption uses a custom key provider

Copy
aws emr create-security-configuration --name "MySecConfig" --security-configuration '{ "EncryptionConfiguration": { "EnableInTransitEncryption" : "true", "EnableAtRestEncryption" : "true", "InTransitEncryptionConfiguration" : { "TLSCertificateConfiguration" : { "CertificateProviderType" : "Custom", "S3Object" : "s3://MyConfig/artifacts/MyCerts.jar", "CertificateProviderClass" : "com.mycompany.MyCertProvider" } }, "AtRestEncryptionConfiguration" : { "S3EncryptionConfiguration" : { "EncryptionMode" : "CSE-Custom", "S3Object" : "s3://MyConfig/artifacts/MyCerts.jar", "EncryptionKeyProviderClass" : "com.mycompany.MyKeyProvider" }, "LocalDiskEncryptionConfiguration" : { "EncryptionKeyProviderType" : "Custom", "S3Object" : "s3://MyConfig/artifacts/MyCerts.jar", "EncryptionKeyProviderClass" : "com.mycompany.MyKeyProvider" } } } }'

AWS CLI Security Configuration JSON Reference

The following table lists the JSON parameters for encryption settings and provides a description of acceptable values for each parameter.

Parameter Description
"EnableInTransitEncryption" : true | false Specify true to enable in-transit encryption and false to disable it. If omitted, false is assumed, and in-transit encryption is disabled.
"EnableAtRestEncryption" : true | false Specify true to enable at-rest encryption and false to disable it. If omitted, false is assumed and at-rest encryption is disabled.
In-transit encryption parameters
"InTransitEncryptionConfiguration" : Specifies a collection of values used to configure in-transit encryption when EnableInTransitEncryption is true.
"CertificateProviderType" : "PEM" | "Custom" Specifies whether to use PEM certificates referenced with a zipped file, or a Custom certificate provider. If PEM is specified, S3Object must be a reference to the location in Amazon S3 of a zip file containing the certificates. If Custom is specified, S3Object must be a reference to the location in Amazon S3 of a JAR file, followed by a CertificateProviderClass entry.
"S3Object" : "ZipLocation" | "JarLocation" Provides the location in Amazon S3 to a zip file when PEM is specified, or to a JAR file when Custom is specified. The format can be a path (for example, s3://MyConfig/articfacts/CertFiles.zip) or an ARN (for example, arn:aws:s3:::Code/MyCertProvider.jar). If a zip file is specified, it must contain files named exactly privateKey.pem and certificateChain.pem. A file named trustedCertificates.pem is optional.
"CertificateProviderClass" : "MyClassID" Required only if Custom is specified for CertificateProviderType. MyClassID specifies a full class name declared in the JAR file, which implements the TLSArtifactsProvider interface. For example, com.mycompany.MyCertProvider.
At-rest encryption parameters
"AtRestEncryptionConfiguration" : Specifies a collection of values for at-rest encryption when EnableAtRestEncryption is true, including Amazon S3 encryption and local disk encryption.
Amazon S3 encryption parameters
"S3EncryptionConfiguration" : Specifies a collection of values used for Amazon S3 encryption with the EMR File System (EMRFS).
"EncryptionMode" : "SSE-S3" | "SSE-KMS" | "CSE-KMS" | "CSE-Custom" Specifies the type of Amazon S3 encryption to use. If SSE-S3 is specified, no further Amazon S3 encryption values are required. If either SSE-KMS or CSE-KMS is specified, an AWS KMS customer master key (CMK) ARN must be specified as the AwsKmsKey value. If CSE-Custom is specified, S3Object and EncryptionKeyProviderClass values must be specified.
"AwsKmsKey" : "MyKeyARN" Required only when either SSE-KMS or CSE-KMS is specified for EncryptionMode. MyKeyARN must be a fully specified ARN to a key (for example, arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012).
"S3Object" : "JarLocation" Required only when CSE-Custom is specified for CertificateProviderType. JarLocation provides the location in Amazon S3 to a JAR file. The format can be a path (for example, s3://MyConfig/articfacts/MyKeyProvider.jar) or an ARN (for example, arn:aws:s3:::Code/MyKeyProvider.jar).
"EncryptionKeyProviderClass" : "MyS3KeyClassID" Required only when CSE-Custom is specified for EncryptionMode. MyS3KeyClassID specifies a full class name of a class declared in the application that implements the EncryptionMaterialsProvider interface; for example, com.mycompany.MyS3KeyProvider.
Local disk encryption parameters
"LocalDiskEncryptionKeyProvider" Specifies the key provider and corresponding values to be used for local disk encryption.
"Type" : "AwsKms" | "Custom" Specifies the key provider. If AwsKms is specified, an AWS KMS CMK ARN must be specified as the AwsKmsKey value. If Custom is specified, S3Object and EncryptionKeyProviderClass values must be specified.
"AwsKmsKey : "MyKeyARN" Required only when AwsKms is specified for Type. MyKeyARN must be a fully specified ARN to a key (for example, arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-456789012123).
"S3Object" : "JarLocation" Required only when CSE-Custom is specified for CertificateProviderType. JarLocation provides the location in Amazon S3 to a JAR file. The format can be a path (for example, s3://MyConfig/articfacts/MyKeyProvider.jar) or an ARN (for example, arn:aws:s3:::Code/MyKeyProvider.jar).

"EncryptionKeyProviderClass" : "MyLocalDiskKeyClassID"

Required only when Custom is specified for Type. MyLocalDiskKeyClassID specifies a full class name of a class declared in the application that implements the EncryptionMaterialsProvider interface; for example, com.mycompany.MyLocalDiskKeyProvider.

Using a Security Configuration to Specify Cluster Encryption Settings

You can specify encryption settings when you create a cluster by specifying the security configuration. You can use the AWS Management Console or the AWS CLI.

Specifying a Security Configuration Using the Console

When using the AWS console to create an Amazon EMR cluster, you choose the security configuration during Step 4: Security of the advanced options creation process.

  1. Sign in to the AWS Management Console and open the Amazon EMR console at https://console.aws.amazon.com/elasticmapreduce/.

  2. Choose Create cluster, Go to advanced options.

  3. On theStep 1: Software and Steps screen, from the Release list, choose emr-4.8.0 or a more recent release. Choose the settings you want and choose Next.

  4. On the Step 2: Hardware screen, choose the settings you want and choose Next. Do the same for Step 3: General Cluster Settings.

  5. On the Step 4: Security screen, under Encryption Options, choose a value for Security configuration.

  6. Configure other security options as desired and choose Create cluster.

Specifying a Security Configuration Using the CLI

When you use aws emr create-cluster, you can optionally apply a security configuration using --security-configuration MySecConfig, where MySecConfig is the name of the security configuration, as shown in the following example. The --release-label specified must be 4.8.0 or later and the --instance-type can be any available.

Copy
aws emr create-cluster --instance-type m3.xlarge --release-label emr-5.0.0 --security-configuration mySecConfig