AWS KMS Hierarchical keyrings

Important

The AWS KMS Hierarchical keyring is only supported by version 4.x of the AWS Encryption SDK for .NET and version 3.x of the AWS Encryption SDK for Java.

With the AWS KMS Hierarchical keyring, you can protect your cryptographic materials under a symmetric encryption KMS key without calling AWS KMS every time you encrypt or decrypt data. It is a good choice for applications that need to minimize calls to AWS KMS, and applications that can reuse some cryptographic materials without violating their security requirements.

The Hierarchical keyring is a cryptographic materials caching solution that reduces the number of AWS KMS calls by using AWS KMS protected branch keys persisted in an Amazon DynamoDB table, and then locally caching branch key materials used in encrypt and decrypt operations. The DynamoDB table serves as the key store that manages and protects branch keys. It stores the active branch key and all previous versions of the branch key. The active branch key is the most recent branch key version. The Hierarchical keyring uses a unique data key to encrypt each message and encrypts each data encryption key for each encrypt request and encrypts each data encryption key with a unique wrapping key derived from the active branch key. The Hierarchical keyring is dependent on the hierarchy established between active branch keys and their derived wrapping keys.

The Hierarchical keyring typically uses each branch key version to satisfy multiple requests. But you control the extent to which active branch keys are reused and determine how often the active branch key is rotated. The active version of the branch key remains active until you rotate it. Previous versions of the active branch key will not be used to perform encrypt operations, but they can still be queried and used in decrypt operations.

When you instantiate the Hierarchical keyring, it creates a local cache. You specify a cache limit that defines the maximum amount of time that the branch key materials are stored within the local cache before they expire and are evicted from the cache. The Hierarchical keyring makes one AWS KMS call to decrypt the branch key and assemble the branch key materials the first time a branch-key-id is specified in an operation. Then, the branch key materials are stored in the local cache and reused for all encrypt and decrypt operations that specify that branch-key-id until the cache limit expires. Storing branch key materials in the local cache reduces AWS KMS calls. For example, consider a cache limit of 15 minutes. If you perform 10,000 encrypt operations within that cache limit, the traditional AWS KMS keyring would need to make 10,000 AWS KMS calls to satisfy 10,000 encrypt operations. If you have one active branch-key-id, the Hierarchical keyring only needs to make one AWS KMS call to satisfy 10,000 encrypt operations.

The local cache separates encryption materials from decryption materials. The encryption materials are assembled from the active branch key and reused for all encrypt operations until the cache limit expires. The decryption materials are assembled from the branch key ID and version that is identified in the encrypted field's metadata, and they are reused for all decrypt operations related to the branch key ID and version until the cache limit expires. The local cache can store multiple versions of the same branch key at a time. When the local cache is configured to use a branch key ID supplier, it can also store branch key materials from multiple active branch keys at a time.

Note

All mentions of Hierarchical keyring in the AWS Encryption SDK refer to the AWS KMS Hierarchical keyring.

How it works

The following walkthroughs describe how the Hierarchical keyring assembles encryption and decryption materials, and the different calls that the keyring makes for encrypt and decrypt operations. For technical details on the wrapping key derivation and plaintext data key encryption processes, see AWS KMS Hierarchical keyring technical details.

Encrypt and sign

The following walkthrough describes how the Hierarchical keyring assembles encryption materials and derives a unique wrapping key.

1. The encryption method asks the Hierarchical keyring for encryption materials. The keyring generates a plaintext data key, then checks to see if there are valid branch materials in the local cache to generate the wrapping key. If there are valid branch key materials, the keyring proceeds to Step 4.

2. If there are no valid branch key materials, the Hierarchical keyring queries the key store for the active branch key.

   1. The key store calls AWS KMS to decrypt the active branch key and returns the plaintext active branch key. Data identifying the active branch key is serialized to provide additional authenticated data (AAD) in the decrypt call to AWS KMS.

   2. The key store returns the plaintext branch key and data that identifies it, such as the branch key version.

3. The Hierarchical keyring assembles branch key materials (the plaintext branch key and branch key version) and stores a copy of them in the local cache.

4. The Hierarchical keyring derives a unique wrapping key from the plaintext branch key and a 16-byte random salt. It uses the derived wrapping key to encrypt a copy of the plaintext data key.

The encryption method uses the encryption materials to encrypt the data. For more information, see How the AWS Encryption SDK encrypts data.

Show moreShow less

Decrypt and verify

The following walkthrough describes how the Hierarchical keyring assembles decryption materials and decrypts the encrypted data key.

1. The decryption method identifies the encrypted data key from the encrypted message, and passes it to the Hierarchical keyring.

2. The Hierarchical keyring deserializes data identifying the encrypted data key, including the branch key version, the 16-byte salt, and other information describing how the data key was encrypted.

   For more information, see AWS KMS Hierarchical keyring technical details.

3. The Hierarchical keyring checks to see if there are valid branch key materials in the local cache that match the branch key version identified in Step 2. If there are valid branch key materials, the keyring proceeds to Step 6.

4. If there are no valid branch key materials, the Hierarchical keyring queries the key store for the branch key that matches the branch key version identified in Step 2.

   1. The key store calls AWS KMS to decrypt the branch key and returns the plaintext active branch key. Data identifying the active branch key is serialized to provide additional authenticated data (AAD) in the decrypt call to AWS KMS.

   2. The key store returns the plaintext branch key and data that identifies it, such as the branch key version.

5. The Hierarchical keyring assembles branch key materials (the plaintext branch key and branch key version) and stores a copy of them in the local cache.

6. The Hierarchical keyring uses the assembled branch key materials and the 16-byte salt identified in Step 2 to reproduce the unique wrapping key that encrypted the data key.

7. The Hierarchical keyring uses the reproduced wrapping key to decrypt the data key and returns the plaintext data key.

The decryption method uses the decryption materials and plaintext data key to decrypt the encrypted message. For more information , see How the AWS Encryption SDK decrypts an encrypted message.

Show moreShow less

Prerequisites

Before you create and use a Hierarchical keyring, ensure the following prerequisites are met.

• You, or your key store administrator, have created a key store and created at least one active branch key.

• You have configured your key store actions.

  Note

  How you configure your key store actions determines what operations you can perform and what KMS keys the Hierarchical keyring can use. For more information, see Key store actions.

• You have the required AWS KMS permissions to access and use the key store and branch keys. For more information, see Required permissions.

• You have reviewed the supported cache types and configured the cache type that best fits your needs. For more information, see Choose a cache

Required permissions

The AWS Encryption SDK doesn't require an AWS account and it doesn't depend on any AWS service. However, to use an Hierarchical keyring, you need an AWS account and the following minimum permissions on the symmetric encryption AWS KMS key(s) in your key store.

• To encrypt and decrypt data with the Hierarchical keyring, you need kms:Decrypt.

• To create and rotate branch keys, you need kms:GenerateDataKeyWithoutPlaintext and kms:ReEncrypt.

For more information on controlling access to your branch keys and key store, see Implementing least privileged permissions.

Choose a cache

The Hierarchical keyring reduces the number of calls made to AWS KMS by locally caching the branch key materials used in encrypt and decrypt operations. Before you create your Hierarchical keyring, you need to decide what type of cache you want to use. You can use the default cache or customize the cache to best fits your needs.

The Hierarchical keyring supports the following cache types:

• Default cache

• MultiThreaded cache

• StormTracking cache

• Shared cache

Default cache

For most users, the Default cache fulfills their threading requirements. The Default cache is designed to support heavily multithreaded environments. When a branch key materials entry expires, the Default cache prevents multiple threads from calling AWS KMS by notifying one thread that the branch key materials entry is going to expire 10 seconds in advance. This ensures that only one thread sends a request to AWS KMS to refresh the cache.

The Default and StormTracking caches support the same threading model, but you only need to specify the entry capacity to use the Default cache. For more granular cache customizations, use the StormTracking cache.

Unless you want to customize the number of branch key materials entries that can be stored in the local cache, you do not need to specify a cache type when you create the Hierarchical keyring. If you do not specify a cache type, the Hierarchical keyring uses the Default cache type and sets the entry capacity to 1000.

To customize the Default cache, specify the following values:

• Entry capacity: limits the number of branch key materials entries that can be stored in the local cache.

Java

.cache(CacheType.builder()
        .Default(DefaultCache.builder()
        .entryCapacity(100)
        .build())

C# / .NET

CacheType defaultCache = new CacheType
{
    Default = new DefaultCache{EntryCapacity = 100}
};

MultiThreaded cache

The MultiThreaded cache is safe to use in multithreaded environments, but it does not provide any functionality to minimize AWS KMS or Amazon DynamoDB calls. As a result, when a branch key materials entry expires, all threads will be notified at the same time. This can result in multiple AWS KMS calls to refresh the cache.

To use the MultiThreaded cache, specify the following values:

• Entry capacity: limits the number of branch key materials entries that can be stored in the local cache.

• Entry pruning tail size: defines the number of entries to prune if the entry capacity is reached.

Java

.cache(CacheType.builder()
        .MultiThreaded(MultiThreadedCache.builder()
        .entryCapacity(100)
        .entryPruningTailSize(1)                                        
        .build())

C# / .NET

CacheType multithreadedCache = new CacheType
{
    MultiThreaded = new MultiThreadedCache
    {
        EntryCapacity = 100,
        EntryPruningTailSize = 1
    }
};

StormTracking cache

The StormTracking cache is designed to support heavily multithreaded environments. When a branch key materials entry expires, the StormTracking cache prevents multiple threads from calling AWS KMS by notifying one thread that the branch key materials entry is going to expire in advance. This ensures that only one thread sends a request to AWS KMS to refresh the cache.

To use the StormTracking cache, specify the following values:

• Entry capacity: limits the number of branch key materials entries that can be stored in the local cache.

  Default value: 1000 entries

• Entry pruning tail size: defines the number of branch key materials entries to prune at a time.

  Default value: 1 entry

• Grace period: defines the number of seconds before expiration that an attempt to refresh branch key materials is made.

  Default value: 10 seconds

• Grace interval: defines the number of seconds between attempts to refresh the branch key materials.

  Default value: 1 second

• Fan out: defines the number of simultaneous attempts that can be made to refresh the branch key materials.

  Default value: 20 attempts

• In flight time to live (TTL): defines the number of seconds until an attempt to refresh the branch key materials times out. Any time the cache returns NoSuchEntry in response to a GetCacheEntry, that branch key is considered to be in flight until the same key is written with a PutCache entry.

  Default value: 20 seconds

• Sleep: defines the number of seconds that a thread should sleep if the fanOut is exceeded.

  Default value: 20 milliseconds

Java

.cache(CacheType.builder()
        .StormTracking(StormTrackingCache.builder()
        .entryCapacity(100)
        .entryPruningTailSize(1)
        .gracePeriod(10)
        .graceInterval(1)
        .fanOut(20) 
        .inFlightTTL(20)
        .sleepMilli(20)                                        
        .build())

C# / .NET

CacheType stormTrackingCache = new CacheType
{
    StormTracking = new StormTrackingCache
    {
        EntryCapacity = 100,
        EntryPruningTailSize = 1,
        FanOut = 20,
        GraceInterval = 1,
        GracePeriod = 10,
        InFlightTTL = 20,
        SleepMilli = 20
    }
};

Shared cache

By default, the Hierarchical keyring creates a new local cache every time you instantiate the keyring. However, the Shared cache can help conserve memory by enabling you to share a cache across multiple Hierarchical keyrings. Rather than creating a new cryptographic materials cache for each Hierarchical keyring you instantiate, the Shared cache stores only one cache in memory, which can be used by all the Hierarchical keyrings that reference it. The Shared cache helps optimize memory usage by avoiding the duplication of cryptographic materials across keyrings. Instead, the Hierarchical keyrings can access the same underlying cache, reducing the overall memory footprint.

When you create your Shared cache, you still define the cache type. You can specify a Default cache, MultiThreaded cache, or StormTracking cache as the cache type, or substitute any compatible custom cache.

Partitions

Multiple Hierarchical keyrings can use a single Shared cache. When you create a Hierarchical keyring with a Shared cache you can define an optional partition ID. The partition ID distinguishes which Hierarchical keyring is writing to the cache. If two Hierarchical keyrings reference the same partition ID, logical key store name, and branch key ID the two keyrings will share the same cache entries in the cache. If you create two Hierarchical keyrings with the same Shared cache, but different partition IDs, each keyring will only access the cache entries from its own designated partition within the Shared cache. The partitions act as logical divisions within the shared cache, allowing each Hierarchical keyring to operate independently on its own designated partition, without interfering with the data stored in the other partition.

If you intend to reuse or share the cache entries in a partition, you must define your own partition ID. When you pass the partition ID to your Hierarchical keyring, the keyring can reuse the cache entries that are already present in the Shared cache, rather than having to retrieve and re-authorize the branch key materials again. If you do not specify a partition ID, a unique partition ID is automatically assigned to the keyring each time you instantiate the Hierarchical keyring.

The following procedures demonstrate how to create a Shared cache with the Default cache type and pass it to a Hierarchical keyring.

1. Create a CryptographicMaterialsCache (CMC) using the Material Providers Library (MPL).

   Java

   // Instantiate the MPL
   final MaterialProviders matProv =
       MaterialProviders.builder()
           .MaterialProvidersConfig(MaterialProvidersConfig.builder().build())
           .build();
   
   // Create a CacheType object for the Default cache
   final CacheType cache =
       CacheType.builder() 
           .Default(DefaultCache.builder().entryCapacity(100).build())
           .build();
   
   // Create a CMC using the default cache
   final CreateCryptographicMaterialsCacheInput cryptographicMaterialsCacheInput =
       CreateCryptographicMaterialsCacheInput.builder()
           .cache(cache)
           .build();
   
   final ICryptographicMaterialsCache sharedCryptographicMaterialsCache =
       matProv.CreateCryptographicMaterialsCache(cryptographicMaterialsCacheInput);

   C# / .NET

   // Instantiate the MPL
   var materialProviders = new MaterialProviders(new MaterialProvidersConfig());
   
   // Create a CacheType object for the Default cache
   var cache = new CacheType { Default = new DefaultCache{EntryCapacity = 100} };
   
   // Create a CMC using the default cache
   var cryptographicMaterialsCacheInput = new CreateCryptographicMaterialsCacheInput {Cache = cache};
   
   var sharedCryptographicMaterialsCache = materialProviders.CreateCryptographicMaterialsCache(cryptographicMaterialsCacheInput);

2. Create a CacheType object for the Shared cache.

   Pass the sharedCryptographicMaterialsCache you created in Step 1 to the new CacheType object.

   Java

   // Create a CacheType object for the sharedCryptographicMaterialsCache
   final CacheType sharedCache =
       CacheType.builder()
           .Shared(sharedCryptographicMaterialsCache)
           .build();

   C# / .NET

   // Create a CacheType object for the sharedCryptographicMaterialsCache
   var sharedCache = new CacheType { Shared = sharedCryptographicMaterialsCache };

3. Pass the sharedCache object from Step 2 to your Hierarchical keyring.

   When you create a Hierarchical keyring with a Shared cache, you can optionally define a partitionID to share cache entries across multiple Hierarchical keyrings. If you do not specify a partition ID, the Hierarchical keyring automatically assigns the keyring a unique partition ID.

   Note

   Your Hierarchical keyrings will share the same cache entries in a Shared cache if you create two or more keyrings that reference the same partition ID, logical key store name, and branch key ID. If you do not want multiple keyrings to share the same cache entries, you must use a unique partition ID for each Hierarchical keyring.

   The following example creates a Hierarchical keyring with a branch key ID supplier, and a cache limit of 600 seconds. For more information on the values defined in following Hierarchical keyring configuration, see Create a Hierarchical keyring.

   Java

   // Create the Hierarchical keyring
   final CreateAwsKmsHierarchicalKeyringInput keyringInput =
       CreateAwsKmsHierarchicalKeyringInput.builder()
           .keyStore(keystore)
           .branchKeyIdSupplier(branchKeyIdSupplier)
           .ttlSeconds(600)
           .cache(sharedCache)
           .partitionID(partitionID)
           .build();
   final IKeyring hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput);

   C# / .NET

   // Create the Hierarchical keyring        
   var createKeyringInput = new CreateAwsKmsHierarchicalKeyringInput
   {
      KeyStore = keystore,
      BranchKeyIdSupplier = branchKeyIdSupplier,
      Cache = sharedCache,
      TtlSeconds = 600,
      PartitionId = partitionID
   };
   var keyring = materialProviders.CreateAwsKmsHierarchicalKeyring(createKeyringInput);

Show moreShow less

Create a Hierarchical keyring

To create a Hierarchical keyring, you must provide the following values:

• A key store name

  The name of the DynamoDB table you, or your key store administrator, created to serve as your key store.

• A cache limit time to live (TTL)

  The amount of time in seconds that a branch key materials entry within the local cache can be used before it expires. The cache limit TTL dictates how often the client calls AWS KMS to authorize use of the branch keys. This value must be greater than zero. After the cache limit TTL expires, the entry is never served, and will be evicted from the local cache.

• A branch key identifier

  You can either statically configure the branch-key-id that identifies a single active branch key in your key store, or provide a branch key ID supplier.

  The branch key ID supplier uses the fields stored in the encryption context to determine which branch key is required to decrypt a record.

  We strongly recommend using a branch key ID supplier for multitenant databases where each tenant has their own branch key. You can use the branch key ID supplier to create a friendly name for your branch key IDs to make it easy to recognize the correct branch key ID for a specific tenant. For example, the friendly name lets you refer to a branch key as tenant1 instead of b3f61619-4d35-48ad-a275-050f87e15122.

  For decrypt operations, you can either statically configure a single Hierarchical keyring to restrict decryption to a single tenant, or you can use the branch key ID supplier to identify which tenant is responsible for decrypting a record.

• (Optional) A cache

  If you want to customize your cache type or the number of branch key materials entries that can be stored in the local cache, specify the cache type and entry capacity when you initialize the keyring.

  The Hierarchical keyring supports the following cache types: Default, MultiThreaded, StormTracking, and Shared. For more information and examples demonstrating how to define each cache type, see Choose a cache.

  If you do not specify a cache, the Hierarchical keyring automatically uses the Default cache type and sets the entry capacity to 1000.

• (Optional) A partition ID

  If you specify the Shared cache, you can optionally define a partition ID. The partition ID distinguishes which Hierarchical keyring is writing to the cache. If you intend to reuse or share the cache entries in a partition, you must define your own partition ID. You can specify any string for the partition ID. If you do not specify a partition ID, a unique partition ID is automatically assigned to the keyring at creation.

  For more information, see Partitions.

  Note

  Your Hierarchical keyrings will share the same cache entries in a Shared cache if you create two or more keyrings that reference the same partition ID, logical key store name, and branch key ID. If you do not want multiple keyrings to share the same cache entries, you must use a unique partition ID for each Hierarchical keyring.

• (Optional) A list of Grant Tokens

  If you control access to the KMS key in your Hierarchical keyring with grants, you must provide all necessary grant tokens when you initialize the keyring.

Create a Hierarchical keyring with a static branch key ID

The following examples demonstrate how to create a Hierarchical keyring with a static branch key ID, the Default cache, and a cache limit TTL of 600 seconds.

Java

final MaterialProviders matProv = MaterialProviders.builder()
        .MaterialProvidersConfig(MaterialProvidersConfig.builder().build())
        .build();
final CreateAwsKmsHierarchicalKeyringInput keyringInput = CreateAwsKmsHierarchicalKeyringInput.builder()
        .keyStore(branchKeyStoreName)
        .branchKeyId(branch-key-id)
        .ttlSeconds(600)
        .build();
final Keyring hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput);

C# / .NET

var matProv = new MaterialProviders(new MaterialProvidersConfig());
var keyringInput = new CreateAwsKmsHierarchicalKeyringInput
{
   KeyStore = keystore,
   BranchKeyIdSupplier = branchKeyIdSupplier,
   TtlSeconds = 600
};
var hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput);

Create a Hierarchical keyring with a branch key ID supplier

The following procedures demonstrate how to create a Hierarchical keyring with a branch key ID supplier.

1. Create a branch key ID supplier

   The following example creates friendly names for the two branch keys created in Step 1, and calls CreateDynamoDbEncryptionBranchKeyIdSupplier to create a branch key ID supplier.

   Java

   // Create friendly names for each branch-key-id 
   class ExampleBranchKeyIdSupplier implements IDynamoDbKeyBranchKeyIdSupplier {
       private static String branchKeyIdForTenant1;
       private static String branchKeyIdForTenant2;
   
       public ExampleBranchKeyIdSupplier(String tenant1Id, String tenant2Id) {
           this.branchKeyIdForTenant1 = tenant1Id;
           this.branchKeyIdForTenant2 = tenant2Id;
       }
   // Create the branch key ID supplier    
   final DynamoDbEncryption ddbEnc = DynamoDbEncryption.builder()
           .DynamoDbEncryptionConfig(DynamoDbEncryptionConfig.builder().build())
           .build();
   final BranchKeyIdSupplier branchKeyIdSupplier = ddbEnc.CreateDynamoDbEncryptionBranchKeyIdSupplier(
       CreateDynamoDbEncryptionBranchKeyIdSupplierInput.builder()
               .ddbKeyBranchKeyIdSupplier(new ExampleBranchKeyIdSupplier(branch-key-ID-tenant1, branch-key-ID-tenant2))
               .build()).branchKeyIdSupplier();

   C# / .NET

   // Create friendly names for each branch-key-id
    class ExampleBranchKeyIdSupplier : DynamoDbKeyBranchKeyIdSupplierBase {
       private String _branchKeyIdForTenant1;
       private String _branchKeyIdForTenant2;
   
       public ExampleBranchKeyIdSupplier(String tenant1Id, String tenant2Id) {
           this._branchKeyIdForTenant1 = tenant1Id;
           this._branchKeyIdForTenant2 = tenant2Id;
       }    
   // Create the branch key ID supplier
   var ddbEnc = new DynamoDbEncryption(new DynamoDbEncryptionConfig());
   var branchKeyIdSupplier = ddbEnc.CreateDynamoDbEncryptionBranchKeyIdSupplier(
       new CreateDynamoDbEncryptionBranchKeyIdSupplierInput
       {
           DdbKeyBranchKeyIdSupplier = new ExampleBranchKeyIdSupplier(branch-key-ID-tenant1, branch-key-ID-tenant2)
       }).BranchKeyIdSupplier;

2. Create a Hierarchical keyring

   The following examples initialize a Hierarchical keyring with the branch key ID supplier created in Step 1, a cache limit TLL of 600 seconds, and a maximum cache size of 1000.

   Java

   final MaterialProviders matProv = MaterialProviders.builder()
           .MaterialProvidersConfig(MaterialProvidersConfig.builder().build())
           .build();
   final CreateAwsKmsHierarchicalKeyringInput keyringInput = CreateAwsKmsHierarchicalKeyringInput.builder()
           .keyStore(keystore)
           .branchKeyIdSupplier(branchKeyIdSupplier)
           .ttlSeconds(600)
           .cache(CacheType.builder() //OPTIONAL
                   .Default(DefaultCache.builder()
                   .entryCapacity(100)
                   .build())
           .build();
   final Keyring hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput);

   C# / .NET

   var matProv = new MaterialProviders(new MaterialProvidersConfig());
   var keyringInput = new CreateAwsKmsHierarchicalKeyringInput
   {
      KeyStore = keystore,
      BranchKeyIdSupplier = branchKeyIdSupplier,
      TtlSeconds = 600, 
      Cache = new CacheType
      {
           Default = new DefaultCache { EntryCapacity = 100 }
      }
   };
   var hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput);