AWS KMS Hierarchical keyrings
Our client-side encryption library was renamed to the AWS Database Encryption SDK. This developer guide still provides information on the DynamoDB Encryption Client. |
Note
As of July 24, 2023, branch keys created during the developer preview are not supported. Create new branch keys to continue using the branch key store that you created during the developer preview.
With the AWS KMS Hierarchical keyring, you can protect your cryptographic materials under a symmetric encryption KMS key without calling AWS KMS every time you encrypt or decrypt a record. It is a good choice for applications that need to minimize calls to AWS KMS, and applications that can reuse some cryptographic materials without violating their security requirements.
The Hierarchical keyring is a cryptographic materials caching solution that reduces the number of AWS KMS calls by using AWS KMS protected branch keys persisted in an Amazon DynamoDB table, and then locally caching branch key materials used in encrypt and decrypt operations. The DynamoDB table serves as the branch key store that manages and protects branch keys. It stores the active branch key and all previous versions of the branch key. The active branch key is the most recent branch key version. The Hierarchical keyring uses a unique data key to encrypt each field and encrypts each data key with a unique wrapping key derived from the active branch key. The Hierarchical keyring is dependent on the hierarchy established between active branch keys and their derived wrapping keys.
The Hierarchical keyring typically uses each branch key version to satisfy multiple requests. But you control the extent to which active branch keys are reused and determine how often the active branch key is rotated. The active version of the branch key remains active until you rotate it. Previous versions of the active branch key will not be used to perform encrypt operations, but they can still be queried and used in decrypt operations.
When you instantiate the Hierarchical keyring, it creates a local cache. You specify a cache limit that defines the maximum amount of time that
the branch key materials are stored within the local cache before they expire and are
evicted from the cache. The Hierarchical keyring makes one AWS KMS call to decrypt the branch key and
assemble the branch key materials the first time a branch-key-id
is specified
in an operation. Then, the branch key materials are stored in the local cache and reused for
all encrypt and decrypt operations that specify that branch-key-id
until the
cache limit expires. Storing branch key materials in the local cache reduces AWS KMS calls.
For example, consider a cache limit of 15 minutes. If you perform 10,000 encrypt operations
within that cache limit, the traditional
AWS KMS keyring would need to make 10,000 AWS KMS calls to satisfy 10,000 encrypt
operations. If you have one active branch-key-id
, the Hierarchical keyring only needs to
make one AWS KMS call to satisfy 10,000 encrypt operations.
The local cache consists of two partitions, one for encrypt operations and a second for decrypt operations. The encrypt partition stores the branch key materials assembled from the active branch key and reuses them for all encrypt operations until the cache limit expires. The decrypt partition stores the branch key materials assembled for other branch key versions identified in decrypt operations. The decryption partition can store multiple active branch key materials versions at a time. When it's configured to use a branch key ID supplier for a multitenant database, the encrypt partition can also store multiple branch key materials versions at a time. For more information, see Using the Hierarchical keyring with multitenant databases.
Note
All mentions of Hierarchical keyring in the AWS Database Encryption SDK refer to the AWS KMS Hierarchical keyring.
Topics
How it works
The following walkthroughs describe how the Hierarchical keyring assembles encryption and decryption materials, and the different calls that the keyring makes for encrypt and decrypt operations. For technical details on the wrapping key derivation and plaintext data key encryption processes, see AWS KMS Hierarchical keyring technical details.
Encrypt and sign
The following walkthrough describes how the Hierarchical keyring assembles encryption materials and derives a unique wrapping key.
-
The encryption method asks the Hierarchical keyring for encryption materials. The keyring generates a plaintext data key, then checks to see if there are valid branch materials in the local cache to generate the wrapping key. If there are valid branch key materials, the keyring proceeds to Step 5.
-
If there are no valid branch key materials, the Hierarchical keyring queries the branch key store for the active branch key.
-
The branch key store calls AWS KMS to decrypt the active branch key and returns the plaintext active branch key. Data identifying the active branch key is serialized to provide additional authenticated data (AAD) in the decrypt call to AWS KMS.
-
The branch key store returns the plaintext branch key and data that identifies it, such as the branch key version.
-
-
The Hierarchical keyring assembles branch key materials (the plaintext branch key and branch key version) and stores a copy of them in the local cache.
-
The Hierarchical keyring derives a unique wrapping key from the plaintext branch key and a 16-byte random salt. It uses the derived wrapping key to encrypt a copy of the plaintext data key.
The encryption method uses the encryption materials to encrypt and sign the record. For more information on how records are encrypted and signed in the AWS Database Encryption SDK, see Encrypt and sign.
Decrypt and verify
The following walkthrough describes how the Hierarchical keyring assembles decryption materials and decrypts the encrypted data key.
-
The decryption method identifies the encrypted data key from the material description field of the encrypted record, and passes it to the Hierarchical keyring.
-
The Hierarchical keyring deserializes data identifying the encrypted data key, including the branch key version, the 16-byte salt, and other information describing how the data key was encrypted.
For more information, see AWS KMS Hierarchical keyring technical details.
-
The Hierarchical keyring checks to see if there are valid branch key materials in the local cache that match the branch key version identified in Step 2. If there are valid branch key materials, the keyring proceeds to Step 6.
-
If there are no valid branch key materials, the Hierarchical keyring queries the branch key store for the branch key that matches the branch key version identified in Step 2.
-
The branch key store calls AWS KMS to decrypt the branch key and returns the plaintext active branch key. Data identifying the active branch key is serialized to provide additional authenticated data (AAD) in the decrypt call to AWS KMS.
-
The branch key store returns the plaintext branch key and data that identifies it, such as the branch key version.
-
-
The Hierarchical keyring assembles branch key materials (the plaintext branch key and branch key version) and stores a copy of them in the local cache.
-
The Hierarchical keyring uses the assembled branch key materials and the 16-byte salt identified in Step 2 to reproduce the unique wrapping key that encrypted the data key.
-
The Hierarchical keyring uses the reproduced wrapping key to decrypt the data key and returns the plaintext data key.
The decryption method uses the decryption materials and plaintext data key to decrypt and verify the record. For more information on how records are decrypted and verified in the AWS Database Encryption SDK, see Decrypt and verify.
Prerequisites
The AWS Database Encryption SDK doesn't require an AWS account and it doesn't depend on any AWS service. However, the Hierarchical keyring depends on AWS KMS and Amazon DynamoDB.
To use a Hierarchical keyring, you need a symmetric encryption AWS KMS key with kms:Decrypt permissions. You can also use a symmetric encryption multi-Region key. For detailed information about permissions for AWS KMS keys, see Authentication and access control in the AWS Key Management Service Developer Guide.
Before you can create and use a Hierarchical keyring, you must create your branch key store and populate it with your first active branch key.
- Step 1: Configure a new key store service
-
The key store service provides several operations, such as
CreateKeyStore
andCreateKey
, to help you assemble the Hierarchical keyring prerequisites and manage your branch key store.The following Java example creates a key store service. You must specify a DynamoDB table name to serve as the name of your branch key store, a logical name for the branch key store, and the KMS key ARN that identifies the KMS key that will protect your branch keys.
The logical key store name is cryptographically bound to all data stored in the table to simplify DynamoDB restore operations. The logical key store name can be the same as your DynamoDB table name, but it does not have to be. We strongly recommend specifying your DynamoDB table name as the logical table name when you first configure your key store service. You must always specify the same logical table name. In the event that your branch key store name changes after restoring your DynamoDB table from a backup, the logical key store name maps to the DynamoDB table name you specify to ensure that the Hierarchical keyring can still access your branch key store.
final KeyStore keystore = KeyStore.builder().KeyStoreConfig( KeyStoreConfig.builder() .ddbClient(DynamoDbClient.create()) .ddbTableName(
keyStoreName
) .logicalKeyStoreName(logicalKeyStoreName
) .kmsClient(KmsClient.create()) .kmsConfiguration(KMSConfiguration.builder() .kmsKeyArn(kmsKeyArn
) .build()) .build()).build(); - Step 2: Call
CreateKeyStore
to create a branch key store -
The following Java operation creates the branch key store that will persist and protect your branch keys.
keystore.CreateKeyStore(CreateKeyStoreInput.builder().build());
The
CreateKeyStore
operation creates a DynamoDB table with the table name you specified in Step 1 and the following required values.Partition key Sort key Base table branch-key-id
version
- Step 3: Call
CreateKey
to create a new active branch key -
The following Java operation creates a new active branch key using the KMS key you specified in Step 1, and adds the active branch key to the DynamoDB table you created in Step 2.
When you call
CreateKey
, you can choose to specify the following optional values.-
branchKeyIdentifier
: defines a custombranch-key-id
.To create a custom
branch-key-id
, you must also include an additional encryption context with theencryptionContext
parameter. -
encryptionContext
: defines an optional set of non-secret key–value pairs that provides additional authenticated data (AAD) in the encryption context included in the kms:GenerateDataKeyWithoutPlaintext call.This additional encryption context is displayed with the
aws-crypto-ec:
prefix.
final Map<String, String>
additionalEncryptionContext
= Collections.singletonMap("contextKey
", "contextValue
"); final String BranchKey = keystore.CreateKey( CreateKeyInput.builder() .branchKeyIdentifier(custom-branch-key-id
) //OPTIONAL .encryptionContext(additionalEncryptionContext
) //OPTIONAL .build()).branchKeyIdentifier();First, the
CreateKey
operation generates the following values.-
A version 4 Universally Unique Identifier
(UUID) for the branch-key-id
(unless you specified a custombranch-key-id
). -
A version 4 UUID for the branch key version
-
A
timestamp
in the ISO 8601 date and time formatin Coordinated Universal Time (UTC).
Then, the
CreateKey
operation calls kms:GenerateDataKeyWithoutPlaintext using the following request.{ "EncryptionContext": { "branch-key-id" : "
branch-key-id
", "type" : "type
", "create-time" : "timestamp
", "logical-key-store-name" : "the logical table name for your branch key store
", "kms-arn" :the KMS key ARN
, "hierarchy-version" : "1", "aws-crypto-ec:contextKey
": "contextValue
" }, "KeyId": "the KMS key ARN you specified in Step 1
", "NumberOfBytes": "32" }Note
The
CreateKey
operation creates an active branch key and a beacon key, even if you have not configured your database for searchable encryption. Both keys are stored in your branch key store. For more information, see Using the Hierarchical keyring for searchable encryption.Next, the
CreateKey
operation calls kms:ReEncrypt to create an active record for the branch key by updating the encryption context.Last, the
CreateKey
operation calls ddb:TransactWriteItems to write a new item that will persist the branch key in the table you created in Step 2. The item has the following attributes.{ "branch-key-id" :
branch-key-id
, "type" : "branch:ACTIVE", "enc" :the branch key returned by the GenerateDataKeyWithoutPlaintext call
, "version": "branch:version:the branch key version UUID
", "create-time" : "timestamp
", "kms-arn" : "the KMS key ARN you specified in Step 1
", "hierarchy-version" : "1", "aws-crypto-ec:contextKey
": "contextValue
" } -
Create a Hierarchical keyring
To initialize the Hierarchical keyring, you must provide the following values:
-
A branch key store name
The name of the DynamoDB table you created to serve as your branch key store.
-
A cache limit time to live (TTL)
The amount of time in seconds that a branch key materials entry within the local cache can be used before it expires. This value must be greater than zero. When the cache limit TTL expires, the entry is evicted from the local cache.
-
A branch key identifier
The
branch-key-id
that identifies the active branch key in your branch key store.Note
To initialize the Hierarchical keyring for multitenant use, you must specify a branch key ID supplier instead of a
branch-key-id
. For more information, see Using the Hierarchical keyring with multitenant databases. -
(Optional) A list of Grant Tokens
If you control access to the KMS key in your Hierarchical keyring with grants, you must provide all necessary grant tokens when you initialize the keyring.
The following Java example demonstrates how to initialize a Hierarchical keyring with the AWS Database Encryption SDK for DynamoDB client. The following example specifies a cache limit TTL of 600 seconds.
final MaterialProviders matProv = MaterialProviders.builder() .MaterialProvidersConfig(MaterialProvidersConfig.builder().build()) .build(); final CreateAwsKmsHierarchicalKeyringInput keyringInput = CreateAwsKmsHierarchicalKeyringInput.builder() .keyStore(
branchKeyStoreName
) .branchKeyId(branch-key-id
) .ttlSeconds(600
) .build(); final Keyring hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput);
Rotate your active branch key
There can only be one active version for each branch key at a time. The Hierarchical keyring typically uses each active branch key version to satisfy multiple requests. But you control the extent to which active branch keys are reused and determine how often the active branch key is rotated.
Branch keys are not used to encrypt plaintext data keys. They are used to derive the unique wrapping keys that encrypt plaintext data keys. The wrapping key derivation process produces a unique 32 byte wrapping key with 28 bytes of randomness. This means that a branch key can derive more than 79 octillion, or 296, unique wrapping keys before cryptographic wear-out occurs. Despite this very low exhaustion risk, you might be required to rotate your active branch keys due to business or contract rules or government regulations.
The active version of the branch key remains active until you rotate it. Previous versions of the active branch key will not be used to perform encrypt operations and cannot be used to derive new wrapping keys. But they can still be queried and provide wrapping keys to decrypt the data keys that they encrypted while active.
Use the key store service VersionKey
operation to rotate your active
branch key. When you rotate the active branch key, a new branch key is created to
replace the previous version. The branch-key-id
does not change when you
rotate the active branch key. You must specify the branch-key-id
that
identifies the current active branch key when you call VersionKey
.
keystore.VersionKey( VersionKeyInput.builder() .branchKeyIdentifier("
branch-key-id
") .build() );
Using the Hierarchical keyring with multitenant databases
You can use the key hierarchy established between active branch keys and their derived wrapping keys to support multitenant databases by creating a branch key for each tenant in your database. The Hierarchical keyring then encrypts and signs all of the data for a given tenant with their distinct branch key. This enables you to store multitenant data in a single database and isolate tenant data by branch key.
Each tenant has their own branch key that is defined by a unique
branch-key-id
. There can only be one active version of each
branch-key-id
at a time.
Before you can initialize your Hierarchical keyring for multitenant use, you must create a
branch key for each tenant and create a branch key ID supplier. Use the branch key ID
supplier to create a friendly name for your branch-key-ids
to make it easy
to recognize the correct branch-key-id
for a tenant. For example, the
friendly name lets you refer to a branch key as tenant1
instead of
b3f61619-4d35-48ad-a275-050f87e15122
.
For decrypt operations, you can either statically configure a single Hierarchical keyring to restrict decryption to a single tenant, or you can use the branch key ID supplier to identify which tenant is responsible for decrypting a record.
First, follow Step 1 and Step 2 of the Prerequisites procedures. Then, use the following procedures to create a branch key for each tenant, create a branch key ID supplier, and initialize your Hierarchical keyring for multitenant use.
- Step 1: Create a branch key for each tenant in your database
-
Call
CreateKey
for each tenant in your database.The following Java operation creates two branch keys using the KMS key you specified when creating your key store service, and adds the branch keys to the DynamoDB table you created to serve as your branch key store. The same KMS key must protect all branch keys.
final String
tenant1BranchKey
= keystore.CreateKey( CreateKeyInput.builder().build()).branchKeyIdentifier(); final String tenant2BranchKey = keystore.CreateKey( CreateKeyInput.builder().build()).branchKeyIdentifier(); - Step 2: Create a branch key ID supplier
-
The following Java example creates friendly names fro the two branch keys created in Step 1, and calls
CreateDynamoDbEncryptionBranchKeyIdSupplier
to create a branch key ID supplier with the AWS Database Encryption SDK for DynamoDB client.// Create friendly names for each branch-key-id class ExampleBranchKeyIdSupplier implements IDynamoDbKeyBranchKeyIdSupplier { private static String branchKeyIdForTenant1; private static String branchKeyIdForTenant2; public ExampleBranchKeyIdSupplier(String tenant1Id, String tenant2Id) { this.branchKeyIdForTenant1 = tenant1Id; this.branchKeyIdForTenant2 = tenant2Id; } // Create the branch key ID supplier final DynamoDbEncryption ddbEnc = DynamoDbEncryption.builder() .DynamoDbEncryptionConfig(DynamoDbEncryptionConfig.builder().build()) .build(); final BranchKeyIdSupplier branchKeyIdSupplier = ddbEnc.CreateDynamoDbEncryptionBranchKeyIdSupplier( CreateDynamoDbEncryptionBranchKeyIdSupplierInput.builder() .ddbKeyBranchKeyIdSupplier(new ExampleBranchKeyIdSupplier(
branch-key-ID-tenant1
,branch-key-ID-tenant2
)) .build()).branchKeyIdSupplier(); - Step 3: Initialize your Hierarchical keyring with the branch key ID supplier
-
To initialize the Hierarchical keyring you must provide the following values:
-
A branch key store name
-
A branch key ID supplier
-
(Optional) A cache
If you want to customize your cache type or the number of branch key materials entries that can be stored in the local cache, specify the cache type and entry capacity when you initialize the keyring.
Cache type defines the threading model. The Hierarchical keyring provides three cache types that support multitenant databases: Default, MultiThreaded, StormTracking.
If you do not specify a cache, the Hierarchical keyring automatically uses the Default cache type and sets the entry capacity to 1000.
-
(Optional) A list of Grant Tokens
If you control access to the KMS key in your Hierarchical keyring with grants, you must provide all necessary grant tokens when you initialize the keyring.
The following Java example initializes a Hierarchical keyring with the branch key ID supplier created in Step 2, a cache limit TLL of 600 seconds, and a maximum cache size of 1000. This example initializes a Hierarchical keyring with the AWS Database Encryption SDK for DynamoDB client.
final MaterialProviders matProv = MaterialProviders.builder() .MaterialProvidersConfig(MaterialProvidersConfig.builder().build()) .build(); final CreateAwsKmsHierarchicalKeyringInput keyringInput = CreateAwsKmsHierarchicalKeyringInput.builder() .keyStore(
branchKeyStoreName
) .branchKeyIdSupplier(branchKeyIdSupplier) .ttlSeconds(600
) .cache(CacheType.builder() //OPTIONAL .Default(DefaultCache.builder() .entryCapacity(100
) .build()) .build(); final Keyring hierarchicalKeyring = matProv.CreateAwsKmsHierarchicalKeyring(keyringInput); -
Using the Hierarchical keyring for searchable encryption
Searchable encryption enables you to search encrypted records without decrypting the entire database. This is accomplished by indexing the plaintext value of an encrypted field with a beacon. To implement searchable encryption, you must use a Hierarchical keyring.
The key store CreateKey
operation generates both a branch key and
beacon key. The branch key is used in record encryption and
decryption operations. The beacon key is used to generate beacons.
The branch key and beacon key are protected by the same AWS KMS key that you
specify when creating your key store service. After the CreateKey
operation
calls AWS KMS to generate the branch key, it calls kms:GenerateDataKeyWithoutPlaintext a second time to generate the beacon
key using the following request.
{ "EncryptionContext": { "branch-key-id" : "
branch-key-id
", "type" :type
, "create-time" : "timestamp
", "logical-key-store-name" : "the logical table name for your branch key store
", "kms-arn" :the KMS key ARN
, "hierarchy-version" : 1 }, "KeyId": "the KMS key ARN
", "NumberOfBytes": "32" }
After generating both keys, the CreateKey
operation calls ddb:TransactWriteItems to
write two new items that will persist the branch key and beacon key in your branch key
store.
When you configure a standard beacon,
the AWS Database Encryption SDK queries the branch key store for the beacon key. Then, it uses an
HMAC-based extract-and-expand key derivation function (HKDF
Unlike branch keys, there is only one beacon key version per
branch-key-id
in a branch key store. The beacon key is never
rotated.
Defining your beacon key source
When you define the beacon version for your
standard and compound beacons, you must identify the beacon key and define a cache
limit time to live (TTL) for the beacon key materials. Beacon key materials are
stored in a separate local cache from the branch keys. The following snippet
demonstrates how to define the keySource
for a single-tenant database.
Identify your beacon key by the branch-key-id
it is associated
with.
keySource(BeaconKeySource.builder() .single(SingleKeyStore.builder() .keyId(
branch-key-id
) .cacheTTL(6000
) .build()) .build())
- Defining beacon source in a multitenant database
-
If you have a multitenant database, you must specify the following values when configuring the
keySource
.-
keyFieldName
Defines the name of the field that stores the
branch-key-id
associated with the beacon key used to generated beacons for a given tenant. ThekeyFieldName
can be any string, but it must be unique to all other fields in your database. When you write new records to your database, thebranch-key-id
that identifies the beacon key used to generate any beacons for that record is stored in this field. You must include this field in your beacon queries and identify the appropriate beacon key materials required to recalculate the beacon. For more information, see Querying beacons in a multitenant database. -
cacheTTL
The amount of time in seconds that a beacon key materials entry within the local beacon cache can be used before it expires. This value must be greater than zero. When the cache limit TTL expires, the entry is evicted from the local cache.
-
(Optional) A cache
If you want to customize your cache type or the number of branch key materials entries that can be stored in the local cache, specify the cache type and entry capacity when you initialize the keyring.
Cache type defines the threading model. The Hierarchical keyring provides three cache types that support multitenant databases: Default, MultiThreaded, StormTracking.
If you do not specify a cache, the Hierarchical keyring automatically uses the Default cache type and sets the entry capacity to 1000.
keySource(BeaconKeySource.builder() .multi(MultiKeyStore.builder() .keyFieldName(
beaconKeys
) .cacheTTL(6000
) .cache(CacheType.builder() // OPTIONAL .Default(DefaultCache.builder() .entryCapacity(100) .build()) .build()) .build()) -