Create an account-wide data protection policy
You can use the CloudWatch Logs console or AWS CLI commands to create a data protection policy to mask sensitive data for all log groups in your account. Doing so affects both current log groups and log groups that you create in the future.
Important
Sensitive data is detected and masked when it is ingested into the log group. When you set a data protection policy, log events ingested to the log group before that time are not masked.
Console
To use the console to create an account-wide data protection policy
-
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
In the navigation pane, choose Settings. It is located near the bottom of the list.
Choose the Logs tab.
Choose Configure.
For Managed data identifiers, select the types of data that you want to audit and mask for all of your log groups. You can type in the selection box to find the identifiers that you want.
We recommend that you select only the data identifiers that are relevant for your log data and your business. Choosing many types of data can lead to false positives.
For details about which types of data that you can protect, see Types of data that you can protect.
(Optional) If you want to audit and mask other types of data by using custom data identifiers, choose Add custom data identifier. Then enter a name for the data type and the regular expression to use to search for that type of data in the log events. For more information, see Custom data identifiers.
A single data protection policy can include up to 10 custom data identifiers. Each regular expression that defines a custom data identifier must be 200 characters or fewer.
(Optional) Choose one or more services to send the audit findings to. Even if you choose not to send audit findings to any of these services, the sensitive data types that you select will still be masked.
Choose Activate data protection.
AWS CLI
To use the AWS CLI to create a data protection policy
Use a text editor to create a policy file named
DataProtectionPolicy.json
. For information about the policy syntax, see the following section.Enter the following command:
aws logs put-account-policy \ --policy-name TEST_POLICY --policy-type "DATA_PROTECTION_POLICY" \ --policy-document file://policy.json \ --scope "ALL" \ --region
us-west-2
Data protection policy syntax for AWS CLI or API operations
When you create a JSON data protection policy to use in an AWS CLI command or API operation, the policy must include two JSON blocks:
The first block must include both a
DataIdentifer
array and anOperation
property with anAudit
action. TheDataIdentifer
array lists the types of sensitive data that you want to mask. For more information about the available options, see Types of data that you can protect.The
Operation
property with anAudit
action is required to find the sensitive data terms. ThisAudit
action must contain aFindingsDestination
object. You can optionally use thatFindingsDestination
object to list one or more destinations to send audit findings reports to. If you specify destinations such as log groups, Amazon Data Firehose streams, and S3 buckets, they must already exist. For an example of an audit findins report, see Audit findings reports.The second block must include both a
DataIdentifer
array and anOperation
property with anDeidentify
action. TheDataIdentifer
array must exactly match theDataIdentifer
array in the first block of the policy.The
Operation
property with theDeidentify
action is what actually masks the data, and it must contain the"MaskConfig": {}
object. The"MaskConfig": {}
object must be empty.
The following is an example of a data protection policy using only managed data identifiers. This policy masks email addresses and United States driver's licenses.
For information about policies that specify custom data identifiers, see Using custom data identifiers in your data protection policy.
{ "Name": "data-protection-policy", "Description": "test description", "Version": "2021-06-01", "Statement": [{ "Sid": "audit-policy", "DataIdentifier": [ "arn:aws:dataprotection::aws:data-identifier/EmailAddress", "arn:aws:dataprotection::aws:data-identifier/DriversLicense-US" ], "Operation": { "Audit": { "FindingsDestination": { "CloudWatchLogs": { "LogGroup": "
EXISTING_LOG_GROUP_IN_YOUR_ACCOUNT
," }, "Firehose": { "DeliveryStream": "EXISTING_STREAM_IN_YOUR_ACCOUNT
" }, "S3": { "Bucket": "EXISTING_BUCKET
" } } } } }, { "Sid": "redact-policy", "DataIdentifier": [ "arn:aws:dataprotection::aws:data-identifier/EmailAddress", "arn:aws:dataprotection::aws:data-identifier/DriversLicense-US" ], "Operation": { "Deidentify": { "MaskConfig": {} } } } ] }