Discovering sensitive data with Amazon Macie - Amazon Macie

Discovering sensitive data with Amazon Macie

With Amazon Macie, you can automate discovery, logging, and reporting of sensitive data in your Amazon Simple Storage Service (Amazon S3) data estate. You can do this in two ways: by configuring Macie to perform automated sensitive data discovery for your account or organization, and by creating and running sensitive data discovery jobs for your account or organization.

Automated sensitive data discovery provides broad visibility into where sensitive data might reside in your Amazon S3 data estate. With this option, Macie continually evaluates your S3 bucket inventory and uses sampling techniques to identify and select representative S3 objects in your buckets. Macie then retrieves and analyzes the selected objects, inspecting them for sensitive data. For more information, see Performing automated sensitive data discovery.

Sensitive data discovery jobs provide deeper, more targeted analysis. With this option, you define the breadth and depth of the analysis—specific S3 buckets that you select or buckets that match specific criteria. You can refine the scope of the analysis by choosing additional options, such as custom criteria that derive from properties of S3 objects. You can also configure a job to run only once for on-demand analysis and assessment, or on a recurring basis for periodic analysis and assessment. For more information, see Running sensitive data discovery jobs.

With either option, you can analyze S3 objects by using managed data identifiers that Macie provides, custom data identifiers that you define, or a combination of the two:

  • A managed data identifier is a set of built-in criteria and techniques that are designed to detect a specific type of sensitive data—for example, credit card numbers, AWS secret access keys, or passport numbers for a particular country or region. These identifiers can detect a large and growing list of sensitive data types for many countries and regions, including multiple types of personally identifiable information (PII), credentials data, and financial data. For a complete list, see Using managed data identifiers.

  • A custom data identifier is a set of criteria that you define to detect sensitive data. The criteria consist of a regular expression (regex) that defines a text pattern to match and, optionally, character sequences and a proximity rule that refine the results. With custom data identifiers, you can detect sensitive data that reflects your organization's particular scenarios, intellectual property, or proprietary data—for example, employee IDs, customer account numbers, or internal data classifications. For more information, see Building custom data identifiers

To fine tune the analysis, you can also define and use allow lists. In Macie, an allow list specifies text or a text pattern to ignore in S3 objects, typically sensitive data exceptions for your particular scenarios or environment—for example, public names or phone numbers for your organization, or sample data that your organization uses for testing. For more information, see Defining sensitive data exceptions with allow lists.

Macie can analyze an S3 object if the following is true:

  • The object uses a supported file or storage format. For more information, see Supported file and storage formats.

  • If the object is encrypted, it’s encrypted with a key that Macie is allowed to use. For more information, see Analyzing encrypted S3 objects.

  • The object is stored directly in Amazon S3 and uses a supported storage class—S3 Intelligent-Tiering, S3 One Zone-IA, S3 Standard, or S3 Standard-IA. Macie can’t analyze data that’s stored in Amazon S3 Glacier or other AWS services.


    Although Macie is optimized for Amazon S3, you can use it to discover sensitive data that you currently store elsewhere. You can do this by moving the data to Amazon S3 temporarily or permanently. For example, export Amazon RDS or Amazon Aurora snapshots to Amazon S3 in Apache Parquet format. Or export an Amazon DynamoDB table to Amazon S3. You can then create a job to analyze the data in Amazon S3.

  • If the object is stored in a bucket that has a restrictive bucket policy, the policy allows Macie to access objects in the bucket. For more information, see Allowing Macie to access S3 buckets and objects.

To help you meet and maintain compliance with your data security and privacy requirements, Macie produces records of the sensitive data that it finds and the analysis that it performs—sensitive data findings and sensitive data discovery results. A sensitive data finding is a detailed report of sensitive data that Macie found in an object. A sensitive data discovery result is a record that logs details about the analysis of an object. Each type of record adheres to a standardized schema, which can help you query, monitor, and process the records by using other applications, services, and systems as necessary.