Menu
Amazon Macie
User Guide

Using the Macie Research Tab

You can use the Research tab in the Macie console to construct and run queries in the Query Parser and conduct in-depth investigative research of your Macie-monitored data and activity. You can navigate to the Research tab at any time and construct queries from scratch in the empty parser. For more information, see Constructing Queries in Macie. Or you can be redirected to the Research tab from various places throughout the Macie console, for example, any of the Dashboard views (see Using the Macie Dashboard) or the Basic alerts list (see Amazon Macie Alerts). When redirected to the Research tab from other places in the console, your data selection is translated into an automatically generated query that is displayed in the query parser.

Constructing Queries in Macie

Macie allows you to construct queries in the Query Parser in the Research tab. This Query Parser is a lexer which interprets a string into a Lucene Query using JavaCC. For more information about query syntax, see Apache Lucene - Query Parser Syntax.

The following are example queries for common searches:

  • You can use the following query to search for any console login not that did not originate from IP addresses owned by Amazon: eventNameIsp.compound:/ConsoleLogin:~(Amazon.*)/

  • You can use the following query to search for PII artifacts inside a public S3 bucket: filesystem_metadata.bucket:"my-public-bucket" AND (pii_impact:”moderate” OR pii_impact:”high”)

Research Filters

In the Macie Research tab, you can apply the following filters to your searches:

The first Research tab filter (pull-down list), with the pre-selected default value of Cloudtrail data, allows you to specifying the index (or the data repository) that you want Macie to search through. This filter includes the following options:

  • CloudTrail data - a collection of 5-minute aggregates of raw Cloudtrail data

  • S3 bucket properties - a collection of metadata about the S3 buckets that Macie is monitoring

  • S3 objects - a collection of metadata about the S3 objects that are stored in the buckets that Macie is monitoring

Number of Results to Display

The next Research tab filter with the pre-selected default value of Top 10, allows you to control the number of results to display when you do your initial search and the number of additional results to display if more results are available. This filter includes the following options:

  • Top 10

  • Top 50

  • Top 100

  • Top 500

Time Range

The third Research tab filter with the pre-selected default value of Past 30 days, allows you to define a time range for which you want to display your search results. This filter includes the following options:

  • Past 7 days

  • Past 30 days

  • Past 90 days

  • Past 365 days

  • All

  • Custom time range

Researching AWS CloudTrail Data

The following section describes the elements of the search results that get displayed when you use the Research tab to investigate your Macie-monitored CloudTrail data.

Complete the following steps in the Research tab:

  1. Select CloudTrail data in the first filter pull-down list.

  2. For this sample procedure, select Top 10 in the second filter pull-down list.

  3. For this sample procedure, select Past 90 days in the third filter pull-down list.

  4. Choose the button with the looking glass icon to start the search.

Your search produces the following elements:

  • The total number of results that matched your CloudTrail data search for the selected time range.

  • The graphical representation of CloudTrail data search results for the selected time range.

    Note

    If your data set is very large and you specify a very wide time range, you data might not render properly and this graph might not be displayed as one of the resulting elements of your search.

    Important

    You can use the graph to further narrow your search and generate and run a query that will produce a subset of the results generated by your original selections in the steps above. Double-click any of the graph's results and your selection is translated into a new query that's automatically displayed in the Query Parser, and the Research tab is refreshed with the results of this new query.

  • Search results summary - this is a list of the most significant fields from your search. The first line includes the top (or bottom) 3 values for each field. The second line includes the top (or bottom) 10 values for each field.

    Important

    You can use the fields in the search results summary to further narrow your search and generate and run a query that will produce a subset of the results generated by your original selections in the steps above. Choose the first or the second line of results for any field, and in the expanded results breakdown, choose the looking glass icon next to any of the results. Your choices are then translated into a new query that's automatically displayed in the Query Parser, and the Research tab is refreshed with the results of this new query.

  • A list of user sessions (5-minute aggregates of CloudTrail data) that match your search criteria. You can choose any user session to expand it and view its details.

The following table includes the complete list of fields that can appear in the results of your CloudTrail data searches. You can use this table to investigate the results of your CloudTrail data searches:

Field name Definition

IP address intelligence

Security-related intelligence on the source IP address of the request.

Objects read

ARNs of S3 objects that are being read by the event.

User agent

The agent through which the request was made.

Error code

The AWS service error if the request returns an error.

Resources

A list of resources accessed in the event.

Macie unique ID

A format that is unique to Macie for specifying users. Macie unique ID is a combination of the IAM UserIdentity element and the recipientAccountId.

Recipient account ID

The ID of the AWS account that received the event.

Account ID

The ID of the AWS account that owns the entity that granted permissions for the request.

IP location

Key pair of city and country that hosts the IP address of the request.

Principal ID

A unique identifier for the entity that made the request.

Source IP address

The IP address that the request was made from.

Objects deleted

ARNs of S3 objects that are being deleted by the event.

ISP

The ISP where the event originated.

Source IP address

IP address that the AWS request came from

Objects written

ARNs of S3 objects that are being written by the event.

Event type

The type of the event that generated the event record (for example, AwsApiCall, AwsServiceEvent, or ConsoleSignin).

Resource owner account ID

Account ID of the S3 bucket owner.

User identity type

The type of the identity that made the request.

Event source

The service that the request was made to.

Event name

The name of the event for which the request was made.

Session name

The name of the session that was created when the request was made through an assumed role.

Source ARN

The ARN used to make the request.

AWS region

The AWS region in which the request is made.

Event name (ISP)

The name of the event for which the request was made and where Macie was able to successfully resolve an ISP.

Timestamp

The timestamp of the request.

Event name by error code

The event name and the corresponding error code.

Count of unique objects deleted

The count of unique S3 objects deleted by the event.

Count of unique objects written

The count of unique S3 objects written by the event.

Count of unique objects read

The count of unique S3 objects read by the event.

Count of unique event names

The count of unique event names.

Earliest event timestamp

The timestamp of the earliest event in the user session.

Latest event timestamp

The timestamp of the latest event in the user session.

Researching S3 Bucket Properties Data

The following section describes the elements of the search results that get displayed when you use the Research tab to investigate your Macie-monitored S3 bucket properties data.

Complete the following steps in the Research tab:

  1. Select S3 bucket properties in the first filter pull-down list.

  2. For this sample procedure, select Top 10 in the second filter pull-down list.

  3. For this sample procedure, select Past 90 days in the third filter pull-down list.

  4. Choose the button with the looking glass icon to start the search.

Your search results contain the following elements:

  • The total number of results that matched your S3 bucket properties data search for the selected time range.

  • The graphical representation of the S3 bucket properties data search results for the selected time range.

    Note

    If your data set is very large and you specify a very wide time range, you data might not render properly and this graph might not be displayed as one of the resulting elements of your search.

    Important

    You can use the graph to further narrow your search and generate and run a query that will produce a subset of the results generated by your original selections in the steps above. Double-click any of the graph's results and your selection is translated into a new query that's automatically displayed in the Query Parser, and the Research tab is refreshed with the results of this new query.

  • Search results summary - this is a list of the most significant fields from your search. The first line includes the top (or bottom) 3 values for each field. The second line includes the top (or bottom) 10 values for each field.

    Important

    You can use the fields in the search results summary to further narrow your search and generate and run a query that will produce a subset of the results generated by your original selections in the steps above. Choose the first or the second line of results for any field, and in the expanded results breakdown, choose the looking glass icon next to any of the results. Your choices are then translated into a new query that's automatically displayed in the Query Parser, and the Research tab is refreshed with the results of this new query.

  • A list of S3 buckets that match your search criteria. You can choose any bucket to expand it and view its details.

The following table includes the complete list of fields that can appear in the results of your S3 buckets searches. You can use this table to investigate the results of your S3 buckets searches:

Field name Definition

S3 fully writable

True or false value (evaluated through an automated reasoning engine) indicative of unrestricted public 'write' access policy to the S3 bucket.

S3 fully readable

True or false value (evaluated through an automated reasoning engine) indicative of unrestricted public 'read' access policy to the S3 bucket.

Bucket tag value

The values in the tag set associated with the S3 bucket.

Bucket region

Specifies the AWS region where the S3 bucket resides.

Bucket permission

Permission given to the grantee for the S3 bucket.

Bucket tag key

The keys in the tag set associated with the S3 bucket.

Bucket grantee type

Describes the S3 bucket's grantee type, such as CanonicalUser or a Group.

Bucket grantee URI

Describes the S3 bucket's grantee URI, such as AllUsers or AuthenticatedAWSusers.

Bucket policy principal ARNs

A list of ARNs for the principals listed in the S3 bucket's IAM policy.

Bucket name

The name of the S3 bucket.

Bucket owner account ID

The ID of the AWS account that owns the S3 bucket.

Bucket owner name

The display name of the S3 bucket owner.

Bucket timestamp

The timestamp when Macie last updated bucket information.

Bucket IAM policy

The IAM policy document attached to the S3 bucket.

Bucket lifecycle configuration

Lifecycle rules attached to the S3 bucket.

Bucket ACL

ACL information attached to the S3 bucket.

Bucket logging configuration

Indicates whether logging is enabled on the S3 bucket.

Bucket versioning

The versioning state of the S3 bucket.

Bucket tagging

Contains the S3 bucket's TagSet and Tag elements.

Bucket activity

If you choose this field, your selection is automatically translated into a query displayed in the Query Parser. When you run this query, it generates all CloudTrail activity on this S3 bucket for the same time range as specified in your original search.

S3 bucket URL

A URL for the S3 bucket in the Amazon S3 console.

Researching S3 Objects Data

The following section describes the elements of the search results that get displayed when you use the Research tab to investigate your Macie-monitored S3 objects.

Complete the following steps in the Research tab:

  1. Select S3 objects in the first filter pull-down list.

  2. For this sample procedure, select Top 10 in the second filter pull-down list.

  3. For this sample procedure, select Past 90 days in the third filter pull-down list.

  4. Choose the button with the looking glass icon to start the search.

Your search results include the following elements:

  • The total number of results that matched your S3 objects search for the selected time range.

  • The graphical representation of the S3 objects search results for the selected time range.

    Note

    If your data set is very large and you specify a very wide time range, you data might not render properly and this graph might not be displayed as one of the resulting elements of your search.

    Important

    You can use the graph to further narrow your search and generate and run a query that will produce a subset of the results generated by your original selections in the steps above. Double-click any of the graph's results and your selection is translated into a new query that's automatically displayed in the Query Parser, and the Research tab is refreshed with the results of this new query.

  • Search results summary - this is a list of the most significant fields from your search. The first line includes the top (or bottom) 3 values for each field. The second line includes the top (or bottom) 10 values for each field.

    Important

    You can use the fields in the search results summary to further narrow your search and generate and run a query that will produce a subset of the results generated by your original selections in the steps above. Choose the first or the second line of results for any field, and in the expanded results breakdown, choose the looking glass icon next to any of the results. Your choices are then translated into a new query that's automatically displayed in the Query Parser, and the Research tab is refreshed with the results of this new query.

  • A list of S3 objects that match your search criteria. You can choose any S3 object to expand it and view its details.

The following table includes the complete list of fields that can appear in the results of your S3 objects searches. You can use this table to investigate the results of your S3 objects searches:

Field name Definition

Object PII artifacts

The PII artifacts discovered in the contents of the S3 object.

Grantee URI

The URI of the S3 object's grantee.

Bucket name

The name of the S3 bucket where the S3 object is stored.

Object file type

S3 object's file type based on its Macie-assigned file extension.

Account ID

The AWS account ID that owns the S3 object.

Last modified

The time when the S3 object was last modified.

Object theme risk

S3 object's risk level determined by its Macie-assigned theme.

Object PII details

The total counts of unique PII artifacts discovered in the contents of the S3 object.

Object key

The key name that uniquely identifies the object in the S3 bucket.

AWS region

The AWS region that hosts the S3 bucket where the object is stored.

Object MD5 digest

The base64-encoded 128-bit MD5 digest of the object.

Object ACL 

ACL information attached to the S3 object.

Object size

S3 object's size in bytes.

Object storage class

Storage class used for storing the S3 object.

Object matching keywords

The matching keywords discovered in the contents of the S3 object for the Macie-assigned theme.

Object KMS key

If the x-amz-server-side-encryption is present and has the value of aws:kms, this indicates the ID of the Key Management Service (KMS) master encryption key that was used for the object.

Matching REGEX

The matching keywords discovered in the contents of the S3 object for the Macie-assigned regex.

Object activity

If you choose this field, your selection is automatically translated into a query displayed in the Query Parser. When you run this query, it generates all CloudTrail activity on this S3 object for the same time range as specified in your original search.

S3 object URL

A URL for the S3 object in the Amazon S3 console.

Object cp CLI command

The cp CLI command to copy the S3 object to another location locally or in S3.

Object PII priority

The PII priority of the S3 object.

Object mimetypes

A mimetype match that describes the S3 object's content file type based on its header.

Bucket owner

The display name of the owner of the S3 bucket where the S3 object is stored.

Object language code

Then language code of the contents of the S3 object.

Object risk level

S3 object's Macie-assigned risk level.

Object encryption

The encryption standard used for encrypting the S3 object.

Object theme

S3 object's Macie-assigned theme.

Path prefix or folder

The first level prefix or folder of the S3 bucket where the S3 object is stored.

Object content type

S3 object's Macie-assigned content type.

Save a Query as an Alert

You can use the following procedure to save a query that is displayed in the query parser as a basic alert. For more information about basic alerts, see Amazon Macie Alerts.

  1. In the Macie console's Research tab, either autogenerate or construct a query in the query parser.

  2. Choose the Save query as alert icon.

  3. Fill out the Basic alert definition form and then choose Save. For more information, see Adding New and Editing Existing Custom Basic Alerts.

Favorite Queries

You can mark queries that you frequently run as favorite and quickly view a list of your favorite queries.

  1. In the Macie console's Research tab, either autogenerate or construct a query in the query parser.

  2. Choose the Mark query as favorite icon.

  3. Fill out the Favorite query definition form by specifying the name and the description for the favorite query, and then choose Save.

  4. To view the list of your favorite queries, in the Macie console's Research tab, choose the Favorite queries icon.