Data Sources - Search - Amazon Macie

Data Sources - Search

The Search Data Sources resource provides statistical data and other information about AWS resources that Amazon Macie monitors and analyzes for your account. The information includes quantitative breakdowns that indicate how much data Macie can analyze to detect sensitive data in a resource. It also indicates whether you configured any classification jobs to analyze data in a resource and, if so, the date and time when one of those jobs most recently ran. The data is available for all the AWS resources that Macie monitors and analyzes for your account.

Note that complete data is available for a resource only if Macie can retrieve and process information about the resource. If a resource's permissions settings or an error prevents Macie from retrieving and processing information about a resource, statistical data and other information is limited to a subset of the resource's properties, such as the name of the resource and the account ID for the AWS account that owns the resource.

In addition to querying data, you can use this resource to build, test, and refine Amazon Simple Storage Service (Amazon S3) bucket criteria that you want to use for new classification jobs. For existing classification jobs, you can use this resource to create a snapshot of the S3 buckets that a job would analyze if the job ran at a certain time—the same time that you query this resource using the bucket criteria for the job. The SearchResourcesBucketCriteria structure for the Search Data Sources resource is the same as the S3BucketCriteriaForJob structure for classification jobs.

You can use the Search Data Sources resource to query (retrieve) statistical data and other information about AWS resources that Macie monitors and analyzes for your account. To customize and refine your query, you can use supported parameters to specify whether and how to filter, sort, and paginate the query results. You can also use this resource to build and test S3 bucket criteria for classification jobs.

URI

/datasources/search-resources

HTTP methods

POST

Operation ID: SearchResources

Retrieves (queries) statistical data and other information about AWS resources that Amazon Macie monitors and analyzes.

Responses
Status code Response model Description
200 SearchResourcesResponse

The request succeeded.

400 ValidationException

The request failed because it contains a syntax error.

402 ServiceQuotaExceededException

The request failed because fulfilling the request would exceed one or more service quotas for your account.

403 AccessDeniedException

The request was denied because you don't have sufficient access to the specified resource.

404 ResourceNotFoundException

The request failed because the specified resource wasn't found.

409 ConflictException

The request failed because it conflicts with the current state of the specified resource.

429 ThrottlingException

The request failed because you sent too many requests during a certain amount of time.

500 InternalServerException

The request failed due to an unknown internal server error, exception, or failure.

Schemas

Request bodies

{ "nextToken": "string", "maxResults": integer, "bucketCriteria": { "excludes": { "and": [ { "tagCriterion": { "comparator": enum, "tagValues": [ { "value": "string", "key": "string" } ] }, "simpleCriterion": { "comparator": enum, "values": [ "string" ], "key": enum } } ] }, "includes": { "and": [ { "tagCriterion": { "comparator": enum, "tagValues": [ { "value": "string", "key": "string" } ] }, "simpleCriterion": { "comparator": enum, "values": [ "string" ], "key": enum } } ] } }, "sortCriteria": { "orderBy": enum, "attributeName": enum } }

Response bodies

{ "nextToken": "string", "matchingResources": [ { "matchingBucket": { "bucketName": "string", "objectCount": integer, "sizeInBytes": integer, "classifiableObjectCount": integer, "objectCountByEncryptionType": { "kmsManaged": integer, "s3Managed": integer, "customerManaged": integer, "unencrypted": integer, "unknown": integer }, "errorMessage": "string", "errorCode": enum, "classifiableSizeInBytes": integer, "unclassifiableObjectCount": { "total": integer, "storageClass": integer, "fileType": integer }, "accountId": "string", "unclassifiableObjectSizeInBytes": { "total": integer, "storageClass": integer, "fileType": integer }, "jobDetails": { "lastJobId": "string", "lastJobRunTime": "string", "isDefinedInJob": enum, "isMonitoredByJob": enum }, "sizeInBytesCompressed": integer } } ] }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }

Properties

AccessDeniedException

Provides information about an error that occurred due to insufficient access to a specified resource.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

BucketMetadataErrorCode

The error code for an error that prevented Amazon Macie from retrieving and processing information about an S3 bucket and the bucket's objects.

  • ACCESS_DENIED

ConflictException

Provides information about an error that occurred due to a versioning conflict for a specified resource.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

InternalServerException

Provides information about an error that occurred due to an unknown internal server error, exception, or failure.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

JobDetails

Specifies whether any one-time or recurring classification jobs are configured to analyze data in an S3 bucket, and, if so, the details of the job that ran most recently.

Property Type Required Description
lastJobId

string

False

The unique identifier for the job that ran most recently and is configured to analyze data in the bucket, either the latest run of a recurring job or the only run of a one-time job.

This value is typically null if the value for the isDefinedInJob property is FALSE or UNKNOWN.

lastJobRunTime

string

Format: date-time

False

The date and time, in UTC and extended ISO 8601 format, when the job (lastJobId) started. If the job is a recurring job, this value indicates when the most recent run started.

This value is typically null if the value for the isDefinedInJob property is FALSE or UNKNOWN.

isDefinedInJob

string

Values: TRUE | FALSE | UNKNOWN

False

Specifies whether any one-time or recurring jobs are configured to analyze data in the bucket. Possible values are:

  • TRUE - The bucket is explicitly included in the bucket definition (S3BucketDefinitionForJob) for one or more jobs and at least one of those jobs has a status other than CANCELLED. Or the bucket matched the bucket criteria (S3BucketCriteriaForJob) for at least one job that previously ran.

  • FALSE - The bucket isn't explicitly included in the bucket definition (S3BucketDefinitionForJob) for any jobs, all the jobs that explicitly include the bucket in their bucket definitions have a status of CANCELLED, or the bucket didn't match the bucket criteria (S3BucketCriteriaForJob) for any jobs that previously ran.

  • UNKNOWN - An exception occurred when Amazon Macie attempted to retrieve job data for the bucket.

isMonitoredByJob

string

Values: TRUE | FALSE | UNKNOWN

False

Specifies whether any recurring jobs are configured to analyze data in the bucket. Possible values are:

  • TRUE - The bucket is explicitly included in the bucket definition (S3BucketDefinitionForJob) for one or more recurring jobs or the bucket matches the bucket criteria (S3BucketCriteriaForJob) for one or more recurring jobs. At least one of those jobs has a status other than CANCELLED.

  • FALSE - The bucket isn't explicitly included in the bucket definition (S3BucketDefinitionForJob) for any recurring jobs, the bucket doesn't match the bucket criteria (S3BucketCriteriaForJob) for any recurring jobs, or all the recurring jobs that are configured to analyze data in the bucket have a status of CANCELLED.

  • UNKNOWN - An exception occurred when Amazon Macie attempted to retrieve job data for the bucket.

MatchingBucket

Provides statistical data and other information about an S3 bucket that Amazon Macie monitors and analyzes for your account. If an error occurs when Macie attempts to retrieve and process information about the bucket or the bucket's objects, the value for most of these properties is null. Exceptions are accountId and bucketName. To identify the cause of the error, refer to the errorCode and errorMessage values.

Property Type Required Description
bucketName

string

False

The name of the bucket.

objectCount

integer

Format: int64

False

The total number of objects in the bucket.

sizeInBytes

integer

Format: int64

False

The total storage size, in bytes, of the bucket.

If versioning is enabled for the bucket, Amazon Macie calculates this value based on the size of the latest version of each object in the bucket. This value doesn't reflect the storage size of all versions of each object in the bucket.

classifiableObjectCount

integer

Format: int64

False

The total number of objects that Amazon Macie can analyze in the bucket. These objects use a supported storage class and have a file name extension for a supported file or storage format.

objectCountByEncryptionType

ObjectCountByEncryptionType

False

The total number of objects that are in the bucket, grouped by server-side encryption type. This includes a grouping that reports the total number of objects that aren't encrypted or use client-side encryption.

errorMessage

string

False

A brief description of the error (errorCode) that prevented Amazon Macie from retrieving and processing information about the bucket and the bucket's objects. This value is null if Macie was able to retrieve and process the information.

errorCode

BucketMetadataErrorCode

False

Specifies the error code for an error that prevented Amazon Macie from retrieving and processing information about the bucket and the bucket's objects. If this value is ACCESS_DENIED, Macie doesn't have permission to retrieve the information. For example, the bucket has a restrictive bucket policy and Amazon S3 denied the request. If this value is null, Macie was able to retrieve and process the information.

classifiableSizeInBytes

integer

Format: int64

False

The total storage size, in bytes, of the objects that Amazon Macie can analyze in the bucket. These objects use a supported storage class and have a file name extension for a supported file or storage format.

If versioning is enabled for the bucket, Macie calculates this value based on the size of the latest version of each applicable object in the bucket. This value doesn't reflect the storage size of all versions of each applicable object in the bucket.

unclassifiableObjectCount

ObjectLevelStatistics

False

The total number of objects that Amazon Macie can't analyze in the bucket. These objects don't use a supported storage class or don't have a file name extension for a supported file or storage format.

accountId

string

False

The unique identifier for the AWS account that owns the bucket.

unclassifiableObjectSizeInBytes

ObjectLevelStatistics

False

The total storage size, in bytes, of the objects that Amazon Macie can't analyze in the bucket. These objects don't use a supported storage class or don't have a file name extension for a supported file or storage format.

jobDetails

JobDetails

False

Specifies whether any one-time or recurring classification jobs are configured to analyze objects in the bucket, and, if so, the details of the job that ran most recently.

sizeInBytesCompressed

integer

Format: int64

False

The total storage size, in bytes, of the objects that are compressed (.gz, .gzip, .zip) files in the bucket.

If versioning is enabled for the bucket, Amazon Macie calculates this value based on the size of the latest version of each applicable object in the bucket. This value doesn't reflect the storage size of all versions of each applicable object in the bucket.

MatchingResource

Provides statistical data and other information about an AWS resource that Amazon Macie monitors and analyzes for your account.

Property Type Required Description
matchingBucket

MatchingBucket

False

The details of an S3 bucket that Amazon Macie monitors and analyzes.

ObjectCountByEncryptionType

Provides information about the number of objects that are in an S3 bucket and use certain types of server-side encryption, use client-side encryption, or aren't encrypted.

Property Type Required Description
kmsManaged

integer

Format: int64

False

The total number of objects that are encrypted with an AWS KMS key, either an AWS managed key or a customer managed key. The objects use AWS KMS encryption (SSE-KMS).

s3Managed

integer

Format: int64

False

The total number of objects that are encrypted with an Amazon S3 managed key. The objects use Amazon S3 managed encryption (SSE-S3).

customerManaged

integer

Format: int64

False

The total number of objects that are encrypted with a customer-provided key. The objects use customer-provided server-side encryption (SSE-C).

unencrypted

integer

Format: int64

False

The total number of objects that aren't encrypted or use client-side encryption.

unknown

integer

Format: int64

False

The total number of objects that Amazon Macie doesn't have current encryption metadata for. Macie can't provide current data about the encryption settings for these objects.

ObjectLevelStatistics

Provides information about the total storage size (in bytes) or number of objects that Amazon Macie can't analyze in one or more S3 buckets. In a BucketMetadata or MatchingBucket object, this data is for a specific bucket. In a GetBucketStatisticsResponse object, this data is aggregated for the buckets in the query results. If versioning is enabled for a bucket, total storage size values are based on the size of the latest version of each applicable object in the bucket.

Property Type Required Description
total

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects use an unsupported storage class or don't have a file name extension for a supported file or storage format.

storageClass

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects use an unsupported storage class.

fileType

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects don't have a file name extension for a supported file or storage format.

ResourceNotFoundException

Provides information about an error that occurred because a specified resource wasn't found.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

SearchResourcesBucketCriteria

Specifies property- and tag-based conditions that define filter criteria for including or excluding S3 buckets from the query results. Exclude conditions take precedence over include conditions.

Property Type Required Description
excludes

SearchResourcesCriteriaBlock

False

The property- and tag-based conditions that determine which buckets to exclude from the results.

includes

SearchResourcesCriteriaBlock

False

The property- and tag-based conditions that determine which buckets to include in the results.

SearchResourcesComparator

The operator to use in a condition that filters the results of a query. Valid values are:

  • EQ

  • NE

SearchResourcesCriteria

Specifies a property- or tag-based filter condition for including or excluding AWS resources from the query results.

Property Type Required Description
tagCriterion

SearchResourcesTagCriterion

False

A tag-based condition that defines an operator and tag keys, tag values, or tag key and value pairs for including or excluding resources from the results.

simpleCriterion

SearchResourcesSimpleCriterion

False

A property-based condition that defines a property, operator, and one or more values for including or excluding resources from the results.

SearchResourcesCriteriaBlock

Specifies property- and tag-based conditions that define filter criteria for including or excluding AWS resources from the query results.

Property Type Required Description
and

Array of type SearchResourcesCriteria

False

An array of objects, one for each property- or tag-based condition that includes or excludes resources from the query results. If you specify more than one condition, Amazon Macie uses AND logic to join the conditions.

SearchResourcesRequest

Specifies criteria for filtering, sorting, and paginating the results of a query for statistical data and other information about AWS resources that Amazon Macie monitors and analyzes.

Property Type Required Description
nextToken

string

False

The nextToken string that specifies which page of results to return in a paginated response.

maxResults

integer

Format: int32

False

The maximum number of items to include in each page of the response. The default value is 50.

bucketCriteria

SearchResourcesBucketCriteria

False

The filter conditions that determine which S3 buckets to include or exclude from the query results.

sortCriteria

SearchResourcesSortCriteria

False

The criteria to use to sort the results.

SearchResourcesResponse

Provides the results of a query that retrieved statistical data and other information about AWS resources that Amazon Macie monitors and analyzes for your account.

Property Type Required Description
nextToken

string

False

The string to use in a subsequent request to get the next page of results in a paginated response. This value is null if there are no additional pages.

matchingResources

Array of type MatchingResource

False

An array of objects, one for each resource that meets the filter criteria specified in the request.

SearchResourcesSimpleCriterion

Specifies a property-based filter condition that determines which AWS resources are included or excluded from the query results.

Property Type Required Description
comparator

SearchResourcesComparator

False

The operator to use in the condition. Valid values are EQ (equals) and NE (not equals).

values

Array of type string

False

An array that lists one or more values to use in the condition. If you specify multiple values, Amazon Macie uses OR logic to join the values. Valid values for each supported property (key) are:

  • ACCOUNT_ID - A string that represents the unique identifier for the AWS account that owns the resource.

  • S3_BUCKET_EFFECTIVE_PERMISSION - A string that represents an enumerated value that Macie defines for the BucketPublicAccess.effectivePermission property of an S3 bucket.

  • S3_BUCKET_NAME - A string that represents the name of an S3 bucket.

  • S3_BUCKET_SHARED_ACCESS - A string that represents an enumerated value that Macie defines for the BucketMetadata.sharedAccess property of an S3 bucket.

Values are case sensitive. Also, Macie doesn't support use of partial values or wildcard characters in values.

key

SearchResourcesSimpleCriterionKey

False

The property to use in the condition.

SearchResourcesSimpleCriterionKey

The property to use in a condition that filters the query results. Valid values are:

  • ACCOUNT_ID

  • S3_BUCKET_NAME

  • S3_BUCKET_EFFECTIVE_PERMISSION

  • S3_BUCKET_SHARED_ACCESS

SearchResourcesSortAttributeName

The property to sort the query results by. Valid values are:

  • ACCOUNT_ID

  • RESOURCE_NAME

  • S3_CLASSIFIABLE_OBJECT_COUNT

  • S3_CLASSIFIABLE_SIZE_IN_BYTES

SearchResourcesSortCriteria

Specifies criteria for sorting the results of a query for information about AWS resources that Amazon Macie monitors and analyzes.

Property Type Required Description
orderBy

string

Values: ASC | DESC

False

The sort order to apply to the results, based on the value for the property specified by the attributeName property. Valid values are: ASC, sort the results in ascending order; and, DESC, sort the results in descending order.

attributeName

SearchResourcesSortAttributeName

False

The property to sort the results by.

SearchResourcesTagCriterion

Specifies a tag-based filter condition that determines which AWS resources are included or excluded from the query results.

Property Type Required Description
comparator

SearchResourcesComparator

False

The operator to use in the condition. Valid values are EQ (equals) and NE (not equals).

tagValues

Array of type SearchResourcesTagCriterionPair

False

The tag keys, tag values, or tag key and value pairs to use in the condition.

SearchResourcesTagCriterionPair

Specifies a tag key, a tag value, or a tag key and value (as a pair) to use in a tag-based filter condition for a query. Tag keys and values are case sensitive. Also, Amazon Macie doesn't support use of partial values or wildcard characters in tag-based filter conditions.

Property Type Required Description
value

string

False

The tag value to use in the condition.

key

string

False

The value for the tag key to use in the condition.

ServiceQuotaExceededException

Provides information about an error that occurred due to one or more service quotas for an account.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

ThrottlingException

Provides information about an error that occurred because too many requests were sent during a certain amount of time.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

ValidationException

Provides information about an error that occurred due to a syntax error in a request.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

See also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

SearchResources