Data Sources - Search - Amazon Macie

Data Sources - Search

The Search Data Sources resource provides statistical data and other information about AWS resources that Amazon Macie monitors and analyzes for your account. The information includes quantitative breakdowns that indicate how much data Macie can analyze to detect sensitive data in a resource, and whether and when that analysis occurred. The data is available for all the AWS resources that Macie monitors and analyzes for your account. If you're the Macie administrator for an organization, this includes resources that your member accounts own.

Note that complete data is available for a resource only if Macie can retrieve and process information about the resource. If a resource's permissions settings or an error prevents Macie from retrieving and processing information about a resource, the available statistical data and other information is limited. It includes only a subset of the resource's properties, such as the name of the resource and the account ID for the AWS account that owns the resource.

In addition to querying data, you can use the Search Data Sources resource to build, test, and refine runtime criteria for new classification jobs. These criteria can determine which Amazon Simple Storage Service (Amazon S3) general purpose buckets a job analyzes when it runs. For existing classification jobs, you can use this resource to create a snapshot of the S3 general purpose buckets that currently match the criteria. This is because the SearchResourcesBucketCriteria structure for this resource is the same as the S3BucketCriteriaForJob structure for classification jobs. The exception is the automatedDiscoveryMonitoringStatus field. Jobs don't support use of that field in runtime criteria. To learn more about specifying runtime criteria for jobs, see Scope options for jobs in the Amazon Macie User Guide.

You can use the Search Data Sources resource to query (retrieve) statistical data and other information about AWS resources that Macie monitors and analyzes for your account. To customize and refine your query, you can use the supported parameters to specify how to filter, sort, and paginate the query results. You can also use this resource to build and test S3 bucket criteria for classification jobs.

URI

/datasources/search-resources

HTTP methods

POST

Operation ID: SearchResources

Retrieves (queries) statistical data and other information about AWS resources that Amazon Macie monitors and analyzes.

Responses
Status codeResponse modelDescription
200SearchResourcesResponse

The request succeeded.

400ValidationException

The request failed because the input doesn't satisfy the constraints specified by the service.

402ServiceQuotaExceededException

The request failed because fulfilling the request would exceed one or more service quotas for your account.

403AccessDeniedException

The request was denied because you don't have sufficient access to the specified resource.

404ResourceNotFoundException

The request failed because the specified resource wasn't found.

409ConflictException

The request failed because it conflicts with the current state of the specified resource.

429ThrottlingException

The request failed because you sent too many requests during a certain amount of time.

500InternalServerException

The request failed due to an unknown internal server error, exception, or failure.

Schemas

Request bodies

{ "bucketCriteria": { "excludes": { "and": [ { "simpleCriterion": { "comparator": enum, "key": enum, "values": [ "string" ] }, "tagCriterion": { "comparator": enum, "tagValues": [ { "key": "string", "value": "string" } ] } } ] }, "includes": { "and": [ { "simpleCriterion": { "comparator": enum, "key": enum, "values": [ "string" ] }, "tagCriterion": { "comparator": enum, "tagValues": [ { "key": "string", "value": "string" } ] } } ] } }, "maxResults": integer, "nextToken": "string", "sortCriteria": { "attributeName": enum, "orderBy": enum } }

Response bodies

{ "matchingResources": [ { "matchingBucket": { "accountId": "string", "automatedDiscoveryMonitoringStatus": enum, "bucketName": "string", "classifiableObjectCount": integer, "classifiableSizeInBytes": integer, "errorCode": enum, "errorMessage": "string", "jobDetails": { "isDefinedInJob": enum, "isMonitoredByJob": enum, "lastJobId": "string", "lastJobRunTime": "string" }, "lastAutomatedDiscoveryTime": "string", "objectCount": integer, "objectCountByEncryptionType": { "customerManaged": integer, "kmsManaged": integer, "s3Managed": integer, "unencrypted": integer, "unknown": integer }, "sensitivityScore": integer, "sizeInBytes": integer, "sizeInBytesCompressed": integer, "unclassifiableObjectCount": { "fileType": integer, "storageClass": integer, "total": integer }, "unclassifiableObjectSizeInBytes": { "fileType": integer, "storageClass": integer, "total": integer } } } ], "nextToken": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }
{ "message": "string" }

Properties

AccessDeniedException

Provides information about an error that occurred due to insufficient access to a specified resource.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

AutomatedDiscoveryMonitoringStatus

Specifies whether automated sensitive data discovery is currently configured to analyze objects in an S3 bucket. Possible values are:

  • MONITORED

  • NOT_MONITORED

BucketMetadataErrorCode

The error code for an error that prevented Amazon Macie from retrieving and processing information about an S3 bucket and the bucket's objects.

  • ACCESS_DENIED

ConflictException

Provides information about an error that occurred due to a versioning conflict for a specified resource.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

InternalServerException

Provides information about an error that occurred due to an unknown internal server error, exception, or failure.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

JobDetails

Specifies whether any one-time or recurring classification jobs are configured to analyze objects in an S3 bucket, and, if so, the details of the job that ran most recently.

PropertyTypeRequiredDescription
isDefinedInJob

string

Values: TRUE | FALSE | UNKNOWN

False

Specifies whether any one-time or recurring jobs are configured to analyze objects in the bucket. Possible values are:

  • TRUE - The bucket is explicitly included in the bucket definition (S3BucketDefinitionForJob) for one or more jobs and at least one of those jobs has a status other than CANCELLED. Or the bucket matched the bucket criteria (S3BucketCriteriaForJob) for at least one job that previously ran.

  • FALSE - The bucket isn't explicitly included in the bucket definition (S3BucketDefinitionForJob) for any jobs, all the jobs that explicitly include the bucket in their bucket definitions have a status of CANCELLED, or the bucket didn't match the bucket criteria (S3BucketCriteriaForJob) for any jobs that previously ran.

  • UNKNOWN - An exception occurred when Amazon Macie attempted to retrieve job data for the bucket.

isMonitoredByJob

string

Values: TRUE | FALSE | UNKNOWN

False

Specifies whether any recurring jobs are configured to analyze objects in the bucket. Possible values are:

  • TRUE - The bucket is explicitly included in the bucket definition (S3BucketDefinitionForJob) for one or more recurring jobs or the bucket matches the bucket criteria (S3BucketCriteriaForJob) for one or more recurring jobs. At least one of those jobs has a status other than CANCELLED.

  • FALSE - The bucket isn't explicitly included in the bucket definition (S3BucketDefinitionForJob) for any recurring jobs, the bucket doesn't match the bucket criteria (S3BucketCriteriaForJob) for any recurring jobs, or all the recurring jobs that are configured to analyze data in the bucket have a status of CANCELLED.

  • UNKNOWN - An exception occurred when Amazon Macie attempted to retrieve job data for the bucket.

lastJobId

string

False

The unique identifier for the job that ran most recently and is configured to analyze objects in the bucket, either the latest run of a recurring job or the only run of a one-time job.

This value is typically null if the value for the isDefinedInJob property is FALSE or UNKNOWN.

lastJobRunTime

string

Format: date-time

False

The date and time, in UTC and extended ISO 8601 format, when the job (lastJobId) started. If the job is a recurring job, this value indicates when the most recent run started.

This value is typically null if the value for the isDefinedInJob property is FALSE or UNKNOWN.

MatchingBucket

Provides statistical data and other information about an S3 bucket that Amazon Macie monitors and analyzes for your account. By default, object count and storage size values include data for object parts that are the result of incomplete multipart uploads. For more information, see How Macie monitors Amazon S3 data security in the Amazon Macie User Guide.

If an error occurs when Macie attempts to retrieve and process information about the bucket or the bucket's objects, the value for most of these properties is null. Key exceptions are accountId and bucketName. To identify the cause of the error, refer to the errorCode and errorMessage values.

PropertyTypeRequiredDescription
accountId

string

False

The unique identifier for the AWS account that owns the bucket.

automatedDiscoveryMonitoringStatus

AutomatedDiscoveryMonitoringStatus

False

Specifies whether automated sensitive data discovery is currently configured to analyze objects in the bucket. Possible values are: MONITORED, the bucket is included in analyses; and, NOT_MONITORED, the bucket is excluded from analyses. If automated sensitive data discovery is disabled for your account, this value is NOT_MONITORED.

bucketName

string

False

The name of the bucket.

classifiableObjectCount

integer

Format: int64

False

The total number of objects that Amazon Macie can analyze in the bucket. These objects use a supported storage class and have a file name extension for a supported file or storage format.

classifiableSizeInBytes

integer

Format: int64

False

The total storage size, in bytes, of the objects that Amazon Macie can analyze in the bucket. These objects use a supported storage class and have a file name extension for a supported file or storage format.

If versioning is enabled for the bucket, Macie calculates this value based on the size of the latest version of each applicable object in the bucket. This value doesn't reflect the storage size of all versions of each applicable object in the bucket.

errorCode

BucketMetadataErrorCode

False

The error code for an error that prevented Amazon Macie from retrieving and processing information about the bucket and the bucket's objects. If this value is ACCESS_DENIED, Macie doesn't have permission to retrieve the information. For example, the bucket has a restrictive bucket policy and Amazon S3 denied the request. If this value is null, Macie was able to retrieve and process the information.

errorMessage

string

False

A brief description of the error (errorCode) that prevented Amazon Macie from retrieving and processing information about the bucket and the bucket's objects. This value is null if Macie was able to retrieve and process the information.

jobDetails

JobDetails

False

Specifies whether any one-time or recurring classification jobs are configured to analyze objects in the bucket, and, if so, the details of the job that ran most recently.

lastAutomatedDiscoveryTime

string

Format: date-time

False

The date and time, in UTC and extended ISO 8601 format, when Amazon Macie most recently analyzed objects in the bucket while performing automated sensitive data discovery. This value is null if automated sensitive data discovery is disabled for your account.

objectCount

integer

Format: int64

False

The total number of objects in the bucket.

objectCountByEncryptionType

ObjectCountByEncryptionType

False

The total number of objects in the bucket, grouped by server-side encryption type. This includes a grouping that reports the total number of objects that aren't encrypted or use client-side encryption.

sensitivityScore

integer

Format: int32

False

The sensitivity score for the bucket, ranging from -1 (classification error) to 100 (sensitive).

If automated sensitive data discovery has never been enabled for your account or it's been disabled for your organization or your standalone account for more than 30 days, possible values are: 1, the bucket is empty; or, 50, the bucket stores objects but it's been excluded from recent analyses.

sizeInBytes

integer

Format: int64

False

The total storage size, in bytes, of the bucket.

If versioning is enabled for the bucket, Amazon Macie calculates this value based on the size of the latest version of each object in the bucket. This value doesn't reflect the storage size of all versions of each object in the bucket.

sizeInBytesCompressed

integer

Format: int64

False

The total storage size, in bytes, of the objects that are compressed (.gz, .gzip, .zip) files in the bucket.

If versioning is enabled for the bucket, Amazon Macie calculates this value based on the size of the latest version of each applicable object in the bucket. This value doesn't reflect the storage size of all versions of each applicable object in the bucket.

unclassifiableObjectCount

ObjectLevelStatistics

False

The total number of objects that Amazon Macie can't analyze in the bucket. These objects don't use a supported storage class or don't have a file name extension for a supported file or storage format.

unclassifiableObjectSizeInBytes

ObjectLevelStatistics

False

The total storage size, in bytes, of the objects that Amazon Macie can't analyze in the bucket. These objects don't use a supported storage class or don't have a file name extension for a supported file or storage format.

MatchingResource

Provides statistical data and other information about an AWS resource that Amazon Macie monitors and analyzes for your account.

PropertyTypeRequiredDescription
matchingBucket

MatchingBucket

False

The details of an S3 bucket that Amazon Macie monitors and analyzes.

ObjectCountByEncryptionType

Provides information about the number of objects that are in an S3 bucket and use certain types of server-side encryption, use client-side encryption, or aren't encrypted.

PropertyTypeRequiredDescription
customerManaged

integer

Format: int64

False

The total number of objects that are encrypted with customer-provided keys. The objects use server-side encryption with customer-provided keys (SSE-C).

kmsManaged

integer

Format: int64

False

The total number of objects that are encrypted with AWS KMS keys, either AWS managed keys or customer managed keys. The objects use dual-layer server-side encryption or server-side encryption with AWS KMS keys (DSSE-KMS or SSE-KMS).

s3Managed

integer

Format: int64

False

The total number of objects that are encrypted with Amazon S3 managed keys. The objects use server-side encryption with Amazon S3 managed keys (SSE-S3).

unencrypted

integer

Format: int64

False

The total number of objects that use client-side encryption or aren't encrypted.

unknown

integer

Format: int64

False

The total number of objects that Amazon Macie doesn't have current encryption metadata for. Macie can't provide current data about the encryption settings for these objects.

ObjectLevelStatistics

Provides information about the total storage size (in bytes) or number of objects that Amazon Macie can't analyze in one or more S3 buckets. In a BucketMetadata or MatchingBucket object, this data is for a specific bucket. In a GetBucketStatisticsResponse object, this data is aggregated for all the buckets in the query results. If versioning is enabled for a bucket, storage size values are based on the size of the latest version of each applicable object in the bucket.

PropertyTypeRequiredDescription
fileType

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects don't have a file name extension for a supported file or storage format.

storageClass

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects use an unsupported storage class.

total

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects use an unsupported storage class or don't have a file name extension for a supported file or storage format.

ResourceNotFoundException

Provides information about an error that occurred because a specified resource wasn't found.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

SearchResourcesBucketCriteria

Specifies property- and tag-based conditions that define filter criteria for including or excluding S3 buckets from the query results. Exclude conditions take precedence over include conditions.

PropertyTypeRequiredDescription
excludes

SearchResourcesCriteriaBlock

False

The property- and tag-based conditions that determine which buckets to exclude from the results.

includes

SearchResourcesCriteriaBlock

False

The property- and tag-based conditions that determine which buckets to include in the results.

SearchResourcesComparator

The operator to use in a condition that filters the results of a query. Valid values are:

  • EQ

  • NE

SearchResourcesCriteria

Specifies a property- or tag-based filter condition for including or excluding AWS resources from the query results.

PropertyTypeRequiredDescription
simpleCriterion

SearchResourcesSimpleCriterion

False

A property-based condition that defines a property, operator, and one or more values for including or excluding resources from the results.

tagCriterion

SearchResourcesTagCriterion

False

A tag-based condition that defines an operator and tag keys, tag values, or tag key and value pairs for including or excluding resources from the results.

SearchResourcesCriteriaBlock

Specifies property- and tag-based conditions that define filter criteria for including or excluding AWS resources from the query results.

PropertyTypeRequiredDescription
and

Array of type SearchResourcesCriteria

False

An array of objects, one for each property- or tag-based condition that includes or excludes resources from the query results. If you specify more than one condition, Amazon Macie uses AND logic to join the conditions.

SearchResourcesRequest

Specifies criteria for filtering, sorting, and paginating the results of a query for statistical data and other information about AWS resources that Amazon Macie monitors and analyzes.

PropertyTypeRequiredDescription
bucketCriteria

SearchResourcesBucketCriteria

False

The filter conditions that determine which S3 buckets to include or exclude from the query results.

maxResults

integer

Format: int32

False

The maximum number of items to include in each page of the response. The default value is 50.

nextToken

string

False

The nextToken string that specifies which page of results to return in a paginated response.

sortCriteria

SearchResourcesSortCriteria

False

The criteria to use to sort the results.

SearchResourcesResponse

Provides the results of a query that retrieved statistical data and other information about AWS resources that Amazon Macie monitors and analyzes for your account.

PropertyTypeRequiredDescription
matchingResources

Array of type MatchingResource

False

An array of objects, one for each resource that matches the filter criteria specified in the request.

nextToken

string

False

The string to use in a subsequent request to get the next page of results in a paginated response. This value is null if there are no additional pages.

SearchResourcesSimpleCriterion

Specifies a property-based filter condition that determines which AWS resources are included or excluded from the query results.

PropertyTypeRequiredDescription
comparator

SearchResourcesComparator

False

The operator to use in the condition. Valid values are EQ (equals) and NE (not equals).

key

SearchResourcesSimpleCriterionKey

False

The property to use in the condition.

values

Array of type string

False

An array that lists one or more values to use in the condition. If you specify multiple values, Amazon Macie uses OR logic to join the values. Valid values for each supported property (key) are:

  • ACCOUNT_ID - A string that represents the unique identifier for the AWS account that owns the resource.

  • AUTOMATED_DISCOVERY_MONITORING_STATUS - A string that represents an enumerated value that Macie defines for the BucketMetadata.automatedDiscoveryMonitoringStatus property of an S3 bucket.

  • S3_BUCKET_EFFECTIVE_PERMISSION - A string that represents an enumerated value that Macie defines for the BucketPublicAccess.effectivePermission property of an S3 bucket.

  • S3_BUCKET_NAME - A string that represents the name of an S3 bucket.

  • S3_BUCKET_SHARED_ACCESS - A string that represents an enumerated value that Macie defines for the BucketMetadata.sharedAccess property of an S3 bucket.

Values are case sensitive. Also, Macie doesn't support use of partial values or wildcard characters in values.

SearchResourcesSimpleCriterionKey

The property to use in a condition that filters the query results. Valid values are:

  • ACCOUNT_ID

  • S3_BUCKET_NAME

  • S3_BUCKET_EFFECTIVE_PERMISSION

  • S3_BUCKET_SHARED_ACCESS

  • AUTOMATED_DISCOVERY_MONITORING_STATUS

SearchResourcesSortAttributeName

The property to sort the query results by. Valid values are:

  • ACCOUNT_ID

  • RESOURCE_NAME

  • S3_CLASSIFIABLE_OBJECT_COUNT

  • S3_CLASSIFIABLE_SIZE_IN_BYTES

SearchResourcesSortCriteria

Specifies criteria for sorting the results of a query for information about AWS resources that Amazon Macie monitors and analyzes.

PropertyTypeRequiredDescription
attributeName

SearchResourcesSortAttributeName

False

The property to sort the results by.

orderBy

string

Values: ASC | DESC

False

The sort order to apply to the results, based on the value for the property specified by the attributeName property. Valid values are: ASC, sort the results in ascending order; and, DESC, sort the results in descending order.

SearchResourcesTagCriterion

Specifies a tag-based filter condition that determines which AWS resources are included or excluded from the query results.

PropertyTypeRequiredDescription
comparator

SearchResourcesComparator

False

The operator to use in the condition. Valid values are EQ (equals) and NE (not equals).

tagValues

Array of type SearchResourcesTagCriterionPair

False

The tag keys, tag values, or tag key and value pairs to use in the condition.

SearchResourcesTagCriterionPair

Specifies a tag key, a tag value, or a tag key and value (as a pair) to use in a tag-based filter condition for a query. Tag keys and values are case sensitive. Also, Amazon Macie doesn't support use of partial values or wildcard characters in tag-based filter conditions.

PropertyTypeRequiredDescription
key

string

False

The value for the tag key to use in the condition.

value

string

False

The tag value to use in the condition.

ServiceQuotaExceededException

Provides information about an error that occurred due to one or more service quotas for an account.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

ThrottlingException

Provides information about an error that occurred because too many requests were sent during a certain amount of time.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

ValidationException

Provides information about an error that occurred due to a syntax error in a request.

PropertyTypeRequiredDescription
message

string

False

The explanation of the error that occurred.

See also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

SearchResources