Data Sources - S3 - Amazon Macie

Data Sources - S3

The S3 Data Sources resource provides statistical data and other information about the Amazon Simple Storage Service (Amazon S3) buckets that you configured Amazon Macie to monitor and analyze. This includes data such as the number of objects that are in an S3 bucket and how many of those objects Macie can analyze. This also includes information about the settings that define who can access data in an S3 bucket and how that data can be accessed. The data is available for all the S3 buckets that you configured Macie to monitor and analyze.

You can use this resource to retrieve (query) statistical data and other information about the settings and contents of one or more S3 buckets that Macie monitors and analyzes. To customize and refine a query, you can use supported parameters that specify whether and how to filter, sort, and paginate the query results.

URI

/datasources/s3

HTTP Methods

POST

Operation ID: DescribeBuckets

Retrieves (queries) statistical data and other information about one or more S3 buckets that Amazon Macie monitors and analyzes.

Responses
Status Code Response Model Description
200 DescribeBucketsResponse

The request succeeded.

400 ValidationException

The request failed because it contains a syntax error.

402 ServiceQuotaExceededException

The request failed because fulfilling the request would exceed one or more service quotas for your account.

403 AccessDeniedException

The request was denied because you don't have sufficient access to the specified resource.

404 ResourceNotFoundException

The request failed because the specified resource wasn't found.

409 ConflictException

The request failed because it conflicts with the current state of the specified resource.

429 ThrottlingException

The request failed because you sent too many requests during a certain amount of time.

500 InternalServerException

The request failed due to an unknown internal server error, exception, or failure.

Schemas

Request Bodies

Example POST

{ "criteria": { }, "nextToken": "string", "maxResults": integer, "sortCriteria": { "orderBy": enum, "attributeName": "string" } }

Response Bodies

Example DescribeBucketsResponse

{ "nextToken": "string", "buckets": [ { "bucketName": "string", "objectCount": integer, "sizeInBytes": integer, "versioning": boolean, "classifiableObjectCount": integer, "publicAccess": { "effectivePermission": enum, "permissionConfiguration": { "accountLevelPermissions": { "blockPublicAccess": { "blockPublicPolicy": boolean, "restrictPublicBuckets": boolean, "blockPublicAcls": boolean, "ignorePublicAcls": boolean } }, "bucketLevelPermissions": { "accessControlList": { "allowsPublicReadAccess": boolean, "allowsPublicWriteAccess": boolean }, "blockPublicAccess": { "blockPublicPolicy": boolean, "restrictPublicBuckets": boolean, "blockPublicAcls": boolean, "ignorePublicAcls": boolean }, "bucketPolicy": { "allowsPublicReadAccess": boolean, "allowsPublicWriteAccess": boolean } } } }, "objectCountByEncryptionType": { "kmsManaged": integer, "s3Managed": integer, "customerManaged": integer, "unencrypted": integer }, "classifiableSizeInBytes": integer, "tags": [ { "value": "string", "key": "string" } ], "unclassifiableObjectCount": { "total": integer, "storageClass": integer, "fileType": integer }, "lastUpdated": "string", "accountId": "string", "bucketArn": "string", "bucketCreatedAt": "string", "replicationDetails": { "replicationAccounts": [ "string" ], "replicatedExternally": boolean, "replicated": boolean }, "unclassifiableObjectSizeInBytes": { "total": integer, "storageClass": integer, "fileType": integer }, "sharedAccess": enum, "region": "string", "sizeInBytesCompressed": integer } ] }

Example ValidationException

{ "message": "string" }

Example ServiceQuotaExceededException

{ "message": "string" }

Example AccessDeniedException

{ "message": "string" }

Example ResourceNotFoundException

{ "message": "string" }

Example ConflictException

{ "message": "string" }

Example ThrottlingException

{ "message": "string" }

Example InternalServerException

{ "message": "string" }

Properties

AccessControlList

Provides information about the permissions settings of the bucket-level access control list (ACL) for an S3 bucket.

Property Type Required Description
allowsPublicReadAccess

boolean

False

Specifies whether the ACL grants the general public with read access permissions for the bucket.

allowsPublicWriteAccess

boolean

False

Specifies whether the ACL grants the general public with write access permissions for the bucket.

AccessDeniedException

Provides information about an error that occurred due to insufficient access to a specified resource.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

AccountLevelPermissions

Provides information about account-level permissions settings that apply to an S3 bucket.

Property Type Required Description
blockPublicAccess

BlockPublicAccess

False

The block public access settings for the bucket.

BlockPublicAccess

Provides information about the block public access settings for an S3 bucket. These settings can apply to a bucket at the account level or bucket level. For detailed information about each setting, see Using Amazon S3 block public access in the Amazon Simple Storage Service Developer Guide.

Property Type Required Description
blockPublicPolicy

boolean

False

Specifies whether Amazon S3 blocks public bucket policies for the bucket.

restrictPublicBuckets

boolean

False

Specifies whether Amazon S3 restricts public bucket policies for the bucket.

blockPublicAcls

boolean

False

Specifies whether Amazon S3 blocks public access control lists (ACLs) for the bucket and objects in the bucket.

ignorePublicAcls

boolean

False

Specifies whether Amazon S3 ignores public ACLs for the bucket and objects in the bucket.

BucketCriteria

Specifies, as a map, one or more attribute-based conditions that filter the results of a query for information about S3 buckets.

Property Type Required Description

*

object

False

BucketCriteriaAdditionalProperties

Specifies the operator to use in an attribute-based condition that filters the results of a query for information about S3 buckets.

Property Type Required Description
prefix

string

False

The prefix of the buckets to include in the results.

lt

integer

Format: int64

False

A less than condition to apply to a specified attribute value for buckets.

gte

integer

Format: int64

False

A greater than or equal to condition to apply to a specified attribute value for buckets.

neq

Array of type string

False

A not equal to condition to apply to a specified attribute value for buckets.

lte

integer

Format: int64

False

A less than or equal to condition to apply to a specified attribute value for buckets.

eq

Array of type string

False

An equal to condition to apply to a specified attribute value for buckets.

gt

integer

Format: int64

False

A greater than condition to apply to a specified attribute value for buckets.

BucketLevelPermissions

Provides information about the bucket-level permissions settings for an S3 bucket.

Property Type Required Description
accessControlList

AccessControlList

False

The permissions settings of the access control list (ACL) for the bucket. This value is null if an ACL hasn't been defined for the bucket.

blockPublicAccess

BlockPublicAccess

False

The block public access settings for the bucket.

bucketPolicy

BucketPolicy

False

The permissions settings of the bucket policy for the bucket. This value is null if a bucket policy hasn't been defined for the bucket.

BucketMetadata

Provides information about an S3 bucket that Amazon Macie monitors and analyzes.

Property Type Required Description
bucketName

string

False

The name of the bucket.

objectCount

integer

Format: int64

False

The total number of objects in the bucket.

sizeInBytes

integer

Format: int64

False

The total storage size, in bytes, of the bucket.

versioning

boolean

False

Specifies whether versioning is enabled for the bucket.

classifiableObjectCount

integer

Format: int64

False

The total number of objects that Amazon Macie can analyze in the bucket. These objects use a supported storage class and have a file name extension for a supported file or storage format.

publicAccess

BucketPublicAccess

False

Specifies whether the bucket is publicly accessible. If this value is true, an access control list (ACL), bucket policy, or block public access settings allow the bucket to be accessed by the general public.

objectCountByEncryptionType

ObjectCountByEncryptionType

False

The total number of objects that are in the bucket, grouped by server-side encryption type. This includes a grouping that reports the total number of objects that aren't encrypted or use client-side encryption.

classifiableSizeInBytes

integer

Format: int64

False

The total storage size, in bytes, of the objects that Amazon Macie can analyze in the bucket. These objects use a supported storage class and have a file name extension for a supported file or storage format.

tags

Array of type KeyValuePair

False

An array that specifies the tags (keys and values) that are associated with the bucket.

unclassifiableObjectCount

ObjectLevelStatistics

False

The total number of objects that Amazon Macie can't analyze in the bucket. These objects don't use a supported storage class or don't have a file name extension for a supported file or storage format.

lastUpdated

string

Format: date-time

False

The date and time, in UTC and extended ISO 8601 format, when Amazon Macie most recently retrieved data about the bucket from Amazon S3.

accountId

string

False

The unique identifier for the AWS account that owns the bucket.

bucketArn

string

False

The Amazon Resource Name (ARN) of the bucket.

bucketCreatedAt

string

Format: date-time

False

The date and time, in UTC and extended ISO 8601 format, when the bucket was created.

replicationDetails

ReplicationDetails

False

Specifies whether the bucket is configured to replicate one or more objects to buckets for other AWS accounts and, if so, which accounts.

unclassifiableObjectSizeInBytes

ObjectLevelStatistics

False

The total storage size, in bytes, of the objects that Amazon Macie can't analyze in the bucket. These objects don't use a supported storage class or don't have a file name extension for a supported file or storage format.

sharedAccess

string

Values: EXTERNAL | INTERNAL | NOT_SHARED | UNKNOWN

False

Specifies whether the bucket is shared with another AWS account. Possible values are:

  • EXTERNAL - The bucket is shared with an AWS account that isn’t part of the same Amazon Macie organization.

  • INTERNAL - The bucket is shared with an AWS account that's part of the same Amazon Macie organization.

  • NOT_SHARED - The bucket isn't shared with other AWS accounts.

  • UNKNOWN - Amazon Macie wasn't able to evaluate the shared access settings for the bucket.

region

string

False

The AWS Region that hosts the bucket.

sizeInBytesCompressed

integer

Format: int64

False

The total compressed storage size, in bytes, of the bucket.

BucketPermissionConfiguration

Provides information about the account-level and bucket-level permissions settings for an S3 bucket.

Property Type Required Description
accountLevelPermissions

AccountLevelPermissions

False

The account-level permissions settings that apply to the bucket.

bucketLevelPermissions

BucketLevelPermissions

False

The bucket-level permissions settings for the bucket.

BucketPolicy

Provides information about the permissions settings of a bucket policy for an S3 bucket.

Property Type Required Description
allowsPublicReadAccess

boolean

False

Specifies whether the bucket policy allows the general public to have read access to the bucket.

allowsPublicWriteAccess

boolean

False

Specifies whether the bucket policy allows the general public to have write access to the bucket.

BucketPublicAccess

Provides information about the permissions settings that determine whether an S3 bucket is publicly accessible.

Property Type Required Description
effectivePermission

string

Values: PUBLIC | NOT_PUBLIC | UNKNOWN

False

Specifies whether the bucket is publicly accessible due to the combination of permissions settings that apply to the bucket. Possible values are:

  • NOT_PUBLIC - The bucket isn't publicly accessible.

  • PUBLIC - The bucket is publicly accessible.

  • UNKNOWN - Amazon Macie can't determine whether the bucket is publicly accessible.

permissionConfiguration

BucketPermissionConfiguration

False

The account-level and bucket-level permissions for the bucket.

BucketSortCriteria

Specifies criteria for sorting the results of a query for information about S3 buckets.

Property Type Required Description
orderBy

string

Values: ASC | DESC

False

The sort order to apply to the results, based on the value for the property specified by the attributeName property. Valid values are: ASC, sort the results in ascending order; and, DESC, sort the results in descending order.

attributeName

string

False

The name of the attribute to sort the results by. This value can be the name of any property that Amazon Macie defines as bucket metadata, such as bucketName or accountId.

ConflictException

Provides information about an error that occurred due to a versioning conflict for a specified resource.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

DescribeBucketsRequest

Specifies criteria for filtering, sorting, and paginating the results of a query for information about S3 buckets.

Property Type Required Description
criteria

BucketCriteria

False

The criteria to use to filter the query results.

nextToken

string

False

The nextToken string that specifies which page of results to return in a paginated response.

maxResults

integer

Format: int32

False

The maximum number of items to include in each page of the response. The default value is 50.

sortCriteria

BucketSortCriteria

False

The criteria to use to sort the query results.

DescribeBucketsResponse

Provides the results of a query that retrieved statistical data and other information about one or more S3 buckets that Amazon Macie monitors and analyzes.

Property Type Required Description
nextToken

string

False

The string to use in a subsequent request to get the next page of results in a paginated response. This value is null if there are no additional pages.

buckets

Array of type BucketMetadata

False

An array of objects, one for each bucket that meets the filter criteria specified in the request.

InternalServerException

Provides information about an error that occurred due to an unknown internal server error, exception, or failure.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

KeyValuePair

Provides information about the tags that are associated with an S3 bucket or object. Each tag consists of a required tag key and an associated tag value.

Property Type Required Description
value

string

False

One part of a key-value pair that comprises a tag. A tag value acts as a descriptor for a tag key. A tag value can be empty or null.

key

string

False

One part of a key-value pair that comprises a tag. A tag key is a general label that acts as a category for more specific tag values.

ObjectCountByEncryptionType

Provides information about the number of objects that are in an S3 bucket and use certain types of server-side encryption, use client-side encryption, or aren't encrypted.

Property Type Required Description
kmsManaged

integer

Format: int64

False

The total number of objects that are encrypted using an AWS Key Management Service (AWS KMS) customer master key (CMK). The objects use AWS KMS AWS-managed (AWS-KMS) encryption or AWS KMS customer-managed (SSE-KMS) encryption.

s3Managed

integer

Format: int64

False

The total number of objects that are encrypted using an Amazon S3-managed key. The objects use Amazon S3-managed (SSE-S3) encryption.

customerManaged

integer

Format: int64

False

The total number of objects that are encrypted using a customer-managed key. The objects use customer-provided server-side (SSE-C) encryption.

unencrypted

integer

Format: int64

False

The total number of objects that aren't encrypted or use client-side encryption.

ObjectLevelStatistics

Provides information about the total storage size (in bytes) or number of objects that Amazon Macie can't analyze in one or more S3 buckets. In a BucketMetadata object, this data is for a specific bucket. In a GetBucketStatisticsResponse object, this data is aggregated for all the buckets in the query results.

Property Type Required Description
total

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects use an unsupported storage class or don’t have a file name extension for a supported file or storage format.

storageClass

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects use an unsupported storage class.

fileType

integer

Format: int64

False

The total storage size (in bytes) or number of objects that Amazon Macie can't analyze because the objects don’t have a file name extension for a supported file or storage format.

ReplicationDetails

Provides information about settings that define whether one or more objects in an S3 bucket are replicated to S3 buckets for other AWS accounts and, if so, which accounts.

Property Type Required Description
replicationAccounts

Array of type string

False

An array of AWS account IDs, one for each AWS account that the bucket is configured to replicate one or more objects to.

replicatedExternally

boolean

False

Specifies whether the bucket is configured to replicate one or more objects to an AWS account that isn't part of the same Amazon Macie organization.

replicated

boolean

False

Specifies whether the bucket is configured to replicate one or more objects to any destination.

ResourceNotFoundException

Provides information about an error that occurred because a specified resource wasn't found.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

ServiceQuotaExceededException

Provides information about an error that occurred due to one or more service quotas for an account.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

ThrottlingException

Provides information about an error that occurred because too many requests were sent during a certain amount of time.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

ValidationException

Provides information about an error that occurred due to a syntax error in a request.

Property Type Required Description
message

string

False

The explanation of the error that occurred.

See Also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

DescribeBuckets