Amazon S3 Storage Lens examples using the AWS CLI - Amazon Simple Storage Service

Amazon S3 Storage Lens examples using the AWS CLI

Amazon S3 Storage Lens aggregates your usage and activity metrics and displays the information in the account snapshot on the Amazon S3 console home (Buckets) page, interactive dashboards, or through a metrics export that you can download in CSV or Parquet format. You can use the dashboard to visualize insights and trends, flag outliers, and receive recommendations for optimizing storage costs and applying data protection best practices. You can use S3 Storage Lens through the AWS Management Console, AWS CLI, AWS SDKs, or REST API.. For more information, see Assessing storage activity and usage with Amazon S3 Storage Lens.

The following examples show how you can use S3 Storage Lens with the AWS Command Line Interface.

Helper files for using Amazon S3 Storage Lens

Use the following JSON files for key inputs for your examples.

S3 Storage Lens sample configuration JSON

Example config.json

Contains details of a S3 Storage Lens Organizations-level Advanced Metrics and Recommendations configuration.

Note

Additional charges apply for Advanced Metrics and Recommendations. For more information, see Advanced Metrics and Recommendations.

{ "Id": "SampleS3StorageLensConfiguration", //Use this property to identify S3 Storage Lens configuration. "AwsOrg": { //Use this property when enabling S3 Storage Lens for AWS Organizations "Arn": "arn:aws:organizations::222222222222:organization/o-abcdefgh" }, "AccountLevel": { "ActivityMetrics": { "IsEnabled":true }, "BucketLevel": { "ActivityMetrics": { "IsEnabled":true //Mark this as false if you only want Free Metrics metrics. }, "PrefixLevel":{ "StorageMetrics":{ "IsEnabled":true, //Mark this as false if you only want Free Metrics metrics. "SelectionCriteria":{ "MaxDepth":5, "MinStorageBytesPercentage":1.25, "Delimiter":"/" } } } } }, "Exclude": { //Replace with include if you prefer to include regions. "Regions": [ "eu-west-1" ], "Buckets": [ //This attribute is not supported for Organizations-level configurations. "arn:aws:s3:::source_bucket1" ] }, "IsEnabled": true, //Whether the configuration is enabled "DataExport": { //Details about the metrics export "S3BucketDestination": { "OutputSchemaVersion": "V_1", "Format": "CSV", //You can add "Parquet" if you prefer. "AccountId": "ExampleAWSAccountNo8", "Arn": "arn:aws:s3:::destination-bucket-name", // The destination bucket for your metrics export must be in the same Region as your S3 Storage Lens configuration. "Prefix": "prefix-for-your-export-destination", "Encryption": { "SSES3": {} } } } }

S3 Storage Lens sample configuration tags JSON

Example tags.json

[ { "Key": "key1", "Value": "value1" }, { "Key": "key2", "Value": "value2" } ]

S3 Storage Lens sample configuration IAM permissions

Example permissions.json - Specific dashboard name

This example policy shows S3 Storage Lens IAM permissions with a specific dashboard name specified. Replace your-dashboard-name and example-account-id with your own values.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetStorageLensConfiguration", "s3:DeleteStorageLensConfiguration", "s3:PutStorageLensConfiguration" ], "Condition": { "StringEquals": { "aws:ResourceTag/key1": "value1" } }, "Resource": "arn:aws:s3:us-east-1:example-account-id:storage-lens/your-dashboard-name" } ] }

Example permissions.json - No specific dashboard name

This example policy shows S3 Storage Lens IAM permissions without a specific dashboard name specified. Replace example-account-id with your AWS account ID.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetStorageLensConfiguration", "s3:DeleteStorageLensConfiguration", "s3:PutStorageLensConfiguration" ], "Condition": { "StringEquals": { "aws:ResourceTag/key1": "value1" } }, "Resource": "arn:aws:s3:us-east-1:example-account-id:storage-lens/*" } ] }

Using Amazon S3 Storage Lens configurations with the AWS CLI

You can use the AWS CLI to list, create, get and update your S3 Storage Lens configurations. The following examples use the helper JSON files for key inputs.

Put an S3 Storage Lens configuration

Example Puts an S3 Storage Lens configuration

aws s3control put-storage-lens-configuration --account-id=222222222222 --config-id=your-configuration-id --region=us-east-1 --storage-lens-configuration=file://./config.json --tags=file://./tags.json

Put an S3 Storage Lens configuration without tags

Example Put an S3 Storage Lens configuration

aws s3control put-storage-lens-configuration --account-id=222222222222 --config-id=your-configuration-id --region=us-east-1 --storage-lens-configuration=file://./config.json

Get an S3 Storage Lens configuration

Example Get an S3 Storage Lens configuration

aws s3control get-storage-lens-configuration --account-id=222222222222 --config-id=your-configuration-id --region=us-east-1

List S3 Storage Lens configurations without next token

Example List S3 Storage Lens configurations without next token

aws s3control list-storage-lens-configurations --account-id=222222222222 --region=us-east-1

List S3 Storage Lens configurations

Example List S3 Storage Lens configurations

aws s3control list-storage-lens-configurations --account-id=222222222222 --region=us-east-1 --next-token=abcdefghij1234

Delete an S3 Storage Lens configuration

Example Delete an S3 Storage Lens configuration

aws s3control delete-storage-lens-configuration --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id

Put tags to an S3 Storage Lens configuration

Example Put tags to an S3 Storage Lens configuration

aws s3control put-storage-lens-configuration-tagging --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id --tags=file://./tags.json

Get tags for an S3 Storage Lens configuration

Example Get tags for an S3 Storage Lens configuration

aws s3control get-storage-lens-configuration-tagging --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id

Delete tags for an S3 Storage Lens configuration

Example Delete tags for an S3 Storage Lens configuration

aws s3control delete-storage-lens-configuration-tagging --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id

Using Amazon S3 Storage Lens with AWS Organizations using the AWS CLI

Use Amazon S3 Storage Lens to collect storage metrics and usage data for all accounts that are part of your AWS Organizations hierarchy. For more information, see Using Amazon S3 Storage Lens with AWS Organizations.

Enable Organizations trusted access for S3 Storage Lens

Example Enable Organizations trusted access for S3 Storage Lens

aws organizations enable-aws-service-access --service-principal storage-lens.s3.amazonaws.com

Disable Organizations trusted access for S3 Storage Lens

Example Disable Organizations trusted access for S3 Storage Lens

aws organizations disable-aws-service-access --service-principal storage-lens.s3.amazonaws.com

Register Organizations delegated administrators for S3 Storage Lens

Example Register Organizations delegated administrators for S3 Storage Lens

aws organizations register-delegated-administrator --service-principal storage-lens.s3.amazonaws.com —account-id 123456789012

Deregister Organizations delegated administrators for S3 Storage Lens

Example Deregister Organizations delegated administrators for S3 Storage Lens

aws organizations deregister-delegated-administrator --service-principal storage-lens.s3.amazonaws.com —account-id 123456789012