Amazon S3 Storage Lens examples using the AWS CLI - Amazon Simple Storage Service
Helper filesUsing S3 Storage Lens configurationsS3 Storage Lens for your Organizations

Amazon S3 Storage Lens examples using the AWS CLI

S3 Storage Lens aggregates your metrics and displays the information in the Account snapshot section on the Amazon S3 console Buckets page. S3 Storage Lens also provides an interactive dashboard that you can use to visualize insights and trends, flag outliers, and receive recommendations for optimizing storage costs and applying data-protection best practices. Your dashboard has drill-down options to generate insights at the organization, account, bucket, object, or prefix level. You can also send a once-daily metrics export in CSV or Parquet format to an S3 bucket. For more information, see Assessing storage activity and usage with Amazon S3 Storage Lens.

The following examples show how you can use S3 Storage Lens with the AWS Command Line Interface.

Helper files for using Amazon S3 Storage Lens

Use the following JSON files for key inputs for your examples.

S3 Storage Lens sample configuration JSON

Example config.json

The config.json file contains the details of a S3 Storage Lens Organizations-level advanced metrics and recommendations configuration. To use the following example, replace the user input placeholders with your own information.

Note

Additional charges apply for advanced metrics and recommendations. For more information, see advanced metrics and recommendations.

{
  "Id": "SampleS3StorageLensConfiguration", //Use this property to identify your S3 Storage Lens configuration.
  "AwsOrg": { //Use this property when enabling S3 Storage Lens for AWS Organizations.
    "Arn": "arn:aws:organizations::123456789012:organization/o-abcdefgh"
  },
  "AccountLevel": {
    "ActivityMetrics": {
      "IsEnabled":true
    },
    "AdvancedCostOptimizationMetrics": {
      "IsEnabled":true
    },
    "AdvancedDataProtectionMetrics": {
      "IsEnabled":true
    },
    "DetailedStatusCodesMetrics": {
      "IsEnabled":true
    },
    "BucketLevel": {
      "ActivityMetrics": {
        "IsEnabled":true //Mark this as false if you want only free metrics.
      },
      "AdvancedDataProtectionMetrics": {
      "IsEnabled":true //Mark this as false if you want only free metrics.
      },
      "AdvancedCostOptimizationMetrics": {
        "IsEnabled":true //Mark this as false if you want only free metrics.
      },
      "DetailedStatusCodesMetrics": {
        "IsEnabled":true //Mark this as false if you want only free metrics.
      },
      "PrefixLevel":{
        "StorageMetrics":{
          "IsEnabled":true, //Mark this as false if you want only free metrics.
          "SelectionCriteria":{
            "MaxDepth":5,
            "MinStorageBytesPercentage":1.25,
            "Delimiter":"/"
          }
        }
      }
    }
  },
  "Exclude": { //Replace with "Include" if you prefer to include Regions.
    "Regions": [
      "eu-west-1"
    ],
    "Buckets": [ //This attribute is not supported for AWS Organizations-level configurations.
      "arn:aws:s3:::source_bucket1"
    ]
  },
  "IsEnabled": true, //Whether the configuration is enabled
  "DataExport": { //Details about the metrics export
    "S3BucketDestination": {
      "OutputSchemaVersion": "V_1",
      "Format": "CSV", //You can add "Parquet" if you prefer.
      "AccountId": "111122223333",
      "Arn": "arn:aws:s3:::destination-bucket-name", // The destination bucket for your metrics export must be in the same Region as your S3 Storage Lens configuration. 
      "Prefix": "prefix-for-your-export-destination",
      "Encryption": {
        "SSES3": {}
      }
    },
    "CloudWatchMetrics": {
      "IsEnabled": true //Mark this as false if you don't want to enable the Amazon CloudWatch publishing option.
    }
  }
}

S3 Storage Lens sample configuration tags JSON

Example tags.json

The tags.json file contains the tags that you want to apply to your S3 Storage Lens configuration. To use this example, replace the user input placeholders with your own information.

[
    {
        "Key": "key1",
        "Value": "value1"
    },
    {
        "Key": "key2",
        "Value": "value2"
    }
]

S3 Storage Lens sample configuration IAM permissions

Example permissions.json – Specific dashboard name

This example policy shows an S3 Storage Lens IAM permissions.json file with a specific dashboard name specified. Replace value1, us-east-1, your-dashboard-name, and example-account-id with your own values.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetStorageLensConfiguration",
                "s3:DeleteStorageLensConfiguration",
                "s3:PutStorageLensConfiguration"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/key1": "value1"
                }
            },
            "Resource": "arn:aws:s3:us-east-1:example-account-id:storage-lens/your-dashboard-name"
        }
    ]
}
Example permissions.json – No specific dashboard name

This example policy shows an S3 Storage Lens IAM permissions.json file without a specific dashboard name specified. Replace value1, us-east-1, and example-account-id with your own values.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetStorageLensConfiguration",
                "s3:DeleteStorageLensConfiguration",
                "s3:PutStorageLensConfiguration"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/key1": "value1"
                }
            },
            "Resource": "arn:aws:s3:us-east-1:example-account-id:storage-lens/*"
        }
    ]
}

Using Amazon S3 Storage Lens configurations with the AWS CLI

You can use the AWS CLI to list, create, delete, get, tag, and update your S3 Storage Lens configurations. The following examples use the helper JSON files for key inputs. To use these examples, replace the user input placeholders with your own information.

Create an S3 Storage Lens configuration

Example Create an S3 Storage Lens configuration
aws s3control put-storage-lens-configuration --account-id=222222222222 --config-id=your-configuration-id --region=us-east-1 --storage-lens-configuration=file://./config.json --tags=file://./tags.json

Create an S3 Storage Lens configuration without tags

Example Create an S3 Storage Lens configuration without tags
aws s3control put-storage-lens-configuration --account-id=222222222222 --config-id=your-configuration-id --region=us-east-1 --storage-lens-configuration=file://./config.json

Get an S3 Storage Lens configuration

Example Get an S3 Storage Lens configuration
aws s3control get-storage-lens-configuration --account-id=222222222222 --config-id=your-configuration-id --region=us-east-1

List S3 Storage Lens configurations without a next token

Example List S3 Storage Lens configurations without a next token
aws s3control list-storage-lens-configurations --account-id=222222222222 --region=us-east-1

List S3 Storage Lens configurations

Example List S3 Storage Lens configurations
aws s3control list-storage-lens-configurations --account-id=222222222222 --region=us-east-1 --next-token=abcdefghij1234

Delete an S3 Storage Lens configuration

Example Delete an S3 Storage Lens configuration
aws s3control delete-storage-lens-configuration --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id

Add tags to an S3 Storage Lens configuration

Example Add tags to an S3 Storage Lens configuration
aws s3control put-storage-lens-configuration-tagging --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id --tags=file://./tags.json

Get tags for an S3 Storage Lens configuration

Example Get tags for an S3 Storage Lens configuration
aws s3control get-storage-lens-configuration-tagging --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id

Delete tags for an S3 Storage Lens configuration

Example Delete tags for an S3 Storage Lens configuration
aws s3control delete-storage-lens-configuration-tagging --account-id=222222222222 --region=us-east-1 --config-id=your-configuration-id

Using Amazon S3 Storage Lens with AWS Organizations examples using the AWS CLI

Use Amazon S3 Storage Lens to collect storage metrics and usage data for all accounts that are part of your AWS Organizations hierarchy. For more information, see Using Amazon S3 Storage Lens with AWS Organizations.

Enable Organizations trusted access for S3 Storage Lens

Example Enable Organizations trusted access for S3 Storage Lens
aws organizations enable-aws-service-access --service-principal storage-lens.s3.amazonaws.com

Disable Organizations trusted access for S3 Storage Lens

Example Disable Organizations trusted access for S3 Storage Lens
aws organizations disable-aws-service-access --service-principal storage-lens.s3.amazonaws.com

Register Organizations delegated administrators for S3 Storage Lens

Example Register Organizations delegated administrators for S3 Storage Lens

To use this example, replace 111122223333 with the appropriate AWS account ID.

aws organizations register-delegated-administrator --service-principal storage-lens.s3.amazonaws.com --account-id 111122223333

Deregister Organizations delegated administrators for S3 Storage Lens

Example Deregister Organizations delegated administrators for S3 Storage Lens

To use this example, replace 111122223333 with the appropriate AWS account ID.

aws organizations deregister-delegated-administrator --service-principal storage-lens.s3.amazonaws.com --account-id 111122223333