AWS Billing and Cost Management
User Guide (Version 2.0)

Getting Started with Amazon Athena

You can use an existing Amazon S3 bucket or AWS Cost and Usage report with Athena, but we strongly recommend that you create both a new Amazon S3 bucket and a new AWS Cost and Usage report for use with Athena. The recommended setup process removes any Amazon S3 events that your bucket might already have, which can negatively affect any existing event-based processes that you have for an existing AWS Cost and Usage report. Setting up a new AWS Cost and Usage report can take up to 8 hours, so we recommend that you plan to do the last two setup steps the next day.

AWS Cost and Usage Report supports only the parquet compression format for Athena and automatically overwrites previous AWS Cost and Usage reports stored in your Amazon S3 bucket.

Important

If you plan to use the provided AWS CloudFormation template, you must create all resources in the same Region. AWS CloudFormation doesn't support cross-Region resources. The Region must support the following services:

  • AWS Lambda

  • Amazon Simple Storage Service

  • AWS Glue

  • Amazon Athena

Setting up to use Athena includes the following steps:

To create an Amazon S3 bucket for your reports

Use this procedure to create a new Amazon S3 bucket for your report.

Note

If you are part of an AWS Organizations organization, only the master account can create this bucket. AWS Cost and Usage reports can be delivered only to a bucket owned by the master account.

  1. Open the Amazon S3 console at https://console.aws.amazon.com/s3/.

  2. Choose Create Bucket.

  3. In the dialog box, for Bucket Name, enter the name for your bucket.

    Note

    Your bucket name must be all lowercase, from 3 to 63 characters long, and can't contain spaces. You can use lowercase letters, numbers, hyphens (-), and periods (.) in your bucket name.

  4. Choose the Region that you want your Amazon S3 bucket to be in.

  5. Choose Next.

  6. Choose Next.

  7. (Optional) If you choose Grant Amazon Simple Storage Service Log Delivery group write access to this bucket, you can enable access logs that track who accesses your Amazon S3 bucket. Choose the bucket that you want the access logs to be delivered to and the name of a folder that you want the logs to be stored in.

  8. Choose Next.

  9. Choose Create bucket.

  10. From the list of buckets, choose the bucket that you want to receive reports in.

  11. Choose Permissions.

  12. Choose Bucket Policy.

  13. Paste the following text into the bucket policy editor.

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "386209384616" }, "Action": [ "s3:GetBucketAcl", "s3:GetBucketPolicy" ], "Resource": "arn:aws:s3:::bucketname" }, { "Effect": "Allow", "Principal": { "AWS": "386209384616" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::bucketname/*" } ] }
  14. Replace bucketname with the name of your bucket. Don't replace the Principal number 386209384616. AWS uses that account to deliver reports to the S3 bucket.

  15. Choose Save.

To create an AWS Cost and Usage report

Use this procedure to create a new AWS Cost and Usage report for use with Athena.

  1. Sign in to the AWS Management Console and open the Billing and Cost Management console at https://console.aws.amazon.com/billing/home#/.

  2. On the navigation pane, choose Reports.

  3. Choose Create report.

  4. For Report name, enter a name for your report.

  5. For Additional report details, to include the IDs of each individual resource in the report, select Include resource IDs.

  6. For Data refresh settings, select whether you want the AWS Cost and Usage report to refresh if AWS applies refunds, credits, or support fees to your account after finalizing your bill. When a report refreshes, a new report is upload to Amazon S3.

  7. Choose Next.

  8. For S3 bucket, enter the name of the Amazon S3 bucket where you want the reports to be delivered and choose Verify. The bucket must have appropriate permissions to be valid. For more information on adding permissions to the bucket, see Setting Bucket and Object Access Permissions in the Amazon Simple Storage Service Console User Guide.

  9. For Report path prefix, enter the report path prefix that you want prepended to the name of your report. You must provide a report path prefix to use Athena with AWS Cost and Usage report.

  10. For Time granularity, choose Hourly if you want the line items in the report to be aggregated by the hour. Choose Daily if you want the line items in the report to be aggregated by the day.

  11. For Report versioning, choose whether you want each version of the report to overwrite the previous version of the report or to be delivered in addition to the previous versions.

  12. For Enable report data integration for, select whether you want to upload your AWS Cost and Usage report to Amazon Redshift, Amazon QuickSight, or Amazon Athena. If you select an Amazon Redshift or Amazon QuickSight manifest, your report is stored with .gz compression. If you select an Athena manifest, your report is stored with parquet compression.

  13. Choose Next.

  14. After you have reviewed the settings for your report, choose Review and Complete.

  15. Before moving on to the next procedure, you must wait for the first AWS Cost and Usage report to be delivered to your Amazon S3 bucket. It might take up to 8 hours for AWS to deliver your first AWS Cost and Usage report.

To use the Athena AWS CloudFormation template

To use Athena, you must set up an AWS Glue crawler, an AWS Glue database, and an AWS Lambda event. Billing and Cost Management provides an AWS CloudFormation template that does this setup for you. AWS CloudFormation doesn't support resources in multiple Regions for one stack, so the Region that you use must support the following services:

  • AWS Lambda

  • Amazon Simple Storage Service

  • AWS Glue

  • Amazon Athena

Important

The AWS CloudFormation template removes all events associated with your Amazon S3 bucket, so we strongly recommend that you use a new Amazon S3 bucket.

  1. Open the Amazon S3 console at https://console.aws.amazon.com/s3/.

  2. From the list of buckets, choose the bucket image next to the bucket where you chose to receive your AWS Cost and Usage report.

  3. Select the crawler-cfn.yml file and copy the link to the template file.

  4. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  5. If you have never used AWS CloudFormation before, choose Create New Stack. Otherwise, choose Create Stack.

  6. Under Choose a template, choose Upload a template to Amazon S3.

  7. Choose Browse..., choose the downloaded template, and then choose Open.

  8. Choose Next.

  9. For Stack name, enter a name for your template and choose Next.

  10. Choose Next.

  11. Select I acknowledge that AWS CloudFormation might create IAM resources. This template creates the following resources:

    • Three IAM roles

    • An AWS Glue database

    • An AWS Glue crawler

    • Two Lambda functions

    • An Amazon S3 notification

  12. Choose Create.