Amazon Athena
User Guide

Setting Data Usage Control Limits

Athena allows you to set two types of cost controls: per-query limit and per-workgroup limit. For each workgroup, you can set only one per-query limit and multiple per-workgroup limits.

  • The per-query control limit specifies the total amount of data scanned per query. If any query that runs in the workgroup exceeds the limit, it is canceled. You can create only one per-query control limit in a workgroup and it applies to each query that runs in it. Edit the limit if you need to change it. For detailed steps, see To create a per-query data usage control.

  • The workgroup-wide data usage control limit specifies the total amount of data scanned for all queries that run in this workgroup during the specified time period. You can create multiple limits per workgroup. The workgroup-wide query limit allows you to set multiple thresholds on hourly or daily aggregates on data scanned by queries running in the workgroup.

    If the aggregate amount of data scanned exceeds the threshold, you can choose to take one of the following actions:

    • Configure an Amazon SNS alarm and an action in the Athena console to notify an administrator when the limit is breached. For detailed steps, see To create a per-workgroup data usage control. You can also create an alarm and an action on any metric that Athena publishes from the CloudWatch console. For example, you can set an alert on a number of failed queries. This alert can trigger an email to an administrator if the number crosses a certain threshold. If the limit is exceeded, an action sends an Amazon SNS alarm notification to the specified users.

    • Invoke a Lambda function. For more information, see Invoking Lambda functions using Amazon SNS notifications in the Amazon Simple Notification Service Developer Guide.

    • Disable the workgroup, stopping any further queries from running.

The per-query and per-workgroup limits are independent of each other. A specified action is taken whenever either limit is exceeded. If two or more users run queries at the same time in the same workgroup, it is possible that each query does not exceed any of the specified limits, but the total sum of data scanned exceeds the data usage limit per workgroup. In this case, an Amazon SNS alarm is sent to the user.

To create a per-query data usage control

The per-query control limit specifies the total amount of data scanned per query. If any query that runs in the workgroup exceeds the limit, it is canceled. Canceled queries are charged according to Amazon Athena pricing.

Note

In the case of canceled or failed queries, Athena may have already written partial results to Amazon S3. In such cases, Athena does not delete partial results from the Amazon S3 prefix where results are stored. You must remove the Amazon S3 prefix with partial results. Athena uses Amazon S3 multipart uploads to write data Amazon S3. We recommend that you set the bucket lifecycle policy to abort multipart uploads in cases when queries fail. For more information, see Aborting Incomplete Multipart Uploads Using a Bucket Lifecycle Policy in the Amazon Simple Storage Service Developer Guide.

You can create only one per-query control limit in a workgroup and it applies to each query that runs in it. Edit the limit if you need to change it.

  1. Open the Athena console at https://console.aws.amazon.com/athena/.

  2. Choose the Workgroup:<name> tab.

    To create a data usage control for a query in a particular workgroup, you don't need to switch to it and can remain in another workgroup. You do need to select the workgroup from the list and have permissions to edit the workgroup.

  3. Select the workgroup from the list, and then choose View details. If you have permissions, the workgroup's details display in the Overview tab.

  4. Choose the Data usage controls tab. The Per Query Data Usage Control dialog displays.

    
                  The screenshot for creating a data usage control per query.
  5. Specify (or update) the field values, as follows:

    • For Data limit, specify a value between 10000 KB (minimum) and 7 EB (maximum).

      Note

      These are limits imposed by the console for data usage controls within workgroups. They do not represent any query limits in Athena.

    • For units, select the unit value from the drop-down list.

    • Review the default Action. The default Action is to cancel the query if it exceeds the limit. This action cannot be changed.

  6. Choose Create if you are creating a new limit, or Update if you are editing an existing limit. If you are editing an existing limit, refresh the Overview tab to see the updated limit.

To create a per-workgroup data usage control

The workgroup-wide data usage control limit specifies the total amount of data scanned for all queries that run in this workgroup during the specified time period. You can create multiple control limits per workgroup. If the limit is exceeded, you can choose to take action, such as send an Amazon SNS alarm notification to the specified users.

  1. Open the Athena console at https://console.aws.amazon.com/athena/.

  2. Choose the Workgroup:<name> tab.

    To create a data usage control for a particular workgroup, you don't need to switch to it and can remain in another workgroup. You do need to select the workgroup from the list and have permissions to edit the workgroup.

  3. Select the workgroup from the list, and then choose View details. If you have edit permissions, the workgroup's details display in the Overview tab.

  4. Choose the Data usage controls tab, and scroll down. Then choose Workgroup Data Usage Control to create a new limit or edit an existing limit. The Create workgroup data usage control dialog displays.

    
                  The screenshot for creating a data usage control per workgroup.
  5. Specify field values as follows:

    • For Data limits, specify a value between 10000 KB (minimum) and 7 EB (maximum).

      Note

      These are limits imposed by the console for data usage controls within workgroups. They do not represent any query limits in Athena.

    • For units, select the unit value from the drop-down list.

    • For time period, choose a time period from the drop-down list.

    • For Action, choose the Amazon SNS topic from the drop-down list, if you have it configured. Or, choose Create an Amazon SNS topic to go directly to the Amazon SNS console, create the Amazon SNS topic, and set up a subscription for it for one of the users in your Athena account. For more information, see Creating an Amazon SNS Topic in the Amazon Simple Notification Service Getting Started Guide.

  6. Choose Create if you are creating a new limit, or Save if you are editing an existing limit. If you are editing an existing limit, refresh the Overview tab for the workgroup to see the updated limit.