Getting started with AWS Cost Anomaly Detection - AWS Cost Management

Getting started with AWS Cost Anomaly Detection

With AWS Cost Anomaly Detection in AWS Billing and Cost Management, you can configure your cost monitors and alert subscriptions in several different ways.

Creating your cost monitors and alert subscriptions

Configure AWS Cost Anomaly Detection so that it detects anomalies at a lower granularity and spend patterns, in context to your monitor type.

For example, your spend patterns for Amazon EC2 usage might be different from your AWS Lambda or Amazon S3 spend patterns. By segmenting spends by AWS services, AWS Cost Anomaly Detection can detect separate spend patterns that help decrease false positive alerts. You can also create cost monitors. They can evaluate specific cost allocation tags, member accounts within an organization (AWS Organizations), and cost categories based on your AWS account structure.

As you create your cost monitors, configure your alert subscriptions specific to each monitor.

To create a cost monitor
  1. Open the AWS Billing and Cost Management console at https://console.aws.amazon.com/costmanagement/home.

  2. In the navigation pane, choose Cost Anomaly Detection.

  3. Choose the Cost monitors tab.

  4. Choose Create monitor.

  5. In Step 1, choose a monitor type and name your monitor.

    For more information about each monitor type and best practices, see Monitor types.

    For Monitor name, enter a name for your anomaly monitor. We recommend that the name is a short description. That way, you know what the monitor represents when you view your monitors on the Cost monitors tab.

  6. (Optional) Add a tag to your monitor. For more information about tags, see Tagging AWS resources in the AWS General Reference guide.

    1. Enter the key value for the tag.

    2. Choose Add new tag to add additional tags. The maximum number of tags that you can add is 50.

  7. Choose Next.

  8. In Step 2, configure your alert subscriptions.

    For Alert subscription, if you don't have an existing subscription, choose Create a new subscription. If you have existing subscriptions, select Choose an existing subscription.

    Note

    An alert subscription notifies you when a cost monitor detects an anomaly. Depending on the alert frequency, you can notify designated individuals by email or Amazon SNS.

    For Amazon SNS topics, configure to create an AWS Chatbot configuration. This configuration maps the SNS topic to a Slack channel or an Amazon Chime chat room. For example, create a subscription for the Finance team in your organization. For more information, see Receiving AWS Cost Anomaly Detection alerts in Amazon Chime and Slack.

    For Subscription name, enter a name that describes your use case. For example, if the subscription is meant for leadership, the subscription name might be “Leadership report.”

    Under Alerting frequency, choose your preferred notification frequency.

    • Individual alerts - The alert notifies you as soon as an anomaly is detected. You might receive multiple alerts throughout a day. These notifications require an Amazon SNS topic.

      You can configure the Amazon SNS topic to create an AWS Chatbot configuration that maps the SNS topic to a Slack channel or an Amazon Chime chat room. For more information, see Receiving AWS Cost Anomaly Detection alerts in Amazon Chime and Slack.

    • Daily summaries - The alert notifies you with a daily summary when anomalies are detected. You receive one email that contains information for multiple anomalies that occurred that day. These notifications require at least one email recipient.

    • Weekly summaries - The alert notifies you with a weekly summary when anomalies are detected. You receive one email that contains information for multiple anomalies that occurred that week. These notifications require at least one email recipient.

    Under Alert recipients, enter email addresses for this subscription.

    For Threshold, enter a number to configure the anomalies that you want to generate alerts for.

    There are two types of thresholds: absolute and percentage. Absolute thresholds trigger alerts when an anomaly's total cost impact exceeds your chosen threshold. Percentage thresholds trigger alerts when an anomaly's total impact percentage exceeds your chosen threshold. Total impact percentage is the percentage difference between the total expected spend and total actual spend.

    (Optional) Choose Add threshold to configure a second threshold on the same subscription. Thresholds can be combined by choosing AND or OR from the dropdown list.

    Note

    AWS Cost Anomaly Detection sends you a notification when an anomaly reaches or exceeds the Threshold. If an anomaly continues over multiple days, then alert recipients will continue to get notifications while the threshold is met.

    Even if an anomaly is below the alert threshold, the machine learning model continues to detect spend anomalies on your account. All the anomalies that the machine learning model detected (with cost impacts that are greater or less than the threshold) are available in the Detection history tab.

  9. (Optional) Add a tag to your alert subscription. For more information about tags, see Tagging AWS resources in the AWS General Reference guide.

    1. Enter the key value for the tag.

    2. Choose Add new tag to add additional tags. The maximum number of tags that you can add is 50.

  10. (Optional) Choose Add alert subscriptions to create another alert subscription. With this option, you can create a new subscription using the same monitor.

  11. Choose Create monitor.

To create an alert subscription

You must create at least one alert subscription for each monitor. The "create cost monitor steps" that are described earlier already include the alert subscription creation process. If you want to create additional subscriptions, follow these steps.

  1. Choose the Alert subscriptions tab.

  2. Choose Create a subscription.

  3. For Subscription name, enter a name that describes your use case. For example, if the subscription is meant for leadership, then the subscription name might be “Leadership report.”

  4. Under Alerting frequency, choose your preferred notification frequency.

    • Individual alerts - The alert notifies you as soon as an anomaly is detected. You might receive multiple alerts throughout a day. These notifications require an Amazon SNS topic.

      You can configure the Amazon SNS topic to create an AWS Chatbot configuration. This configuration maps the SNS topic to a Slack channel or an Amazon Chime chat room. For more information, see Receiving AWS Cost Anomaly Detection alerts in Amazon Chime and Slack.

    • Daily summaries - The alert notifies you with a daily summary when anomalies are detected. You receive one email that contains information for multiple anomalies that occurred that day. These notifications require at least one email recipient.

    • Weekly summaries - The alert notifies you with a weekly summary when anomalies are detected. You receive one email that contains information for multiple anomalies that occurred that week. These notifications require at least one email recipient.

  5. Under Alert recipients, enter email addresses for this subscription.

  6. For Threshold, enter a number to configure the anomalies that you want to generate alerts for.

    There are two types of thresholds: absolute and percentage. Absolute thresholds trigger alerts when an anomaly's total cost impact exceeds your chosen threshold. Percentage thresholds trigger alerts when an anomaly's total impact percentage exceeds your chosen threshold. Total impact percentage is the percentage difference between the total expected spend and total actual spend.

    (Optional) Choose Add threshold to configure a second threshold on the same subscription. Thresholds can be combined by choosing AND or OR from the dropdown list.

    Note

    AWS Cost Anomaly Detection sends you a notification when an anomaly reaches or exceeds the Threshold. If an anomaly continues over multiple days, then alert recipients will continue to get notifications while the threshold is met.

    Even if an anomaly is below the alert threshold, the machine learning model continues to detect spend anomalies on your account. All the anomalies that the machine learning model detected (with cost impacts that are greater or less than the threshold) are available in the Detection history tab.

  7. In the Cost monitors section, select the monitors that you want to be associated with the alert subscription.

  8. (Optional) Add a tag to your alert subscription. For more information about tags, see Tagging AWS resources in the AWS General Reference guide.

    1. Enter the key value for the tag.

    2. Choose Add new tag to add additional tags. The maximum number of tags that you can add is 50.

  9. Choose Create subscription.

Note

You can only access cost monitors and alert subscriptions under the account that created them. For example, suppose that the cost monitor was created under a member account. Then, the management account can't view or edit the cost monitors, alert subscriptions, or detected anomalies.

Detection history values

On the Detection history tab, you can view a list of all the anomalies detected over the time frame that you selected. By default, you can see the anomalies that are detected in the last 90 days. You can search by Severity, Assessment, Service, Account, Usage type, Region, or Monitor type. You can sort by Start date, Last detected date, Actual spend, Expected spend, Total cost impact, and Impact percentage.

The following information is included on the Detection history tab:

Time frame

The options are Last 30 days, Last 60 days, and Last 90 days.

Start date

The day that the anomaly started.

Last detected date

The last time that the anomaly was detected.

Severity

Represents how abnormal a certain anomaly is accounting for historical spending patterns. A low severity generally suggests a small spike compared to historical spend and a high severity suggests a big spike. However, a small spike with historically consistent spend is categorized as high severity. And, similarly, a big spike with irregular historical spend is categorized as low severity.

Duration

The duration that the anomaly lasted. An anomaly can be on-going.

Monitor name

The name of the anomaly monitor.

Service

The service that caused the anomaly. If the service field is empty, AWS has detected an anomaly, but the root cause is unclear.

Account

The account ID and account name that caused the anomaly. If the account is empty, AWS has detected an anomaly, but the root cause is undetermined.

Actual spend

The total amount you actually spent during the anomaly’s duration.

Expected spend

The amount our machine learning models expected you to spend during the anomaly’s duration, based on your historical spending pattern.

Total cost impact

The spend increase detected compared to the expected spend amount. It is calculated as actual spend - expected spend. For example, a total cost impact of $20 on a service monitor means that there was a $20 increase detected in a particular service with a total duration of the specified days.

Impact percentage

The percentage difference between the actual spend and expected spend. It is calculated as (total cost impact / expected spend) * 100. For example, if the total cost impact was $20 and the expected spend was $60, then the impact percentage would be 33.33%. This value cannot be calculated when expected spend is zero, so in those situations the value will show as “N/A”.

Assessment

For each detected anomaly, you can submit an assessment to help improve our anomaly detection systems. The possible values are Not submitted, Not an issue, or Accurate anomaly.

Viewing your detected anomalies and root causes

After you create your monitors, AWS Cost Anomaly Detection evaluates your future spend. Based on your defined alert subscriptions, you might start receiving alerts within 24 hours.

To view your anomalies from an email alert
  1. Choose the provided View in Anomaly Detection link.

  2. On the Anomaly details page, you can view the root cause analysis and cost impact of the anomaly.

  3. (Optional) Choose View in Cost Explorer to view a time series graph of the cost impact.

  4. (Optional) Choose View root cause in the Top ranked potential root causes table to see a time series graph that's filtered by the root cause.

  5. (Optional) Choose Submit assessment in the Did you find this detected anomaly to be helpful? information alert to provide feedback and help improve our detection accuracy.

To view your anomalies from the AWS Billing and Cost Management console
  1. Open the AWS Billing and Cost Management console at https://console.aws.amazon.com/costmanagement/home.

  2. In the navigation pane, choose Cost Anomaly Detection.

  3. (Optional) On the Detection history tab, use the search area to narrow the list of detected anomalies for a particular category. The categories that you can choose are Severity, Assessment, Service, Account, Usage type, Region, and Monitor type.

  4. (Optional) Choose the Start date for a particular anomaly to view the details.

  5. On the Anomaly details page, you can view the root cause analysis and cost impact of the anomaly.

  6. (Optional) Choose View in Cost Explorer to view a time series graph of the cost impact and, if necessary, dive deeper into the data.

  7. (Optional) Choose View root cause in the Top ranked potential root causes table to see a time series graph that's filtered by the root cause.

  8. (Optional) Choose Submit assessment in the Did you find this detected anomaly to be helpful? information alert to provide feedback and help improve our detection accuracy.

To view your anomalies from an Amazon SNS topic
  1. Subscribe an endpoint to the Amazon SNS topic that you created for a cost monitor with individual alerts. For instructions, see Subscribing to an Amazon SNS topic in the Amazon Simple Notification Service Developer Guide.

  2. After your endpoint receives messages from the Amazon SNS topic, open a message and then find the anomalyDetailsLink URL. The following example is a message from AWS Cost Anomaly Detection through Amazon SNS.

    { "accountId": "123456789012", "anomalyDetailsLink": "https://console.aws.amazon.com/cost-management/home#/anomaly-detection/monitors/abcdef12-1234-4ea0-84cc-918a97d736ef/anomalies/12345678-abcd-ef12-3456-987654321a12", "anomalyEndDate": "2021-05-25T00:00:00Z", "anomalyId": "12345678-abcd-ef12-3456-987654321a12", "anomalyScore": { "currentScore": 0.47, "maxScore": 0.47 }, "anomalyStartDate": "2021-05-25T00:00:00Z", "dimensionalValue": "ServiceName", "impact": { "maxImpact": 151, "totalActualSpend": 1301, "totalExpectedSpend": 300, "totalImpact": 1001, "totalImpactPercentage": 333.67 }, "monitorArn": "arn:aws:ce::123456789012:anomalymonitor/abcdef12-1234-4ea0-84cc-918a97d736ef", "rootCauses": [ { "linkedAccount": "AnomalousLinkedAccount", "linkedAccountName": "AnomalousLinkedAccountName", "region": "AnomalousRegionName", "service": "AnomalousServiceName", "usageType": "AnomalousUsageType" } ], "subscriptionId": "874c100c-59a6-4abb-a10a-4682cc3f2d69", "subscriptionName": "alertSubscription" }
  3. Open the anomalyDetailsLink URL in a web browser. The URL takes you to the associated Anomaly details page. This page shows the root cause analysis and cost impact of the anomaly.

Monitor types

You can choose the monitor type that fits your account structure. Currently, we offer the following monitor types:

  • AWS services - We recommend this monitor if you don't need to segment your spend by internal organizations or environments. This single monitor evaluates all the AWS services that are used by your individual AWS account for anomalies. When you add new AWS services, the monitor automatically begins to evaluate the new service for anomalies. That way, you don't have to manually configure your settings.

    Note

    Management accounts can have one AWS services monitor and up to 500 custom monitors (linked account, cost allocation tag, and cost category) for a total of 501 anomaly monitors. Member accounts only have access to the AWS services monitor.

  • Linked account - This monitor evaluates the total spend of an individual, or group of, member accounts. If your Organizations need to segment spend by team, product, services, or environment, this monitor is useful. The maximum number of member accounts that you can select for each monitor is 10.

  • Cost category - This monitor is recommended if you use cost categories to organize and manage your spend. This monitor type is restricted to one key:value pair.

  • Cost allocation tag - This monitor is similar to Linked account. If you to need to segment your spend by team, product, services, or environment, this monitor is useful. This monitor type is restricted to one key, but accepts multiple values. The maximum number of values that you can select for each monitor is 10.

We recommend that you do not create monitors that span multiple monitor types. This might lead to evaluating overlapping spends that generate duplicate alerts.

For more information about creating your Amazon SNS topic, see Creating an Amazon SNS topic for anomaly notifications.