Configuring CloudWatch to create OpsItems from alarms - AWS Systems Manager

Configuring CloudWatch to create OpsItems from alarms

You can configure Amazon CloudWatch to automatically create an OpsItem in Systems Manager OpsCenter when an alarm enters the ALARM state. Doing so enables you to quickly diagnose and remediate issues with AWS resources from a single console.

For example, you can configure an alarm to automatically create an OpsItem if there is a spike in HTTP errors generated by your application load balancer. To help you diagnose the issue, the OpsItem includes contextually relevant information such as the name and ID of the monitored AWS resource, alarm details, alarm history, and alarm timeline graph. For the monitored AWS resource, OpsCenter aggregates information from AWS Config, AWS CloudTrail logs, and Amazon CloudWatch Events, so you don't have to navigate across multiple console pages during your investigation. You can run Systems Manager Automation documents in OpsCenter for easy remediation.

This feature is available in all AWS Regions where Systems Manager is available. Note the following important details about this feature:

  • CloudWatch can create OpsItems in OpsCenter for metric and composite alarms.

  • Alarms must use the default aws namespace, such as AWS/EC2 (metric alarms only).

  • CloudWatch automatically creates a new service-linked role in AWS Identity and Access Management (IAM) when you configure an alarm to create OpsItems. The new role is named AWSServiceRoleForCloudWatchAlarms_ActionSSM. For more information about CloudWatch service-linked roles, see Using Service-Linked Roles for CloudWatch in the Amazon CloudWatch User Guide.

  • OpsCenter uses a deduplication feature to prohibit a single alarm from creating multiple OpsItems. For more information, see Reducing duplicate OpsItems.

For information about how to create a new alarm that automatically creates OpsItems in OpsCenter, see Create a CloudWatch alarm based on a static threshold in the Amazon CloudWatch User Guide. In Step 8 of that procedure, choose Systems Manager OpsCenter action and then complete the procedure.

Manually configure an existing alarm to create OpsItems (console)

Use the following procedure to edit an existing alarm and configure Systems Manager as the target of that alarm. When the alarm enters the ALARM state, CloudWatch creates a new OpsItem in OpsCenter.

To edit an existing alarm and configure Systems Manager as a target of that alarm

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the navigation pane, choose Alarms.

  3. Select the alarm, and then choose Actions, Edit.

  4. (Optional) Change settings in the Metrics and Conditions sections, and then choose Next.

  5. In the Systems Manager section, choose Add Systems Manager OpsCenter action.

  6. For Severity, choose a number.

    Note

    Severity is a user-defined value. You or your organization determine what each severity value means and any service-level agreement associated with each severity.

  7. (Optional) For Category, choose an option.

  8. Choose Next and complete the wizard.

An OpsItem created from an alarm shows CloudWatch alarm - 'alarm_name' is in ALARM state. To view details about a specific OpsItem created from an alarm, choose the OpsItem and then choose the Related resource details tab.

Note

If an alarm created an OpsItem and if you specified a deduplication string, the alarm won't create additional OpsItems even if you edit the alarm in CloudWatch. (If the OpsItem is resolved in OpsCenter, CloudWatch will create a new OpsItem.)

If you edit an alarm and change the severity or the category for any new OpsItems created from it, Systems Manager won't change the severity or category of OpsItems already created from that alarm. You can manually edit OpsItems to change details such as the severity or category.

Programmatically configure CloudWatch alarms to create OpsItems

You can programmatically configure Amazon CloudWatch alarms to create OpsItems by using the AWS CLI, AWS CloudFormation templates, or Java code snippets.

Before you begin

If you programmatically edit an existing alarm or create a new alarm that creates OpsItems, you must specify an Amazon Resource Name (ARN). This ARN identifies Systems Manager OpsCenter as the target for OpsItems created from the alarm. You can customize the ARN so that OpsItems created from the alarm include specific information such as severity or category. Each ARN includes the information described in the following table.

Parameter Details

Region (required)

The AWS Region where the alarm exists. For example: us-west-2. For information about AWS Regions where you can use OpsCenter, see AWS Systems Manager endpoints and quotas.

AWS account ID (required)

The same AWS account ID used to create the alarm. For example: 123456789012. Note that the account ID must be followed by a colon (:) and the parameter opsitem as shown in the examples below.

Severity (required)

A user-defined severity level for OpsItems created from the alarm. Valid values: 1,2,3,4

Note that because this is a user-defined value, you or your organization determine what each severity value means and any service-level agreement associated with each severity.

Category (optional)

A category for OpsItems created from the alarm. Valid values: Availability, Cost, Performance, Recovery, Security.

Create the ARN by using the following syntax. This ARN doesn't include the optional Category parameter.

arn:aws:ssm:Region:account_ID:opsitem:severity

Following is an example.

arn:aws:ssm:us-west-2:123456789012:opsitem:3

To create an ARN that uses the optional Category parameter, use the following syntax.

arn:aws:ssm:Region:account_ID:opsitem:severity#CATEGORY=category_name

Following is an example.

arn:aws:ssm:us-west-2:123456789012:opsitem:3#CATEGORY=Security

Manually configure an existing alarm to create OpsItems (AWS CLI)

Use the following command to configure an existing alarm to create OpsItems by using the AWS CLI. This command requires that you specify an Amazon Resource Name (ARN) for the alarm-actions parameter. For information about how to create the ARN, see Before you begin.

To configure an existing alarm to create OpsItems

  1. Install and configure the AWS CLI, if you have not already.

    For information, see Install or upgrade AWS command line tools.

  2. Run the following command to collect information about the alarm that you want to configure.

    aws cloudwatch describe-alarms --alarm-names "alarm_name"
  3. Run the following command to update an alarm.

    aws cloudwatch put-metric-alarm --alarm-name name \ --alarm-description "description" \ --metric-name name --namespace namespace \ --statistic statistic --period value --threshold value \ --comparison-operator value \ --dimensions "dimensions" --evaluation-periods value \ --alarm-actions arn:aws:ssm:Region:account_ID:opsitem:severity#CATEGORY=category_name \ --unit unit

    Here's an example.

    Linux
    aws cloudwatch put-metric-alarm --alarm-name cpu-mon \ --alarm-description "Alarm when CPU exceeds 70 percent" \ --metric-name CPUUtilization --namespace AWS/EC2 \ --statistic Average --period 300 --threshold 70 \ --comparison-operator GreaterThanThreshold \ --dimensions "Name=InstanceId,Value=i-12345678" --evaluation-periods 2 \ --alarm-actions arn:aws:ssm:us-east-1:123456789012:opsitem:3#CATEGORY=Security \ --unit Percent
    Windows
    aws cloudwatch put-metric-alarm --alarm-name cpu-mon ^ --alarm-description "Alarm when CPU exceeds 70 percent" ^ --metric-name CPUUtilization --namespace AWS/EC2 ^ --statistic Average --period 300 --threshold 70 ^ --comparison-operator GreaterThanThreshold ^ --dimensions "Name=InstanceId,Value=i-12345678" --evaluation-periods 2 ^ --alarm-actions arn:aws:ssm:us-east-1:123456789012:opsitem:3#CATEGORY=Security ^ --unit Percent

An OpsItem created from an alarm shows CloudWatch alarm - 'alarm_name' is in ALARM state. To view details about a specific OpsItem created from an alarm, choose the OpsItem and then choose the Related resource details tab.

Note

If an alarm created an OpsItem and if you specified a deduplication string, the alarm won't create additional OpsItems even if you edit the alarm in CloudWatch. (If the OpsItem is resolved in OpsCenter, CloudWatch will create a new OpsItem.)

If you edit an alarm and change the severity or the category for any new OpsItems created from it, Systems Manager won't change the severity or category of OpsItems already created from that alarm. You can manually edit OpsItems to change details such as the severity or category.

Use AWS CloudFormation templates to configure Amazon CloudWatch alarms to automatically create OpsItems

This section includes AWS CloudFormation templates that you can use to configure CloudWatch alarms to automatically create OpsItems. Each template requires that you specify an Amazon Resource Name (ARN) for the AlarmActions parameter. For information about how to create the ARN, see Before you begin.

Metric alarm

Use the following AWS CloudFormation template to create or update an Amazon CloudWatch metric alarm. The alarm specified in this template monitors Amazon EC2 instance status checks. If the alarm enters the ALARM state, it creates an OpsItem in OpsCenter.

{ "AWSTemplateFormatVersion": "2010-09-09", "Parameters" : { "RecoveryInstance" : { "Description" : "The EC2 instance ID to associate this alarm with.", "Type" : "AWS::EC2::Instance::Id" } }, "Resources": { "RecoveryTestAlarm": { "Type": "AWS::CloudWatch::Alarm", "Properties": { "AlarmDescription": "Run a recovery action when instance status check fails for 15 consecutive minutes.", "Namespace": "AWS/EC2" , "MetricName": "StatusCheckFailed_System", "Statistic": "Minimum", "Period": "60", "EvaluationPeriods": "15", "ComparisonOperator": "GreaterThanThreshold", "Threshold": "0", "AlarmActions": [ {"Fn::Join" : ["", ["arn:arn:aws:ssm:Region:account_ID:opsitem:severity#CATEGORY=category_name", { "Ref" : "AWS::Partition" }, ":ssm:", { "Ref" : "AWS::Region" }, { "Ref" : "AWS:: AccountId" }, ":opsitem:3" ]]} ], "Dimensions": [{"Name": "InstanceId","Value": {"Ref": "RecoveryInstance"}}] } } } }

Composite alarm

Use the following AWS CloudFormation template to create or update a composite alarm. A composite alarm consists of multiple metric alarms. If the alarm enters the ALARM state, it creates an OpsItem in OpsCenter.

"Resources":{ "HighResourceUsage":{ "Type":"AWS::CloudWatch::CompositeAlarm", "Properties":{ "AlarmName":"HighResourceUsage", "AlarmRule":"(ALARM(HighCPUUsage) OR ALARM(HighMemoryUsage)) AND NOT ALARM(DeploymentInProgress)", "AlarmActions":"arn:aws:ssm:Region:account_ID:opsitem:severity#CATEGORY=category_name", "AlarmDescription":"Indicates that the system resource usage is high while no known deployment is in progress" }, "DependsOn":[ "DeploymentInProgress", "HighCPUUsage", "HighMemoryUsage" ] }, "DeploymentInProgress":{ "Type":"AWS::CloudWatch::CompositeAlarm", "Properties":{ "AlarmName":"DeploymentInProgress", "AlarmRule":"FALSE", "AlarmDescription":"Manually updated to TRUE/FALSE to disable other alarms" } }, "HighCPUUsage":{ "Type":"AWS::CloudWatch::Alarm", "Properties":{ "AlarmDescription":"CPUusageishigh", "AlarmName":"HighCPUUsage", "ComparisonOperator":"GreaterThanThreshold", "EvaluationPeriods":1, "MetricName":"CPUUsage", "Namespace":"CustomNamespace", "Period":60, "Statistic":"Average", "Threshold":70, "TreatMissingData":"notBreaching" } }, "HighMemoryUsage":{ "Type":"AWS::CloudWatch::Alarm", "Properties":{ "AlarmDescription":"Memoryusageishigh", "AlarmName":"HighMemoryUsage", "ComparisonOperator":"GreaterThanThreshold", "EvaluationPeriods":1, "MetricName":"MemoryUsage", "Namespace":"CustomNamespace", "Period":60, "Statistic":"Average", "Threshold":65, "TreatMissingData":"breaching" } } }

An OpsItem created from an alarm shows CloudWatch alarm - 'alarm_name' is in ALARM state. To view details about a specific OpsItem created from an alarm, choose the OpsItem and then choose the Related resource details tab.

Note

If an alarm created an OpsItem and if you specified a deduplication string, the alarm won't create additional OpsItems even if you edit the alarm in CloudWatch. (If the OpsItem is resolved in OpsCenter, CloudWatch will create a new OpsItem.)

If you edit an alarm and change the severity or the category for any new OpsItems created from it, Systems Manager won't change the severity or category of OpsItems already created from that alarm. You can manually edit OpsItems to change details such as the severity or category.

Use Java code snippets to configure CloudWatch alarms to automatically create OpsItems

This section includes Java code snippets that you can use to configure CloudWatch alarms to automatically create OpsItems. Each snippet requires that you specify an Amazon Resource Name (ARN) for the validSsmActionStr parameter. For information about how to create the ARN, see Before you begin.

A specific alarm

Use the following Java code snippet to create or update a CloudWatch alarm. The alarm specified in this template monitors Amazon EC2 instance status checks. If the alarm enters the ALARM state, it creates an OpsItem in OpsCenter.

import com.amazonaws.services.cloudwatch.AmazonCloudWatch; import com.amazonaws.services.cloudwatch.AmazonCloudWatchClientBuilder; import com.amazonaws.services.cloudwatch.model.ComparisonOperator; import com.amazonaws.services.cloudwatch.model.Dimension; import com.amazonaws.services.cloudwatch.model.PutMetricAlarmRequest; import com.amazonaws.services.cloudwatch.model.PutMetricAlarmResult; import com.amazonaws.services.cloudwatch.model.StandardUnit; import com.amazonaws.services.cloudwatch.model.Statistic; private void putMetricAlarmWithSsmAction() { final AmazonCloudWatch cw = AmazonCloudWatchClientBuilder.defaultClient(); Dimension dimension = new Dimension() .withName("InstanceId") .withValue(instanceId); String validSsmActionStr = "arn:aws:ssm:Region:account_ID:opsitem:severity#CATEGORY=category_name"; PutMetricAlarmRequest request = new PutMetricAlarmRequest() .withAlarmName(alarmName) .withComparisonOperator( ComparisonOperator.GreaterThanThreshold) .withEvaluationPeriods(1) .withMetricName("CPUUtilization") .withNamespace("AWS/EC2") .withPeriod(60) .withStatistic(Statistic.Average) .withThreshold(70.0) .withActionsEnabled(false) .withAlarmDescription( "Alarm when server CPU utilization exceeds 70%") .withUnit(StandardUnit.Seconds) .withDimensions(dimension) .withAlarmActions(validSsmActionStr); PutMetricAlarmResult response = cw.putMetricAlarm(request); }

Update all alarms

Use the following Java code snippet to update all CloudWatch alarms in your AWS account to create OpsItems when an alarm enters the ALARM state.

import com.amazonaws.services.cloudwatch.AmazonCloudWatch; import com.amazonaws.services.cloudwatch.AmazonCloudWatchClientBuilder; import com.amazonaws.services.cloudwatch.model.DescribeAlarmsRequest; import com.amazonaws.services.cloudwatch.model.DescribeAlarmsResult; import com.amazonaws.services.cloudwatch.model.MetricAlarm; private void listMetricAlarmsAndAddSsmAction() { final AmazonCloudWatch cw = AmazonCloudWatchClientBuilder.defaultClient(); boolean done = false; DescribeAlarmsRequest request = new DescribeAlarmsRequest(); String validSsmActionStr = "arn:aws:ssm:Region:account_ID:opsitem:severity#CATEGORY=category_name"; while(!done) { DescribeAlarmsResult response = cw.describeAlarms(request); for(MetricAlarm alarm : response.getMetricAlarms()) { // assuming there are no alarm actions added for the metric alarm alarm.setAlarmActions(ImmutableList.of(validSsmActionStr)); } request.setNextToken(response.getNextToken()); if(response.getNextToken() == null) { done = true; } } }

An OpsItem created from an alarm shows CloudWatch alarm - 'alarm_name' is in ALARM state. To view details about a specific OpsItem created from an alarm, choose the OpsItem and then choose the Related resource details tab.

Note

If an alarm created an OpsItem and if you specified a deduplication string, the alarm won't create additional OpsItems even if you edit the alarm in CloudWatch. (If the OpsItem is resolved in OpsCenter, CloudWatch will create a new OpsItem.)

If you edit an alarm and change the severity or the category for any new OpsItems created from it, Systems Manager won't change the severity or category of OpsItems already created from that alarm. You can manually edit OpsItems to change details such as the severity or category.