Monitored Metrics Defaults - AMS Advanced Onboarding Guide

Monitored Metrics Defaults

The following table shows what is monitored and the default alerting thresholds. You can change the defaults with a change management request for change (RFC).

Note

CloudWatch launched extended retention of metrics in November 1, 2016. For more information, see CloudWatch Limits.

Alerts from baseline monitoring

Resource

Security alert

Alert name and trigger condition

Notes

For starred (*) alerts, AMS proactively assesses impact and remediates when possible; if remediation is not possible, AMS creates an incident. Where automation fails to remediate the issue, AMS informs you of the incident case and an AMS engineer is engaged. In addition, these alerts can be sent directly to your email (if you have opted in to the Direct-Customer-Alerts SNS topic).

Application Load Balancer (ALB) instance

No

RejectedConnectionCount

sum > 0 for 1 min, 5 consecutive times.

CloudWatch alarm if the number of connections that were rejected because the load balancer reached its maximum.

Application Load Balancer (ALB) target

No

TargetConnectionErrorCount

sum > 0 for 1 min, 5 consecutive times.

CloudWatch alarm if number of connections were unsuccessfully established between the load balancer and the registered instances.

HTTPCode_Target_5XX_Count

sum > 0 for 1 min, 5 consecutive times.

CloudWatch alarm on excess number of HTTP 5XX response codes generated by the targets.

Aurora instance

No

CPUUtilization

> 85% for 5 mins, 2 consecutive times.

CloudWatch alarm.

EC2 instance - all OSs

No

CPUUtilization*

>= 95% for 5 mins, 6 consecutive times.

CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as dead locks, infinite loops, malicious attacks, and other anomalies.

StatusCheckFailed

> 0 for 5 minutes, 3 consecutive times.

CloudWatch alarm.

Root Volume Usage

>= 85% for 5 mins, 6 consecutive times.

Memory Free*

MemoryFree < 5% for 5 minutes, 6 consecutive times.

Yes

EPS Malware

Malware found on instance.

CloudWatch event.

Amazon EC2 instance - Linux

No

Root Volume Inode Usage

Average >= 95% for 5 mins, 6 consecutive times.

CloudWatch alarm. Applied to Linux instances only.

Swap Free*

Memory Swap < 5% for 5 minutes, 6 consecutive times.

ElastiCache Cluster

No

CurrConnections = 65000

This alarm notifies AMS of the maximum connection limit of an ElastiCache Host.

CloudWatch Alarm. If you would like to update this threshold, contact AMS support.

ElastiCache Node

No

CPUUtilization

Average > predefined value for 15 mins, 2 consecutive times.

CloudWatch alarm. Default is 90. If Redis, use one the following values based on instance type:

  • cache.t1.micro: 90%

  • cache.m1.small: 90%

  • cache.m1.medium: 90%

  • cache.m1.large: 45%

  • cache.m1.xlarge: 22.5%

  • cache.m2.xlarge: 45%

  • cache.m2.4xlarge: 11.25%

  • cache.c1.xlarge: 11.25%

  • cache.t2.micro: 90%

  • cache.t2.small: 90%

  • cache.t2.medium: 45%

  • cache.m3.medium: 90%

  • cache.m3.large: 45%

  • cache.m3.xlarge: 22.5%

  • cache.m3.2xlarge: 11.25%

  • cache.r3.large: 45%

  • cache.r3.xlarge: 22.5%

  • cache.r3.2xlarge: 11.25%

  • cache.r3.4xlarge: 5.625%

  • cache.r3.8xlarge: 2.8125%

ElastiCache Node - memcached

No

SwapUsage

maximum > 50,000,000 bytes for 5 mins, 5 consecutive times.

CloudWatch alarm. Applied to memcached only.

OpenSearch cluster

No

ClusterStatus.red

maximum is >= 1 for 1 minute, 1 consecutive time.

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

CloudWatch alarm. At least one primary shard and its replicas are not allocated to a node. To learn more, see Red Cluster Status.

OpenSearch domain

No

KMSKeyError

>= 1 for 1 minute, 1 consecutive time.

CloudWatch alarm. The KMS encryption key that is used to encrypt data at rest in your domain is disabled. Re-enable it to restore normal operations. To learn more, see Encryption of Data at Rest for OpenSearch Service Service.

ClusterStatus.yellow

maximum is >= 1 for 1 minute, 1 consecutive time

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

At least one replica shard is not allocated to a node. To learn more, see Yellow Cluster Status.

FreeStorageSpace

minimum is <= 20480 for 1 minute, 1 consecutive time

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

A node in your cluster is down to 20 GiB of free storage space. To learn more, see Lack of Available Storage Space.

ClusterIndexWritesBlocked

>= 1 for 5 minutes, 1 consecutive time

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

The cluster is blocking write requests. To learn more, see ClusterBlockException.

Nodes

minimum is < x for 1 day, 1 consecutive time

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

x is the number of nodes in your cluster. This alarm indicates that at least one node in your cluster has been unreachable for one day. To learn more, see Failed Cluster Nodes.

CPUUtilization

average is >= 80% for 15 minutes, 3 consecutive times

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

100% CPU utilization is common, but sustained high averages are problematic. Consider using larger instance types or adding instances.

JVMMemoryPressure

maximum is >= 80% for 5 minutes, 3 consecutive times

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

The cluster could encounter out of memory errors if usage increases. Consider scaling vertically. Amazon ES uses half of an instance's RAM for the Java heap, up to a heap size of 32 GiB. You can scale instances vertically up to 64 GiB of RAM, at which point you can scale horizontally by adding instances.

MasterCPUUtilization

average is >= 50% for 15 minutes, 3 consecutive times

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

Consider using larger instance types for your dedicated master nodes. Because of their role in cluster stability and blue/green deployments, dedicated master nodes should have lower average CPU usage than data nodes.

MasterJVMMemoryPressure

maximum is >= 80% for 15 minutes, 1 consecutive time

AMS takes pro-active actions to reduce operational impact, when this alert is triggered.

Consider using larger instance types for your dedicated master nodes. Because of their role in cluster stability and blue/green deployments, dedicated master nodes should have lower average CPU usage than data nodes.

OpenSearch instance

No

AutomatedSnapshotFailure

maximum is >= 1 for 1 minute, 1 consecutive time.

CloudWatch alarm. An automated snapshot failed. This failure is often the result of a red cluster health status. See Red Cluster Status.

Elastic Load Balancing instance

No

SurgeQueueLength

> 100 for 1 minute, 15 consecutive times.

CloudWatch alarm if an excess number of requests are pending routing.

SpilloverCount

> 1 for 1 minute, 15 consecutive times.

CloudWatch alarm if an excess number of requests that were rejected because the surge queue is full.

GuardDuty service

Yes

Not applicable; all findings (threat purposes) are monitored. Each finding corresponds to an alert.

Changes in the GuardDuty findings. These changes include newly generated findings or subsequent occurrences of existing findings.

List of supported GuardDuty finding types are on GuardDuty Active Finding Types.

Health

Varies

Notifications sent when there are changes in the status of AWS Personal Health Dashboard (AWS Health) events.

Service event. Example: Scheduled EC2 instance store retirement.

AWS Managed Microsoft AD

No

Active Directory Status

AWS Managed Microsoft AD instance sends an active status event.

Service event. Emitted when the directory is operating normally after an event.

Impaired Directory Status

AWS Managed Microsoft AD instance sends an impaired directory status event.

Service event. Emitted when the directory is running in a degraded state. One or more issues have been detected, and not all directory operations may be working at full operational capacity.

Inoperable Directory Status

AWS Managed Microsoft AD instance sends an inoperable status event.

Service event. Emitted when the directory is not functional. All directory endpoints have reported issues.

Deleting Directory Status

AWS Managed Microsoft AD instance sends a deleting directory status event.

Service event. Emitted when the directory is currently being deleted.

Failed Directory Status

AWS Managed Microsoft AD instance sends a failed status event.

Service event. Emitted when the directory could not be created.

RestoreFailed Directory Status

AWS Managed Microsoft AD instance sends a restore failed directory status event.

Service event. Emitted when restoring the directory from a snapshot failed.

Amazon RDS instance

No

Failover not attempted

Amazon RDS is not attempting a requested failover because a failover recently occurred on the DB instance.

Service event. RDS-EVENT-0034, Amazon RDS Event Categories and Event Messages.

DB instance partial failover recovery complete

The instance has recovered from a partial failover.

Service event. RDS-EVENT-0065, Amazon RDS Event Categories and Event Messages.

DB instance fail

The DB instance has failed due to an incompatible configuration or an underlying storage issue. Begin a point-in-time-restore for the DB instance.

Service event. RDS-EVENT-0031, Amazon RDS Event Categories and Event Messages.

Invalid subnet IDs DB instance

The DB instance is in an incompatible network. Some of the specified subnet IDs are invalid or do not exist.

Service event. RDS-EVENT-0036, Amazon RDS Event Categories and Event Messages.

DB instance invalid parameters

For example, MySQL could not start because a memory-related parameter is set too high for this instance class, so the customer action would be to modify the memory parameter and reboot the DB instance.

Service event. RDS-EVENT-0035, Amazon RDS Event Categories and Event Messages.

Error create statspack user account

Error while creating Statspack user account PERFSTAT. Drop the account before adding the Statspack option.

Service event. RDS-EVENT-0058, Amazon RDS Event Categories and Event Messages.

DB instance without enhanced monitoring

Enhanced Monitoring can't be enabled without the enhanced monitoring IAM role. For information about creating the enhanced monitoring IAM role, see To create an IAM role for Amazon RDS Enhanced Monitoring.

Service event. RDS-EVENT-0079, Amazon RDS Event Categories and Event Messages.

DB instance enhanced monitoring disabled

Enhanced Monitoring was disabled due to an error making the configuration change. It's likely that the enhanced monitoring IAM role is configured incorrectly. For information about creating the enhanced monitoring IAM role, see To create an IAM role for Amazon RDS Enhanced Monitoring.

Service event. RDS-EVENT-0080, Amazon RDS Event Categories and Event Messages.

Invalid permissions recovery S3 bucket

The IAM role that you use to access your Amazon S3 bucket for SQL Server native backup and restore is configured incorrectly. For more information, see Setting Up for Native Backup and Restore.

Service event. RDS-EVENT-0081, Amazon RDS Event Categories and Event Messages.

DB instance read replica error

An error has occurred in the read replication process. For more information, see the event message. For information on troubleshooting Read Replica errors, see Troubleshooting a MySQL Read Replica Problem.

Service event. RDS-EVENT-0045, Amazon RDS Event Categories and Event Messages.

DB instance read replication ended

Replication on the Read Replica was ended.

Service event. RDS-EVENT-0057, Amazon RDS Event Categories and Event Messages.

DB instance recovery start

The SQL Server DB instance is re-establishing its mirror. Performance will be degraded until the mirror is reestablished. A database was found with non-FULL recovery model. The recovery model was changed back to FULL and mirroring recovery was started. (<dbname>: <recovery model found>[,…])”.

Service event. RDS-EVENT-0066, Amazon RDS Event Categories and Event Messages.

Low Storage alert triggers when the allocated storage for the DB instance has been exhausted.

RDS-EVENT-0007, see details at Using Amazon RDS event notification.

Low storage alert when the DB instance has consumed more than 90% of its allocated storage

RDS-EVENT-0089, see details at Amazon RDS Event Categories and Event Messages.

Notification service when scaling failed for the Aurora Serverless DB cluster.

RDS-EVENT-0143, see details at Amazon RDS Event Categories and Event Messages.

CPUUtilization

Average CPU utilization > 75% for 15 mins, 2 consecutive times.

CloudWatch alarm.

DiskQueueDepth

Sum is > 75 for 1 mins, 2 consecutive times.

FreeStorageSpace

Average < 1,073,741,824 bytes for 5 mins, 2 consecutive times.

ReadLatency

Average >= 1.001 seconds for 5 mins, 2 consecutive times.

WriteLatency

Average >= 1.005 seconds for 5 mins, 2 consecutive times.

SwapUsage

Average >= 104,857,600 bytes for 5 mins, 2 consecutive times.

Amazon Redshift cluster

No

HealthStatus

The health of the cluster <= 0 for 5 min, 1 consecutive times.

1 represents a healthy cluster.

MaintenanceMode

Cluster maintenance mode >= 1 for 5 min, 1 consecutive time.

1 represents ON state.

ReadLatency

The average time for disk read >= 1 for 5 min, 1 consecutive time.

None.

WriteLatency

The average time for disk write >= 1 for 5 min, 1 consecutive time.

None.

Amazon Macie

Yes

Newly generated alerts and updates to existing alerts.

Macie finds any changes in the findings. These changes include newly generated findings or subsequent occurrences of existing findings.

Amazon Macie alert. For a list of supported Macie alert types, see Macie Alerts. Note that Macie is not configured for all accounts.