Configuring alerts in Amazon OpenSearch Service - Amazon OpenSearch Service (successor to Amazon Elasticsearch Service)

Configuring alerts in Amazon OpenSearch Service

Configure alerts in Amazon OpenSearch Service to get notified when data from one or more indices meets certain conditions. For example, you might want to receive an email if your application logs more than five HTTP 503 errors in one hour, or you might want to page a developer if no new documents have been indexed in the last 20 minutes.

Alerting requires OpenSearch or Elasticsearch 6.2 or later. For full documentation, including API descriptions, see the OpenSearch documentation. This topic highlights the differences in alerting in OpenSearch Service compared to the open-source version.

To get started with alerting

  1. Choose Alerting from the OpenSearch Dashboards main menu.

  2. Set up a destination for the alert. Choose between Slack, Amazon Chime, a custom webhook, or Amazon SNS. As you might imagine, notifications require connectivity to the destination. For example, your OpenSearch Service domain must be able to connect to the internet to notify a Slack channel or send a custom webhook to a third-party server. The OpenSearch Service domain must have a public IP address to send alerts to a custom webhook.

  3. Create a monitor in one of three ways: visually, using a query, or using an anomaly detector.

  4. Define a condition to trigger the monitor.

  5. (Optional) Add one or more actions to the monitor.

    Tip

    After an action successfully sends a message, securing access to that message (for example, access to a Slack channel) is your responsibility. If your domain contains sensitive data, consider using triggers without actions and periodically checking Dashboards for alerts.

For detailed steps, see Monitors in the OpenSearch documentation.

Differences

Compared to OpenSearch, alerting in Amazon OpenSearch Service has some notable differences.

Amazon SNS support

OpenSearch Service supports Amazon Simple Notification Service (Amazon SNS) for notifications. This integration means that in addition to standard destinations (Slack, custom webhooks, and Amazon Chime), you can also send emails, text messages, and even run AWS Lambda functions using SNS topics. For more information about Amazon SNS, see the Amazon Simple Notification Service Developer Guide.

To add Amazon SNS as a destination

  1. Choose Alerting from the OpenSearch Dashboards main menu.

  2. Go to the Destinations tab and then choose Add destination.

  3. Provide a unique name for the destination.

  4. For Type, choose Amazon SNS.

  5. Provide the SNS topic ARN.

  6. Provide the ARN for an IAM role within your account that has the following trust relationship and permissions (at minimum):

    { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "es.amazonaws.com" }, "Action": "sts:AssumeRole" }] }
    { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Action": "sns:Publish", "Resource": "sns-topic-arn" }] }

    For more information, see Adding IAM Identity Permissions in the IAM User Guide.

  7. Choose Create.

Alerting settings

OpenSearch Service lets you modify the following alerting settings:

  • plugins.scheduled_jobs.enabled

  • plugins.alerting.alert_history_enabled

  • plugins.alerting.alert_history_max_age

  • plugins.alerting.alert_history_max_docs

  • plugins.alerting.alert_history_retention_period

  • plugins.alerting.alert_history_rollover_period

  • plugins.alerting.filter_by_backend_roles

All other settings use the default values which you can't change.

To disable alerting, send the following request:

PUT _cluster/settings { "persistent" : { "plugins.scheduled_jobs.enabled" : false } }

The following request configures alerting to automatically delete history indices after seven days, rather than the default 30 days:

PUT _cluster/settings { "persistent": { "plugins.alerting.alert_history_retention_period": "7d" } }

If you previously created monitors and want to stop the creation of daily alerting indices, delete all alert history indices:

DELETE .plugins-alerting-alert-history-*

To reduce shard count for history indices, create an index template. The following request sets history indices for both alerting and Index State Management to one shard and one replica:

PUT _template/template-name { "index_patterns": [".opendistro-alerting-alert-history-*", ".opendistro-ism-managed-index-history-*"], "template": { "settings": { "number_of_shards": 1, "number_of_replicas": 1 } } }

Depending on your tolerance for data loss, you might even consider using zero replicas. For more information about creating and managing index templates, see Index templates in the OpenSearch documentation.

Alerting permissions

Alerting supports fine-grained access control. For details on mixing and matching permissions to fit your use case, see Alerting security in the OpenSearch documentation.