Detect Metrics Export - AWS IoT Core
How detect metric export worksSetting up Detect metrics exportCreate a Security Profile and enable metrics exportUpdate a Security Profile and turn on metrics exportUpdate a Security Profile to turn off metrics exportMetrics export CLI commandsMetrics export API reference

Detect Metrics Export

With the metrics export feature, you can export cloud-side, device-side, or custom metrics from AWS IoT Device Defender and publish them to an MQTT topic that you configure. This feature gives you the option to either choose your MQTT topic as an AWS IoT Rules Basic Ingest Topic, or create and subscribe to your own MQTT topic. You can configure metrics export via the AWS IoT Device Defender console, API, or CLI. This feature is available in all AWS Regions where AWS IoT Device Defender is available.

The first diagram below demonstrates how you can configure AWS IoT Device Defender to export metrics on a Basic Ingest topic. You can then leverage AWS IoT Rules and actions to route the exported metrics to various AWS IoT Rules-supported destinations.The second diagram demonstrates how you can configure AWS IoT Device Defender to publish data to a MQTT topic and consume the data by subscribing to that MQTT topic. It depicts a reference architecture where you configure AWS IoT Device Defender to export metrics on an MQTT topic. You can run an MQTT client in a container on Amazon Elastic Container Service, Lambda, or an Amazon EC2 instance that subscribes to the same MQTT topic. Whenever AWS IoT Device Defender publishes the data, the MQTT client receives it and processes it.

How detect metric export works

When setting up your Security Profile you can choose the metrics for export, specify the MQTT topic, and configure an IAM role that grants AWS IoT Device Defender Detect the necessary permissions to publish messages to the configured MQTT topic. You can configure an AWS IoT Rules Basic Ingest MQTT topic and send the exported metrics to AWS IoT Rules supported destinations. Refer to Rules for AWSAWS IoT in the AWS IoT Core Developer Guide for detailed instructions on setting up and configuring IoT Rules.

Device Defender Detect batches metric values for each configured metric and publishes them to configured MQTT topic at regular intervals. Cloud-side metrics are aggregated by summing metric values for the batched duration except for message byte size and disconnect duration, while custom and device-side metrics are not aggregated. For message byte size, the export values are the minimum, maximum and total byte size for the batched duration. For disconnect duration, the export value is the disconnect duration—in seconds— for all tracked devices. This occurs whenever there is a connection or a disconnection event, as well as for every one hour interval. For connected devices or connection events, the value will be zero. For more information on cloud-side metrics, device-side metrics, and custom metrics, see the AWS IoT Device Defender Guide:

As AWS IoT Device Defender batches metrics, you can configure AWS IoT Rules to export them to different destinations. Refer to AWS IoT rule actionsfor a list of supported destinations. To send individual metrics within a batched export message to an AWS IoT Rules destination, you can use the batchMode option for AWS IoT rules actions. If your preferred AWS IoT Rules destination lacks batchMode support and you want to send individual metrics within a batched message, you can use intermediary actions such as Lambda or Kinesis Data Streams to extract and send the individual metrics.

Setting up Detect metrics export

Setting up Detect metrics export in the AWS console

In this tutorial, you learn how to create, view, and edit a new Security Profile that includes metrics export in the console.

Prerequisites

Before you start this tutorial, you must create an IAM role.For more information on how to create an IAM role, see Creating IAM role in the IAM User Guide.You must also have an AWS account that you can sign in to as an AWS Identity and Access Management (IAM) user with correct permissions. For more information on the AWS IoT Device Defender Detect permissions needed for metrics export, see Permissions

Creating a new Security Profile including metrics export (console)

In order for metric behavior data to be exported, you must first configure a Security Profile to include metric exporting. Although Device Defender supports both non-reserved topic and rule engine basic ingest topic, the following procedure details how to set up a Rule-based Security Profile that includes Detect metrics export in the console.

  1. Open the AWS IoT console. In the navigation pane, expand Security, Detect, and then choose Create Security Profile.

  2. Select Create Rule-based anomaly Detect profile from the dropdown.

  3. To specify your security profile properties, enter your Security Profile name and choose a group of devices to target for anomalies from the Target dropdown. (Optional) Include a description and tags to label AWS resources. Then, choose Next.

  4. To configure metric behaviors, select a metric from the dropdown. Then, you can either define the thresholds to receive alerts or retain the metric. You can define the behavior threshold to be alerted on when your device does not meet behavior expectations.

  5. If you choose to be alerted, then specify the behavior name and the behavior conditions. Choose Next.

  6. To configure metrics export, select the option to Turn on metrics export.

  7. Enter an MQTT topic name that will be used to publish your metric data to AWS IoT Core. Choose an IAM role to grant AWS IoT the permission "AWS IoT:Publish" to publish messages to the configured topic. Then select the metrics you want to export. Choose Next.

    NOTE: Use the forward slash to represent hierarchical information when entering your MQTT topic name. For example, $aws/rules/rule-name.

  8. To configure the Amazon SNS notifications to have alerts sent to your AWS console when a device violates a set behavior, choose or create an Amazon SNS topic and IAM role. Then, choose Next.

  9. Review your configurations, then choose Next.

Viewing and editing Security Profile details (console)

  1. To view or edit Security Profile details, choose Detect then choose Security Profiles in the navigation bar in the AWS IoT Device Defender console.

  2. Choose the Security Profile you created to include metrics export. Choose Edit from the Actions button dropdown.

  3. To edit the target device group(s) remove or add targets from the dropdown menu. Then choose Next.

  4. To edit metric behavior configurations, select the option to be alerted and then define the conditions when the metric behaviors are met. Then choose Next.

  5. To turn off metrics export configurations, choose Turn off export metrics. Then choose Next

  6. To configure the Amazon SNS notifications to have alerts sent to your AWS console when a device violates a set behavior, choose or create an Amazon SNS topic and IAM role. Then, choose Next.

  7. Review your configurations, then choose Next.

Create a Security Profile and enable metrics export

  • Use the create-security-profile command to create your Security Profile and enable metrics export.

    1. To turn on metrics export, set the value exportMetric as true in both Behavior and AdditionalMetricsToRetainV2 to indicate if Detect needs to export the corresponding metrics.

    2. You also need to include the value for MetricsExportConfig.This specifies the MQTT topic and role ARN required for metrics export.

      Note

      The mqttTopic is included so that Device Defender Detect can publish messages. The role ARN has permission to publish MQTT messages, after which Device Defender Detect can assume the role and publish messages on your behalf.

aws iot create-security-profile \
    --security-profile-name CreateSecurityProfileWithMetricsExport \
    --security-profile-description "create security profile with metrics export enabled"  \
    --behaviors "[{\"name\":\"BehaviorNumAuthz\",\"metric\":\"aws:num-authorization-failures\",\"criteria\":{\"comparisonOperator\":\"less-than\",\"value\":{\"count\":5}, \"consecutiveDatapointsToAlarm\":1,\"consecutiveDatapointsToClear\":1,\"durationSeconds\":300},\"exportMetric\":true}]" \
    --metrics-export-config "{\"mqttTopic\":\"\$aws/rules/metricsExportRule\",\"roleArn\":\"arn:aws:iam::123456789012:role/iot-test-role\"}" \
    --region us-east-1

Output:

{
    "securityProfileName": "CreateSecurityProfileWithMetricsExport",
    "securityProfileArn": "arn:aws:iot:us-east-1:123456789012:securityprofile/CreateSecurityProfileWithMetricsExport"
}

Update a Security Profile and turn on metrics export

Use the update-security-profile command to update an existing Security Profile and enable metrics export.

  1. To turn on metrics export, set the value exportMetric as true in both Behavior and AdditionalMetricsToRetainV2 to indicate if Detect needs to export the corresponding metrics.

  2. You also need to include the value for MetricsExportConfig.This specifies the MQTT topic and role ARN required for metrics export.

aws iot update-security-profile \
    --security-profile-name UpdateSecurityProfileWithMetricsExport \
    --security-profile-description "update an existing security profile to enable metrics export"  \
    --behaviors "[{\"name\":\"BehaviorNumAuthz\",\"metric\":\"aws:num-authorization-failures\",\"criteria\":{\"comparisonOperator\":\"less-than\",\"value\":{\"count\":5}, \"consecutiveDatapointsToAlarm\":1,\"consecutiveDatapointsToClear\":1,\"durationSeconds\":300},\"exportMetric\":true}]" \
    --metrics-export-config "{\"mqttTopic\":\"\$aws/rules/metricsExportRule\",\"roleArn\":\"arn:aws:iam::123456789012:role/iot-test-role\"}" \
    --region us-east-1

Output:

{
    "securityProfileName": "UpdateSecurityProfileWithMetricsExport",
    "securityProfileArn": "arn:aws:iot:us-east-1:123456789012:securityprofile/UpdateSecurityProfileWithMetricsExport",
    "securityProfileDescription": "update an existing security profile to enable metrics export",
    "behaviors": [
        {
            "name": "BehaviorNumAuthz",
            "metric": "aws:num-authorization-failures",
            "criteria": {
                "comparisonOperator": "less-than",
                "value": {
                    "count": 5
                },
                "durationSeconds": 300,
                "consecutiveDatapointsToAlarm": 1,
                "consecutiveDatapointsToClear": 1
            },
            "exportMetric": true
        }
    ],
    "version": 2,
    "creationDate": "2023-11-09T16:18:37.183000-08:00",
    "lastModifiedDate": "2023-11-09T16:20:15.486000-08:00",
    "metricsExportConfig": {
        "mqttTopic": "$aws/rules/metricsExportRule",
        "roleArn": "arn:aws:iam::123456789012:role/iot-test-role"
    }
}

Update a Security Profile to turn off metrics export

Use the update-security-profile command to update an existing Security Profile and turn off metrics export.

  • Use the option --delete-metrics-export-config to update your Security Profile and remove the metrics export configuration.

aws iot update-security-profile \
    --security-profile-name UpdateSecurityProfileToDisableMetricsExport \
    --security-profile-description "update an existing security profile to disable metrics export"  \
    --behaviors "[{\"name\":\"BehaviorNumAuthz\",\"metric\":\"aws:num-authorization-failures\",\"criteria\":{\"comparisonOperator\":\"less-than\",\"value\":{\"count\":5}, \"consecutiveDatapointsToAlarm\":1,\"consecutiveDatapointsToClear\":1,\"durationSeconds\":300}}]" \
    --delete-metrics-export-config \
    --region us-east-1

Output:

{
    "securityProfileName": "UpdateSecurityProfileToDisableMetricsExport",
    "securityProfileArn": "arn:aws:iot:us-east-1:123456789012:securityprofile/UpdateSecurityProfileWithMetricsExport",
    "securityProfileDescription": "update an existing security profile to disable metrics export",
    "behaviors": [
        {
            "name": "BehaviorNumAuthz",
            "metric": "aws:num-authorization-failures",
            "criteria": {
                "comparisonOperator": "less-than",
                "value": {
                    "count": 5
                },
                "durationSeconds": 300,
                "consecutiveDatapointsToAlarm": 1,
                "consecutiveDatapointsToClear": 1
            }
        }
    ],
    "version": 2,
    "creationDate": "2023-11-09T16:18:37.183000-08:00",
    "lastModifiedDate": "2023-11-09T16:31:16.265000-08:00"
}

For more information, see Detect Commands in the AWS IoT Developer Guide.

Metrics export CLI commands

You can use the following CLI commands to create and manage Detect metrics export.

Metrics export API reference

The following APIs can be used to create and manage Detect metrics export.