Device-side metrics - AWS IoT Core

Device-side metrics

When creating a Security Profile, you can specify your IoT device's expected behavior by configuring behaviors and thresholds for metrics generated by IoT devices. The following are device-side metrics, which are metrics from agents that you install on your devices.

Bytes out (aws:all-bytes-out)

The number of outbound bytes from a device during a given time period.

Use this metric to specify the maximum or minimum amount of outbound traffic that a device should send, measured in bytes, in a given period of time.

Can be used with ML Detect: No

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: bytes

Duration: a non-negative integer. Valid values are 300, 600, 900, 1800, or 3600 seconds.

Example

{ "name": "TCP outbound traffic", "metric": "aws:all-bytes-out", "criteria": { "comparisonOperator": "less-than", "value": { "count": 4096 }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 5, "consecutiveDatapointsToClear": 4 } }

Example using a statisticalThreshold

{ "name": "TCP outbound traffic", "metric": "aws:all-bytes-out", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p50" }, "durationSeconds": 900, "consecutiveDatapointsToAlarm": 5, "consecutiveDatapointsToClear": 4 } }

Bytes in (aws:all-bytes-in)

The number of inbound bytes to a device during a given time period.

Use this metric to specify the maximum or minimum amount of inbound traffic that a device should receive, measured in bytes, in a given period of time.

Can be used with ML Detect: No

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: bytes

Duration: a non-negative integer. Valid values are 300, 600, 900, 1800, or 3600 seconds.

Example

{ "name": "TCP inbound traffic", "metric": "aws:all-bytes-in", "criteria": { "comparisonOperator": "less-than", "value": { "count": 4096 }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 1, "consecutiveDatapointsToClear": 3 } }

Example using a statisticalThreshold

{ "name": "TCP inbound traffic", "metric": "aws:all-bytes-in", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p90" }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 1, "consecutiveDatapointsToClear": 3 } }

Listening TCP port count (aws:num-listening-tcp-ports)

The number of TCP ports the device is listening on.

Use this metric to specify the maximum number of TCP ports that each device should monitor.

Can be used with ML Detect: No

Unit: failures

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: failures

Duration: a non-negative integer. Valid values are 300, 600, 900, 1800, or 3600 seconds.

Example

{ "name": "Max TCP Ports", "metric": "aws:num-listening-tcp-ports", "criteria": { "comparisonOperator": "less-than", "value": { "count": 5 }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 2, "consecutiveDatapointsToClear": 1 } }

Example using a statisticalThreshold

{ "name": "Max TCP Ports", "metric": "aws:num-listening-tcp-ports", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p50" }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 2, "consecutiveDatapointsToClear": 1 } }

Listening UDP port count (aws:num-listening-udp-ports)

The number of UDP ports the device is listening on.

Use this metric to specify the maximum number of UDP ports that each device should monitor.

Can be used with ML Detect: No

Unit: failures

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: failures

Duration: a non-negative integer. Valid values are 300, 600, 900, 1800, or 3600 seconds.

Example

{ "name": "Max UDP Ports", "metric": "aws:num-listening-udp-ports", "criteria": { "comparisonOperator": "less-than", "value": { "count": 5 }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 2, "consecutiveDatapointsToClear": 1 } }

Example using a statisticalThreshold

{ "name": "Max UDP Ports", "metric": "aws:num-listening-udp-ports", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p50" }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 2, "consecutiveDatapointsToClear": 1 } }

Packets out (aws:all-packets-out)

The number of outbound packets from a device during a given time period.

Use this metric to specify the maximum or minimum amount of total outbound traffic that a device should send in a given period of time.

Can be used with ML Detect: No

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: packets

Duration: a non-negative integer. Valid values are 300, 600, 900, 1800, or 3600 seconds.

Example

{ "name": "TCP outbound traffic", "metric": "aws:all-packets-out", "criteria": { "comparisonOperator": "less-than", "value": { "count": 100 }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 1, "consecutiveDatapointsToClear": 3 } }

Example using a statisticalThreshold

{ "name": "TCP outbound traffic", "metric": "aws:all-packets-out", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p90" }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 1, "consecutiveDatapointsToClear": 3 } }

Packets in (aws:all-packets-in)

The number of inbound packets to a device during a given time period.

Use this metric to specify the maximum or minimum amount of total inbound traffic that a device should receive in a given period of time.

Can be used with ML Detect: No

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: packets

Duration: a non-negative integer. Valid values are 300, 600, 900, 1800 or 3600 seconds.

Example

{ "name": "TCP inbound traffic", "metric": "aws:all-packets-in", "criteria": { "comparisonOperator": "less-than", "value": { "count": 100 }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 2, "consecutiveDatapointsToClear": 1 } }

Example using a statisticalThreshold

{ "name": "TCP inbound traffic", "metric": "aws:all-packets-in", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p90" }, "durationSeconds": 300, "consecutiveDatapointsToAlarm": 2, "consecutiveDatapointsToClear": 1 } }

Destination IPs (aws:destination-ip-addresses)

A set of IP destinations.

Use this metric to specify a set of allowed (formerly referred to as whitelisted) or denied (formerly referred to as blacklisted) Classless Inter-Domain Routings (CIDR) from which each device must or must not connect to AWS IoT.

Can be used with ML Detect: No

Operators: in-cidr-set | not-in-cidr-set

Values: a list of CIDRs

Units: n/a

Example

{ "name": "Denied source IPs", "metric": "aws:source-ip-address", "criteria": { "comparisonOperator": "not-in-cidr-set", "value": { "cidrs": [ "12.8.0.0/16", "15.102.16.0/24" ] } } }

Listening TCP ports (aws:listening-tcp-ports)

The TCP ports that the device is listening on.

Use this metric to specify a set of allowed (formerly referred to as whitelisted) or denied (formerly referred to as blacklisted) TCP ports on which each device must or must not listen.

Can be used with ML Detect: No

Operators: in-port-set | not-in-port-set

Values: a list of ports

Units: n/a

{ "name": "Listening TCP Ports", "metric": "aws:listening-tcp-ports", "criteria": { "comparisonOperator": "in-port-set", "value": { "ports": [ 443, 80 ] } } }

Listening UDP ports (aws:listening-udp-ports)

The UDP ports that the device is listening on.

Use this metric to specify a set of allowed (formerly referred to as whitelisted) or denied (formerly referred to as blacklisted) UDP ports on which each device must or must not listen.

Can be used with ML Detect: No

Operators: in-port-set | not-in-port-set

Values: a list of ports

Units: n/a

{ "name": "Listening UDP Ports", "metric": "aws:listening-udp-ports", "criteria": { "comparisonOperator": "in-port-set", "value": { "ports": [ 1025, 2000 ] } } }

Established TCP connections count (aws:num-established-tcp-connections)

The number of TCP connections for a device.

Use this metric to specify the maximum or minimum number of active TCP connections that each device should have. (All TCP states)

Can be used with ML Detect: No

Operators: less-than | less-than-equals | greater-than | greater-than-equals

Value: a non-negative integer

Units: connections

Example

{ "name": "TCP Connection Count", "metric": "aws:num-established-tcp-connections", "criteria": { "comparisonOperator": "less-than", "value": { "count": 3 }, "consecutiveDatapointsToAlarm": 3, "consecutiveDatapointsToClear": 3 } }

Example using a statisticalThreshold

{ "name": "TCP Connection Count", "metric": "aws:num-established-tcp-connections", "criteria": { "comparisonOperator": "less-than", "statisticalThreshold": { "statistic": "p90" }, "durationSeconds": 900, "consecutiveDatapointsToAlarm": 3, "consecutiveDatapointsToClear": 3 } }

Device metrics document specification

Overall structure

Long name

Short name

Required

Type

Constraints

Notes

header

hed

Y

Object

Complete block required for well-formed report.

metrics

met

Y

Object

Complete block required for well-formed report.

Header block

Long name

Short name

Required

Type

Constraints

Notes

report_id

rid

Y

Integer

Monotonically increasing value. Epoch timestamp recommended.

version

v

Y

String

Major.Minor

Minor increments with addition of field. Major increments if metrics removed.

Metrics block:

TCP connections

Long name

Short name

Parent element

Required

Type

Constraints

Notes

tcp_connections

tc

metrics

N

Object

established_connections

ec

tcp_connections

N

List<Connection>

Established TCP state

connections

cs

established_connections

N

List<Object>

remote_addr

rad

connections

Y

Number

ip:port

IP can be IPv6 or IPv4

local_port

lp

connections

N

Number

>= 0

local_interface

li

connections

N

String

Interface name

total

t

established_connections

N

Number

>= 0

Number of established connections

Listening TCP ports

Long name

Short name

Parent element

Required

Type

Constraints

Notes

listening_tcp_ports

tp

metrics

N

Object

ports

pts

listening_tcp_ports

N

List<Port>

> 0

port

pt

ports

N

Number

> 0

ports should be numbers greater than 0

interface

if

ports

N

String

Interface name

total

t

listening_tcp_ports

N

Number

>= 0

Listening UDP ports

Long name

Short name

Parent element

Required

Type

Constraints

Notes

listening_udp_ports

up

metrics

N

Object

ports

pts

listening_udp_ports

N

List<Port>

> 0

port

pt

ports

N

Number

> 0

Ports should be numbers greater than 0

interface

if

ports

N

String

Interface name

total

t

listening_udp_ports

N

Number

>= 0

Network statistics

Long name

Short name

Parent element

Required

Type

Constraints

Notes

network_stats

ns

metrics

N

Object

bytes_in

bi

network_stats

N

Number

Delta Metric, >= 0

bytes_out

bo

network_stats

N

Number

Delta Metric, >= 0

packets_in

pi

network_stats

N

Number

Delta Metric, >= 0

packets_out

po

network_stats

N

Number

Delta Metric, >= 0

Example

The following JSON structure uses long names.

{ "header": { "report_id": 1530304554, "version": "1.0" }, "metrics": { "listening_tcp_ports": { "ports": [ { "interface": "eth0", "port": 24800 }, { "interface": "eth0", "port": 22 }, { "interface": "eth0", "port": 53 } ], "total": 3 }, "listening_udp_ports": { "ports": [ { "interface": "eth0", "port": 5353 }, { "interface": "eth0", "port": 67 } ], "total": 2 }, "network_stats": { "bytes_in": 29358693495, "bytes_out": 26485035, "packets_in": 10013573555, "packets_out": 11382615 }, "tcp_connections": { "established_connections": { "connections": [ { "local_interface": "eth0", "local_port": 80, "remote_addr": "192.168.0.1:8000" }, { "local_interface": "eth0", "local_port": 80, "remote_addr": "192.168.0.1:8000" } ], "total": 2 } ], "total":3 } }, "custom_metrics":{ "MyMetricOfType_Number":[ { "number":1.0 } ], "MyMetricOfType_NumberList":[ { "number_list":[ 1.0, 2.0, 3.0 ] } ], "MyMetricOfType_StringList":[ { "string_list":[ "value_1", "value_2" ] } ], "MyMetricOfType_IpList":[ { "ip_list":[ "172.0.0.0", "172.0.0.10" ] } ] } }

Example JSON structure using short names

{ "hed": { "rid": 1530305228, "v": "1.0" }, "met": { "tp": { "pts": [ { "if": "eth0", "pt": 24800 }, { "if": "eth0", "pt": 22 }, { "if": "eth0", "pt": 53 } ], "t": 3 }, { "if": "eth0", "pt": 22 }, { "if": "eth0", "pt": 53 } ], "t": 3 }, "up": { "pts": [ { "if": "eth0", "pt": 5353 }, { "if": "eth0", "pt": 67 } ], "t": 2 }, "ns": { "bi": 29359307173, "bo": 26490711, "pi": 10014614051, "po": 11387620 }, "tc": { "ec": { "cs": [ { "li": "eth0", "lp": 80, "rad": "192.168.0.1:8000" }, { "li": "eth0", "lp": 80, "rad": "192.168.0.1:8000" } ], "t": 2 } } }, "cmet": { "MyMetricOfType_Number": [ { "number": 1 } ], "MyMetricOfType_NumberList": [ { "number_list": [ 1, 2, 3 ] } ], "MyMetricOfType_StringList": [ { "string_list": [ "value_1", "value_2" ] } ], "MyMetricOfType_IpList": [ { "ip_list": [ "172.0.0.0", "172.0.0.10" ] } ] } }

Sending metrics from devices

AWS IoT Device Defender Detect can collect, aggregate, and monitor metrics data generated by AWS IoT devices to identify devices that exhibit abnormal behavior. This section shows you how to send metrics from a device to AWS IoT Device Defender.

You must securely deploy the AWS IoT SDK version two on your AWS IoT connected devices or device gateways to collect device-side metrics. AWS IoT Device Defender provides sample agents to use as examples when you create your own. If you can't provide device metrics, you can still get limited functionality based on cloud-side metrics. See the full list of SDKs here.

There are two methods for configuring your device to publish metrics, through AWS IoT Device Client and AWS IoT Device Defender sample agent. Generally you should use AWS IoT Device Client because it provides a single agent that covers the features present in both AWS IoT Device Defender and AWS IoT Device Management. These features include jobs, secure tunneling, AWS IoT Device Defender metrics publishing and more. If you define custom metrics for your device to monitor, then you should use AWS IoT Device Defender sample agent in Python to send data.

Using the AWS IoT Device Client to publish metrics

To install AWS IoT Device Client, you can download it from Github. After you've installed the AWS IoT Device Client on the device for which you want to collect device-side data, you must configure it to send device-side metrics to AWS IoT Device Defender. Verify that the AWS IoT Device Client configuration file has the following parameters set in the device-defender section:

"device-defender": { "enabled": true, "interval-in-seconds": 300 }
Warning

At a minimum, you should set the time interval to 300 seconds. If you set the time interval to anything less than 300 seconds, your metric data may be throttled.

After you've updated your configuration, you can create security profiles and behaviors in the AWS IoT Device Defender console to monitor the metrics that your devices publish to the cloud. You can find published metrics in the AWS IoT Core console by choosing Defend, Detect, and then Metrics.

Using the AWS IoT Device Defender sample agent to publish metrics

You can use the AWS IoT Device Defender sample agent to monitor device-side metrics and custom metrics from AWS IoT devices.

A sample device metric reporting agent is currently available in C at https://github.com/aws-samples/aws-iot-device-defender-agent-c. There is also a sample device metric reporting agent available in Python on GitHub at https://github.com/aws-samples/aws-iot-device-defender-agent-sdk-python. Custom metrics are only supported by the Python sample agent. Specifically see the greengrass_defender_agent.py file for a sample agent used with AWS IoT Greengrass devices.

To use the sample agents or create your own custom agent, you must install the AWS IoT Device SDK. To find resources for your development language, see AWS IoT Core Resources.

  • All agents must create a connection to AWS IoT and publish metrics to one of these reserved AWS IoT Device Defender MQTT topics:

    $aws/things/THING_NAME/defender/metrics/json

    or

    $aws/things/THING_NAME/defender/metrics/cbor

    AWS IoT Device Defender uses one of these topics to reply with the receipt status of your metrics reports:

    $aws/things/THING_NAME/defender/metrics/json/accepted
    $aws/things/THING_NAME/defender/metrics/cbor/accepted
    $aws/things/THING_NAME/defender/metrics/json/rejected
    $aws/things/THING_NAME/defender/metrics/cbor/rejected
  • To report metrics, a device must be registered as a thing in AWS IoT.

  • A device should, generally, send a metric report once every five minutes. Devices are throttled so they can't make more than one metric report every five minutes.

  • Devices must report current metrics. Historical metrics reporting isn't supported.

  • You can optionally use Jobs to change how often Device Defender sends metrics. An example is included with the AWS IoT Device Defender Agent C samples. For more information, see the README.md.