Real-Time IoT Device Monitoring with Kinesis Data Analytics
Real-Time IoT Device Monitoring with Kinesis Data Analytics

The AWS Documentation website is getting a new look!
Try it now and let us know what you think. Switch to the new look >>

You can return to the original look by selecting English in the language selector above.

Solution Components

Amazon Kinesis Data Analytics Application

This solution includes an Amazon Kinesis Data Analytics application with SQL statements that compute metrics for the built-in dashboard. The application reads records from the Amazon Kinesis Data Firehose delivery stream and runs the SQL queries to emit specific IoT device metrics including the number of unique connected devices; the average, minimum, and maximum amount of time a device has been connected; the minimum, maximum, and total number of data points sent per device over one-minute time windows; the number of unique devices that are not connected; the average, minimum, and maximum amount of time a device has been disconnected; and anomaly detection, which are stored in Amazon DynamoDB. For more information, see Appendix A.

Amazon DynamoDB

The Real-Time IoT Device Monitoring solution creates an Amazon DynamoDB table: AnalyticsTable that stores the following information on metrics computed by the Kinesis data analytics application:

  • MetricType: The name of the computed metric

  • EventTime: The time the event was generated

  • ConcurrencyToken: The token used in the event of updates for optimistic locking

  • Data: The metric data, in JSON format

Device Monitoring Dashboard

The solution features a simple dashboard that loads data from Amazon DynamoDB into line charts every 10 seconds and bar charts every minute. The dashboard leverages Amazon Cognito for user authentication and is powered by web assets hosted in an Amazon Simple Storage Service (Amazon S3) bucket. Amazon CloudFront is used to restrict access to the solution’s website bucket contents.

The dashboard uses the open-source chart.js JavaScript library to draw charts using HTML5. The index.html file contains the HTML elements that render the charts in the dashboard. The dash.js file in the js folder contains the JavaScript that populates the dashboard with metrics. The Kinesis data application contains the SQL queries that compute metrics. For more information, see Appendix A.

After you successfully launch the solution, you will receive an email with instructions for logging into the dashboard.

The dashboard can also be customized to include additional metrics. For more information, see Appendix B.


        Real-Time IoT Device Monitoring with Kinesis Data Analytics dashboard

Figure 2: Real-Time IoT Device Monitoring with Kinesis Data Analytics dashboard

Anomaly Detection

The Real-Time IoT Device Monitoring with Kinesis Data Analytics solution leverages the built-in anomaly detection of Amazon Kinesis. When an anomaly score hits the threshold, the events are displayed in the solution’s anomaly detection graph. For example, the solution might normally record temperatures that range from 18°C to 24°C. If the solution then records a temperature of 37°C, the solution will detect the anomaly and display it in the graph.