Real-Time Web Analytics with Kinesis Data Analytics
Real-Time Web Analytics with Kinesis Data Analytics

Architecture Overview

Deploying this solution builds the following environment in the AWS Cloud.


        Real-Time Web Analytics with Kinesis Data Analytics architectural overview

Figure 1: Real-Time Web Analytics with Kinesis Data Analytics architecture

The solution's primary AWS CloudFormation template (real-time-web-analytics-with-kinesis) deploys a multi-AZ Amazon Virtual Private Cloud (Amazon VPC) network topology with two public subnets, a multi-AZ Application Load Balancer (ALB), two Amazon Elastic Compute Cloud (Amazon EC2) instances in an Auto Scaling group, an Amazon Kinesis Data Firehose delivery stream, Amazon Simple Storage Service (Amazon S3) buckets, a Kinesis data analytics application, a Kinesis data stream, an AWS Lambda function, Amazon DynamoDB tables, an Amazon Cognito user pool, an optional Amazon CloudWatch dashboard, and a real-time web usage dashboard to securely read and display the clickstream metrics stored in the DynamoDB table.

This solution also includes a template that deploys the solution in an existing VPC (real-time-web-analytics-with-kinesis-existing-vpc). For more information, see Appendix C.

When a user accesses your website, client-side JavaScript code on your web server sends a web analytic beacon to the solution's beacon web servers which log the request. An Application Load Balancer manages the incoming traffic.


        Request process

Figure 2: Request process

The beacon web servers send the data to the Kinesis data delivery stream, which archives the events in an Amazon S3 bucket and sends the data to the Kinesis data analytics application for processing. Once the data is processed, it is sent to the Kinesis data stream. A Lambda function (ProcessMetricsFunction) reads data from the stream and sends the data in real-time to a DynamoDB table to be stored for up to one week.