Gaming Analytics Pipeline
Gaming Analytics Pipeline

Architecture Overview

Deploying this solution with the default parameters builds the following environment in the AWS Cloud.

        Gaming Analytics Pipeline architecture

Figure 1: Gaming Analytics Pipeline architecture

The AWS CloudFormation template deploys an Amazon Virtual Private Cloud (Amazon VPC) network topology with two private subnets; two Amazon Kinesis streams; two KCL consumer applications running on top of AWS Elastic Beanstalk; an application for database tasks and maintenance; an Amazon Redshift cluster; and Amazon Simple Storage Service (Amazon S3) buckets for telemetry, errors, code, and configuration information; and an Amazon S3 endpoint.

Game servers transmit JSON-formatted gameplay events to an Amazon Kinesis stream (called the telemetry stream) that collects and processes those events. A KCL consumer application (called the S3Connector) validates, sanitizes, and enriches the events, and archives the events as a batch telemetry file in Amazon S3. The application also sends a pointer to the location of the batch telemetry file to a separate Amazon Kinesis stream (called the file stream) that initiates the process of loading the data into Amazon Redshift. A second KCL consumer application (called the RedshiftConnector) loads batches of events from Amazon S3, deletes duplicate events, and inserts events into tables in Amazon Redshift. A third application (called the CronConnector) performs routine database tasks and maintenance.

Once the data has been loaded into Amazon Redshift, you can use analytics tools and services, including Amazon QuickSight, for reporting, ad-hoc queries, and more detailed analysis.

AWS Elastic Beanstalk is used to manage the deployment of the solution’s three applications, from capacity provisioning, load balancing, auto-scaling to application health monitoring.