Solution Components - AI-Driven Social Media Dashboard

Solution Components

Tweet Ingestion

The solution's Amazon Elastic Compute Cloud (Amazon EC2) instance has a Node.js application that monitors tweets for a list of terms you specify during initial deployment. When the solution finds a tweet containing one or more of the terms, the solution will ingest that tweet. You can modify the terms that will be pulled from the Twitter streaming API. By default, this solution uses stream processing for tweets. After tweet ingestion, AWS Lambda analyzes your tweets using Amazon Translate and Amazon Comprehend.

To retrieve tens or hundreds of tweets per second, you can perform batch calls or leverage AWS Glue with triggers to perform batch processing.

Social Media Data Lake

This solution includes a data lake to store your tweet data. The data lake consists of Amazon S3 to store raw and enriched datasets, Amazon Kinesis Data Firehose delivery streams to write the ingested tweet data to the data lake, and AWS Glue Data Catalog to be the metadata catalog for analytics. By default, this solution uses Amazon Athena to query data in the data lake. But, you can extend this solution to use Amazon Redshift Spectrum, Amazon EMR, and Amazon SageMaker.