Component details

As described in the Architecture diagram, four of this solution’s components use automations to inspect IP addresses and add them to the AWS WAF block list. The following sections explain each of these components in more detail.

Log parser - Application

The Application log parser helps protect against scanners and probes.

Application log parser flow

1. When CloudFront or an ALB receives requests on behalf of your web application, it sends access logs to an Amazon S3 bucket.

   1. (Optional) If you select Yes - Amazon Athena log parser for the template parameters Activate HTTP Flood Protection and Activate Scanner & Probe Protection, a Lambda function moves access logs from their original folder <customer-bucket>/AWSLogs to a newly partitioned folder <customer-bucket>/AWSLogs-partitioned/<optional-prefix> /year=<YYYY>/month=<MM> /day=<DD>/hour=<HH>/ upon their arrival in Amazon S3.

   2. (Optional) If you select yes for the Keep Data in Original S3 location template parameter, logs remain in their original location and are copied to their partitioned folder, duplicating your log storage.

   Note

   For the Athena log parser, this solution only partitions new logs that arrive in your Amazon S3 bucket after you deploy this solution. If you have existing logs that you want to partition, you must manually upload those logs to Amazon S3 after you deploy this solution.

2. Based on your selection for the template parameters Activate HTTP Flood Protection and Activate Scanner & Probe Protection, this solution processes logs using one of the following:

   1. Lambda – Each time a new access log is stored in the Amazon S3 bucket, the Log Parser Lambda function is initiated.

   2. Athena – By default, every five minutes the Scanner & Probe Protection Athena query runs, and the output pushes to AWS WAF. This process is initiated by a CloudWatch event, which starts the Lambda function responsible for running the Athena query and pushes the result into AWS WAF.

3. The solution analyzes the log data to identify IP addresses that generated more errors than the defined quota. The solution then updates an AWS WAF IP set condition to block those IP addresses for a customer-defined period of time.

Log parser - AWS WAF

If you select yes - AWS Lambda log parser or yes - Amazon Athena log parser for Activate HTTP Flood Protection, this solution provisions the following components, which parse AWS WAF logs to identify and block origins that flood the endpoint with a request rate greater than the quota you defined.

AWS WAF log parser flow

1. When AWS WAF receives access logs, it sends the logs to an Firehose endpoint. Firehose then delivers the logs to a partitioned bucket in Amazon S3 named <customer-bucket>/AWSLogs/ <optional-prefix>/year=<YYYY> /month=<MM>/day=<DD>/hour= <HH>/

2. Based on your selection for the template parameters Activate HTTP Flood Protection and Activate Scanner & Probe Protection, this solution processes logs using one of the following:

   1. Lambda: Each time a new access log is stored in the Amazon S3 bucket, the Log Parser Lambda function is initiated.

   2. Athena: By default, every five minutes the scanner and probe Athena query is run and the output is pushed to AWS WAF. This process is initiated by an Amazon CloudWatch event, that then starts the Lambda function responsible for executing the Amazon Athena query, and pushes the result into AWS WAF.

3. The solution analyses the log data to identify IP addresses that sent more requests than the defined quota. The solution then updates an AWS WAF IP set condition to block those IP addresses for a customer-defined period of time.

IP lists parser

The IP Lists Parser Lambda function helps protect against known attackers identified in third-party IP reputation lists.

IP reputation lists parser flow

1. An hourly Amazon CloudWatch event invokes the IP Lists Parser Lambda function.

2. The Lambda function gathers and parses data from three sources:

   • Spamhaus DROP and EDROP lists

   • Proofpoint Emerging Threats IP list

   • Tor exit node list

3. The Lambda function updates the AWS WAF block list with the current IP addresses.

Access Handler

The Access Handler Lambda function inspects requests to the honeypot endpoint to extract their source IP address.

Access Handler and the honeypot endpoint

1. Embed the honeypot endpoint in your website and update your robots exclusion standard, as described in Embed the Honeypot Link in Your Web Application (Optional).

2. When a content scraper or bad bot accesses the honeypot endpoint, it invokes the Access Handler Lambda function.

3. The Lambda function intercepts and inspects the request headers to extract the IP address of the source that accessed the trap endpoint.

4. The Lambda function updates an AWS WAF IP set condition to block those IP addresses.