Appendix B: Component Details - AWS WAF Security Automations

Appendix B: Component Details

As described in the Architecture Overview, four of this solution’s components use automations to inspect IP addresses and add them to the AWS WAF block list. The following sections explain each of these functions in more detail.

Log Parser - Application

The Application Log Parser helps protect against Scanners and Probes.


         App Log Parser flow

Figure 4: App Log Parser flow

  1. Once Amazon CloudFront or an Application Load Balancer receives requests on behalf of your web application, it sends access logs to an Amazon S3 bucket.

    (Optional) If you select Yes - Amazon Athena log parser for the template parameters Activate HTTP Flood Protection and Activate Scanner & Probe Protection, a Lambda moves access logs from their original folder customer-bucket/AWSLogs to a newly partitioned folder customer-bucket/AWSLogs-partitioned/optional-prefix/year=YYYY/month=MM/day=DD/hour=HH/, upon their arrival in S3. If you select yes for the Keep Data in Original S3 location template parameter, logs will be kept in their original location as well as being copied to their partitioned folder, and this will duplicate your log storage.

    Note

    For Athena Log Parser, this solution only partitions new logs that arrive in your Amazon S3 bucket after you deploy this solution. If you have existing logs that you would like to be partitioned, you must manually upload those logs to S3 after you deploy this solution.

  2. Based on your selection for the template parameters Activate HTTP Flood Protection and Activate Scanner & Probe Protection, this solution processes logs using one of the following:

    1. AWS Lambda: each time a new access log is stored in the Amazon S3 bucket, the Log Parser Lambda function is triggered.

    2. Amazon Athena: every five minutes the Scanner and Probes Athena query is executed and the output is pushed to AWS WAF. This process is triggered by an Amazon CloudWatch event, that then triggers the Lambda function responsible for executing the Amazon Athena query, and pushes the result into AWS WAF.

  3. The log data is analyzed in order to identify IP addresses that have generated more errors than the defined threshold, it then updates an AWS WAF IP Set condition to block those IP addresses for a customer-defined period of time.

Log Parser - AWS WAF

If you select yes - AWS Lambda log parser or yes - Amazon Athena log parser for HTTP flood protection, this solution will provision the following components, which will be responsible for parsing AWS WAF logs in order to identify and block origins that flood the endpoint with a request rate above the threshold you defined.


        AWS WAF Log Parser flow

Figure 5: AWS WAF Log Parser flow

  1. As AWS WAF receives access logs, it sends the logs to an Amazon Kinesis Data Firehose endpoint. Firehose then delivers the logs to a partitioned folder in S3: customer-bucket/AWSLogs/optional-prefix/year=YYYY/month=MM/day=DD/hour=HH/.

  2. Based on your selection for the template parameters Activate HTTP Flood Protection and Activate Scanner & Probe Protection, the solution processes logs using one of the following:

    1. AWS Lambda: each time a new access log is stored in the Amazon S3 bucket, the Log Parser Lambda function is triggered.

    2. Amazon Athena: by default, every five minutes the scanner and probe Athena query is executed and the output is pushed to AWS WAF. This process is triggered by an Amazon CloudWatch event, that then triggers the Lambda function responsible for executing the Amazon Athena query, and pushes the result into AWS WAF.

  3. The log data is analyzed in order to identify IP addresses that have sent more requests than the defined threshold, it then updates an AWS WAF IP Set condition to block those IP addresses for a customer-defined period of time.

IP Lists Parser

The IP Lists Parser AWS Lambda function helps protect against known attackers identified in third-party IP reputation lists.


        IP Reputation Lists Parser flow

Figure 6: IP Reputations Lists Parser flow

  1. An hourly Amazon CloudWatch event triggers the IP Lists Parser Lambda function.

  2. The Lambda function gathers and parses data from three sources:

  3. The Lambda function updates the AWS block list with the most current IP addresses.

Access Handler

The Access Handler AWS Lambda function inspects requests to the honeypot endpoint in order to extract their source IP address.


          Access Handler and the honeypot endpoint

Figure 7: Access Handler and the honeypot endpoint

  1. Embed the honeypot endpoint in your website and update your robots exclusion standard, as described in Step 3. Embed the Honeypot Link in Your Web Application (Optional).

  2. When a content scraper or bad bot accesses the honeypot endpoint, it triggers the Access Handler Lambda function.

  3. The Lambda function intercepts and inspects the request headers to extract the IP address of the source that accessed the trap endpoint.

  4. The Lambda function updates an AWS WAF IP Set condition to block those IP addresses.