Appendix A: Log Parser Options - AWS WAF Security Automations

Appendix A: Log Parser Options

As described in the Architecture Overview, there are three options to handle HTTP flood and scanner and probe protections. The following sections explain each of these options in more detail.

AWS WAF Rate-based Rule

Rate-based rules are available for HTTP flood protection and can be configured in AWS WAF. This feature allows you to specify the maximum number of web requests to allow from any single IP address in a trailing, continuously updated five-minute period. If an IP address breaches the configured limit, new requests will be blocked until the request rate falls below the configured threshold. For details, refer to AWS CloudFormation service role in the AWS CloudFormation User Guide.

Amazon Athena Log Parser

Both HTTP Flood and Scanner & Probe protection template parameters provide the Amazon Athena Log Parser option. When activated, AWS CloudFormation provisions an Amazon Athena query and a scheduled AWS Lambda function responsible for orchestrating Athena to execute, process result output, and update AWS WAF. This Lambda function is triggered by an Amazon CloudWatch event configured to trigger every five minutes. You can configure their run schedules by changing QueryScheduledRunTime in aws-waf-security-automations.template.

We recommend selecting this option when AWS WAF rate-based rules cannot be used and if you have familiarity with SQL language to implement customizations. For more information about how to change the default query, see Appendix D.

Note that HTTP flood protection is based on AWS WAF access log processing and uses Amazon CloudFront/ALB log files. The WAF access log type has a lower lag time which can be used to identify HTTP flood origins more quickly when compared to CloudFront/ALB log delivery time. However, you must select the CloudFront/ALB log type in the Activate Scanner & Probe Protection template parameter to receive response status codes.

AWS Lambda Log Parser

Both HTTP Flood and Scanner & Probe template parameters provide this option. Use the Lambda log parser only when the previous two options are not available. A known limitation of this option is that information is processed within the context of the file being processed. For example, an IP may generate more requests/errors than the defined threshold but because this info is split into different files, each file doesn’t store enough data to exceed the threshold.