Testing and tuning

After your initial implementation of WAF, there is normally a phase of tuning to mitigate potential false negatives and false positives. False negatives are attacks that were not caught by your WAF and require you to harden your rules. False positives represent legitimate requests that were considered by WAF wrongly as attacks and blocked as a consequence.

False negatives

To identify false negatives based on your security requirements, you can use penetration testing providers, automated vulnerabilities scanners from third parties, or open source web application security scanners. Although automated vulnerability scanners are great tools to test your architecture quickly against known vulnerabilities and predefined attack vectors, they only cover known cases and may falsely flag an issue when there isn’t one. Therefore, keep in mind that testing your WAF rules using a vulnerability scanner is not a guarantee that your application is fully protected. However, you can use a vulnerability scanner for sanity-checking. Make sure to regularly update your scanner and set a regular cadence for running it so that you are aware of new attacks and can update the WAF rules whenever necessary.

False positives

False positives are generally identified by quality assurance teams by testing the application after changes are introduced to the application’s code or WAF configuration. In many cases, this testing requires a deep understanding of the application to tell the difference between suspicious request signatures and a malicious attack. In some cases, false positives may only appear in production owing to a shortcoming in quality assurance test coverage. To help identify these cases, consider the following:

• Set up alarming on selected WAF rules in CloudWatch to be notified when a rule is triggered when predefined thresholds are exceeded.

• Update your application experience to allow real users to report unexpected unauthorized access to your application. For example, when WAF is deployed with CloudFront, you can use custom error pages to catch 403 error codes and serve a friendly response. This page can prompt the user to provide information on the issue they’ve encountered.

• Enable WAF logging. Have security and application teams review and baseline blocked traffic on a regular cadence to identify threat patterns and anomalies in blocked traffic.

When you detect a false positive, you must understand what has initiated it. A quick way to identify the trigger is by using the AWS Management Console to check sample logs that corresponds to the rule being initiated. A more robust way to identify the false positive is to check the logs generated by AWS WAF and filter on the blocked request by URL, IP, and timestamp. If you are using AWS WAF with CloudFront when a request is blocked, CloudFront returns a Request ID within the response headers and body.

You can use this Request ID to find the corresponding record in WAF logs.

Error page displayed by CloudFront

Error page response headers

With the log record, you have the information about which rule was initiated—for example, a specific header doesn’t respect your size constraint rule. In some cases, the information is not sufficient to identify the issue. For example, a SQLi rule triggers on a cookie in the request, but it’s not obvious which part of the cookie is causing this trigger. For SQLi and XSS rules, the terminatingRuleMatchDetails field in the log record provides more details on the match.

Resolving false positives

After you understand the issue, you can resolve it. The best approach is to change the application code that is generating requests that look similar to attacks, but that may take some time and effort. For a quick fix, you can create an exception to the rule in WAF, but note that creating exceptions in WAF exposes your application to potential attacks.

Example 1: Override rules using AWS CLI

Suppose that you have a legitimate URL pattern xxxx that is blocked by an XSS rule on the path of the URI. You can override this block by adding an additional rule to include an exception on the xxxx URL pattern:

BLOCK [XSS condition] AND NOT[String Match Condition on Path]

The following WAF rule statement illustrates this example. This solution works for SQLi and XSS rules, and for any custom rules.

{
    "Name": "XSSprotection",
    "Priority": 0,
    "Action":  {
        "Block":  {}
    },
    "VisibilityConfig":  {
        "SampledRequestsEnabled": true,
        "CloudWatchMetricsEnabled": true,
        "MetricName": "XSSprotection" 
    },
    "Statement":  {
        "AndStatement":  {
            "Statements":  [
                {
                    "XssMatchStatement":  {
                        "FieldToMatch":  {
                            "UriPath":  {}
                        },
                        "TextTransformations":  [
                            {
                                "Type": "URL_DECODE",
                                "Priority": 0 
                            }
                        ]
                    }
                },
                {
                    "NotStatement":  {
                        "Statement":  {
                            "ByteMatchStatement":  {
                                "FieldToMatch":  {
                                    "UriPath":  {}
                                },
                                "PositionalConstraint": "CONTAINS",
                                "SearchString": "xxxx",
                                "TextTransformations":  [
                                    {
                                        "Type": "LOWERCASE",
                                        "Priority": 0 
                                    }
                                ]
                            }
                        }
                    }
                }
            ]
        }
    }
}

Example 2: Override rules using AWS Managed Rules

Suppose you have a legitimate URL pattern xxxx that is blocked by a managed rule such as a Core rule set by AWS. You can override the action of the blocking sub rule by setting it to count mode to mitigate false positives temporarily. However, you open your environment to attacks previously stopped by this sub rule.

Overriding the action of a sub rule in AWS Managed Rules

Another approach is to create an allowed list rule before the managed rule, which allows traffic when it matches the false positive xxxx pattern and stops evaluating the rest of rules. With this approach, you risk exposing your application to the attacks normally stopped by the rules coming after your false positive allowed list rule when it matches.

One way to reduce this risk is by setting the allowed list rule to the lowest priority in the web ACL rules order.

Adding a rule to handle exceptions

After you deal with false positives, either by adding exceptions to WAF rules or by changing your application code, you can validate your remediation by replaying the requests that caused the false positive by using tools such as cURL or Postman. If the false positive is mitigated properly, the request should not be blocked.

Example 3: Using labels for creating exceptions for known false positives

Suppose you have a legitimate traffic (for example, XSS matching in cookie header) that is blocked by a managed rule such as a Core rule set by AWS. You can use labels with AWS Managed Rules groups to create exception rules. When a web request matches a rule, AWS WAF adds the rule’s labels to the request. The labels remain available on the request as long as AWS WAF is evaluating it against the web ACL. This means that you can use Amazon Managed Core rule set in count mode to create labels and create custom rules to run after the rule group for known false positives.

For example, you can create Amazon Managed Core set rule group in count mode and then create custom rules for allowing known XSS patterns in the cookie header and blocking known XSS patterns in URI path as shown in the following figure.

AWS WAF rules for creating false-positive exceptions

The JSON representation of the amr-exception-rule-XSS-cookie is shown in the following figure.

{
    "Name": "amr-exception-rule-XSS-cookie",
    "Priority": 1,
    "Statement":  {
        "LabelMatchStatement":  {
            "Scope": "LABEL",
            "Key": "awswaf:managed:aws:core-ruleset:CrossSiteScripting_Cookie" 
        }
    },
    "Action":  {
        "Allow":  {
            "CustomRequestHandling":  {
                "InsertHeaders":  [
                    {
                        "Name": "XSS",
                        "Value": "false-positive" 
                    }
                ]
            }
        }
    },
    "VisibilityConfig":  {
        "SampledRequestsEnabled": true,
        "CloudWatchMetricsEnabled": true,
        "MetricName": "amr-exception-rule" 
    }
}

The JSON representation of the amr-exception-rule-URI-cookie is shown in the following figure.

{
    "Name": "amr-exception-rule-URI",
    "Priority": 2,
    "Statement":  {
        "LabelMatchStatement":  {
            "Scope": "LABEL",
            "Key": "awswaf:managed:aws:core-ruleset:CrossSiteScripting_URIPath" 
        }
    },
    "Action":  {
        "Block":  {}
    },
    "VisibilityConfig":  {
        "SampledRequestsEnabled": true,
        "CloudWatchMetricsEnabled": true,
        "MetricName": "amr-exception-rule-URI" 
    }
}