Filter policy constraints in Amazon SNS

When you create a filter policy for an Amazon SNS subscription, it's essential to understand how keys are counted, especially when filtering on the message body (payload-based filtering). Here's what you need to know:

Key counting in filter policies

When creating a filter policy for an Amazon SNS subscription, you can include up to five top-level keys (parent keys) for message filtering, and in payload-based filtering, only these top-level keys count toward the limit, while nested keys under them do not.

Maximum of five top-level keys

• Limit – You can have up to five top-level keys (also called parent keys) in a filter policy.

• Top-Level keys only – In payload-based filtering, only the top-level keys are counted toward this limit.

• Nested keys don't count – Any keys nested under the top-level keys do not count toward the five-key limit.

Comparing attribute-based to payload-based filtering

• Attribute-based filtering:

  • No nested keys – Only supports top-level keys (nesting is not allowed).

  • Key limit – All keys are counted, with a maximum of five keys.

• Payload-based filtering:

  • Supports nested keys – Allows nesting within the message body.

  • Key limit – Only counts top-level keys toward the limit.

Key counting examples

Amazon SNS counts keys in filter policies differently for payload-based and attribute-based filtering—counting only top-level keys in payload-based filtering and all keys in attribute-based filtering toward the key limit.

Example 1: Payload-based Filtering

Filter policy:

{
  "state": ["SUCCESS"],
  "severity": [{ "exists": true }],
  "message": [{ "exists": true }],
  "finding": {
    "standard_control": [{ "exists": true }],
    "region": [{ "exists": true }],
    "account": [{ "exists": true }]
  }
}

Key counting:

• Top-level keys counted:

  1. state

  2. severity

  3. message

  4. finding

• Total top-level keys – 4 (within the limit of 5)

• Nested keys not counted standard_control, region, account under finding are not counted.

Example 2: Attribute-based filtering

Filter policy:

 {
  "state": ["SUCCESS"],
  "severity": [{ "exists": true }],
  "message": [{ "exists": true }],
  "standard_control": [{ "exists": true }],
  "region": [{ "exists": true }]
}

Key counting:

• Keys counted:

  1. state

  2. severity

  3. message

  4. standard_control

  5. region

• Total keys – 5 (maximum limit reached)

Note

Since attribute-based filtering doesn't support nesting, all keys are at the top level and count toward the limit.

Filter policy complexity

Amazon SNS filter policy complexity is calculated by multiplying the number of values in each array at each level of the policy, ensuring that the total number of attribute combinations does not exceed the maximum limit of 150.

• Maximum combinations – The total number of attribute combinations in your filter policy must not exceed 150.

• Calculation method – Multiply the number of values in each array at each level.

• Example calculation:

  • For finding key in Example 1:

    • standard_control: 1 condition

    • region: 1 condition

    • account: 1 condition

    • Combinations under finding: 1 × 1 × 1 = 1

  • Total combinations:

    • state: 1 value

    • severity: 1 condition

    • message: 1 condition

    • finding: 1 combination

    • Total: 1 × 1 × 1 × 1 = 1 (well below the limit)

Valid values for keys

In Amazon SNS filter policies, the complexity is affected by using valid value types for keys—such as strings, arrays of strings, numbers, and conditions—which keeps the policy manageable, whereas using invalid values like JSON objects at the top level in attribute-based filtering (where nesting isn't supported) can lead to increased complexity and issues.

• Accepted value types:

  • A string

  • An array of strings

  • A number

  • A condition (for example, { "exists": true }, { "numeric": [">", 100] }

• Invalid value types:

  • JSON objects as values at the top level in attribute-based filtering (nesting not supported).

Filter policy syntax

By adhering to the syntax requirements for Amazon SNS filter policies—using allowed elements like strings, numbers, and specific keywords in valid JSON format encoded as a UTF-8 string—you effectively manage filter policy complexity and ensure your policies function correctly without unnecessary complications.

• Allowed elements:

  • Strings enclosed in quotation marks

  • Numbers

  • The keywords true, false, and null (without quotation marks)

• JSON format – The filter policy must be valid JSON.

• Encoding – When using the Amazon SNS API, pass the JSON as a valid UTF-8 string.

Filter policy limits

To effectively manage Amazon SNS filter policy complexity, you need to ensure your policies stay within the specified limits—such as a maximum size of 256 KB and up to 200 filter policies per topic—since exceeding these limits will result in failures when attaching filter policies using the Subscribe or SetSubscriptionAttributes API calls.

• Policy size – Maximum size is 256 KB.

• Per-topic limit – Up to 200 filter policies per topic.

• Per-account limit – Up to 10,000 filter policies per AWS account.

• Exceeding limits – Attaching a filter policy that exceeds these limits will fail when using the Subscribe or SetSubscriptionAttributes API calls.

• Increasing limits – Request an increase through AWS service quotas if necessary.

String and number matching

String comparisons in Amazon SNS filter policies are case-sensitive, and numeric matching must adhere to specific range and precision constraints.

String matching

• Case-sensitive – String comparisons are case-sensitive. "SUCCESS" is different from "success".

Numeric matching

• Range – Values can range from -1e9 to 1e9 (-1,000,000,000 to 1,000,000,000).

• Precision – Supports up to five decimal places.

Switching to payload-based filtering

Switching to payload-based filtering in Amazon SNS affects filter policy complexity because you must adjust your filter policy to align with the message body structure, use nested keys as needed, and ensure you stay within the top-level key limit to manage complexity effectively.

1. Set FilterPolicyScope to MessageBody in your subscription attributes.

2. Ensure message body Is valid JSON – The message body must be a valid JSON object.

3. Adjust your filter policy – Modify your filter policy to align with the structure of your message body. Use nested keys as needed, keeping in mind the top-level key limit.

4. Example:

   aws sns subscribe \
     --topic-arn arn:aws:sns:us-east-1:123456789012:MyTopic \
     --protocol sqs \
     --notification-endpoint arn:aws:sqs:us-east-1:123456789012:MyQueue \
     --attributes '{"FilterPolicyScope":"MessageBody", "FilterPolicy":"{...}"}'

Best practices

Learn about best practices for crafting efficient Amazon SNS filter policies.

• Keep top-level keys within limit. Ensure you have no more than five top-level keys in your filter policy.

• Leverage nested keys. Use nested keys to create complex filtering logic without exceeding the top-level key limit.

• Test your policies. Use the Amazon SNS console or AWS CLI to validate your filter policies before deploying them.

• Monitor policy complexity. Keep the total combinations of attribute values under 150 to avoid performance issues.

• Consistent message structure. Ensure that all messages published to the topic have a consistent JSON structure, especially when using payload-based filtering.