Use Network Firewall to capture the DNS domain names from the Server Name Indication (SNI) for outbound traffic - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpics

Use Network Firewall to capture the DNS domain names from the Server Name Indication (SNI) for outbound traffic

Created by Kirankumar Chandrashekar (AWS)

Environment: PoC or pilot

Technologies: Security, identity, compliance; Networking; Web & mobile apps

Workload: All other workloads

AWS services: AWS Lambda; AWS Network Firewall; Amazon VPC; Amazon CloudWatch Logs

Summary

This pattern shows you how to use Amazon Web Services (AWS) Network Firewall to collect the DNS domain names that are provided by the Server Name Indication (SNI) in the HTTPS header of your outbound network traffic. Network Firewall is a managed service that makes it easy to deploy critical network protections for Amazon Virtual Private Cloud (Amazon VPC), including the ability to secure outbound traffic with a firewall that blocks packets that fail to meet certain security requirements. Securing outbound traffic to specific DNS domain names is called egress filtering, which is the practice of monitoring and potentially restricting the flow of outbound information from one network to another.

After you capture the SNI data that passes through Network Firewall, you can use Amazon CloudWatch Logs and AWS Lambda to publish the data to an Amazon Simple Notification Service (Amazon SNS) topic that generates email notifications. The email notifications include the server name and other relevant SNI information. Additionally, you can use the output of this pattern to allow or restrict outbound traffic by domain name in the SNI by using firewall rules. For more information, see Working with stateful rule groups in AWS Network Firewall in the Network Firewall documentation.

Prerequisites and limitations

Prerequisites

Note: Network Firewall can use any of the following VPC configurations:

Architecture

The following diagram shows how to use Network Firewall to collect SNI data from outbound network traffic, and then publish that data to an SNS topic by using CloudWatch Logs and Lambda.

Workflow between Network Firewall, CloudWatch Logs, Lambda, and Amazon SNS.

The diagram shows the following workflow:

  1. Network Firewall collects domain names from the SNI data in the HTTPS header of your outbound network traffic.

  2. CloudWatch Logs monitors the SNI data and invokes a Lambda function whenever the outbound network traffic passes through Network Firewall.

  3. The Lambda function reads the SNI data captured by CloudWatch Logs and then publishes that data to an SNS topic.

  4. The SNS topic sends you an email notification that includes the SNI data.

Automation and scale

Technology stack

  • Amazon CloudWatch Logs

  • Amazon SNS

  • Amazon VPC

  • AWS Lambda 

  • AWS Network Firewall

Tools

AWS services

  • Amazon CloudWatch Logs – You can use Amazon CloudWatch Logs to monitor, store, and access your log files from Amazon Elastic Compute Cloud (Amazon EC2) instances, AWS CloudTrail, Amazon Route 53, and other sources.

  • Amazon SNS – Amazon Simple Notification Service (Amazon SNS) is a managed service that provides message delivery from publishers to subscribers (also known as producers and consumers).

  • Amazon VPC – Amazon Virtual Private Cloud (Amazon VPC) provisions a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you've defined. This virtual network closely resembles a traditional network that you'd operate in your own data center, with the benefits of using the scalable infrastructure of AWS.

  • AWS Lambda – AWS Lambda is a compute service that lets you run code without provisioning or managing servers.

  • AWS Network Firewall – AWS Network Firewall is a managed service that makes it easy to deploy essential network protections for all of your Amazon VPCs.

Epics

TaskDescriptionSkills required

Create a CloudWatch log group.

  1. Sign in to the AWS Management Console and open the CloudWatch console.

  2. In the navigation pane, choose Log groups.

  3. Choose Actions, and then choose Create log group.

  4. Enter a name for the log group, and then choose Create log group.

For more information, see Working with log groups and log streams in the CloudWatch documentation.

Cloud administrator
TaskDescriptionSkills required

Create an SNS topic.

To create an SNS topic, follow the instructions in the Amazon SNS documentation.

Cloud administrator

Subscribe an endpoint to the SNS topic.

To subscribe an email address as an endpoint to the SNS topic that you created, follow the instructions in the Amazon SNS documentation. For Protocol, choose Email/Email-JSON. Note: You can also choose a different endpoint based on your requirements.

Cloud administrator
TaskDescriptionSkills required

Enable firewall logging.

  1. Sign in to the AWS Management Console and open the Amazon VPC console.

  2. In the navigation pane, under NETEWORK FIREWALL, choose Firewalls.

  3. In the Firewalls section, choose the firewall where you want to capture the server name from the SNI for outbound traffic.

  4. Choose the Firewall details tab, and then choose Edit in the Logging section. 

  5. For Log type, select Alert. For Log destination for alerts, select CloudWatch log group. 

  6. For CloudWatch log group, search for and choose the log group that you created earlier, and then choose Save.

For more information about using CloudWatch Logs as a log destination for Network Firewall, see Amazon CloudWatch Logs in the Network Firewall documentation. 

Cloud administrator
TaskDescriptionSkills required

Create a stateful rule.

  1. Sign in to the AWS Management Console and open the Amazon VPC console.

  2. In the navigation pane, under NETWORK FIREWALL, choose Network Firewall Rule Groups.

  3. Choose Create Network Firewall rule group.

  4. On the Create Network Firewall rule group page, for the Rule group type, choose Stateful rule group. Note: For more information, see Working with stateful rule groups in AWS Network Firewall.

  5. In the Stateful rule group section, enter a name and description for the rule group.

  6. For Capacity, set the maximum capacity that you want to allow for the stateful rule group (up to the maximum of 30,000). Note: You can't change this setting after you create the rule group. For information about how to calculate capacity, see Setting rule group capacity in AWS Network Firewall. For information about the maximum setting, see AWS Network Firewall quotas.

  7. For Stateful rule group options, select 5-tuple.

  8. In the Stateful rule order section, choose Default.

  9. In the Rule variables section, keep the default values.

  10. In the Add rule section, choose TLS for Protocol. For Source, choose Any. For Source port, choose Any port. For Destination, choose Any. For Destination port, choose Any port. For Traffic direction, choose Forward. For Action, choose Alert. Choose Add rule.

  11. Choose Create stateful rule group.

Cloud administrator

Associate the stateful rule to Network Firewall.

  1. Sign in to the AWS Management Console and open the Amazon VPC console.

  2. In the navigation pane, under NETWORK FIREWALL, choose Firewalls.

  3. Choose the firewall where you want to capture the server name from the SNI for outbound traffic.

  4. In the Stateful rule groups section, choose Actions, and then choose Add unmanaged stateful rule groups. 

  5. On the Add unmanaged stateful rule groups page, select the stateful rule group that you created earlier, and then choose Add stateful rule group.

Cloud administrator
TaskDescriptionSkills required

Create the code for the Lambda function.

In an integrated development environment (IDE) that can read the CloudWatch Logs event from Network Firewall for outbound traffic, paste in the following Python 3 code and replace <SNS-topic-ARN> with your value:

import json
import gzip
import base64
import boto3
sns_client = boto3.client('sns')
def lambda_handler(event, context):
    decoded_event = json.loads(gzip.decompress(base64.b64decode(event['awslogs']['data'])))
    body = '''
    {filtermatch}
    '''.format(
        loggroup=decoded_event['logGroup'],
        logstream=decoded_event['logStream'],
        filtermatch=decoded_event['logEvents'][0]['message'],
    )
    print(body)
    filterMatch = json.loads(body)
    data = []
    if 'http' in filterMatch['event']:
        data.append(filterMatch['event']['http']['hostname'])
    elif 'tls' in filterMatch['event']:
        data.append(filterMatch['event']['tls']['sni'])
    result = 'Domain accessed ' + 1*' ' + (data[0]) + 1*' ' 'via AWS Network Firewall ' + 1*' '  + (filterMatch['firewall_name'])
    print(result)
    message = {'ServerName': result}
    send_to_sns = sns_client.publish(
        TargetArn=<SNS-topic-ARN>,          #Replace with the SNS topic ARN
        Message=json.dumps({'default': json.dumps(message),
                        'sms': json.dumps(message),
                        'email': json.dumps(message)}),
        Subject='Server Name passed through the Network Firewall',
        MessageStructure='json'
    )

This code sample parses the CloudWatch Logs content and captures the server name provided by the SNI in the HTTPS header.

App developer

Create the Lambda function.

To create the Lambda function, follow the instructions in the Lambda documentation and choose Python 3.9 for Runtime.

Cloud administrator

Add the code to the Lambda function.

To add your Python code to the Lambda function that you created earlier, follow the instructions in the Lambda documentation.

Cloud administrator

Add CloudWatch Logs as a trigger to the Lambda function.

  1. Sign in to the AWS Management Console and open the Lambda console.

  2. In the navigation pane, choose Functions, and then choose the function that you created earlier.

  3. In the Function overview section, choose Add trigger.

  4. On the Add trigger page, in the Trigger configuration section, choose CloudWatch Logs, and then choose Add.

  5. For Log group, choose the CloudWatch log group that you created earlier.

  6. For Filter name, enter a name for your filter.

  7. Choose Add.

  8. On the Configuration tab of your function’s page, in the Triggers section, select the trigger that you just added, and then choose Enable.

For more information, see Using Lambda with CloudWatch Logs in the Lambda documentation.

Cloud administrator

Add SNS publish permissions.

Add the sns:Publish permission to the Lambda execution role, so that Lambda can make API calls to publish messages to SNS.  

  1. Find the execution role of the Lambda function that you created earlier.

  2. Add the following policy to your AWS Identity and Access Management (IAM) role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSNSPublish",
            "Effect": "Allow",
            "Action": [
                "sns:GetTopicAttributes",
                "sns:Subscribe",
                "sns:Unsubscribe",
                "sns:Publish"
            ],
            "Resource": "*"
        }
    ]
}
Cloud administrator
TaskDescriptionSkills required

Send traffic through Network Firewall.

  1. Send or wait for HTTPS traffic to pass through Network Firewall.

  2. Check the SNS notification email that you receive from AWS when traffic passes through Network Firewall. The email includes the SNI details for outbound traffic. For example, the email generated from the Lambda code above will have the following content if the accessed domain name is https://aws.amazon.com and the subscription protocol is EMAIL-JSON:

{
    "Type": "Notification",
    "MessageId": "<messageID>",
    "TopicArn": "arn:aws:sns:us-west-2:123456789:testSNSTopic",
    "Subject": "Server Name passed through the Network Firewall",
    "Message": "{\"ServerName\": \"Domain 'aws.amazon.com' accessed via AWS Network Firewall 'AWS-Network-Firewall-Multi-AZ-firewall\"}",
    "Timestamp": "2022-03-22T04:10:04.217Z",
    "SignatureVersion": "1",
    "Signature": "<Signature>",
    "SigningCertURL": "<SigningCertUrl>",
    "UnsubscribeURL": "<UnsubscribeURL>"
}

Then, check the Network Firewall alert log in Amazon CloudWatch by following the instructions in the Amazon CloudWatch documentation. The alert log shows the following output:

{
    "firewall_name": "AWS-Network-Firewall-Multi-AZ-firewall",
    "availability_zone": "us-east-2b",
    "event_timestamp": "<event timestamp>",
    "event": {
        "timestamp": "2021-03-22T04:10:04.214222+0000",
        "flow_id": <flow ID>,
        "event_type": "alert",
        "src_ip": "10.1.3.76",
        "src_port": 22761,
        "dest_ip": "99.86.59.73",
        "dest_port": 443,
        "proto": "TCP",
        "alert": {
            "action": "allowed",
            "signature_id": 2,
            "rev": 0,
            "signature": "",
            "category": "",
            "severity": 3
        },
        "tls": {
            "subject": "CN=aws.amazon.com",
            "issuerdn": "C=US, O=Amazon, OU=Server CA 1B, CN=Amazon",
            "serial": "<serial number>",
            "fingerprint": "<fingerprint ID>",
            "sni": "aws.amazon.com",
            "version": "TLS 1.2",
            "notbefore": "2020-09-30T00:00:00",
            "notafter": "2021-09-23T12:00:00",
            "ja3": {},
            "ja3s": {}
        },
        "app_proto": "tls"
    }
}
Test engineer