Application logging and monitoring using VPC Flow Logs - AWS Prescriptive Guidance
Using VPC Flow LogsUse cases for VPC Flow Logs

Application logging and monitoring using VPC Flow Logs

VPC Flow Logs is a feature of Amazon Virtual Private Cloud (Amazon VPC) that helps you capture information about the IP traffic going to and from network interfaces in your VPC.

Using VPC Flow Logs

You can create a flow log for a virtual private cloud (VPC), a subnet, or a network interface. If you create a flow log for a subnet or VPC, each network interface in that subnet or VPC is monitored. For more information, see Work with flow logs (Amazon VPC documentation).

Flow log data for a monitored network interface is recorded as flow log records. A flow log record represents a network flow in your VPC. By default, each record captures a network IP traffic flow that occurs within an aggregation interval. Each record is a string with fields separated by spaces. A record includes values for the different components of the IP flow, for example, the source, destination, and protocol. When you create a flow log, you can use the default format for the flow log record, or you can specify a custom format. For more information, see Flow log record examples (Amazon VPC documentation).

Flow logs don't capture the following information:

  • Traffic generated by instances when they contact the Amazon Domain Name System (DNS) server. If you use your own DNS server, then all traffic to that DNS server is logged.

  • Traffic generated by a Windows instance for Amazon Windows license activation.

  • Traffic to and from 254.169.254, for instance metadata.

  • Traffic to and from 254.169.123, for the Amazon Time Sync Service.

  • Dynamic Host Configuration Protocol (DHCP) traffic.

  • Traffic to the reserved IP address for the default VPC router.

  • Traffic between an endpoint network interface and a Network Load Balancer network interface.

Flow log data can be published to several AWS services, including Amazon CloudWatch Logs. After you create a flow log, you can retrieve and view the flow log records in CloudWatch Logs in the log group that you configure. For more information, see Publish flow logs to CloudWatch Logs (Amazon VPC documentation).

Flow log data is collected outside of the path of your network traffic, and therefore does not affect network throughput or latency. You can create or delete flow logs without any risk of impact to network performance.

Use cases for VPC Flow Logs

  • Diagnose overly restrictive security group rules

  • Monitor the traffic that is reaching your application instance

  • Determine the direction of the traffic