View a markdown version of this page

PERF04-BP07 Optimize network configuration based on metrics - AWS Well-Architected Framework

PERF04-BP07 Optimize network configuration based on metrics

Base your network configuration on measured behavior. When you collect and analyze network telemetry, you can tune configuration to real traffic and confirm each change improves performance.

Desired outcome:

  • You collect the network metrics that reveal how your workload uses the network.

  • You base configuration changes on analyzed data rather than assumptions or overprovisioning.

  • You can distinguish network-caused performance issues from application-caused ones.

Common anti-patterns:

  • Teams assume every performance issue is application-related.

  • Teams test network performance only from a location close to the deployment.

  • Teams use default configurations for all network services.

  • Teams overprovision network resources to cover for a lack of measurement.

  • Teams read a throughput ceiling as a network limit without checking for CPU overhead in packet processing.

Benefits of establishing this best practice:

  • You understand real network performance instead of guessing at it.

  • You optimize configuration to match actual traffic patterns.

  • You avoid paying for capacity that does not address the real bottleneck.

Level of risk exposed if this best practice is not established: Low

Implementation guidance

Without measurement, network optimization defaults to overprovisioning: more bandwidth, larger instances, or extra connections that do not address the actual bottleneck. Metrics show whether a slow response is the network, the path, or the application.

The metrics that matter are latency, packet loss, throughput, and traffic volume by source. Testing near your deployment hides the latency your distant users experience, so the workload looks fast to the team and slow to those users. Measure from your users' locations, and capture both real-user data and synthetic tests.

Flow logs record which sources exchanged traffic and how much, revealing top talkers and traffic patterns. Path and reachability analyzers confirm that traffic takes the intended route, catching misconfiguration. Metric services aggregate counters over time so you can set alarms on trends. IP address management tracks allocation so you do not exhaust address space.

Traffic patterns shift as the workload grows and users move, so last quarter's configuration might not fit now. A regular review of metrics keeps topology, routing, and capacity aligned with demand. When application and network teams disagree about the cause of a slowdown, shared network metrics settle the question.

When throughput per node plateaus well below the provisioned network bandwidth, profile the CPU cost of packet processing before you add network capacity. Kernel-to-user copies and per-packet system calls can use up CPU before the link saturates. If the data path is CPU-bound, reduce that overhead in software or scale out, and validate either change under load.

Implementation steps

  1. Identify and collect key metrics: Decide which metrics reveal your workload's network behavior, such as latency and packet loss, and collect them with the tools below.

    Tool Where to use it
    Amazon VPC IP Address Manager Plan, track, and monitor IP address usage for AWS and on-premises workloads.
    VPC Flow Logs Capture traffic to and from network interfaces to diagnose security group rules and traffic direction.
    AWS Transit Gateway Flow Logs Capture IP traffic going to and from your transit gateways.
    DNS query logging Log public and private DNS queries to understand requested domains and responding edge locations.
    Reachability Analyzer Test connectivity between a source and destination to confirm the network matches intended connectivity.
    Network Access Analyzer Specify access requirements and find paths that do not meet them.
    Amazon CloudWatch Turn on metrics for network options such as NAT gateway, Transit Gateway, VPN tunnels, and Elastic Load Balancing, and monitor them continuously.
    Infrastructure Performance Obtain near real-time and historical network latency across AWS Regions and across or within Availability Zones for a specified time period.
    Amazon CloudWatch RUM Collect real-user metrics that show where to improve user experience.
  2. Identify top talkers and traffic patterns: Use VPC Flow Logs and AWS Transit Gateway Flow Logs to find which sources generate the most traffic and how patterns change over time.

  3. Assess and optimize your network architecture: Review VPCs, subnets, and routing against the traffic you observed. For example, evaluate whether VPC peering or AWS Transit Gateway improves the connections in your architecture. When a performance issue surfaces, use AWS DevOps Agent and CloudWatch AI Operations: Investigations to trace the root cause back to a specific hop or configuration, and feed what you learn back into your architecture and instrumentation so the same class of issue is caught earlier next time.

  4. Verify routing paths: Confirm that traffic takes the shortest path between destinations. Use Network Access Analyzer to check the paths against your intended design.

  5. Check for CPU-bound packet processing: When per-node throughput plateaus below the provisioned bandwidth, profile the data path, for example with flame graphs, to see whether CPU overhead rather than the network is the limit.

Resources

Related best practices:

Related documents:

Related videos:

Related examples:

Related workshops: