Scalability - Hybrid Connectivity



Scalability refers to the size of the solution, this size can be related to the entire solution or to a specific component(s) of it.

When designing a solution and hybrid networking connectivity in particular, you need to consider the current size, as well as the anticipated growth of the solution. This growth can be either an organic growth of a solution, or might be related to a merger & acquisition type of scenarios, where the size can be increased dramatically within a short period of time. Furthermore, scalability, in the context of a hybrid network connectivity design, refers to the design ability to support the current and future requirements related with 1) number of on-premises sites to be connected to AWS 2) number of AWS Regions to be used 3) Number of Amazon VPCs within each Region 4) number of routes to be exchanged and 5) bandwidth requirements.

Impact on the design

The understanding of the current and the anticipated future scale requirements is critical, because it will influence the optimal design decision with regard to the connectivity option selection and design. To be able to analyze the connectivity options in relation to the targeted solution scale, first we need to identify what are the influencing factors that need to be considered as part of the scale analysis when selecting and designing a hybrid connectivity solution architecture. The following list summarizes the key elements that impact the design decision regarding scale:

  • Number of Amazon VPCs: the number of VPCs that need to be connected to an on-premises site(s). Different connectivity options have limits that need to be taken into consideration, at the decision-making stage.

  • Number of AWS Regions to be used: for multi-Region architectures, identifying the required number of Regions to connect or anticipated to be connected to on-premises site(s) is important. This helps to minimize unexpected limitations in terms of the supported number of Regions to connect to, and major design changes.

  • Number of on-premises sites to be connected to AWS: the number of sites to be connected to AWS impacts the decision of how many connections are required to be set up in which it will influence the entire hybrid connectivity design. From a logical design point of view, this also has implications on the control plane complexity such as number of BGP peering sessions.

  • The number of advertised prefixes: this element is related to the control plane aspect (IP routing). The required scale of IP routing information (routes) to be advertised to or from AWS can influence the deign decision, as different AWS services have different quotas.

  • Bandwidth: in the context of scalability, refers to the ability of a connection or link to support increase of bandwidth capacity for example from 1G to 10G over same fiber link, or more VPN bandwidth capacity by adding more VPN Tunnels.


Depending on the targeted solution architecture, not all of the preceding elements might need to be taken into consideration. However, they can serve as the foundational elements to identify scalability requirements of most common hybrid network solutions. This whitepaper focuses on the hybrid connectivity selection and design. Therefore, the preceding scale elements are mainly focused on the hybrid connectivity selection and design. Nevertheless, it’s recommended to look at the big picture. Therefore, its recommended to expand the scale consideration of the hybrid connectivity to the VPCs networking architecture. To expand on the selected hybrid connectivity design refer to the AWS whitepaper Building a Scalable and Secure Multi-VPC AWS Network Infrastructure.

Requirement definition

  • What is the current or anticipated number of VPCs which require connectivity to on-premises site(s)?

  • Are these VPCs deployed in a single AWS Region or multiple Regions?

  • How many on-premises sites need to be connected to AWS?

  • How many customer gateway devices (e.g. routers) you have per site that need to connect to AWS?

  • How many routes are expected to be advertised to Amazon VPCs as well as the number of expected routes to be received from AWS side?

  • Is there a requirement to increase bandwidth to AWS overtime?

Technical solutions

To address scale requirements as part of the hybrid connectivity design, ideally it should be combined with the connectivity model design. Therefore the subsequent section (Connectivity Models) will incorporate scale as part of the targeted connectivity model design.

The following are recommended best practices to minimize scale complexity of hybrid network connectivity design:

  • Route summarization should be used to reduce the number of routes advertised to and received from AWS. This indicates that IP addressing schema needs to be planned to consider a structured IP addressing that ultimately will simplify the use of route summarization. This should be designed with traffic engineering in mind, refer to Traffic engineering subsection under Reliability section in this whitepaper for more information about traffic engineering.

  • Minimize the number of BGP peering sessions by using DXGW with VGW or AWS Transit Gateway, where a single BGP session can provide connectivity to multiple VPCs.