Unified hybrid cloud management

The hybrid cloud management layer provides a unified set of interfaces to consume hybrid cloud services like compute, storage, networking, databases, analytics, and others. These interfaces provide capabilities for provisioning, editing, deleting, monitoring, and operating resources and services on the hybrid cloud. This section describes design practices, components, and AWS services that address the needs of building a unified hybrid cloud management layer in support of hybrid cloud services.

AWS Outposts natively provides unified hybrid cloud management through the use of the same APIs and management tools across on-premises and AWS infrastructure. AWS Outposts supports several AWS services, including compute, storage, networking, and higher-level services allowing consistent operations across the hybrid cloud, eliminating the need to build and manage custom software.

Compute services

Compute services in the hybrid cloud provide the interfaces to manage compute (instances, containers, functions) resources. Figure 2 provides an example customer software implementation of a unified management interface for compute services.

In this example, a hybrid cloud user authenticates with the Identity, security, and access management service of the hybrid cloud to gain authorization to the management interfaces of the compute service. The compute service provides a unified provisioning, monitoring, and operating interface for the user. Internally, the compute service interacts with the core fleet or device management layer for on-premises infrastructure management, AWS EC2 APIs for EC2 management, as well as the core services of metrics and logging services for metrics and logging needs and identity, security and access management for gaining access authorization to on-premises and AWS resources through their respective APIs.

Example of compute service on a hybrid cloud

A consistent mechanism for managing guest operating systems on AWS instances and in on-premises virtual machines across the hybrid cloud provides seamless operations for activities such as software/patch management and policy enforcement. You can manage on-premises servers and AWS EC2 instances with AWS Systems Manager. Systems Manager provides several features, such as remote command execution, patch management, inventory management, state management, and automation, to help with host management functions.

Storage services

Outside of providing core storage services for block, file, and objects, hybrid cloud use-cases often require moving data between on-premises data centers and AWS. These use cases are cloud bursting for storage, disaster recovery (data replication and backups), distributed data processing (for analytics processing on AWS), or geographic expansion (move data closer to customers). Data movement is required for files, block storage, transactional data in databases, and streaming data.

• Files: The File Gateway interface, AWS DataSync, AWS Transfer for SFTP, Amazon EFS, Amazon FSx for Lustre, and Amazon FSx for Windows File Server are used for integrating files between the environments and enabling cloud bursting, disaster recovery, and application migration use-cases.

• Block storage: Volume Gateway provides an on-premises iSCSI interface to provide S3-based storage in AWS (in gateway-cache mode). AWS Storage Gateway can be used for cloud bursting, storage extension, migration, or backups of block stores.

• Transactional data: AWS Database Migration Service (AWS DMS) provides integration between on-premises SQL/NoSQL databases and AWS-based databases (on EC2 or AWS RDS, DynamoDB, DocumentDB) by providing migration and synchronization between the databases. Additionally, DMS can be used to migrate data from on-premises databases to S3 directly for analytics workflow integration.

• Streaming data: Streaming records from on-premises data centers and AWS sources can be collected and analyzed in managed stream stores on AWS, including Amazon Kinesis and Amazon MSK.

The AWS Outposts service also provides on-premises storage with EBS. S3 on AWS Outposts enables customers to store object data on premises using the S3 API.

Networking and security services

Networking and security services enable you to create and manage networks for applications and secure them on the hybrid cloud. A few major components are discussed here:

• Virtual Networking: Virtual networking enables you to provision logically isolated sections of the infrastructure where they can launch resources. You can define your own IP addressing, subnets, routing policies, securities, and gateways in the virtual network based on the application requirements. On the hybrid cloud, these virtual networks extend between AWS and on-premises infrastructures, allowing applications to function across the environment.

  Amazon VPC enables you to create virtual networks in AWS Regions. AWS Direct Connect private virtual interfaces (VIFs), transit VIFs, and site-to-site VPN provide mechanisms to extend the virtual network between Amazon VPC and on-premises networks.

• Load balancing: In a hybrid environment, load balancers are used to distribute traffic to targets across on-premises and public cloud environments. Load balancers abstract the location of the physical resources in the hybrid cloud by presenting a unified front end for an application service. AWS Application Load Balancer and Network Load Balancer, deployed in AWS regions, support targets in AWS regions as well as on-premises. They also support containers as targets deployed across the hybrid environment. Application load balancing on AWS Outposts is fully managed, operates in a single subnet, and scales automatically up to the capacity available on the Outposts rack to meet varying levels of application load without manual intervention.

• Unified DNS: As a best practice, internal DNS resolutions for applications and services deployed in virtual networks on the hybrid cloud must be unified across the infrastructure. AWS Route53 resolver and conditional forwarding rules provide a mechanism to unify DNS resolutions across on-premises DNS servers and resolvers hosted on AWS.

  For public DNS resolutions, internet traffic is routed to the front-ends of web applications deployed on the hybrid cloud through public DNS services like Amazon Route53. In the hybrid cloud, the application front ends (implemented using load balancers or API endpoints on instances) reside either on-premises, in AWS Regions, or split across the infrastructure. AWS Route53 features routing mechanisms for active-backup and active-active hybrid environments.

• Infrastructure Security: Infrastructure security in a hybrid cloud must be applied to all layers of the technology stack across both on-premises and AWS environments. These layers include security at the edge network, perimeter, load balancers, network devices, host and guest operating systems, applications, virtual networks, subnets, and compute. Customers require a common set of security policies that they can apply to AWS or on-premises infrastructure. AWS provides tools like AWS Web Application Firewall (WAF), AWS Shield, VPC Security Groups, and VPC Network ACLs to enforce security boundaries.