Buat jaringan VPC

Alur Kerja Terkelola Amazon untuk Apache Airflow memerlukan VPC Amazon dan komponen jaringan tertentu untuk mendukung lingkungan. Panduan ini menjelaskan berbagai opsi untuk membuat jaringan Amazon VPC untuk lingkungan Alur Kerja Terkelola Amazon untuk Apache Airflow.

catatan

Apache Airflow bekerja paling baik di lingkungan jaringan latensi rendah. Jika Anda menggunakan Amazon VPC yang sudah ada yang merutekan lalu lintas ke wilayah lain atau ke lingkungan di lokasi, sebaiknya tambahkanAWS PrivateLinktitik akhir untuk Amazon SQS, CloudWatchAmazon S3,AWS KMS, dan Amazon ECR. Untuk informasi lebih lanjut tentang mengkonfigurasiAWS PrivateLinkuntuk Amazon MWAA, lihatMembuat jaringan Amazon VPC tanpa akses internet.

Daftar Isi

• Prasyarat
• Sebelum Anda memulai
• Opsi untuk membuat jaringan Amazon VPC
  • Opsi satu: Membuat jaringan VPC di konsol Amazon MWAA
  • Opsi dua: Membuat jaringan Amazon VPCbersamaAkses internet
  • Opsi tiga: Membuat jaringan Amazon VPCtanpaAkses internet
• Apa selanjutnya?

Prasyarat

TheAWS Command Line Interface(AWS CLI) adalah alat open source yang memungkinkan Anda untuk berinteraksi denganAWSlayanan menggunakan perintah di shell baris perintah Anda. Untuk menyelesaikan langkah-langkah di halaman ini, Anda memerlukan yang berikut:

• AWS CLI— Instal versi 2.

• AWS CLI— Konfigurasi cepat denganaws configure.

Sebelum Anda memulai

• TheJaringan VPCAnda tentukan untuk lingkungan Anda tidak dapat diubah setelah lingkungan dibuat.

• Anda dapat menggunakan perutean pribadi atau publik untuk Amazon VPC dan Apache AirflowServer web. Untuk melihat daftar opsi, lihatContoh kasus penggunaan untuk mode akses Amazon VPC dan Apache Airflow.

Opsi untuk membuat jaringan Amazon VPC

Bagian berikut menjelaskan opsi yang tersedia untuk membuat jaringan Amazon VPC untuk suatu lingkungan.

Opsi satu: Membuat jaringan VPC di konsol Amazon MWAA

Bagian berikut menunjukkan cara membuat jaringan Amazon VPC di konsol Amazon MWAA. Opsi ini menggunakanRouting publik melalui Internet. Hal ini dapat digunakan untuk Apache AirflowServer webdenganJaringan pribadiatauJaringan publikmode akses.

Gambar berikut menunjukkan di mana Anda dapat menemukanBuat MWAA VPCtombol di konsol Amazon MWAA.

Opsi dua: Membuat jaringan Amazon VPCbersamaAkses internet

Berikut iniAWS CloudFormationtemplate membuat jaringan Amazon VPCdengan akses internetdalam default AndaAWSWilayah. Opsi ini menggunakanRouting publik melalui Internet. Template ini dapat digunakan untuk Apache AirflowServer webdenganJaringan pribadiatauJaringan publikmode akses.

1. Salin isi template berikut dan simpan secara lokal sebagaicfn-vpc-public-private.yaml. Anda juga bisaunduh templatnya.

   Description:  This template deploys a VPC, with a pair of public and private subnets spread
     across two Availability Zones. It deploys an internet gateway, with a default
     route on the public subnets. It deploys a pair of NAT gateways (one in each AZ),
     and default routes for them in the private subnets.
   
   Parameters:
     EnvironmentName:
       Description: An environment name that is prefixed to resource names
       Type: String
       Default: mwaa-
   
     VpcCIDR:
       Description: Please enter the IP range (CIDR notation) for this VPC
       Type: String
       Default: 10.192.0.0/16
   
     PublicSubnet1CIDR:
       Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone
       Type: String
       Default: 10.192.10.0/24
   
     PublicSubnet2CIDR:
       Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone
       Type: String
       Default: 10.192.11.0/24
   
     PrivateSubnet1CIDR:
       Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone
       Type: String
       Default: 10.192.20.0/24
   
     PrivateSubnet2CIDR:
       Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone
       Type: String
       Default: 10.192.21.0/24
   
   Resources:
     VPC:
       Type: AWS::EC2::VPC
       Properties:
         CidrBlock: !Ref VpcCIDR
         EnableDnsSupport: true
         EnableDnsHostnames: true
         Tags:
           - Key: Name
             Value: !Ref EnvironmentName
   
     InternetGateway:
       Type: AWS::EC2::InternetGateway
       Properties:
         Tags:
           - Key: Name
             Value: !Ref EnvironmentName
   
     InternetGatewayAttachment:
       Type: AWS::EC2::VPCGatewayAttachment
       Properties:
         InternetGatewayId: !Ref InternetGateway
         VpcId: !Ref VPC
   
     PublicSubnet1:
       Type: AWS::EC2::Subnet
       Properties:
         VpcId: !Ref VPC
         AvailabilityZone: !Select [ 0, !GetAZs '' ]
         CidrBlock: !Ref PublicSubnet1CIDR
         MapPublicIpOnLaunch: true
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Public Subnet (AZ1)
   
     PublicSubnet2:
       Type: AWS::EC2::Subnet
       Properties:
         VpcId: !Ref VPC
         AvailabilityZone: !Select [ 1, !GetAZs  '' ]
         CidrBlock: !Ref PublicSubnet2CIDR
         MapPublicIpOnLaunch: true
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Public Subnet (AZ2)
   
     PrivateSubnet1:
       Type: AWS::EC2::Subnet
       Properties:
         VpcId: !Ref VPC
         AvailabilityZone: !Select [ 0, !GetAZs  '' ]
         CidrBlock: !Ref PrivateSubnet1CIDR
         MapPublicIpOnLaunch: false
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Private Subnet (AZ1)
   
     PrivateSubnet2:
       Type: AWS::EC2::Subnet
       Properties:
         VpcId: !Ref VPC
         AvailabilityZone: !Select [ 1, !GetAZs  '' ]
         CidrBlock: !Ref PrivateSubnet2CIDR
         MapPublicIpOnLaunch: false
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Private Subnet (AZ2)
   
     NatGateway1EIP:
       Type: AWS::EC2::EIP
       DependsOn: InternetGatewayAttachment
       Properties:
         Domain: vpc
   
     NatGateway2EIP:
       Type: AWS::EC2::EIP
       DependsOn: InternetGatewayAttachment
       Properties:
         Domain: vpc
   
     NatGateway1:
       Type: AWS::EC2::NatGateway
       Properties:
         AllocationId: !GetAtt NatGateway1EIP.AllocationId
         SubnetId: !Ref PublicSubnet1
   
     NatGateway2:
       Type: AWS::EC2::NatGateway
       Properties:
         AllocationId: !GetAtt NatGateway2EIP.AllocationId
         SubnetId: !Ref PublicSubnet2
   
     PublicRouteTable:
       Type: AWS::EC2::RouteTable
       Properties:
         VpcId: !Ref VPC
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Public Routes
   
     DefaultPublicRoute:
       Type: AWS::EC2::Route
       DependsOn: InternetGatewayAttachment
       Properties:
         RouteTableId: !Ref PublicRouteTable
         DestinationCidrBlock: 0.0.0.0/0
         GatewayId: !Ref InternetGateway
   
     PublicSubnet1RouteTableAssociation:
       Type: AWS::EC2::SubnetRouteTableAssociation
       Properties:
         RouteTableId: !Ref PublicRouteTable
         SubnetId: !Ref PublicSubnet1
   
     PublicSubnet2RouteTableAssociation:
       Type: AWS::EC2::SubnetRouteTableAssociation
       Properties:
         RouteTableId: !Ref PublicRouteTable
         SubnetId: !Ref PublicSubnet2
   
   
     PrivateRouteTable1:
       Type: AWS::EC2::RouteTable
       Properties:
         VpcId: !Ref VPC
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Private Routes (AZ1)
   
     DefaultPrivateRoute1:
       Type: AWS::EC2::Route
       Properties:
         RouteTableId: !Ref PrivateRouteTable1
         DestinationCidrBlock: 0.0.0.0/0
         NatGatewayId: !Ref NatGateway1
   
     PrivateSubnet1RouteTableAssociation:
       Type: AWS::EC2::SubnetRouteTableAssociation
       Properties:
         RouteTableId: !Ref PrivateRouteTable1
         SubnetId: !Ref PrivateSubnet1
   
     PrivateRouteTable2:
       Type: AWS::EC2::RouteTable
       Properties:
         VpcId: !Ref VPC
         Tags:
           - Key: Name
             Value: !Sub ${EnvironmentName} Private Routes (AZ2)
   
     DefaultPrivateRoute2:
       Type: AWS::EC2::Route
       Properties:
         RouteTableId: !Ref PrivateRouteTable2
         DestinationCidrBlock: 0.0.0.0/0
         NatGatewayId: !Ref NatGateway2
   
     PrivateSubnet2RouteTableAssociation:
       Type: AWS::EC2::SubnetRouteTableAssociation
       Properties:
         RouteTableId: !Ref PrivateRouteTable2
         SubnetId: !Ref PrivateSubnet2
   
     SecurityGroup:
       Type: AWS::EC2::SecurityGroup
       Properties:
         GroupName: "mwaa-security-group"
         GroupDescription: "Security group with a self-referencing inbound rule."
         VpcId: !Ref VPC
   
     SecurityGroupIngress:
       Type: AWS::EC2::SecurityGroupIngress
       Properties:
         GroupId: !Ref SecurityGroup
         IpProtocol: "-1"
         SourceSecurityGroupId: !Ref SecurityGroup
   
   Outputs:
     VPC:
       Description: A reference to the created VPC
       Value: !Ref VPC
   
     PublicSubnets:
       Description: A list of the public subnets
       Value: !Join [ ",", [ !Ref PublicSubnet1, !Ref PublicSubnet2 ]]
   
     PrivateSubnets:
       Description: A list of the private subnets
       Value: !Join [ ",", [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]]
   
     PublicSubnet1:
       Description: A reference to the public subnet in the 1st Availability Zone
       Value: !Ref PublicSubnet1
   
     PublicSubnet2:
       Description: A reference to the public subnet in the 2nd Availability Zone
       Value: !Ref PublicSubnet2
   
     PrivateSubnet1:
       Description: A reference to the private subnet in the 1st Availability Zone
       Value: !Ref PrivateSubnet1
   
     PrivateSubnet2:
       Description: A reference to the private subnet in the 2nd Availability Zone
       Value: !Ref PrivateSubnet2
   
     SecurityGroupIngress:
       Description: Security group with self-referencing inbound rule
       Value: !Ref SecurityGroupIngress

2. Di prompt perintah Anda, arahkan ke direktori tempatcfn-vpc-public-private.yamldisimpan. Misalnya:

   cd mwaaproject

3. Gunakanaws cloudformation create-stackperintah untuk membuat tumpukan menggunakanAWS CLI.

   aws cloudformation create-stack --stack-name mwaa-environment --template-body file://cfn-vpc-public-private.yaml

   catatan

   Dibutuhkan sekitar 30 menit untuk membuat infrastruktur Amazon VPC.

Opsi tiga: Membuat jaringan Amazon VPCtanpaAkses internet

Berikut iniAWS CloudFormationtemplate membuat jaringan Amazon VPCtanpa akses internetdalam default AndaAWSwilayah.

penting

Saat menggunakan Amazon VPC tanpa akses internet, Anda harus memberikan izin ke Amazon ECR untuk mengakses Amazon S3 menggunakan titik akhir gateway. Anda dapat membuat titik akhir gateway dengan melakukan hal berikut:

1. Salin berikut iniJSONKebijakan IAM, dan simpan secara lokal sebagais3-gw-endpoint-policy.json. Kebijakan ini memberikan izin minimum yang diperlukan untuk Amazon ECR untuk mengakses sumber daya Amazon S3.

   {
     "Statement": [
       {
         "Sid": "Access-to-specific-bucket-only",
         "Principal": "*",
         "Action": [
           "s3:GetObject"
         ],
         "Effect": "Allow",
         "Resource": ["arn:aws:s3:::prod-region-starport-layer-bucket/*"]
       }
     ]
   }

2. Buat titik akhir menggunakan yang berikutAWS CLIperintah. Ganti nilai untuk--vpc-iddan--route-table-idsdengan informasi untuk Amazon VPC Anda. Ganti--service-namedengan nama sesuai dengan wilayah Anda.

   $ aws ec2 create-vpc-endpoint --vpc-id vpc-1a2b3c4d \
   --service-name com.amazonaws.us-west-2.s3 \
   --route-table-ids rtb-11aa22bb \
   --vpc-endpoint-type Gateway \
   --policy-document file://s3-gw-endpoint-policy.json

Untuk informasi selengkapnya tentang membuat titik akhir gateway Amazon S3 untuk Amazon ECR, lihatBuat titik akhir gateway Amazon S3diPanduan Pengguna Registri Kontainer Elastis Amazon.

Opsi ini menggunakanPerutean pribadi tanpa akses Internet. Template ini dapat digunakan untuk Apache AirflowServer webdenganJaringan pribadimode akses saja. Ini menciptakan yang dibutuhkanTitik akhir VPC untukAWSlayanan yang digunakan oleh lingkungan.

1. Salin isi template berikut dan simpan secara lokal sebagaicfn-vpc-private.yaml. Anda juga bisaunduh templatnya.

   AWSTemplateFormatVersion: "2010-09-09"
        
   Parameters:
      VpcCIDR:
        Description: The IP range (CIDR notation) for this VPC
        Type: String
        Default: 10.192.0.0/16
        
      PrivateSubnet1CIDR:
        Description: The IP range (CIDR notation) for the private subnet in the first Availability Zone
        Type: String
        Default: 10.192.10.0/24
        
      PrivateSubnet2CIDR:
        Description: The IP range (CIDR notation) for the private subnet in the second Availability Zone
        Type: String
        Default: 10.192.11.0/24
        
   Resources:
      VPC:
        Type: AWS::EC2::VPC
        Properties:
          CidrBlock: !Ref VpcCIDR
          EnableDnsSupport: true
          EnableDnsHostnames: true
          Tags:
           - Key: Name
             Value: !Ref AWS::StackName
        
      RouteTable:
        Type: AWS::EC2::RouteTable
        Properties:
          VpcId: !Ref VPC
          Tags:
           - Key: Name
             Value: !Sub "${AWS::StackName}-route-table"
        
      PrivateSubnet1:
        Type: AWS::EC2::Subnet
        Properties:
          VpcId: !Ref VPC
          AvailabilityZone: !Select [ 0, !GetAZs  '' ]
          CidrBlock: !Ref PrivateSubnet1CIDR
          MapPublicIpOnLaunch: false
          Tags:
           - Key: Name
             Value: !Sub "${AWS::StackName} Private Subnet (AZ1)"
        
      PrivateSubnet2:
        Type: AWS::EC2::Subnet
        Properties:
          VpcId: !Ref VPC
          AvailabilityZone: !Select [ 1, !GetAZs  '' ]
          CidrBlock: !Ref PrivateSubnet2CIDR
          MapPublicIpOnLaunch: false
          Tags:
           - Key: Name
             Value: !Sub "${AWS::StackName} Private Subnet (AZ2)"
        
      PrivateSubnet1RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
          RouteTableId: !Ref RouteTable
          SubnetId: !Ref PrivateSubnet1
        
      PrivateSubnet2RouteTableAssociation:
        Type: AWS::EC2::SubnetRouteTableAssociation
        Properties:
          RouteTableId: !Ref RouteTable
          SubnetId: !Ref PrivateSubnet2
        
      S3VpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.s3"
          VpcEndpointType: Gateway
          VpcId: !Ref VPC
          RouteTableIds:
           - !Ref RouteTable
        
      SecurityGroup:
        Type: AWS::EC2::SecurityGroup
        Properties:
          VpcId: !Ref VPC
          GroupDescription: Security Group for Amazon MWAA Environments to access VPC endpoints
          GroupName: !Sub "${AWS::StackName}-mwaa-vpc-endpoints"
      
      SecurityGroupIngress:
        Type: AWS::EC2::SecurityGroupIngress
        Properties:
          GroupId: !Ref SecurityGroup
          IpProtocol: "-1"
          SourceSecurityGroupId: !Ref SecurityGroup
      
      SqsVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.sqs"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
        
      CloudWatchLogsVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.logs"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
        
      CloudWatchMonitoringVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.monitoring"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
        
      KmsVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.kms"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
        
      EcrApiVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.ecr.api"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
        
      EcrDkrVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.ecr.dkr"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
   
      AirflowApiVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.airflow.api"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup  
           
      AirflowEnvVpcEndoint:
        Type: AWS::EC2::VPCEndpoint
        Properties:
          ServiceName: !Sub "com.amazonaws.${AWS::Region}.airflow.env"
          VpcEndpointType: Interface
          VpcId: !Ref VPC
          PrivateDnsEnabled: true
          SubnetIds:
           - !Ref PrivateSubnet1
           - !Ref PrivateSubnet2
          SecurityGroupIds:
           - !Ref SecurityGroup
   
   Outputs:
      VPC:
        Description: A reference to the created VPC
        Value: !Ref VPC
        
      MwaaSecurityGroupId:
        Description: Associates the Security Group to the environment to allow access to the VPC endpoints 
        Value: !Ref SecurityGroup
        
      PrivateSubnets:
        Description: A list of the private subnets
        Value: !Join [ ",", [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ]]
        
      PrivateSubnet1:
        Description: A reference to the private subnet in the 1st Availability Zone
        Value: !Ref PrivateSubnet1
        
      PrivateSubnet2:
        Description: A reference to the private subnet in the 2nd Availability Zone
        Value: !Ref PrivateSubnet2

2. Di prompt perintah Anda, arahkan ke direktori tempatcfn-vpc-private.ymldisimpan. Misalnya:

   cd mwaaproject

3. Gunakanaws cloudformation create-stackperintah untuk membuat tumpukan menggunakanAWS CLI.

   aws cloudformation create-stack --stack-name mwaa-private-environment --template-body file://cfn-vpc-private.yml

   catatan

   Dibutuhkan sekitar 30 menit untuk membuat infrastruktur Amazon VPC.

4. Anda harus membuat mekanisme untuk mengakses titik akhir VPC ini dari komputer Anda. Untuk mempelajari selengkapnya, lihat Mengelola akses ke titik akhir VPC Amazon khusus layanan di Amazon MWAA.

catatan

Anda selanjutnya dapat membatasi akses keluar di CIDR grup keamanan Amazon MWAA Anda. Misalnya, Anda dapat membatasi dirinya sendiri dengan menambahkan aturan keluar referensi diri,daftar awalanuntuk Amazon S3, dan CIDR Amazon VPC Anda.

Apa selanjutnya?

• Pelajari cara membuat lingkungan Amazon MWAA diBuat lingkungan Amazon MWAA.

• Pelajari cara membuat terowongan VPN dari komputer Anda ke Amazon VPC Anda dengan perutean pribadi diTutorial: Mengkonfigurasi akses jaringan pribadi menggunakanAWS Client VPN.