Activating a gateway in a virtual private cloud
You can create a private connection between your on-premises software appliance and cloud-based storage infrastructure. You can then use the software appliance to transfer data to AWS storage without your gateway communicating with AWS storage services over the public internet. Using the Amazon VPC service, you can launch AWS resources in a custom virtual network. You can use a virtual private cloud (VPC) to control your network settings, such as the IP address range, subnets, route tables, and network gateways. For more information about VPCs, see What is Amazon VPC? in the Amazon VPC User Guide.
To use a gateway with a Storage Gateway VPC endpoint in your VPC, do the following:
-
Use the VPC console to create a VPC endpoint for Storage Gateway and get the VPC endpoint ID.
-
If you are activating a file gateway, create a VPC endpoint for Amazon S3.
-
If you are activating a file gateway, set up a HTTP proxy and configure it in the file gateway VM local console. You need this proxy for on-premises file gateways that are hypervisor-based, such as those based on VMware, Microsoft HyperV, and Linux Kernel-based Virtual Machine (KVM) . In these cases, you need the proxy to enable your gateway access Amazon S3 private endpoints from outside your VPC. For information about how to configure a HTTP proxy, see Configuring an HTTP proxy.
-
Use the VPC endpoint ID to activate the gateway.
Your gateway must be activated in the same region where your VPC endpoint was created.
For file gateway, the Amazon S3 that is configured for the file share must be in the same region where you created the VPC endpoint for S3.
Creating a gateway using a VPC endpoint
In this section, you can find instructions about how to download, deploy, and activate your file gateway using a VPC endpoint.
Topics
Creating a VPC endpoint for Storage Gateway
Follow these instructions to create a VPC endpoint. If you already have a VPC endpoint for Storage Gateway, you can use it.
To create a VPC endpoint for AWS Storage Gateway
-
Sign in to the AWS Management Console and open the Amazon VPC console at https://console.aws.amazon.com/vpc/
. -
In the navigation pane, choose Endpoints, and then choose Create Endpoint.
-
On the Create Endpoint page, choose AWS Services for Service category.
-
For Service Name, choose
com.amazonaws.
. For exampleregion
.storagegatewaycom.amazonaws.us-east-2.storagegateway
. -
For VPC, choose your VPC and note its Availability Zones and subnets.
-
Verify that Enable Private DNS Name is not selected.
-
For Security group, choose the security group that you want to use for your VPC. You can accept the default security group. Verify that all of the following TCP ports are allowed in your security group:
-
TCP 443
-
TCP 1026
-
TCP 1027
-
TCP 1028
-
TCP 1031
-
TCP 2222
-
-
Choose Create endpoint. The initial state of the endpoint is pending. When the endpoint is created, note the ID of the VPC endpoint that you just created.
-
When the endpoint is created, choose Endpoints, then choose the new VPC endpoint.
-
In the DNS Names section, use the first DNS name that doesn't specify an Availability Zone. Your DNS name look similar to this:
vpce-1234567e1c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
Now that you have a VPC endpoint, you can create your gateway.
If you are creating file gateway, you need to create an endpoint for Amazon S3
also. Follow the same steps as shown in To create a VPC endpoint for AWS Storage Gateway
section above but you choose com.amazonaws.us-east-2.s3
under
Service Name instead. Then you select the route table that you want the S3
endpoint associated with instead of subnet/security group. For instructions, see
Creating a gateway endpoint.
Choosing a gateway type
To choose a gateway type
-
Open the AWS Management Console at https://console.aws.amazon.com/storagegateway/home
, and choose the AWS Region that you want to create your gateway in. If you have previously created a gateway in this AWS Region, the console shows your gateway. Otherwise, the service homepage appears.
-
If you haven't created a gateway in the AWS Region that you chose, choose Get started. If you already have a gateway in the AWS Region that you chose, choose Gateways from the navigation pane, and then choose Create gateway.
-
For Select gateway type, choose a gateway type, and then choose Next. In this example file gateway is selected.
Choosing a host platform and downloading the VM
If you create your gateway on-premises, you deploy the hardware appliance, or download and deploy a gateway VM, and then activate the gateway. If you create your gateway on an Amazon EC2 instance, you launch an Amazon Machine Image (AMI) that contains the gateway VM image and then activate the gateway. For information about supported host platforms, see Supported hypervisors and host requirements.
You can run only file, cached volume, and tape gateways on an Amazon EC2 instance.
To choose a host platform and download the VM
-
For Select host platform, choose the virtualization platform that you want to run your gateway on.
-
Do one of the following:
-
If you choose the hardware appliance, activate it by following the instructions in Activating your hardware appliance.
-
If you choose one of the other options, choose Download image next to your virtualization platform to download a .zip file that contains the .ova file for your virtualization platform.
Note The .zip file is over 500 MB in size and might take some time to download, depending on your network connection.
For Amazon EC2, you create an instance from the provided AMI.
-
-
If you choose a hypervisor option, deploy the downloaded image to your hypervisor. Add at least one local disk for your cache and one local disk for your upload buffer during the deployment. A file gateway requires only one local disk for a cache. For information about local disk requirements, see Hardware and storage requirements.
Depending your hypervisor, set certain options:
-
If you choose VMware, do the following:
-
Store your disk using the Thick provisioned format option. When you use thick provisioning, the disk storage is allocated immediately, resulting in better performance. In contrast, thin provisioning allocates storage on demand. On-demand allocation can affect the normal functioning of AWS Storage Gateway. For Storage Gateway to function properly, the VM disks must be stored in thick-provisioned format.
-
Configure your gateway VM to use paravirtualized disk controllers. For more information, see Configuring the AWS Storage Gateway VM to Use Paravirtualized Disk Controllers.
-
-
If you choose Microsoft Hyper-V, do the following:
-
Configure the disk type using the Fixed size option. When you use fixed-size provisioning, the disk storage is allocated immediately, resulting in better performance. If you don't use fixed-size provisioning, the storage is allocated on demand. On-demand allocation can affect the functioning of Storage Gateway. For Storage Gateway to function properly, the VM disks must be stored in fixed-size provisioned format.
-
When allocating disks, choose virtual hard disk (.vhd) file. Storage Gateway supports the .vhdx file type. By using this file type, you can create larger virtual disks than with other file types. If you create a .vhdx type virtual disk, make sure that the size of the virtual disks that you create doesn't exceed the recommended disk size for your gateway.
-
-
If you choose Linux Kernel-bases Virtual Machine (KVM), do the following:
-
Don't configure your disk to use
sparse
formatting. When you use fixed-size (nonsparse) provisioning, the disk storage is allocated immediately, resulting in better performance. -
Use the parameter
sparse=false
to store your disk in nonsparse format when creating new virtual disks in the VM with thevirt-install
command for provisioning new virtual machines. -
Use
virtio
drivers for disk and network devices. -
We recommend that you don't set the
current_memory
option. If necessary, set it equal to the RAM provisioned to the gateway in the--ram
parameter.
Following is an example
virt-install
command for installing KVM.virt-install --name "SGW_KVM" --description "SGW KVM" --os-type=generic --ram=32768 --vcpus=16 --disk path=fgw-kvm.qcow2,bus=virtio,size=80,sparse=false --disk path=fgw-kvm-cache.qcow2,bus=virtio,size=1024,sparse=false --network default,model=virtio --graphics none --import
-
-
For VMware, Microsoft Hyper-V, and KVM, synchronizing the VM time with the host time is required for successful gateway activation. Make sure that your host clock is set to the correct time and synchronize it with a Network Time Protocol (NTP) server.
For information about deploying your gateway to an Amazon EC2 host, see Deploy your gateway to an Amazon EC2 host.
Choosing a service endpoint
You can activate your gateway using a private VPC endpoint. If you use a VPC endpoint, all communication from your gateway to AWS services occurs through the VPC endpoint in your VPC in AWS.
Connecting to your gateway
To connect to your gateway, first get the IP address or activation key of your gateway VM. You use the IP address or activation key to activate your gateway. For gateways deployed and activated on an on-premises host, you can get the IP address or activation key from your gateway VM local console or your hypervisor client. For gateways deployed and activated on an Amazon EC2 instance, you can get the IP address or activation key from the Amazon EC2 console.
The activation process associates your gateway with your AWS account. Your gateway VM must be running for activation to succeed.
Make sure that you select the correct gateway type. The .ova files and Amazon Machine Images (AMIs) for the gateway types are different and are not interchangeable.
To get the IP address or activation key for your gateway VM from the local console
-
Log on to your gateway VM local console. For detailed instructions, see the following:
-
VMware ESXi – Accessing the Gateway Local Console with VMware ESXi.
-
Microsoft Hyper-V – Access the Gateway Local Console with Microsoft Hyper-V.
-
Linux KVM – Accessing the Gateway Local Console with Linux KVM.
-
-
Get the IP address from the top of the menu page, and note it for later use.
To get the IP address or activation key from an EC2 instance
-
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/
. -
In the navigation pane, choose Instances, and then choose the EC2 instance.
-
Choose the Details tab at the bottom, and then note the IP address or activation key. You use one of these to activate the gateway.
For activation with an IP address, you can use the public or private IP address assigned to a gateway. You must be able to reach the IP address that you use from the browser from which you perform the activation.
Setting up and configuring a HTTP proxy (on-premises file gateways only)
If you are activating a file gateway, you need to set up an HTTP proxy and configure it by using the file gateway VM local console. You need this proxy for an on-premises file gateway to access Amazon S3 private endpoints from outside your VPC. If you already have a HTTP proxy in Amazon EC2, you can use it. However, you need to verify that all of the following TCP ports are allowed in your security group:
-
TCP 443
-
TCP 1026
-
TCP 1027
-
TCP 1028
-
TCP 1031
-
TCP 2222
If you don't have an Amazon EC2 proxy, use the following procedure to set up and configure a HTTP proxy.
To set up a proxy server
-
Launch an Amazon EC2 Linux AMI. We recommend using an instance family that is network-optimized, such as the c5n.large.
-
Use the following command to install squid:
sudo yum install squid
. Doing this creates a default config file in/etc/squid/squid.conf
. -
Replace the contents of this config file with the following.
# # Recommended minimum configuration: # # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl SSL_ports port 1026 acl SSL_ports port 1027 acl SSL_ports port 1028 acl SSL_ports port 1031 acl SSL_ports port 2222 acl CONNECT method CONNECT # # Recommended minimum Access Permission configuration: # # Deny requests to certain unsafe ports http_access deny !SSL_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow localnet http_access allow localhost # And finally deny all other access to this proxy http_access deny all # Squid normally listens to port 3128 http_port 3128 # Leave coredumps in the first cache dir coredump_dir /var/spool/squid # # Add any of your own refresh_pattern entries above these. # refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320
-
If you don't need to lock down the proxy server and don't need to make any changes, then enable and start the proxy server using the following commands. These commands start the server when it boots up.
sudo chkconfig squid on sudo service squid start
You now configure the HTTP proxy for Storage Gateway to use it. When configuring the gateway to use a proxy, use the default squid port 3128. The squid.conf file that is generated covers the following required TCP ports by default:
-
TCP 443
-
TCP 1026
-
TCP 1027
-
TCP 1028
-
TCP 1031
-
TCP 2222
To use the VM local console to configure the HTTP proxy
-
Log in to your gateway's VM local console. For information about how to log in, see Logging in to the file gateway local console.
-
In the main menu, choose Configure HTTP proxy.
-
In the Configuration menu, choose Configure HTTP proxy.
-
Provide the host name and port for your proxy server.
For detailed information on how to configure a HTTP proxy, see Configuring an HTTP proxy.
For detailed information about how to get a gateway IP address, see Connecting to Your Gateway.
Activate your gateway in a VPC
Activating a file gateway requires additional setup.
The following, shown on the activation page, are the gateway settings that you selected. The activation page appears after you associate your gateway with your AWS account, as described preceding.
-
Gateway type specifies the type of gateway that you are activating.
-
Endpoint type specifies the type of endpoint that you selected for your gateway.
-
AWS Region specifies the AWS Region where your gateway will be activated and where your data will be stored. If Endpoint type is VPC, the AWS Region should be same as the Region where your VPC endpoint is located.
If activation isn't successful, see Troubleshooting your gateway for possible solutions.
To associate your gateway with your AWS account
If you don’t have internet access and private network access from your browser, you can still do the following.
-
Enter the fully qualified DNS name of the VPC endpoint or elastic network interface to get the activation key from the gateway. You can use curl with the following URL, or just enter this URL into your web browser.
http://
VM IP ADDRESS
/?gatewayType=FILE_S3&activationRegion=REGION
&vpcEndpoint=VPCEndpointDNSname
&no_redirectAn example curl command follows.
curl "http://203.0.113.100/?gatewayType=FILE_S3&activationRegion=us-east-1&vpcEndpoint=vpce-12345678e91c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com&no_redirect"
An example activation key follows.
BME11-LQPTD-DF11P-BLLQ0-111V1
-
Use the AWS CLI to activate the gateway by specifying the activation key you received in previous step, for example:
aws --region us-east-1 storagegateway activate-gateway --activation-key BME11-LQPTD-DF11P-BLLQ0-111V1 --gateway-type FILE_S3 --gateway-name user-ec2-iad-pl-fgw2 --gateway-timezone GMT-4:00 --gateway-region us-east-1 --endpoint-url https://vpce-12345678e91c24a1fe9-62qntt8k.storagegateway.us-east-1.vpce.amazonaws.com
Following is an example response.
{"GatewayARN": "arn:aws:storagegateway:us-east-1:123456789012:gateway/sgw-FFF12345"}
Configure local disks
When you deployed the VM, you allocated local disks for your gateway. Now you configure your gateway to use these disks.
To configure local disks
-
For Configure local disks, identify the disks you added and decide which ones you want to allocate for cached storage. For information about disk size quotas, see Recommended local disk sizes for your gateway.
-
For Allocated to, choose Cache for the disk that you want to configure as cache storage.
If you don't see your disks, choose Refresh.
-
Choose Save and continue to save your configuration settings.
Allowing traffic to required ports in your HTTP proxy
If you use a HTTP proxy, make sure that you allow traffic from Storage Gateway to the destinations and ports listed following.
When Storage Gateway is communicating through the public endpoints, it communicates with the following Storage Gateway services.
anon-cp.storagegateway.
region
.amazonaws.com:443 client-cp.storagegateway.region
.amazonaws.com:443 proxy-app.storagegateway.region
.amazonaws.com:443 dp-1.storagegateway.region
.amazonaws.com:443 storagegateway.region
.amazonaws.com:443 (Required for making API calls) s3.region
.amazonaws.com (Required only for File Gateway)
Depending on your gateway's AWS Region, replace
region
in the endpoint with the corresponding
region string. For example, if you create a gateway in the US West (Oregon)
region, the endpoint looks like this:
storagegateway.us-west-2.amazonaws.com:443
.
When Storage Gateway is communicating through the VPC endpoint, it communicates with the AWS services through multiple ports on the Storage Gateway VPC endpoint and port 443 on the Amazon S3 private endpoint.
-
TCP ports on Storage Gateway VPC endpoint.
-
443, 1026, 1027, 1028, 1031, and 2222
-
-
TCP port on S3 private endpoint
-
443
-
You are now ready to create resources for your gateway.
Next Step
-
File gateway: Creating a file share
-
Volume gateway: Creating a Volume
-
Tape gateway: Creating Tapes