Menu
AWS Storage Gateway
User Guide (API Version 2013-06-30)

Using Your Volume

Following, you can find instructions about how to use your volumes. To use your volumes, you first connect it to your clients as iSCSI targets, initialize and format volume.

Connecting Your Volumes to Your Client

You use the iSCSI initiator in your client to connect to your volumes. At the end of the following procedure, the volumes become available as local devices on your client.

Important

AWS Storage Gateway supports connecting multiple hosts to the same volume if the hosts coordinate access by using Windows Server Failover Clustering (WSFC). However, you can't connect multiple hosts to that same volume (for example, sharing a non-clustered NTFS/ext4 file system) without using WSFC.

Connecting to Microsoft Windows Client

The following procedure shows a summary of the step you follow to connect your targets. For more information, see Connecting iSCSI Initiators.

To connect to a Windows client

  1. Start iscsicpl.exe.

  2. In the iSCSI Initiator Properties dialog box, choose the Discovery tab, and then choose the Discovery Portal button.

  3. In the Discover Target Portal dialog box, type the IP address of your iSCSI target for IP address or DNS name,

  4. Connect the new target portal to the storage volume target on the gateway:

  5. Select the target, and then choose Connect.

  6. In the Targets tab, ensure that the target Status has the value Connected indicating the target is connected, and then choose OK.

Connecting to Red Hat Enterprise Linux Client

The following procedure shows a summary of the step you follow to connect your targets. For more information, see Connecting iSCSI Initiators.

To connect a Linux client to the iSCSI targets

  1. Install the iscsi-initiator-utils RPM package.

    You can use the following command to install the package.

    Copy
    sudo yum install iscsi-initiator-utils
  2. Ensure that the iSCSI daemon is running.

    For RHEL 5 or 6, use the following command.

    Copy
    sudo /etc/init.d/iscsi status

    For RHEL 7, use the following command.

    Copy
    sudo service iscsid status
  3. Discover the volume or VTL device targets defined for a gateway. Use the following discovery command.

    Copy
    sudo /sbin/iscsiadm --mode discovery --type sendtargets --portal [GATEWAY_IP]:3260

    The output of the discovery command will look like the following example output.

    For volume gateways: [GATEWAY_IP]:3260, 1 iqn.1997-05.com.amazon:myvolume

    For tape gateways: iqn.1997-05.com.amazon:[GATEWAY_IP]-tapedrive-01

  4. Connect to a target.

    Note that you need to specify the correct [GATEWAY_IP] and IQN in the connect command.

    Use the following command.

    Copy
    sudo /sbin/iscsiadm --mode node --targetname iqn.1997-05.com.amazon:[ISCSI_TARGET_NAME] --portal [GATEWAY_IP]:3260,1 --login
  5. Verify that the volume is attached to the client machine (the initiator). use the following command.

    Copy
    ls -l /dev/disk/by-path

    The output of the command will look like the following example output.

    lrwxrwxrwx. 1 root root 9 Apr 16 19:31 ip-[GATEWAY_IP]:3260-iscsi-iqn.1997-05.com.amazon:myvolume-lun-0 -> ../../sda

    After setting up your initiator, we highly recommend that you customize your iSCSI settings as discussed in Customizing Your Linux iSCSI Settings.

Initializing and Formatting Your Volume

Before testing your set up, you initialize and format the your volume.

Initializing and formatting on Microsoft Windows

To initialize and format the storage volume you just mapped

  1. Start diskmgmt.msc to open the Disk Management console.

  2. In the Initialize Disk dialog box, initialize the volume as a MBR (Master Boot Record) partition. When selecting the partition style, you should take into account the type of volume you are connecting to—cached or stored—as shown in the following table.

    Partition Style Use in the Following Conditions
    MBR (Master Boot Record)
    • If your gateway is a stored volume and the storage volume is limited to 1 TiB in size.

    • If your gateway is a cached volume and the storage volume is less than 2 TiB in size.

    GPT (GUID Partition Table)

    If your gateway's storage volume is 2 TiB or greater in size.

  3. Create a simple volume:

    1. Bring the volume online to initialize it. All the available volumes are displayed in the disk management console.

    2. Open the context (right-click) menu for the disk, and then choose New Simple Volume.

      Important

      Be careful not to format the wrong disk. Check to ensure that the disk you are formatting matches the size of the local disk you allocated to the gateway VM and that it has a status of Unallocated.

    3. Specify the maximum disk size.

    4. Assign drive letter or Path to your volume and format the volume with Perform a quick format.

      Warning

      Selecting Perform a quick format is highly recommended for cached volumes because it results in less initialization I/O, smaller initial snapshot size, and the fastest time to a usable volume. It also avoids cached volume usage that comes from the full format process and not from application data activity.

      Note

      The time that it takes to format the volume depends on the size of the volume. The process might take several minutes to complete.

Initializing and formatting on Red Hat Enterprise Linux

  1. Change directory to the /dev folder.

  2. Run the sudo cfdiskcommand.

  3. Identify your new volume. You can list the partition layout of your volumes to find new volumes.

    $ lsblk

    Unrecognized volumes label error for the new unpartitioned volume is displayed.

  4. Initialize the new volume

    initialize the volume as a MBR (Master Boot Record) partition. When selecting the partition style, you should take into account the type of volume you are connecting to—cached or stored—as shown in the following table.

    Partition Style Use in the Following Conditions
    MBR (Master Boot Record)
    • If your gateway is a stored volume and the storage volume is limited to 1 TiB in size.

    • If your gateway is a cached volume and the storage volume is less than 2 TiB in size.

    GPT (GUID Partition Table)

    If your gateway's storage volume is 2 TiB or greater in size.

    For MBR partition use the following command: sudo parted /dev/your volume mklabel msdos

    For GPT partition use the following command: sudo parted /dev/your volume mklabel gpt

  5. Create a partition

    sudo parted -a opt /dev/your volume mkpart primary file system 0% 100%

  6. Assign a drive letter to the partition and create a file system.

    sudo mkfs drive letter datapartition /dev/your volume

  7. Mount the file system.

    sudo mount -o defaults /dev/your volume /mnt/your directory

Testing Your Gateway

You test your volume gateway setup by performing the following tasks:

  1. Write data to the volume.

  2. Take a snapshot.

  3. Restore the snapshot to another volume.

You verify the setup for a gateway by taking a snapshot backup of your volume and storing the snapshot in AWS. You then restore the snapshot to a new volume. Your gateway copies the data from the specified snapshot in AWS to the new volume.

Note

Restoring data from Amazon Elastic Block Store (Amazon EBS) volumes that are encrypted is not supported.

To create a snapshot of a storage volume on Microsoft Windows

  1. On your Windows computer, copy some data to your mapped storage volume.

    The amount of data copied doesn't matter for this demonstration. A small file is enough to demonstrate the restore process.

  2. In the navigation pane of the AWS Storage Gateway console, choose Volumes.

  3. Select the storage volume that you created for the gateway.

    This gateway should have only one storage volume. Selecting the volume displays its properties.

  4. For Actions, choose Create Snapshot to create a snapshot of the volume.

    Depending on the amount of data on the disk and the upload bandwidth, it might take a few seconds to complete the snapshot. Note the volume ID for the volume from which you create a snapshot. You will use the ID to find the snapshot.

  5. In the Create Snapshot dialog box, provide a description for your snapshot, and then choose Create Snapshot.

    Your snapshot is stored as an Amazon EBS snapshot. Take note of your snapshot ID.

    The number of snapshots created for your volume is displayed in the snapshot column.

  6. For Snapshot, choose the link for the volume you created the snapshot for to see your EBS snapshot on the Amazon EC2 console.

Write to the volume, Take Snapshots and Restore

Where Do I Go from Here?

The AWS Storage Gateway service provides an easy way for you to back your application storage with the storage infrastructure of the AWS Cloud. In Using Your Volume, you created and provisioned a gateway, and then connected your Windows host to the gateway's storage volume. You added data to the gateway's iSCSI volume, took a snapshot of the volume and restored it to a new volume, and connected to the new volume and verified that the data shows up on it.

After you finish the exercise, consider the following:

Other sections of this guide include information about how to do the following:

Sizing Your Gateway's Storage for Real-World Workloads

By this point, you have a simple, working gateway. However, the assumptions used to create this gateway are not appropriate for real-world workloads. If you want to use this gateway for real-world workloads, you need to do two things:

  1. Size your upload buffer appropriately.

  2. Set up monitoring for your upload buffer, if you haven't done so already.

Following, you can find how to do both of these tasks. If you activated a gateway for cached volumes, you also need to size your cache storage for real-world workloads.

To size your upload buffer and cache storage for a gateway-cached setup

  • Use the formula shown in Adding and Removing Upload Buffer for sizing the upload buffer. We strongly recommend that you allocate at least 150 GiB for the upload buffer. If the upload buffer formula yields a value less than 150 GiB, use 150 GiB as your allocated upload buffer.

    The upload buffer formula takes into account the difference between throughput from your application to your gateway and throughput from your gateway to AWS, multiplied by how long you expect to write data. For example, assume that your applications write text data to your gateway at a rate of 40 MB per second for 12 hours a day and your network throughput is 12 MB per second. Assuming a compression factor of 2:1 for the text data, the formula specifies that you need to allocate approximately 675 GiB of upload buffer space.

To size your upload buffer for a stored setup

  • Use the formula discussed in Adding and Removing Upload Buffer. We strongly recommend that you allocate at least 150 GiB for your upload buffer. If the upload buffer formula yields a value less than 150 GiB, use 150 GiB as your allocated upload buffer.

    The upload buffer formula takes into account the difference between throughput from your application to your gateway and throughput from your gateway to AWS, multiplied by how long you expect to write data. For example, assume that your applications write text data to your gateway at a rate of 40 MB per second for 12 hours a day and your network throughput is 12 MB per second. Assuming a compression factor of 2:1 for the text data, the formula specifies that you need to allocate approximately 675 GiB of upload buffer space.

To monitor your upload buffer

  1. View your gateway's current upload buffer.

    1. In the Gateway tab in the AWS Storage Gateway console, Choose the Details tab and find the Upload Buffer Used field.

  2. Set one or more alarms to notify you about upload buffer use.

    We highly recommend that you create one or more upload buffer alarms in the CloudWatch console. For example, you can set an alarm for a level of use you want to be warned about and an alarm for a level of use that, if exceeded, is cause for action. The action might be adding more upload buffer space. For more information, see To set an upper threshold alarm for a gateway's upload buffer.

Clean Up Resources You Don't Need

If you created the gateway as example exercise or a test, consider cleaning up to avoid incurring unexpected or unnecessary charges.

If you plan to continue using your gateway, see additional information in Where Do I Go from Here?

To clean up resources you don't need

  1. Delete any snapshots. For instructions, see Deleting a Snapshot.

  2. Unless you plan to continue using the gateway, delete it. For more information, see Deleting Your Gateway by Using the AWS Storage Gateway Console and Removing Associated Resources.

  3. Delete the AWS Storage Gateway VM from your on-premises host. If you created your gateway on an Amazon EC2 instance, terminate the instance.