Menu
AWS Storage Gateway
User Guide (API Version 2013-06-30)

Monitoring Your Gateway

In this section, you can find information about how to monitor a gateway, including monitoring the volumes or tapes associated with the gateway and monitoring the upload buffer. You use the AWS Management Console to view metrics for your gateway. For example, you can view the number of bytes used in read and write operations, the time spent in read and write operations, and the time taken to retrieve data from the AWS cloud. With metrics, you can track the health of your gateway and set up alarms to notify you when one or more metrics fall outside a defined threshold.

AWS Storage Gateway provides Amazon CloudWatch metrics at no additional charge. Storage Gateway metrics are recorded for a period of two weeks. By using these metrics, you can access historical information and get a better perspective on how your gateway and volumes are performing. For detailed information about CloudWatch, see the Amazon CloudWatch User Guide.

Monitoring Your Volume Gateway

In this section, you can find information about how to monitor a gateway in a cached volumes or stored volumes setup, including monitoring the volumes associated with the gateway and monitoring the upload buffer. You use the AWS Management Console to view metrics for your gateway. For example, you can view the number of bytes used in read and write operations, the time spent in read and write operations, and the time taken to retrieve data from the AWS cloud. With metrics, you can track the health of your gateway and set up alarms to notify you when one or more metrics fall outside a defined threshold.

Storage Gateway provides CloudWatch metrics at no additional charge. Storage Gateway metrics are recorded for a period of two weeks. By using these metrics, you can access historical information and get a better perspective on how your gateway and volumes are performing. For detailed information about CloudWatch, see the Amazon CloudWatch User Guide.

Using Amazon CloudWatch Metrics

You can get monitoring data for your gateway using either the AWS Management Console or the CloudWatch API. The console displays a series of graphs based on the raw data from the CloudWatch API. You can also use the CloudWatch API through one of the Amazon AWS Software Development Kits (SDKs) or the Amazon CloudWatch API tools. Depending on your needs, you might prefer to use either the graphs displayed in the console or retrieved from the API.

Regardless of which method you choose to use to work with metrics, you must specify the following information:

  • The metric dimension to work with. A dimension is a name-value pair that helps you to uniquely identify a metric. The dimensions for Storage Gateway are GatewayId, GatewayName, and VolumeId. In the CloudWatch console, you can use the Gateway Metrics and Volume Metrics views to easily select gateway-specific and volume-specific dimensions. For more information about dimensions, see Dimensions in the Amazon CloudWatch User Guide>.

  • The metric name, such as ReadBytes.

The following table summarizes the types of Storage Gateway metric data that you can use.

CloudWatch Namespace Dimension Description
AWS/StorageGateway GatewayId, GatewayName

These dimensions filter for metric data that describes aspects of the gateway. You can identify a gateway to work with by specifying both the GatewayId and the GatewayName dimensions.

Throughput and latency data of a gateway are based on all the volumes in the gateway.

Data is available automatically in 5-minute periods at no charge.

VolumeId

This dimension filters for metric data that is specific to a volume. Identify a volume to work with by its VolumeId dimension.

Data is available automatically in 5-minute periods at no charge.

Working with gateway and volume metrics is similar to working with other service metrics. You can find a discussion of some of the most common metrics tasks in the CloudWatch documentation listed following:

Measuring Performance Between Your Application and Gateway

Data throughput, data latency, and operations per second are three measures that you can use to understand how your application storage that is using your gateway is performing. When you use the correct aggregation statistic, you can use Storage Gateway metrics to measure these values.

A statistic is an aggregation of a metric over a specified period of time. When you view the values of a metric in CloudWatch, use the Average statistic for data latency (milliseconds), use the Sum statistic for data throughput (bytes per second), and use the Samples statistic for input/output operations per second (IOPS). For more information, see Statistics in the Amazon CloudWatch User Guide.

The following table summarizes the metrics and corresponding statistic you can use to measure the throughput, latency, and IOPS between your applications and gateways.

Item of Interest How to Measure
Throughput

Use the ReadBytes and WriteBytes metrics with the Sum CloudWatch statistic. For example, the Sum value of the ReadBytes metric over a sample period of 5 minutes divided by 300 seconds gives you the throughput as a rate in bytes per second.

Latency Use the ReadTime and WriteTime metrics with the Average CloudWatch statistic. For example, the Average value of the ReadTime metric gives you the latency per operation over the sample period of time.
IOPS Use the ReadBytes and WriteBytes metrics with the Samples CloudWatch statistic. For example, the Samples value of the ReadBytes metric over a sample period of 5 minutes divided by 300 seconds gives you IOPS.

For the average latency graphs and average size graphs, the average is calculated over the total number of operations (read or write, whichever is applicable to the graph) that completed during the period.

To measure the data throughput from an application to a volume

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Metrics, then choose the All metrics tab and then choose Storage Gateway.

  3. Choose the Volume metrics dimension, and find the volume that you want to work with.

  4. Choose the ReadBytes and WriteBytes metrics.

  5. For Time Range, choose a value.

  6. Choose the Sum statistic.

  7. For Period, choose a value of 5 minutes or greater.

  8. In the resulting time-ordered sets of data points (one for ReadBytes and one for WriteBytes), divide each data point by the period (in seconds) to get the throughput at the sample point. The total throughput is the sum of the throughputs.

The following image shows the ReadBytes and WriteBytes metrics for a volume with the Sum statistic. In the image, the cursor over a data point displays information about the data point including its value and the number of bytes. Divide the bytes value by the Period value (5 minutes) to get the data throughput at that sample point. For the point highlighted, the read throughput is 2,384,199,680 bytes divided by 300 seconds, which is 7.6 megabytes per second.

To measure the data input/output operations per second from an application to a volume

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Metrics, then choose the All metrics tab and then choose Storage Gateway.

  3. Choose the Volume metrics dimension, and find the volume that you want to work with.

  4. Choose the ReadBytes and WriteBytes metrics.

  5. For Time Range, choose a value.

  6. Choose the Samples statistic.

  7. For Period, choose a value of 5 minutes or greater.

  8. In the resulting time-ordered sets of data points (one for ReadBytes and one for WriteBytes), divide each data point by the period (in seconds) to get IOPS.

The following image shows the ReadBytes and WriteBytes metrics for a storage volume with the Samples statistic. In the image, the cursor over a data point displays information about the data point, including its value and the number of samples. Divide the samples value by the Period value (5 minutes) to get the operations per second at that sample point. For the point highlighted, the number of write operations is 24,373 bytes divided by 300 seconds, which is 81 write operations per second.

Measuring Performance Between Your Gateway and AWS

Data throughput, data latency, and operations per second are three measures that you can use to understand how your application storage using the Storage Gateway is performing. These three values can be measured using the Storage Gateway metrics provided for you when you use the correct aggregation statistic. The following table summarizes the metrics and corresponding statistic to use to measure the throughput, latency, and input/output operations per second (IOPS) between your gateway and AWS.

Item of Interest How to Measure
Throughput

Use the ReadBytes and WriteBytes metrics with the Sum CloudWatch statistic. For example, the Sum value of the ReadBytes metric over a sample period of 5 minutes divided by 300 seconds gives you the throughput as a rate in bytes per second.

Latency Use the ReadTime and WriteTime metrics with the Average CloudWatch statistic. For example, the Average value of the ReadTime metric gives you the latency per operation over the sample period of time.
IOPS Use the ReadBytes and WriteBytes metrics with the Samples CloudWatch statistic. For example, the Samples value of the ReadBytes metric over a sample period of 5 minutes divided by 300 seconds gives you IOPS.
Throughput to AWS Use the CloudBytesDownloaded and CloudBytesUploaded metrics with the Sum CloudWatch statistic. For example, the Sum value of the CloudBytesDownloaded metric over a sample period of 5 minutes divided by 300 seconds gives you the throughput from AWS to the gateway as bytes per second.
Latency of data to AWS Use the CloudDownloadLatency metric with the Average statistic. For example, the Average statistic of the CloudDownloadLatency metric gives you the latency per operation.

To measure the upload data throughput from a gateway to AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Metrics, then choose the All metrics tab and then choose Storage Gateway.

  3. Choose the Gateway metrics dimension, and find the volume that you want to work with.

  4. Choose the CloudBytesUploaded metric.

  5. For Time Range, choose a value.

  6. Choose the Sum statistic.

  7. For Period, choose a value of 5 minutes or greater.

  8. In the resulting time-ordered set of data points, divide each data point by the period (in seconds) to get the throughput at that sample period.

The following image shows the CloudBytesUploaded metric for a gateway volume with the Sum statistic. In the image, the cursor over a data point displays information about the data point, including its value and bytes uploaded. Divide this value by the Period value (5 minutes) to get the throughput at that sample point. For the point highlighted, the throughput from the gateway to AWS is 555,544,576 bytes divided by 300 seconds, which is 1.7 megabytes per second.

To measure the latency per operation of a gateway

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Metrics, then choose the All metrics tab and then choose Storage Gateway.

  3. Choose the Gateway metrics dimension, and find the volume that you want to work with.

  4. Choose the ReadTime and WriteTime metrics.

  5. For Time Range, choose a value.

  6. Choose the Average statistic.

  7. For Period, choose a value of 5 minutes to match the default reporting time.

  8. In the resulting time-ordered set of points (one for ReadTime and one for WriteTime), add data points at the same time sample to get to the total latency in milliseconds.

To measure the data latency from a gateway to AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Metrics, then choose the All metrics tab and then choose Storage Gateway.

  3. Choose the Gateway metrics dimension, and find the volume that you want to work with.

  4. Choose the CloudDownloadLatency metric.

  5. For Time Range, choose a value.

  6. Choose the Average statistic.

  7. For Period, choose a value of 5 minutes to match the default reporting time.

The resulting time-ordered set of data points contains the latency in milliseconds.

To set an upper threshold alarm for a gateway's throughput to AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Alarms.

  3. Choose Create Alarm to start the Create Alarm Wizard.

  4. Choose the Storage Gateway dimension, and find the gateway that you want to work with.

  5. Choose the CloudBytesUploaded metric.

  6. To define the alarm, define the alarm state when the CloudBytesUploaded metric is greater than or equal to a specified value for a specified time. For example, you can define an alarm state when the CloudBytesUploaded metric is greater than 10 MB for 60 minutes.

  7. Configure the actions to take for the alarm state. For example, you can have an email notification sent to you.

  8. Choose Create Alarm.

To set an upper threshold alarm for reading data from AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Create Alarm to start the Create Alarm Wizard.

  3. Choose the StorageGateway: Gateway Metrics dimension, and find the gateway that you want to work with.

  4. Choose the CloudDownloadLatency metric.

  5. Define the alarm by defining the alarm state when the CloudDownloadLatency metric is greater than or equal to a specified value for a specified time. For example, you can define an alarm state when the CloudDownloadLatency is greater than 60,000 milliseconds for greater than 2 hours.

  6. Configure the actions to take for the alarm state. For example, you can have an email notification sent to you.

  7. Choose Create Alarm.

Monitoring Your Tape Gateway

In this section, you can find information about how to monitor your tape gateway, virtual tapes associated with your tape gateway, cache storage, and the upload buffer. You use the AWS Management Console to view metrics for your tape gateway. With metrics, you can track the health of your tape gateway and set up alarms to notify you when one or more metrics are outside a defined threshold.

Storage Gateway provides CloudWatch metrics at no additional charge. Storage Gateway metrics are recorded for a period of two weeks. By using these metrics, you can access historical information and get a better perspective of how your tape gateway and virtual tapes are performing. For detailed information about CloudWatch, see the Amazon CloudWatch User Guide.

Using Amazon CloudWatch Metrics

You can get monitoring data for your tape gateway by using either the AWS Management Console or the CloudWatch API. The console displays a series of graphs based on the raw data from the CloudWatch API. The CloudWatch API can also be used through one of the Amazon AWS Software Development Kits (SDKs) or the Amazon CloudWatch API tools. Depending on your needs, you might prefer to use either the graphs displayed in the console or retrieved from the API.

Regardless of which method you choose to use to work with metrics, you must specify the following information:

  • The metric dimension to work with. A dimension is a name-value pair that helps you to uniquely identify a metric. The dimensions for Storage Gateway are GatewayId and GatewayName. In the CloudWatch console, you can use the Gateway Metrics view to easily select gateway-specific and tape-specific dimensions. For more information about dimensions, see Dimensions in the Amazon CloudWatch User Guide.

  • The metric name, such as ReadBytes.

The following table summarizes the types of Storage Gateway metric data that are available to you.

Amazon CloudWatch Namespace Dimension Description
AWS/StorageGateway GatewayId, GatewayName

These dimensions filter for metric data that describes aspects of the tape gateway. You can identify a tape gateway to work with by specifying both the GatewayId and the GatewayName dimensions.

Throughput and latency data of a tape gateway is based on all the virtual tapes in the tape gateway.

Data is available automatically in 5-minute periods at no charge.

Working with gateway and tape metrics is similar to working with other service metrics. You can find a discussion of some of the most common metrics tasks in the CloudWatch documentation listed following:

Measuring Performance Between Your Tape Gateway and AWS

Data throughput, data latency, and operations per second are measures that you can use to understand how your application storage that is using your tape gateway is performing. When you use the correct aggregation statistic, these values can be measured by using the Storage Gateway metrics that are provided for you.

A statistic is an aggregation of a metric over a specified period of time. When you view the values of a metric in CloudWatch, use the Average statistic for data latency (milliseconds), and use the Samples statistic for input/output operations per second (IOPS). For more information, see Statistics in the Amazon CloudWatch User Guide

The following table summarizes the metrics and the corresponding statistic you can use to measure the throughput, latency, and IOPS between your tape gateway and AWS.

Item of Interest How to Measure
Latency Use the ReadTime and WriteTime metrics with the Average CloudWatch statistic. For example, the Average value of the ReadTime metric gives you the latency per operation over the sample period of time.
Throughput to AWS Use the CloudBytesDownloaded and CloudBytesUploaded metrics with the Sum CloudWatch statistic. For example, the Sum value of the CloudBytesDownloaded metric over a sample period of 5 minutes divided by 300 seconds gives you the throughput from AWS to the tape gateway as a rate in bytes per second.
Latency of data to AWS Use the CloudDownloadLatency metric with the Average statistic. For example, the Average statistic of the CloudDownloadLatency metric gives you the latency per operation.

To measure the upload data throughput from a tape gateway to AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose the Metrics tab.

  3. Choose the StorageGateway: Gateway Metrics dimension, and find the tape gateway that you want to work with.

  4. Choose the CloudBytesUploaded metric.

  5. For Time Range, choose a value.

  6. Choose the Sum statistic.

  7. For Period, choose a value of 5 minutes or greater.

  8. In the resulting time-ordered set of data points, divide each data point by the period (in seconds) to get the throughput at that sample period.

The following image shows the CloudBytesUploaded metric for a gateway tape with the Sum statistic. In the image, placing the cursor over a data point displays information about the data point, including its value and the number of bytes uploaded. Divide this value by the Period value (5 minutes) to get the throughput at that sample point. For the point highlighted, the throughput from the tape gateway to AWS is 555,544,576 bytes divided by 300 seconds, which is 1.7 megabytes per second.

To measure the data latency from a tape gateway to AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose the Metrics tab.

  3. Choose the StorageGateway: GatewayMetrics dimension, and find the tape gateway that you want to work with.

  4. Choose the CloudDownloadLatency metric.

  5. For Time Range, choose a value.

  6. Choose the Average statistic.

  7. For Period, choose a value of 5 minutes to match the default reporting time.

The resulting time-ordered set of data points contains the latency in milliseconds.

To set an upper threshold alarm for a tape gateway's throughput to AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Create Alarm to start the Create Alarm Wizard.

  3. Choose the StorageGateway: Gateway Metrics dimension, and find the tape gateway that you want to work with.

  4. Choose the CloudBytesUploaded metric.

  5. Define the alarm by defining the alarm state when the CloudBytesUploaded metric is greater than or equal to a specified value for a specified time. For example, you can define an alarm state when the CloudBytesUploaded metric is greater than 10 megabytes for 60 minutes.

  6. Configure the actions to take for the alarm state. For example, you can have an email notification sent to you.

  7. Choose Create Alarm.

To set an upper threshold alarm for reading data from AWS

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Create Alarm to start the Create Alarm Wizard.

  3. Choose the StorageGateway: Gateway Metrics dimension, and find the tape gateway that you want to work with.

  4. Choose the CloudDownloadLatency metric.

  5. Define the alarm by defining the alarm state when the CloudDownloadLatency metric is greater than or equal to a specified value for a specified time. For example, you can define an alarm state when the CloudDownloadLatency is greater than 60,000 milliseconds for greater than 2 hours.

  6. Configure the actions to take for the alarm state. For example, you can have an email notification sent to you.

  7. Choose Create Alarm.

Monitoring the Upload Buffer

You can find information following about how to monitor a gateway's upload buffer and how to create an alarm so that you get a notification when the buffer exceeds a specified threshold. By using this approach, you can proactively add buffer storage to a gateway before it fills completely and your storage application stops backing up to AWS.

You monitor the upload buffer in the same way in both the cached volumes and stored volumes architectures. For more information, see How AWS Storage Gateway Works (Architecture).

Note

The WorkingStoragePercentUsed, WorkingStorageUsed, and WorkingStorageFree metrics represent the upload buffer for the stored volumes setup only before the release of the cached-volume feature in Storage Gateway. Now you should use the equivalent upload buffer metrics UploadBufferPercentUsed, UploadBufferUsed, and UploadBufferFree. These metrics apply to both gateway architectures.

Item of Interest How to Measure
Upload buffer usage

Use the UploadBufferPercentUsed, UploadBufferUsed, and UploadBufferFree metrics with the Average statistic. For example, use the UploadBufferUsed with the Average statistic to analyze the storage usage over a time period.

To measure upload buffer percent used

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose the StorageGateway: Gateway Metrics dimension, and find the gateway that you want to work with.

  3. Choose the UploadBufferPercentUsed metric.

  4. For Time Range, choose a value.

  5. Choose the Average statistic.

  6. For Period, choose a value of 5 minutes to match the default reporting time.

The resulting time-ordered set of data points contains the percent used of the upload buffer.

Using the following procedure, you can create an alarm using the CloudWatch console. To learn more about alarms and thresholds, see Creating CloudWatch Alarms.

To set an upper threshold alarm for a gateway's upload buffer

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose Create Alarm to start the Create Alarm Wizard.

  3. Specify a metric for your alarm.

    1. On the Select Metric page of the Create Alarm Wizard, choose the AWS/StorageGateway:GatewayId,GatewayName dimension, and then find the gateway that you want to work with.

    2. Choose the UploadBufferPercentUsed metric. Use the Average statistic and a period of 5 minutes.

    3. Choose Continue.

  4. Define the alarm name, description, and threshold.

    1. On the Define Alarm page of the Create Alarm Wizard, identify your alarm by giving it a name and description in the Name and Description boxes.

    2. Define the alarm threshold.

    3. Choose Continue.

  5. Configure an email action for the alarm.

    1. In the Configure Actions page of the Create Alarm Wizard, choose Alarm for Alarm State.

    2. Choose Choose or create email topic for Topic.

      To create an email topic means that you set up an Amazon Simple Notification Service (Amazon SNS) topic. For more information about Amazon SNS, see Set Up Amazon SNS.

    3. For Topic, type a descriptive name for the topic.

    4. Choose Add Action.

    5. Choose Continue.

  6. Review the alarm settings, and then create the alarm.

    1. In the Review page of the Create Alarm Wizard, review the alarm definition, metric, and associated actions from this step. Associated actions include, for example, sending an email notification.

    2. After reviewing the alarm summary, choose Save Alarm.

  7. Confirm your subscription to the alarm topic.

    1. Open the Amazon Simple Notification Service (Amazon SNS) email topic that is sent to the email address that you specified when creating the topic.

      The following image shows a notification.

    2. Confirm your subscription by clicking the link in the email.

      A subscription confirmation appears.

Monitoring Cache Storage

You can find information following about how to monitor a gateway's cache storage and how to create an alarm so that you get a notification when parameters of the cache pass specified thresholds. Using this alarm, you know when to proactively add cache storage to a gateway.

You only monitor cache storage in the cached volumes architecture. For more information, see How AWS Storage Gateway Works (Architecture).

Item of Interest How to Measure
Total usage of cache

Use the CachePercentUsed and TotalCacheSize metrics with the Average statistic. For example, use the CachePercentUsed with the Average statistic to analyze the cache usage over a period of time.

The TotalCacheSize metric changes only when you add cache to the gateway.

Percentage of read requests that are served from the cache

Use the CacheHitPercent metric with the Average statistic.

Typically, you want CacheHitPercent to remain high.

Percentage of cache that is dirty—that is, it contains content that has not been uploaded to AWS

Use the CachePercentDirty metrics with the Average statistic.

Typically, you want CachePercentDirty to remain low.

To measure the cache's percentage dirty for a gateway and all its volumes

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose the StorageGateway: Gateway Metrics dimension, and find the gateway that you want to work with.

  3. Choose the CachePercentDirty metric.

  4. For Time Range, choose a value.

  5. Choose the Average statistic.

  6. For Period, choose a value of 5 minutes to match the default reporting time.

The resulting time-ordered set of data points contains the percentage of the cache that is dirty over the 5 minutes.

To measure the cache's percentage dirty for a volume

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Choose the StorageGateway: Volume Metrics dimension, and find the volume that you want to work with.

  3. Choose the CachePercentDirty metric.

  4. For Time Range, choose a value.

  5. Choose the Average statistic.

  6. For Period, choose a value of 5 minutes to match the default reporting time.

The resulting time-ordered set of data points contains the percentage of the cache that is dirty over the 5 minutes.