Amazon S3 File Gateway documentation has been moved to What is Amazon S3 File Gateway?
Amazon FSx File Gateway documentation has been moved to What is Amazon FSx File Gateway?
Tape Gateway documentation has been moved to What is Tape Gateway?
Monitoring Storage Gateway
This section describes how to monitor a gateway, including monitoring resources associated with the gateway, using Amazon CloudWatch. You can monitor the gateway's upload buffer and cache storage. You use the Storage Gateway console to view metrics and alarms for your gateway. For example, you can view the number of bytes used in read and write operations, the time spent in read and write operations, and the time taken to retrieve data from the Amazon Web Services Cloud. With metrics, you can track the health of your gateway and set up alarms to notify you when one or more metrics fall outside a defined threshold.
Storage Gateway provides CloudWatch metrics at no additional charge. Storage Gateway metrics are
recorded for a period of two weeks. By using these metrics, you can access historical
information and get a better perspective on how your gateway and volumes are performing.
Storage Gateway also provides CloudWatch alarms, except high-resolution alarms, at no additional
charge. For more information about CloudWatch pricing, see Amazon CloudWatch pricing
Topics
Understanding gateway metrics
For the discussion in this topic, we define gateway metrics as
metrics that are scoped to the gateway—that is, they measure something about the
gateway. Because a gateway contains one or more volumes, a gateway-specific metric is
representative of all volumes on the gateway. For example, the
CloudBytesUploaded
metric is the total number of bytes that the gateway
sent to the cloud during the reporting period. This metric includes the activity of all
the volumes on the gateway.
When working with gateway metric data, you specify the unique identification of the
gateway that you are interested in viewing metrics for. To do this, you specify both the
GatewayId
and the GatewayName
values. When you want to work with metric for a gateway, you specify the gateway
dimension in the metrics namespace, which distinguishes a
gateway-specific metric from a volume-specific metric. For more information, see Using Amazon CloudWatch Metrics.
Some metrics return data points only when new data has been generated during the most recent monitoring period.
Metric | Description |
---|---|
AvailabilityNotifications |
Number of availability-related health notifications generated by the gateway. Use this metric with the Unit: Number |
CacheHitPercent |
Percent of application reads served from the cache. The sample is taken at the end of the reporting period. Unit: Percent |
CacheUsed |
The total number of bytes being used in the gateway's cache storage. The sample is taken at the end of the reporting period. Unit: Bytes |
IoWaitPercent |
Percent of time that the gateway is waiting on a response from the local disk. Unit: Percent |
MemTotalBytes |
Amount of RAM provisioned to the gateway VM, in bytes. Unit: Bytes |
MemUsedBytes |
Amount of RAM currently in use by the gateway VM, in bytes. Unit: Bytes |
QueuedWrites |
The number of bytes waiting to be written to AWS, sampled at the end of the reporting period for all volumes in the gateway. These bytes are kept in your gateway's working storage. Unit: Bytes |
ReadBytes |
The total number of bytes read from your on-premises applications in the reporting period for all volumes in the gateway. Use this metric with the Unit: Bytes |
ReadTime |
The total number of milliseconds spent to do read operations from your on-premises applications in the reporting period for all volumes in the gateway. Use this metric with the Unit: Milliseconds |
TimeSinceLastRecoveryPoint |
The time since the last available recovery point. For more information, see Your Cached Gateway is Unreachable And You Want to Recover Your Data. Unit: Seconds |
TotalCacheSize |
The total size of the cache in bytes. The sample is taken at the end of the reporting period. Unit: Bytes |
UploadBufferPercentUsed |
Percent use of the gateway's upload buffer. The sample is taken at the end of the reporting period. Unit: Percent |
UploadBufferUsed |
The total number of bytes being used in the gateway's upload buffer. The sample is taken at the end of the reporting period. Unit: Bytes |
UserCpuPercent |
Percent of CPU time spent on gateway processing, averaged across all cores. Unit: Percent |
WorkingStorageFree |
The total amount of unused space in the gateway's working storage. The sample is taken at the end of the reporting period. Unit: Bytes |
WorkingStoragePercentUsed |
Percent use of the gateway's upload buffer. The sample is taken at the end of the reporting period. Unit: Percent |
WorkingStorageUsed |
The total number of bytes being used in the gateway's upload buffer. The sample is taken at the end of the reporting period. Unit: Bytes |
WriteBytes |
The total number of bytes written to your on-premises applications in the reporting period for all volumes in the gateway. Use this metric with the Unit: Bytes |
WriteTime |
The total number of milliseconds spent to do write operations from your on-premises applications in the reporting period for all volumes in the gateway. Use this metric with the Unit: Milliseconds |
Dimensions for Storage Gateway metrics
The CloudWatch namespace for the Storage Gateway service is AWS/StorageGateway
.
Data is available automatically in 5-minute periods at no charge.
Dimension |
Description |
---|---|
GatewayId , GatewayName |
These dimensions filter the data that you request to
gateway-specific metrics. You can identify a gateway to work by the
value for Throughput and latency data of a gateway is based on all the volumes for the gateway. For information about working with gateway metrics, see Measuring Performance Between Your Gateway and AWS. |
VolumeId
|
This dimension filters the data you request to volume-specific
metrics. Identify a storage volume to work with by its
|
Monitoring the upload buffer
You can find information following about how to monitor a gateway's upload buffer and how to create an alarm so that you get a notification when the buffer exceeds a specified threshold. By using this approach, you can add buffer storage to a gateway before it fills completely and your storage application stops backing up to AWS.
You monitor the upload buffer in the same way in both the cached-volume and Tape Gateway architectures. For more information, see How Volume Gateway works (architecture).
The WorkingStoragePercentUsed
, WorkingStorageUsed
, and
WorkingStorageFree
metrics represent the upload buffer for stored
volumes only before the release of the cached-volume feature in Storage Gateway. Now,
use the equivalent upload buffer metrics UploadBufferPercentUsed
,
UploadBufferUsed
, and UploadBufferFree
. These metrics
apply to both gateway architectures.
Item of Interest | How to Measure |
---|---|
Upload buffer usage |
Use the |
To measure the percent of the upload buffer that is used
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
Choose the StorageGateway: Gateway Metrics dimension, and find the gateway that you want to work with.
-
Choose the
UploadBufferPercentUsed
metric. -
For Time Range, choose a value.
-
Choose the
Average
statistic. -
For Period, choose a value of 5 minutes to match the default reporting time.
The resulting time-ordered set of data points contains the percent used of the upload buffer.
Using the following procedure, you can create an alarm using the CloudWatch console. To learn more about alarms and thresholds, see Creating CloudWatch Alarms in the Amazon CloudWatch User Guide.
To set an upper threshold alarm for a gateway's upload buffer
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
Choose Create Alarm to start the Create Alarm wizard.
-
Specify a metric for your alarm:
-
On the Select Metric page of the Create Alarm wizard, choose the AWS/StorageGateway:GatewayId,GatewayName dimension, and then find the gateway that you want to work with.
-
Choose the
UploadBufferPercentUsed
metric. Use theAverage
statistic and a period of 5 minutes. -
Choose Continue.
-
-
Define the alarm name, description, and threshold:
-
On the Define Alarm page of the Create Alarm wizard, identify your alarm by giving it a name and description in the Name and Description boxes.
-
Define the alarm threshold.
-
Choose Continue.
-
-
Configure an email action for the alarm:
-
On the Configure Actions page of the Create Alarm wizard, choose Alarm for Alarm State.
-
Choose Choose or create email topic for Topic.
To create an email topic means that you set up an Amazon SNS topic. For more information about Amazon SNS, see Set Up Amazon SNS in the Amazon CloudWatch User Guide.
-
For Topic, enter a descriptive name for the topic.
-
Choose Add Action.
-
Choose Continue.
-
-
Review the alarm settings, and then create the alarm:
-
On the Review page of the Create Alarm wizard, review the alarm definition, metric, and associated actions to take (for example, sending an email notification).
-
After reviewing the alarm summary, choose Save Alarm.
-
-
Confirm your subscription to the alarm topic:
-
Open the Amazon SNS email that was sent to the email address that you specified when creating the topic.
The following image shows a typical email notification.
-
Confirm your subscription by clicking the link in the email.
A subscription confirmation appears.
-
Monitoring cache storage
You can find information following about how to monitor a gateway's cache storage and how to create an alarm so that you get a notification when parameters of the cache pass specified thresholds. Using this alarm, you know when to add cache storage to a gateway.
You only monitor cache storage in the cached volumes architecture. For more information, see How Volume Gateway works (architecture).
Item of Interest | How to Measure |
---|---|
Total usage of cache |
Use the The |
Percent of read requests that are served from the cache |
Use the Typically, you want |
Percent of the cache that is dirty—that is, it contains content that has not been uploaded to AWS |
Use the Typically, you want |
To measure the percent of a cache that is dirty for a gateway and all its volumes
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
Choose the StorageGateway: Gateway Metrics dimension, and find the gateway that you want to work with.
-
Choose the
CachePercentDirty
metric. -
For Time Range, choose a value.
-
Choose the
Average
statistic. -
For Period, choose a value of 5 minutes to match the default reporting time.
The resulting time-ordered set of data points contains the percentage of the cache that is dirty over the 5 minutes.
To measure the percent of the cache that is dirty for a volume
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
Choose the StorageGateway: Volume Metrics dimension, and find the volume that you want to work with.
-
Choose the
CachePercentDirty
metric. -
For Time Range, choose a value.
-
Choose the
Average
statistic. -
For Period, choose a value of 5 minutes to match the default reporting time.
The resulting time-ordered set of data points contains the percentage of the cache that is dirty over the 5 minutes.
Understanding CloudWatch alarms
CloudWatch alarms monitor information about your gateway based on metrics and expressions. You can add CloudWatch alarms for your gateway and view their statuses in the Storage Gateway console. For more information about the metrics that are used to monitor Volume Gateway, see Understanding gateway metrics and Understanding Volume Metrics. For each alarm, you specify conditions that will initiate its ALARM state. Alarm status indicators in the Storage Gateway console turn red when in the ALARM state, making it easier for you to monitor status proactively. You can configure alarms to invoke actions automatically based on sustained changes in state. For more information about CloudWatch alarms, see Using Amazon CloudWatch alarms in the Amazon CloudWatch User Guide.
If you don't have permission to view CloudWatch, you can't view the alarms.
For each activated gateway, we recommend that you create the following CloudWatch alarms:
-
High IO wait:
IoWaitpercent
>= 20 for 3 datapoints in 15 minutes -
Cache percent dirty:
CachePercentDirty
> 80 for 4 datapoints within 20 minutes
For gateways on VMware host platforms with HA mode activated, we also recommend these additional CloudWatch alarms:
-
Availability notifications:
AvailabilityNotifications
>= 1 for 1 datapoint within 5 minutes -
Health notifications:
HealthNotifications
>= 1 for 1 datapoint within 5 minutesNote You can set a health notification alarm only if the gateway had a previous health notification in CloudWatch.
The following table describes the state of an alarm.
State | Description |
---|---|
OK |
The metric or expression is within the defined threshold. |
Alarm |
The metric or expression is outside of the defined threshold. |
Insufficient data |
The alarm has just started, the metric is not available, or not enough data is available for the metric to determine the alarm state. |
None |
No alarms are created for the gateway. To create a new alarm, see Creating a custom CloudWatch alarm for your gateway. |
Unavailable |
The state of the alarm is unknown. Choose Unavailable to view error information in the Monitoring tab. |
Creating recommended CloudWatch alarms for your gateway
When you create a new gateway using the Storage Gateway console, you can choose to create all recommended CloudWatch alarms automatically as part of the initial setup process. For more information, see Configure your Volume Gateway. If you want to add or update recommended CloudWatch alarms for an existing gateway, use the following procedure.
To add or update recommended CloudWatch alarms for an existing gateway
This feature requires CloudWatch policy permissions, which are not automatically granted as part of the preconfigured Storage Gateway full access policy. Make sure your security policy grants the following permissions before you attempt to create recommended CloudWatch alarms:
-
cloudwatch:PutMetricAlarm
- create alarms -
cloudwatch:DisableAlarmActions
- turn alarm actions off -
cloudwatch:EnableAlarmActions
- turn alarm actions on -
cloudwatch:DeleteAlarms
- delete alarms
-
Open the Storage Gateway console at https://console.aws.amazon.com/storagegateway/home/
. -
In the navigation pane, choose Gateways, then choose the gateway for which you want to create recommended CloudWatch alarms.
-
On the gateway details page, choose the Monitoring tab.
-
Under Alarms, choose Create recommended alarms. The recommended alarms are created automatically.
The Alarms section lists all CloudWatch alarms for a specific gateway. From here, you can select and delete one or more alarms, turn alarm actions on or off, and create new alarms.
Creating a custom CloudWatch alarm for your gateway
CloudWatch uses Amazon Simple Notification Service (Amazon SNS) to send alarm notifications when an alarm changes state. An alarm watches a single metric over a time period that you specify, and performs one or more actions based on the value of the metric relative to a given threshold over a number of time periods. The action is a notification that's sent to an Amazon SNS topic. You can create an Amazon SNS topic when you create a CloudWatch alarm. For more information about Amazon SNS, see What is Amazon SNS? in the Amazon Simple Notification Service Developer Guide.
To create a CloudWatch alarm in the Storage Gateway console
-
Open the Storage Gateway console at https://console.aws.amazon.com/storagegateway/home/
. -
In the navigation pane, choose Gateways, then choose the gateway for which you want to create an alarm.
-
On the gateway details page, choose the Monitoring tab.
-
Under Alarms, choose Create alarm to open the CloudWatch console.
-
Use the CloudWatch console to create the type of alarm that you want. You can create the following types of alarms:
-
Static threshold alarm: An alarm based on a set threshold for a chosen metric. The alarm enters the ALARM state when the metric breaches the threshold for a specified number of evaluation periods.
To create a static threshold alarm, see Creating a CloudWatch alarm based on a static threshold in the Amazon CloudWatch User Guide.
-
Anomaly detection alarm: Anomaly detection mines past metric data and creates a model of expected values. You set a value for the anomaly detection threshold, and CloudWatch uses this threshold with the model to determine the "normal" range of values for the metric. A higher value for the threshold produces a thicker band of "normal" values. You can choose to activate the alarm only when the metric value is above the band of expected values, only when it's below the band, or when it's above or below the band.
To create an anomaly detection alarm, see Creating a CloudWatch alarm based on anomaly detection in the Amazon CloudWatch User Guide.
-
Metric math expression alarm: An alarm based one or more metrics used in a math expression. You specify the expression, threshold, and evaluation periods.
To create a metric math expression alarm, see Creating a CloudWatch alarm based on a metric math expression in the Amazon CloudWatch User Guide.
-
Composite alarm: An alarm that determines its alarm state by watching the alarm states of other alarms. A composite alarm can help you reduce alarm noise.
To create a composite alarm, see Creating a composite alarm in the Amazon CloudWatch User Guide.
-
-
After you create the alarm in the CloudWatch console, return to the Storage Gateway console. You can view the alarm by doing one of the following:
-
In the navigation pane, choose Gateways, then choose the gateway for which you want to view alarms. On the Details tab, under Alarms, choose CloudWatch Alarms.
-
In the navigation pane, choose Gateways, choose the gateway for which you want to view alarms, then choose the Monitoring tab.
The Alarms section lists all of the CloudWatch alarms for a specific gateway. From here, you can select and delete one or more alarms, turn alarm actions on or off, and create new alarms.
-
In the navigation pane, choose Gateways, then choose the alarm state of the gateway for which you want to view alarms.
-
For information about how to edit or delete an alarm, see Editing or deleting a CloudWatch alarm.
When you delete a gateway using the Storage Gateway console, all CloudWatch alarms associated with the gateway are also automatically deleted.