Menu
Amazon Kinesis Firehose
Developer Guide

Monitoring with Amazon CloudWatch Metrics

Kinesis Firehose integrates with CloudWatch metrics so that you can collect, view, and analyze CloudWatch metrics for your Kinesis Firehose delivery streams. For example, you can monitor the IncomingBytes and IncomingRecords metrics to keep track of data ingested into Kinesis Firehose from data producers.

The metrics that you configure for your Kinesis Firehose delivery streams and agents are automatically collected and pushed to CloudWatch every five minutes. Metrics are archived for two weeks; after that period, the data is discarded.

The metrics collected for Kinesis Firehose delivery streams are free of charge. For information about Kinesis agent metrics, see Monitoring Kinesis Agent Health.

Service-Level CloudWatch Metrics

Kinesis Firehose sends the following service-level metrics to CloudWatch.

Metric Description
DeliveryToElasticsearch.Bytes

The number of bytes indexed to Amazon ES over the specified time period.

Units: Bytes

DeliveryToElasticsearch.Records

The number of records indexed to Amazon ES over the specified time period.

Units: Count

DeliveryToElasticsearch.Success

Sum of the successfully indexed records over the sum of records that were attempted.

DeliveryToRedshift.Bytes

The number of bytes copied to Amazon Redshift over the specified time period.

Units: Bytes

DeliveryToRedshift.Records

The number of records copied to Amazon Redshift over the specified time period.

Units: Count

DeliveryToRedshift.Success

Sum of successful Amazon Redshift COPY commands over the sum of all Amazon Redshift COPY commands.

DeliveryToS3.Bytes

The number of bytes delivered to Amazon S3 over the specified time period.

Units: Bytes

DeliveryToS3.DataFreshness

Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to the S3 bucket.

Units: Seconds

DeliveryToS3.Records

The number of records delivered to Amazon S3 over the specified time period.

Units: Count

DeliveryToS3.Success

Sum of successful Amazon S3 put commands over sum of all Amazon S3 put commands.

DeliveryToSplunk.Bytes

The number of bytes delivered to Splunk over the specified time period.

Units: Bytes

DeliveryToSplunk.DataFreshness

Age (from getting into Kinesis Firehose to now) of the oldest record in Kinesis Firehose. Any record older than this age has been delivered to Splunk.

Units: Seconds

DeliveryToSplunk.Records

The number of records delivered to Splunk over the specified time period.

Units: Count

DeliveryToSplunk.Success

The sum of the successfully indexed records over the sum of records that were attempted.

IncomingBytes

The number of bytes ingested into the Kinesis Firehose Kinesis Firehose delivery stream over the specified time period.

Units: Bytes

IncomingRecords

The number of records ingested into the Kinesis Firehose Kinesis Firehose delivery stream over the specified time period.

Units: Count

API-Level CloudWatch Metrics

Kinesis Firehose sends the following API-level metrics to CloudWatch.

Metric Description
DescribeDeliveryStream.Latency

The time taken per DescribeDeliveryStream operation, measured over the specified time period.

Units: Milliseconds

DescribeDeliveryStream.Requests

Total number of DescribeDeliveryStream requests.

Units: Count

ListDeliveryStreams.Latency

The time taken per ListDeliveryStream operation, measured over the specified time period.

Units: Milliseconds

ListDeliveryStreams.Requests

Total number of ListFirehose requests.

Units: Count

PutRecord.Bytes

The number of bytes put to the Kinesis Firehose Kinesis Firehose delivery stream using PutRecord over the specified time period.

Units: Bytes

PutRecord.Latency

The time taken per PutRecord operation, measured over the specified time period.

Units: Milliseconds

PutRecord.Requests

Total number of PutRecord requests, which is equal to total number of records from PutRecord operations.

Units: Count

PutRecordBatch.Bytes

The number of bytes put to the Kinesis Firehose Kinesis Firehose delivery stream using PutRecordBatch over the specified time period.

Units: Bytes

PutRecordBatch.Latency

The time taken per PutRecordBatch operation, measured over the specified time period.

Units: Milliseconds

PutRecordBatch.Records

Total number of records from PutRecordBatch operations.

Units: Count

PutRecordBatch.Requests

Total number of PutRecordBatch requests.

Units: Count

UpdateDeliveryStream.Latency

The time taken per UpdateDeliveryStream operation, measured over the specified time period.

Units: Milliseconds

UpdateDeliveryStream.Requests

Total number of UpdateDeliveryStream requests.

Units: Count

Dimensions for Kinesis Firehose

To filter metrics by delivery stream, use the DeliveryStreamName dimension.

Accessing CloudWatch Metrics for Kinesis Firehose

You can monitor metrics for Kinesis Firehose using the CloudWatch console, command line, or CloudWatch API. The following procedures show you how to access metrics using these different methods.

To access metrics using the CloudWatch console

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. On the navigation bar, choose a region.

  3. In the navigation pane, choose Metrics.

  4. Choose the Firehose namespace.

  5. Choose Delivery Stream Metrics or Firehose Metrics.

  6. Select a metric to add to the graph.

To access metrics using the AWS CLI

Use the list-metrics and get-metric-statistics commands.

Copy
aws cloudwatch list-metrics --namespace "AWS/Firehose"
Copy
aws cloudwatch get-metric-statistics --namespace "AWS/Firehose" \ --metric-name DescribeDeliveryStream.Latency --statistics Average --period 3600 \ --start-time 2017-06-01T00:00:00Z --endtime 2017-06-30T00:00:00Z