Menu
Amazon Kinesis Firehose
Developer Guide

Troubleshooting Amazon Kinesis Firehose

You may not see data delivered to your specified destinations. Use the following troubleshooting steps to solve common issues you might encounter while using Kinesis Firehose.

Data Not Delivered to Amazon S3

Check the following if data is not delivered to your S3 bucket.

  • Check the Kinesis Firehose IncomingBytes and IncomingRecords metrics to make sure that data is sent to your Kinesis Firehose delivery stream successfully. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • If data transformation with Lambda is enabled, check the Kinesis Firehose ExecuteProcessingSuccess metric to make sure that Kinesis Firehose has attempted to invoke your Lambda function. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • Check the Kinesis Firehose DeliveryToS3.Success metric to make sure that Kinesis Firehose has attempted putting data to your S3 bucket. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • Enable error logging if it is not already enabled, and check error logs for delivery failure. For more information, see Monitoring with Amazon CloudWatch Logs.

  • Make sure that the S3 bucket specified in your Kinesis Firehose delivery stream still exists.

  • If data transformation with Lambda is enabled, make sure that the Lambda function specified in your delivery stream still exists.

  • Make sure that the IAM role specified in your Kinesis Firehose delivery stream has access to your S3 bucket and your Lambda function (if data transformation is enabled). For more information, see Grant Kinesis Firehose Access to an Amazon S3 Destination.

Data Not Delivered to Amazon Redshift

Check the following if data is not delivered to your Amazon Redshift cluster.

Note that data is delivered to your S3 bucket before loading into Amazon Redshift If the data was not delivered to your S3 bucket, see Data Not Delivered to Amazon S3.

  • Check the Kinesis Firehose DeliveryToRedshift.Success metric to make sure that Kinesis Firehose has attempted to copy data from your S3 bucket to the Amazon Redshift cluster. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • Enable error logging if it is not already enabled, and check error logs for delivery failure. For more information, see Monitoring with Amazon CloudWatch Logs.

  • Check the Amazon Redshift STL_CONNECTION_LOG table to see if Kinesis Firehose is able to make successful connections. In this table you should be able to see connections and their status based on a user name. For more information, see STL_CONNECTION_LOG in the Amazon Redshift Database Developer Guide.

  • If the previous check shows that connections are being established, check the Amazon Redshift STL_LOAD_ERRORS table to verify the reason of the COPY failure. For more information, see STL_LOAD_ERRORS in the Amazon Redshift Database Developer Guide.

  • Make sure that the Amazon Redshift configuration in your Kinesis Firehose delivery stream is accurate and valid.

  • Make sure that the IAM role specified in your Kinesis Firehose delivery stream has access to the S3 bucket from which Amazon Redshift copies data and the Lambda function for data transformation (if data transformation is enabled). For more information, see Grant Kinesis Firehose Access to an Amazon S3 Destination.

  • If your Amazon Redshift cluster is in a VPC, make sure that the cluster allows access from Kinesis Firehose IP addresses. For more information, see Grant Kinesis Firehose Access to an Amazon Redshift Destination .

  • Make sure that the Amazon Redshift cluster is publicly accessible.

Data Not Delivered to Amazon Elasticsearch Service

Check the following if data is not delivered to your Elasticsearch domain.

Note that data can be backed up to your S3 bucket concurrently. If data was not delivered to your S3 bucket, see Data Not Delivered to Amazon S3.

  • Check the Kinesis Firehose IncomingBytes and IncomingRecords metrics to make sure that data is sent to your Kinesis Firehose delivery stream successfully. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • If data transformation with Lambda is enabled, check the Kinesis Firehose ExecuteProcessingSuccess metric to make sure that Kinesis Firehose has attempted to invoke your Lambda function. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • Check the Kinesis Firehose DeliveryToElasticsearch.Success metric to make sure that Kinesis Firehose has attempted to index data to the Amazon ES cluster. For more information, see Monitoring with Amazon CloudWatch Metrics.

  • Enable error logging if it is not already enabled, and check error logs for delivery failure. For more information, see Monitoring with Amazon CloudWatch Logs.

  • Make sure that the Amazon ES configuration in your delivery stream is accurate and valid.

  • If data transformation with Lambda is enabled, make sure that the Lambda function specified in your delivery stream still exists.

  • Make sure that the IAM role specified in your delivery stream has access to your Amazon ES cluster and Lambda function (if data transformation is enabled). For more information, see Grant Kinesis Firehose Access to an Amazon ES Destination.

Delivery Stream Not Available as a Target for CloudWatch Logs, CloudWatch Events, or AWS IoT Action

Some AWS services can only send messages and events to a Kinesis Firehose delivery stream that is in the same Region. Verify that your Kinesis Firehose delivery stream is located in the same Region as your other services.