Troubleshooting HTTP Endpoints - Amazon Data Firehose

Amazon Data Firehose was previously known as Amazon Kinesis Data Firehose

Troubleshooting HTTP Endpoints

This section describes common troubleshooting steps when dealing with Amazon Data Firehose delivering data to generic HTTP Endpoints destinations and to partner destinations, including Datadog, Dynatrace, LogicMonitor, MongoDB, New Relic, Splunk, or Sumo Logic. For the purposes of this section, all applicable destinations are referred to as HTTP endpoints. Make sure that the IAM role that is specified in your Firehose delivery stream can access the S3 backup bucket and the Lambda function for data transformation (if data transformation is enabled). Also, make sure that the IAM role has access to CloudWatch log group and log streams to check error logs. For more information, see Grant Firehose Access to an HTTP Endpoint Destination.


The information in this section does not apply to the following destinations: Splunk, OpenSearch Service, S3, and Redshift.

CloudWatch Logs

It is highly recommended that you enable CloudWatch Logging for Firehose. Logs are only published when there are errors delivering to your destination.

Destination Exceptions

ErrorCode: HttpEndpoint.DestinationException

{ "deliveryStreamARN": "arn:aws:firehose:us-east-1:123456789012:deliverystream/ronald-test", "destination": "", "deliveryStreamVersionId": 1, "message": "The following response was received from the endpoint destination. 413: {\"requestId\": \"43b8e724-dbac-4510-adb7-ef211c6044b9\", \"timestamp\": 1598556019164, \"errorMessage\": \"Payload too large\"}", "errorCode": "HttpEndpoint.DestinationException", "processor": "arn:aws:lambda:us-east-1:379522611494:function:httpLambdaProcessing" }

Destination exceptions indicate that Firehose is able to establish a connection to your endpoint and make an HTTP request, but did not receive a 200 response code. 2xx responses that are not 200s will also result in a destination exception. Amazon Data Firehose logs the response code and a truncated response payload received from the configured endpoint to CloudWatch Logs. Because Amazon Data Firehose logs the response code and payload without modification or interpretation, it is up to the endpoint to provide the exact reason why it rejected Amazon Data Firehose's HTTP delivery request. The following are the most common troubleshooting recommendations for these exceptions:

  • 400: Indicates that you are sending a bad request due to a misconfiguration of your Amazon Data Firehose. Make sure that you have the correct url, common attributes, content encoding, access key, and buffering hints for your destination. See the destination specific documentation on the required configuration.

  • 401: Indicates that the access key you configured for your Firehose stream is incorrect or missing.

  • 403: Indicates that the access key you configured for your Firehose stream does not have permissions to deliver data to the configured endpoint.

  • 413: Indicates that the request payload that Amazon Data Firehose sends to the endpoint is too large for the endpoint to handle. Try lowering the buffering hint to the recommended size for your destination.

  • 429: Indicates that Amazon Data Firehose is sending requests at a greater rate than the destination can handle. Fine tune your buffering hint by increasing your buffering time and/or increasing your buffering size (but still within the limit of your destination).

  • 5xx: Indicates that there is a problem with the destination. The Amazon Data Firehose service is still working properly.


Important: While these are the common troubleshooting recommendations, specific endpoints may have different reasons for providing the response codes and the endpoint specific recommendations should be followed first.

Invalid Response

ErrorCode: HttpEndpoint.InvalidResponseFromDestination

{ "deliveryStreamARN": "arn:aws:firehose:us-east-1:123456789012:deliverystream/ronald-test", "destination": "", "deliveryStreamVersionId": 1, "message": "The response received from the specified endpoint is invalid. Contact the owner of the endpoint to resolve the issue. Response for request 2de9e8e9-7296-47b0-bea6-9f17b133d847 is not recognized as valid JSON or has unexpected fields. Raw response received: 200 {\"requestId\": null}", "errorCode": "HttpEndpoint.InvalidResponseFromDestination", "processor": "arn:aws:lambda:us-east-1:379522611494:function:httpLambdaProcessing" }

Invalid response exceptions indicate that Amazon Data Firehose received an invalid response from the endpoint destination. The response must conform to the response specifications or Amazon Data Firehose will consider the delivery attempt a failure and will redeliver the same data until the configured retry duration is exceeded. Amazon Data Firehose treats responses that do not follow the response specifications as failures even if the response has a 200 status. If you are developing a Amazon Data Firehose compatible endpoint, follow the response specifications to ensure data is successfully delivered.

Below are some of the common types of invalid responses and how to fix them:

  • Invalid JSON or Unexpected Fields: Indicates that the response can not be properly deserialized as JSON or has unexpected fields. Ensure that the response is not content-encoded.

  • Missing RequestId: Indicates that the response does not contain a requestId.

  • RequestId does not match: Indicates that the requestId in the response does not match the outgoing requestId.

  • Missing Timestamp: Indicates that the response does not contain a timestamp field. The timestamp field must be a number and not a string.

  • Missing Content-Type Header: Indicates that the response does not contain a “content-type: application/json” header. No other content-type is accepted.


Important: Amazon Data Firehose can only deliver data to endpoints that follow the Firehose request and response specifications. If you are configuring your destination to a third party service, ensure that you are using the correct Amazon Data Firehose compatible endpoint which will likely be different than the public ingestion endpoint. For example Datadog’s Amazon Data Firehose endpoint is while its public endpoint is

Other Common Errors

Additional error codes and definitions are listed below.

  • Error Code: HttpEndpoint.RequestTimeout - Indicates that the endpoint took longer than 3 minutes to respond. If you are the owner of the destination, decrease the response time of the destination endpoint. If you are not the owner of the destination, contact the owner and ask if anything can be done to lower the response time (i.e. decrease the buffering hint so there is less data being processed per request).

  • Error Code: HttpEndpoint.ResponseTooLarge - Indicates that the response is too large. The response must be less than 1 MiB including headers.

  • Error Code: HttpEndpoint.ConnectionFailed - Indicates a connection could not be established with the configured endpoint. This could be due to a typo in the configured url, the endpoint not being accessible to Amazon Data Firehose, or the endpoint taking too long to respond to the connection request.

  • Error Code: HttpEndpoint.ConnectionReset - Indicates a connection was made but reset or prematurely closed by the endpoint.

  • Error Code: HttpEndpoint.SSLHandshakeFailure - Indicates an SSL handshake could not be successfully completed with the configured endpoint.