Troubleshooting data delivery

Use this section to resolve common issues with data delivery.

Delivery stuck in CREATING state

When you create a delivery, it enters the CREATING state while resources are provisioned. Provisioning typically completes within a few minutes. If the delivery remains in CREATING for an extended period or transitions to FAILED, a configuration error is likely the cause.

Call DescribeChannel to check the current status and status reason. Common causes include:

• Invalid IAM role ARN or insufficient permissions in the role policy.

• The destination Amazon S3 bucket does not exist or is in a different region.

• The schema ARN in AWS Glue Schema Registry cannot be resolved.

Delivery in FAILED state

You cannot recover a delivery whose ChannelStatus is FAILED (as returned by DescribeChannel). Read the ChannelStatusReason field from DescribeChannel to identify the root cause. Fix the underlying issue, delete the failed delivery, and recreate it with corrected configuration.

High data freshness

The DataFreshness metric measures the age of the oldest undelivered record. High values indicate delivery is falling behind ingestion. Common causes:

• High partition count on the destination table increases metadata overhead.

• Table metadata growth from many small commits reduces commit throughput.

• Low stream throughput combined with a low freshness setting causes frequent small deliveries.

Resolution: For streaming tables on Apache Iceberg, enable Amazon S3 Tables maintenance (compaction and snapshot expiration) to manage metadata growth. For low-throughput streams, increase the DataFreshnessInSeconds value to allow more data to batch for each delivery cycle.

The tightest data freshness setting requires a minimum sustained stream throughput so that enough data accumulates for efficient delivery and inline compaction each cycle. If your stream produces less than that throughput, use a higher DataFreshnessInSeconds value.

Failed records greater than zero

When the failed-record metric is non-zero (DeliveryToS3.FailedRecordCount for Amazon S3 deliveries or DeliveryToIceberg.FailedRowCount for streaming table deliveries), records are being sent to the dead-letter queue instead of the destination.

For streaming tables on Apache Iceberg:

• Schema mismatch – the record does not conform to the registered schema.

• Missing required field – a non-nullable column has no value in the record.

• Not using the AWS Glue Schema Registry serializer for GSR_JSON format – producers must use the AWS Glue Schema Registry producer library.

For general purpose Amazon S3 buckets:

• Format mismatch – the record format does not match the configured input format.

Resolution: Inspect the dead-letter queue entries for detailed error information. Check CloudWatch Logs for the delivery to see specific parsing or validation errors. Fix the producer to send conforming records.

No data appearing at the destination

If the delivery is in ACTIVE state but no data appears at the destination, the most common causes are:

• Permission problems – the IAM role cannot write to the destination. Check CloudWatch Logs for AccessDenied errors.

• Output key prefix mismatch (general purpose Amazon S3 buckets) – if your s3:PutObject permission is scoped to a prefix such as arn:aws:s3:::my-bucket/data*, the keys generated by your output key template must start with data/. A mismatch causes every write to be denied.

• Missing table permission (streaming tables) – when you encrypt the destination table with a customer managed AWS KMS key, verify the service execution role includes s3tables:PutTableEncryption in addition to the other s3tables actions. Without it, CreateTable succeeds but table encryption fails, the table is never created, and no data is delivered.

• Missing logging permissions – if the service execution role lacks logs:CreateLogStream and logs:PutLogEvents, delivery failures are not recorded in CloudWatch Logs, which can make a permissions problem appear as a silent failure. Verify the logging permissions first when no logs are present.

• No new data after creation – the delivery does not backfill existing data from the stream. Only records written after the delivery becomes ACTIVE are delivered.

Delivery suspended

A delivery enters a suspended state when the destination becomes unavailable or incompatible. Common causes:

• The streaming tables on Apache Iceberg destination table was deleted.

• The Amazon S3 bucket owner does not match the expected account (ownership mismatch).

• An incompatible partition column was detected on the destination table.

A suspended delivery cannot be resumed. You must create a new delivery with a valid destination configuration.

Permission denied errors in CloudWatch Logs

AccessDenied errors in the delivery's CloudWatch Logs indicate a permissions problem. Common causes:

• The IAM role policy was modified after the delivery was created.

• The Amazon S3 bucket policy was changed to deny access from the role.

• The trust policy on the role does not allow the Kinesis Data Streams service to assume it.

• The AWS KMS key policy denies encrypt or decrypt access to the delivery's role.

Review and correct the relevant policy, then verify that the delivery resumes.

Delivery not available for my stream

Streaming tables and Amazon S3 delivery require the Kinesis Data Streams stream to be in On-Demand Standard or On-Demand Advantage capacity mode. If your stream uses Provisioned mode, you must switch it to On-Demand mode before you can create a delivery.

Delivery stops after a schema change

Deliveries do not support schema evolution. If you update the schema in AWS Glue Schema Registry after creating a delivery, records produced with the new schema version might fail validation and be routed to the dead-letter queue.

To resolve: revert the schema change at the producer, or delete the existing delivery and recreate it with the updated schema.

Cannot delete a stream

A DeleteStream request fails with ResourceInUseException when the stream has one or more active deliveries. A stream cannot be deleted while deliveries are attached to it.

To resolve: list the deliveries on the stream with ListChannels (using a stream filter), delete each delivery with DeleteChannel, and then delete the stream.