View a markdown version of this page

Key concepts - Amazon Managed Streaming for Apache Kafka

Key concepts

This section describes the key concepts behind Amazon MSK Data Delivery.

  • Data freshness vs. throughput — A Channel needs at least 2.4 MBps of uncompressed throughput for the minimum 5-minute freshness. For lower-throughput topics, increase freshness (up to 15 minutes) so the service can accumulate enough data for efficient delivery and inline compaction.

  • Input formats — JSON (plain JSON; you provide a GSR schema ARN) and JSON_SCHEMA_GSR (GSR-serialized JSON with the schema ID embedded in each record) are supported. The AWS Glue Schema Registry (GSR) is the source of truth, and the Channel fails to create if the schema cannot be resolved.

  • Schema evolution — Not supported. Changing the schema after creation can cause delivery failures.

  • Partitioning — S3 Tables delivery supports time-based partitioning only.

  • No backfill — only records produced after enablement are delivered.

  • New table for each channel — each Channel creates its own Iceberg table.

  • Dead-letter queue — both destinations require a DLQ S3 bucket. The Channel writes the identifiers (sequence numbers) of unprocessable records, along with error context — not the full record payloads.