View a markdown version of this page

Important considerations - Amazon Kinesis Data Streams

Important considerations

  • KPL upgrade for null partition keys – Customers who want to send records without a partition key should upgrade to the latest KPL version. For customers who continue to provide partition keys, no KPL or KCL changes are required. For more information, see KPL and KCL with service-managed streams.

  • AWS SDK considerations – If you use the AWS SDK directly (rather than the KPL or KCL) to produce to or consume from a service-managed stream, review the following before you enable AUTO:

    • Producers – Upgrade to the latest SDK version to omit the partition key when writing to a service-managed stream. On the latest SDK, PartitionKey is optional, so you can call PutRecord and PutRecords without it. If you continue to supply a partition key, no SDK change is required — the service accepts the value and ignores it for shard placement. Before you enable AUTO and start sending null partition keys on a stream that existing consumers read from, update those consumers to handle an absent PartitionKey.

    • Consumers – On a service-managed stream, GetRecords and SubscribeToShard return PartitionKey as an absent (null) value for records written without one. Any consumer that reads PartitionKey and assumes it is always present can fail at runtime when it reads such a record — this applies regardless of SDK version. Upgrade to the latest SDK version to read these records safely, and update your code to handle an absent value. For the AWS SDK for Go, AWS SDK for Kotlin, and AWS SDK for Rust, upgrading additionally changes PartitionKey to an optional type, so an unhandled absent value surfaces as a compile-time error when you upgrade. Consumers that do not read PartitionKey are unaffected.

  • Aggregation not supported – KPL aggregation is not supported on service-managed streams in any KPL version. Consider disabling aggregation if you want to use service-managed record distribution. For details, see KPL and KCL with service-managed streams.

  • Cannot be mixed within a stream – The record distribution strategy applies to the entire stream. You cannot use both service-managed and user-managed partition keys on the same stream simultaneously. For workloads that require both ordered and unordered data, create separate streams.

  • No new service limits – Service-managed record distribution does not introduce any new limits. Existing Kinesis Data Streams limits continue to apply, including the 1 MB per record maximum, 1,000 records per second per shard for writes, and 500 records per PutRecords request.