Quotas and limits
The following table describes stream and shard quotas and limits for Amazon Kinesis Data Streams.
| Quota | On-demand mode | Provisioned mode |
|---|---|---|
|
Number of data streams |
There's no upper quota on the number of streams within your AWS account. By default, you
can create up to 50 data streams with the on-demand capacity mode. If
you require an increase of this quota, raise a support ticket |
There's no upper quota on the number of streams with the provisioned mode within an account. |
|
Number of shards |
There's no upper limit. Number of shards depends on the amount of data ingested and the level of throughput you require. Kinesis Data Streams automatically scales the number of shards in response to changes in data volume and traffic. |
There's no upper limit. The default shard quota is 20,000 shards per AWS account for the following AWS Regions:
For all other Regions, the default shard quota is 1,000 or 6,000 shards per AWS account. You can view your account's shard quota and utilization through the Service Quotas console at https://console.aws.amazon.com/servicequotas/ To request an increase to the shard quota, use the Service Quotas console or AWS CLI. For more information, see Requesting a quota increase. |
|
Data stream throughput |
By default, new data streams created with the on-demand capacity mode have 4 MB/s of
write and 8 MB/s of read throughput. In US East (N.
Virginia), US West (Oregon), and
Europe (Ireland) AWS Regions, data streams with
the on-demand capacity mode scale up to 10 GB/s of write and 20 GB/s read
throughput. For other Regions, data streams with the on-demand capacity mode
scale up to 200 MB/s of write and 400 MB/s read throughput. If you require
an increase up to 10 GB/s write and 20 GB/s read capacity for these Regions,
submit a support ticket |
There is no upper limit. Maximum throughput depends on the number of shards provisioned for the stream. Each shard can support up to 1 MB/sec or 1,000 records/sec write throughput or up to 2 MB/sec or 2,000 records/sec read throughput. If you need more ingest capacity, you can easily scale up the number of shards in the stream using the AWS Management Console or the UpdateShardCount API. |
|
Data payload size |
The maximum size of the data payload of a record before |
|
|
|
GetRecords can retrieve up to 10 MB of data per call
from a single shard, and up to 10,000 records per call. Each call to |
|
|
Data read rate per shard |
Each shard can support up to a maximum total data read rate of 2 MB per second via GetRecords. If a call to |
|
|
Number of registered consumers per data stream |
You can create up to 20 registered consumers (Enhanced Fan-out Limit) for each data stream. |
|
|
Switching between provisioned and on-demand modes |
For each data stream in your AWS account, you can switch between the on-demand and provisioned capacity modes twice within 24 hours. |
|
API Limits
Like most AWS APIs, Kinesis Data Streams API operations are rate-limited. The following limits apply per AWS account per Region. For more information on Kinesis Data Streams APIs, see the Amazon Kinesis API Reference.
KDS Control Plane API Limits
The following section describes limits for the KDS control plane APIs. KDS control plane APIs let you create and manage your data streams. These limits apply per AWS account per Region.
| API | API call limit | Per Account/Stream | Description |
|---|---|---|---|
| AddTagsToStream | 5 transactions per second (TPS) | Per Account | 50 tags per data stream |
| CreateStream | 5 TPS | Per Account | There is no upper quota on the number of streams you can have in an account. You receive a
|
| DecreaseStreamRetentionPeriod | 5 TPS | Per Stream | The minimum value of a data stream's retention period is 24 hours. |
| DeleteResourcePolicy | 5 TPS | Per Account | If you require an increase of this limit, raise a Support ticket |
| DeleteStream | 5 TPS | Per Account | |
| DeregisterStreamConsumer | 5 TPS | Per Stream | |
| DescribeLimits | 1 TPS | Per Account | |
| DescribeStream | 10 TPS | Per Account | |
| DescribeStreamConsumer | 20 TPS | Per Stream | |
| DescribeStreamSummary | 20 TPS | Per Account | |
| DisableEnhancedMonitoring | 5 TPS | Per Stream | |
| EnableEnhancedMonitoring | 5 TPS | Per Stream | |
| GetResourcePolicy | 5 TPS | Per Account | If you require an increase of this limit, raise a Support ticket |
| IncreaseStreamRetentionPeriod | 5 TPS | Per Stream | The maximum value of a stream's retention period is 8760 hours (365 days). |
| ListShards | 1000 TPS | Per Stream | |
| ListStreamConsumers | 5 TPS | Per Stream | |
| ListStreams | 5 TPS | Per Account | |
| ListTagsForStream | 5 TPS | Per Stream | |
| MergeShards | 5 TPS | Per Stream | Only applicable for provisioned. |
| PutResourcePolicy | 5 TPS | Per Account | If you require an increase of this limit, raise a Support ticket |
| RegisterStreamConsumer | 5 TPS | Per Stream | You can register up to 20 consumers per data stream. A given
consumer can only be registered with one data stream at a time. Only
5 consumers can be created simultaneously. In other words, you
cannot have more than 5 consumers in a CREATING status at the same
time. Registering a 6th consumer while there are 5 in a CREATING |
| RemoveTagsFromStream | 5 TPS | Per Stream | |
| SplitShard | 5 TPS | Per Stream | Only applicable for provisioned |
| StartStreamEncryption | Per Stream | You can successfully apply a new AWS KMS key for server-side encryption 25 times in a rolling 24-hour period. | |
| StopStreamEncryption | Per Stream | You can successfully disable server-side encryption 25 times in a rolling 24-hour period. | |
| UpdateShardCount | Per Stream | Only applicable for provisioned. The default limit on number of shards is 10,000. There are additional limits on this API. For more information, see UpdateShardCount. | |
| UpdateStreamMode | Per stream |
For each data stream in your AWS account, you can switch between the on-demand and provisioned capacity modes twice within 24 hours. |
KDS Data Plane API Limits
The following section describes the limits for the KDS data plane APIs. KDS data plane APIs enable you to use your data streams for collecting and processing data records in real time. These limits apply per shard within your data streams.
| API | API call limit | Payload limit | Additional details |
|---|---|---|---|
| GetRecords | 5 TPS | The maximum number of records that can be returned per call is 10,000. The maximum size of data that GetRecords
can return is 10 MB.
|
If a call returns this amount of data, subsequent calls made within the next 5 seconds
throw ProvisionedThroughputExceededException. If there
is insufficient provisioned throughput on the stream, subsequent
calls made within the next 1 second throw
ProvisionedThroughputExceededException. |
| GetShardIterator | 5 TPS | A shard iterator expires 5 minutes after it is returned to the requester. If a GetShardIterator request is made too often, you receive a ProvisionedThroughputExceededException. | |
| PutRecord | 1000 TPS | Each shard can support writes up to 1,000 records per second, up to a maximum data write total of 1 MB per second. | |
| PutRecords | Each PutRecords request can support up to 500 records. Each record in the request can be as large as 1 MB, up to a limit of 5 MB for the entire request, including partition keys. Each shard can support writes up to 1,000 records per second, up to a maximum data write total of 1 MB per second. | ||
| SubscribeToShard | You can make one call to SubscribeToShard per second per registered consumer per shard. | If you call SubscribeToShard again with the same ConsumerARN and ShardId within 5 seconds of a successful call, you'll get a ResourceInUseException. |
Increasing Quotas
You can use Service Quotas to request an increase for a quota, if the quota is adjustable. Some requests are automatically resolved, while others are submitted to AWS Support. You can track the status of a quota increase request that is submitted to AWS Support. Requests to increase Service Quotas do not receive priority support. If you have an urgent request, contact AWS Support. For more information, see What Is Service Quotas?
To request a service quota increase, follow the procedure outlined in Requesting a Quota Increase.
