Amazon MSK quota

Amazon MSK quota

• Up to 90 brokers per account and 30 brokers per cluster. To request higher quota, create a support case.

• A minimum of 1 GiB of storage per broker.

• A maximum of 16384 GiB of storage per broker.

• A cluster that uses IAM access control can have up to 3000 TCP connections per broker at any given time. To increase this limit, you can adjust the listener.name.client_iam.max.connections or the listener.name.client_iam_public.max.connections configuration property using the Kafka AlterConfig API or the kafka-configs.sh tool. It's important to note that increasing either property to a high value can result in unavailability.

• Limits on TCP connections. A cluster that uses IAM access control can accept new connections at a rate of up to 20 TCP connections per broker per second for all broker types, except for the type kafka.t3.small. Brokers of type kafka.t3.small are limited to 4 TCP connections per broker per second. If you created your cluster after May 25, 2022, it also supports connection rate bursts. If you want an older cluster to support connection rate bursts, you can create a support case.

  To handle retries on failed connections, you can set the reconnect.backoff.ms configuration parameter on the client side. For example, if you want a client to retry connections after 1 second, set reconnect.backoff.ms to 1000. For more information, see reconnect.backoff.ms in the Apache Kafka documentation.

• Up to 100 configurations per account. To request higher quota, create a support case.

• A maximum of 50 revisions per configuration.

• To update the configuration or the Apache Kafka version of an MSK cluster, first ensure the number of partitions per broker is under the limits described in Right-size your cluster: Number of partitions per broker.

MSK Serverless quota

Note

If you experience any issues with quota limits, contact AWS Support by creating a support case.

Limits are per cluster, unless otherwise stated.
Dimension	Quota	Quota violation result
Maximum ingress throughput	200 MBps	Slowdown with throttle duration in response
Maximum egress throughput	400 MBps	Slowdown with throttle duration in response
Maximum retention duration	Unlimited	N/A
Maximum number of client connections	3000	Connection close
Maximum connection attempts	100 per second	Connection close
Maximum message size	8 MB	Request fails with ErrorCode: INVALID_REQUEST
Maximum request rate	15,000 per second	Slowdown with throttle duration in response
Maximum rate of topic management APIs requests rate	2 per second	Slowdown with throttle duration in response
Maximum fetch bytes per request	55 MB	Request fails with ErrorCode: INVALID_REQUEST
Maximum number of consumer groups	500	JoinGroup request fails
Maximum number of partitions	2400 for non-compacted topics. 120 for compacted topics. To request a quota adjustment, create a support case.	Request fails with ErrorCode: INVALID_REQUEST
Maximum rate of partition creation and deletion	250 in 5 minutes	Request fails with ErrorCode: THROUGHPUT_QUOTA_EXCEEDED
Maximum ingress throughput per partition	5 MBps	Slowdown with throttle duration in response
Maximum egress throughput per partition	10 MBps	Slowdown with throttle duration in response
Maximum partition size (for compacted topics)	250 GB	Request fails with ErrorCode: THROUGHPUT_QUOTA_EXCEEDED
Maximum number of client VPCs per serverless cluster	5
Maximum number of serverless clusters per account	10. To request a quota adjustment, create a support case.

MSK Connect quota

• Up to 100 custom plugins.

• Up to 100 worker configurations.

• Up to 60 connect workers. If a connector is set up to have auto scaled capacity, then the maximum number of workers that the connector is set up to have is the number MSK Connect uses to calculate the quota for the account.

• Up to 10 workers per connector.

To request higher quota for MSK Connect, create a support case.