Clusters - Amazon MSK

Clusters

Represents the Amazon MSK clusters in an account.

URI

/api/v2/clusters

HTTP methods

GET

Operation ID: ListClustersV2

Lists all serverless and provisioned clusters.

Query parameters
NameTypeRequiredDescription
nextTokenStringFalse

If the response of ListClusters is truncated, it returns a nextToken in the response. This nextToken should be sent in the subsequent request to ListClusters.

clusterNameFilterStringFalse

Returns clusters starting with given name.

maxResultsStringFalse

Maximum number of clusters to fetch in one get request.

clusterTypeFilterStringFalse

Returns clusters with the given type.

Responses
Status codeResponse modelDescription
200 ListClustersV2Response

HTTP Status Code 200: OK.

400None

HTTP Status Code 400: Bad request due to incorrect input. Correct your request and then retry it.

401None

HTTP Status Code 401: Unauthorized request. The provided credentials couldn't be validated.

403None

HTTP Status Code 403: Access forbidden. Correct your credentials and then retry your request.

404None

HTTP Status Code 404: Resource not found due to incorrect input. Correct your request and then retry it.

429None

HTTP Status Code 429: Limit exceeded. Resource limit reached.

500None

HTTP Status Code 500: Unexpected internal server error. Retrying your request might resolve the issue.

503None

HTTP Status Code 503: Service Unavailable. Retrying your request in some time might resolve the issue.

POST

Operation ID: CreateClusterV2

Creates a serverless or provisioned cluster.

Responses
Status codeResponse modelDescription
200 CreateClusterV2Response

HTTP Status Code 200: OK.

400None

HTTP Status Code 400: Bad request due to incorrect input. Correct your request and then retry it.

401None

HTTP Status Code 401: Unauthorized request. The provided credentials couldn't be validated.

403None

HTTP Status Code 403: Access forbidden. Correct your credentials and then retry your request.

404None

HTTP Status Code 404: Resource not found due to incorrect input. Correct your request and then retry it.

409None

HTTP Status Code 409: Conflict. This cluster name already exists. Retry your request with another name.

429None

HTTP Status Code 429: Limit exceeded. Resource limit reached.

500None

HTTP Status Code 500: Unexpected internal server error. Retrying your request might resolve the issue.

503None

HTTP Status Code 503: Service Unavailable. Retrying your request in some time might resolve the issue.

OPTIONS

Enable CORS by returning correct headers

Responses
Status codeResponse modelDescription
200None

200 response

Schemas

Request bodies

{ "provisioned": { "encryptionInfo": { "encryptionInTransit": { "inCluster": boolean, "clientBroker": enum }, "encryptionAtRest": { "dataVolumeKMSKeyId": "string" } }, "numberOfBrokerNodes": integer, "configurationInfo": { "arn": "string", "revision": integer }, "enhancedMonitoring": enum, "storageMode": enum, "kafkaVersion": "string", "clientAuthentication": { "sasl": { "iam": { "enabled": boolean }, "scram": { "enabled": boolean } }, "unauthenticated": { "enabled": boolean }, "tls": { "certificateAuthorityArnList": [ "string" ], "enabled": boolean } }, "loggingInfo": { "brokerLogs": { "s3": { "bucket": "string", "prefix": "string", "enabled": boolean }, "firehose": { "deliveryStream": "string", "enabled": boolean }, "cloudWatchLogs": { "logGroup": "string", "enabled": boolean } } }, "brokerNodeGroupInfo": { "clientSubnets": [ "string" ], "zoneIds": [ "string" ], "instanceType": "string", "connectivityInfo": { "vpcConnectivity": { "clientAuthentication": { "sasl": { "iam": { "enabled": boolean }, "scram": { "enabled": boolean } }, "tls": { "enabled": boolean } } }, "publicAccess": { "type": "string" } }, "securityGroups": [ "string" ], "brokerAZDistribution": enum, "storageInfo": { "ebsStorageInfo": { "provisionedThroughput": { "volumeThroughput": integer, "enabled": boolean }, "volumeSize": integer } } }, "openMonitoring": { "prometheus": { "nodeExporter": { "enabledInBroker": boolean }, "jmxExporter": { "enabledInBroker": boolean } } } }, "clusterName": "string", "serverless": { "vpcConfigs": [ { "securityGroupIds": [ "string" ], "subnetIds": [ "string" ] } ], "clientAuthentication": { "sasl": { "iam": { "enabled": boolean } } } }, "tags": { } }

Response bodies

{ "nextToken": "string", "clusterInfoList": [ { "clusterType": enum, "clusterArn": "string", "activeOperationArn": "string", "provisioned": { "encryptionInfo": { "encryptionInTransit": { "inCluster": boolean, "clientBroker": enum }, "encryptionAtRest": { "dataVolumeKMSKeyId": "string" } }, "zookeeperConnectString": "string", "customerActionStatus": enum, "zookeeperConnectStringTls": "string", "loggingInfo": { "brokerLogs": { "s3": { "bucket": "string", "prefix": "string", "enabled": boolean }, "firehose": { "deliveryStream": "string", "enabled": boolean }, "cloudWatchLogs": { "logGroup": "string", "enabled": boolean } } }, "numberOfBrokerNodes": integer, "enhancedMonitoring": enum, "storageMode": enum, "clientAuthentication": { "sasl": { "iam": { "enabled": boolean }, "scram": { "enabled": boolean } }, "unauthenticated": { "enabled": boolean }, "tls": { "certificateAuthorityArnList": [ "string" ], "enabled": boolean } }, "brokerNodeGroupInfo": { "clientSubnets": [ "string" ], "zoneIds": [ "string" ], "instanceType": "string", "connectivityInfo": { "vpcConnectivity": { "clientAuthentication": { "sasl": { "iam": { "enabled": boolean }, "scram": { "enabled": boolean } }, "tls": { "enabled": boolean } } }, "publicAccess": { "type": "string" } }, "securityGroups": [ "string" ], "brokerAZDistribution": enum, "storageInfo": { "ebsStorageInfo": { "provisionedThroughput": { "volumeThroughput": integer, "enabled": boolean }, "volumeSize": integer } } }, "openMonitoring": { "prometheus": { "nodeExporter": { "enabledInBroker": boolean }, "jmxExporter": { "enabledInBroker": boolean } } }, "currentBrokerSoftwareInfo": { "configurationRevision": integer, "kafkaVersion": "string", "configurationArn": "string" } }, "creationTime": "string", "clusterName": "string", "serverless": { "vpcConfigs": [ { "securityGroupIds": [ "string" ], "subnetIds": [ "string" ] } ], "kafkaVersion": "string", "clientAuthentication": { "sasl": { "iam": { "enabled": boolean } } } }, "stateInfo": { "code": "string", "message": "string" }, "state": enum, "currentVersion": "string", "tags": { } } ] }
{ "clusterType": enum, "clusterArn": "string", "clusterName": "string", "state": enum }

Properties

BrokerAZDistribution

The distribution of broker nodes across Availability Zones.

  • DEFAULT

BrokerLogs

Broker Logs details for cluster.

PropertyTypeRequiredDescription
cloudWatchLogs

CloudWatchLogs

False

CloudWatch Log destination details.

firehose

Firehose

False
s3

S3

False

S3 Log destination details.

BrokerNodeGroupInfo

Describes the setup to be used for the brokers.

PropertyTypeRequiredDescription
brokerAZDistribution

BrokerAZDistribution

False

The distribution of broker nodes across Availability Zones.

clientSubnets

Array of type string

True

The list of subnets in the client VPC to connect to.

connectivityInfo

ConnectivityInfo

False

Information about the cluster access configuration.

instanceType

string

MinLength: 5

MaxLength: 32

True

The type of broker used for the cluster.

securityGroups

Array of type string

False

The security groups to attach to the ENIs for the broker nodes.

storageInfo

StorageInfo

False

Data volume information.

zoneIds

Array of type string

False

The zoneIds for brokers in customer account.

BrokerSoftwareInfo

Information about current software installed in the cluster.

PropertyTypeRequiredDescription
configurationArn

string

False

ARN of the configuration used on the cluster.

configurationRevision

integer

Format: int64

False

Revision of the configuration to use.

kafkaVersion

string

False

The version of Apache Kafka to install and run on the cluster.

ClientAuthentication

Includes all client authentication information.

PropertyTypeRequiredDescription
sasl

Sasl

False

Details for ClientAuthentication using SASL.

tls

Tls

False

Details for ClientAuthentication using TLS.

unauthenticated

Unauthenticated

False

Details for ClientAuthentication using no authentication.

ClientBroker

Client-broker encryption in transit setting.

  • TLS

  • TLS_PLAINTEXT

  • PLAINTEXT

CloudWatchLogs

CloudWatchLogs details for BrokerLogs.

PropertyTypeRequiredDescription
enabled

boolean

True

Broker logs for destination CW enabled or not.

logGroup

string

False

CloudWatch LogGroup where the logs will be delivered.

Cluster

Returns information about a cluster.

PropertyTypeRequiredDescription
activeOperationArn

string

False

Arn of active cluster operation.

clusterArn

string

False

The Amazon Resource Name (ARN) of the cluster.

clusterName

string

False

The name of the cluster.

clusterType

ClusterType

False

Type of the backend cluster.

creationTime

string

False

The time when the cluster was created.

currentVersion

string

False

Current version of cluster.

provisioned

Provisioned

False

Properties of a provisioned cluster.

serverless

Serverless

False

Properties of a serverless cluster.

state

ClusterState

False

State of the cluster.

stateInfo

StateInfo

False

Includes information of the cluster state.

tags

object

False

Tags attached to the cluster.

ClusterState

The sate of an MSK cluster.

  • ACTIVE

  • CREATING

  • UPDATING

  • DELETING

  • FAILED

  • MAINTENANCE

  • REBOOTING_BROKER

  • HEALING

ClusterType

The type of backend cluster.

  • PROVISIONED

  • SERVERLESS

ConfigurationInfo

Specifies the configuration to be used on the brokers.

PropertyTypeRequiredDescription
arn

string

True

ARN of the configuration to use.

revision

integer

Format: int64

Minimum: 1

True

The revision of the configuration to use.

ConnectivityInfo

Broker access controls

PropertyTypeRequiredDescription
publicAccess

PublicAccess

False

Access control settings for brokers

vpcConnectivity

VpcConnectivity

False

VPC connection control settings for brokers

CreateClusterV2Request

Creates a provisioned or serverless cluster.

PropertyTypeRequiredDescription
clusterName

string

MinLength: 1

MaxLength: 64

True

The name of the cluster.

provisioned

ProvisionedRequest

False

Properties required for creating a provisioned cluster.

serverless

ServerlessRequest

False

Properties required for creating a serverless cluster.

tags

object

False

Create tags when creating the cluster.

CreateClusterV2Response

Returns information about the created cluster.

PropertyTypeRequiredDescription
clusterArn

string

False

The Amazon Resource Name (ARN) of the cluster.

clusterName

string

False

Name of the cluster provided by the customer.

clusterType

ClusterType

False

Type of the backend cluster.

state

ClusterState

False

State of the cluster.

CustomerActionStatus

A type of an action required from the customer.

  • CRITICAL_ACTION_REQUIRED

  • ACTION_RECOMMENDED

  • NONE

EBSStorageInfo

Contains information about the EBS storage volumes that are attached to the brokers.

PropertyTypeRequiredDescription
provisionedThroughput

ProvisionedThroughput

False

EBS volume provisioned throughput information.

volumeSize

integer

Minimum: 1

Maximum: 16384

False

The size of the EBS volumes for the data drive on each of the brokers in GiB.

EncryptionAtRest

Details for encryption at rest.

PropertyTypeRequiredDescription
dataVolumeKMSKeyId

string

True

KMS key used for data volume encryption.

EncryptionInTransit

Details for encryption in transit.

PropertyTypeRequiredDescription
clientBroker

ClientBroker

False

Client-broker encryption in transit setting.

inCluster

boolean

False

In-cluster encryption in transit setting.

EncryptionInfo

Includes all encryption related information.

PropertyTypeRequiredDescription
encryptionAtRest

EncryptionAtRest

False

Details for encryption at rest.

encryptionInTransit

EncryptionInTransit

False

Details for encryption in transit.

EnhancedMonitoring

Controls level of cluster metrics Amazon pushes to customer's cloudwatch account.

  • DEFAULT

  • PER_BROKER

  • PER_TOPIC_PER_BROKER

  • PER_TOPIC_PER_PARTITION

Firehose

Firehose details for BrokerLogs.

PropertyTypeRequiredDescription
deliveryStream

string

False

Firehose delivery stream where the logs will be delivered.

enabled

boolean

True

Broker logs for destination firehose enabled or not.

IAM

Details for SASL/IAM client authentication.

PropertyTypeRequiredDescription
enabled

boolean

False

SASL/IAM authentication is enabled or not.

JmxExporter

JMX Exporter details.

PropertyTypeRequiredDescription
enabledInBroker

boolean

True

JMX Exporter being enabled in broker.

JmxExporterInfo

JMX Exporter details.

PropertyTypeRequiredDescription
enabledInBroker

boolean

True

JMX Exporter being enabled in broker.

ListClustersV2Response

The response contains an array of cluster information and a nextToken if the response is truncated.

PropertyTypeRequiredDescription
clusterInfoList

Array of type Cluster

False

An array of clusters.

nextToken

string

False

If the response of ListClustersV2 is truncated, it returns a NextToken in the response. Pass this NextToken in a subsequent request to ListClustersV2 to get the next set of results.

LoggingInfo

Logging info details for the cluster.

PropertyTypeRequiredDescription
brokerLogs

BrokerLogs

True

Broker Logs details.

NodeExporter

Node Exporter details.

PropertyTypeRequiredDescription
enabledInBroker

boolean

True

Node Exporter being enabled in broker.

NodeExporterInfo

Node Exporter details.

PropertyTypeRequiredDescription
enabledInBroker

boolean

True

Node Exporter being enabled in broker.

OpenMonitoring

JMX and Node monitoring for cluster.

PropertyTypeRequiredDescription
prometheus

Prometheus

True

Prometheus details.

OpenMonitoringInfo

JMX and Node monitoring for the cluster.

PropertyTypeRequiredDescription
prometheus

PrometheusInfo

True

Prometheus details.

Prometheus

Prometheus details.

PropertyTypeRequiredDescription
jmxExporter

JmxExporter

False

JMX Exporter details.

nodeExporter

NodeExporter

False

Node Exporter details.

PrometheusInfo

Prometheus details.

PropertyTypeRequiredDescription
jmxExporter

JmxExporterInfo

False

JMX Exporter details.

nodeExporter

NodeExporterInfo

False

Node Exporter details.

Provisioned

Properties of a provisioned cluster.

PropertyTypeRequiredDescription
brokerNodeGroupInfo

BrokerNodeGroupInfo

False

Information about the brokers of the cluster.

clientAuthentication

ClientAuthentication

False

Includes all client authentication information.

currentBrokerSoftwareInfo

BrokerSoftwareInfo

False

Information about the version of the software that is deployed on the brokers of the cluster.

customerActionStatus

CustomerActionStatus

False

Determines if there is an action required from the customer.

encryptionInfo

EncryptionInfo

False

Includes all encryption related information.

enhancedMonitoring

EnhancedMonitoring

False

This knob controls level of metrics pushed customer's cloudwatch account.

loggingInfo

LoggingInfo

False

Logging Info details.

numberOfBrokerNodes

integer

False

The number of brokers to create in the cluster.

openMonitoring

OpenMonitoring

False

Open monitoring details.

storageMode

StorageMode

False

This controls storage mode for supported storage tiers.

zookeeperConnectString

string

False

The connection string to use to connect to zookeeper cluster on plaintext port.

zookeeperConnectStringTls

string

False

The connection string to use to connect to zookeeper cluster on Tls port.

ProvisionedRequest

Properties for creating a provisioned cluster.

PropertyTypeRequiredDescription
brokerNodeGroupInfo

BrokerNodeGroupInfo

False

Information about the brokers of the cluster.

clientAuthentication

ClientAuthentication

False

Includes all client authentication related information.

configurationInfo

ConfigurationInfo

False

The configuration to be used on the brokers.

encryptionInfo

EncryptionInfo

False

Includes all encryption related information.

enhancedMonitoring

EnhancedMonitoring

False

Specifies the monitoring level for the cluster.

kafkaVersion

string

MinLength: 1

MaxLength: 128

False

The version of Apache Kafka to install and run on the cluster.

loggingInfo

LoggingInfo

False

Logging Info details.

numberOfBrokerNodes

integer

False

The number of brokers to create in the cluster.

openMonitoring

OpenMonitoringInfo

False

The open-monitoring details.

storageMode

StorageMode

False

Specifies the storage mode for supported storage tiers.

ProvisionedThroughput

Contains information about provisioned throughput for the EBS storage volumes that are attached to the brokers.

PropertyTypeRequiredDescription
enabled

boolean

False

Whether provisioned throughput is turned on.

volumeThroughput

integer

False

Throughput value of the EBS volumes for the data drive on each broker in MiB per second.

PublicAccess

Broker access controls

PropertyTypeRequiredDescription
type

string

False

If public access is disabled, or if enabled the EIP provider

S3

S3 details for BrokerLogs.

PropertyTypeRequiredDescription
bucket

string

False

Name of the bucket where the logs will be delivered.

enabled

boolean

True

Broker logs for destination S3 enabled or not.

prefix

string

False

prefix to the S3 bucket where the logs will be delivered.

Sasl

Details for client authentication using SASL.

PropertyTypeRequiredDescription
iam

IAM

False

Details for ClientAuthentication using IAM.

scram

Scram

False

Details for SASL/SCRAM client authentication.

Scram

Details for SASL/SCRAM client authentication.

PropertyTypeRequiredDescription
enabled

boolean

False

SASL/SCRAM authentication is enabled or not.

Serverless

Properties to create a serverless cluster

PropertyTypeRequiredDescription
clientAuthentication

ServerlessClientAuthentication

True

Includes all client authentication related information.

kafkaVersion

string

False

The version of Apache Kafka for the serverless cluster.

vpcConfigs

Array of type VpcConfig

True

VPC configuration information

ServerlessClientAuthentication

Details for client authentication using SASL.

PropertyTypeRequiredDescription
sasl

ServerlessSasl

False

Details for ClientAuthentication using IAM.

ServerlessRequest

Properties for creating a serverless cluster.

PropertyTypeRequiredDescription
clientAuthentication

ServerlessClientAuthentication

True

Includes all client authentication related information.

vpcConfigs

Array of type VpcConfig

True

VPC configuration information

ServerlessSasl

Details for client authentication using SASL for Serverless Cluster.

PropertyTypeRequiredDescription
iam

IAM

False

Details for ClientAuthentication using IAM for Serverless Cluster.

StateInfo

Includes information about the cluster state.

PropertyTypeRequiredDescription
code

string

False

Code for cluster state.

message

string

False

Message for cluster state.

StorageInfo

Contains information about the storage volumes that are attached to the brokers.

PropertyTypeRequiredDescription
ebsStorageInfo

EBSStorageInfo

False

EBS volume information.

StorageMode

Controls storage mode for various supported storage tiers.

  • LOCAL

  • TIERED

Tls

The details of client authentication using TLS.

PropertyTypeRequiredDescription
certificateAuthorityArnList

Array of type string

False

List of ACM CertificateAuthority ARNs.

enabled

boolean

False

Whether TLS authentication is turned on.

Unauthenticated

Details for allowing no client authentication.

PropertyTypeRequiredDescription
enabled

boolean

False

Unauthenticated is enabled or not.

VpcConfig

Includes information about subnets and security groups for a VPC.

PropertyTypeRequiredDescription
securityGroupIds

Array of type string

False

The security groups to attach to the ENIs for the broker nodes.

subnetIds

Array of type string

True

The list of subnets in the client VPC to connect to. Client subnets can't occupy the Availability Zone with ID use1-az3.

VpcConnectivity

VPC connection control settings for brokers

PropertyTypeRequiredDescription
clientAuthentication

VpcConnectivityClientAuthentication

False

VPC connection control settings for brokers

VpcConnectivityClientAuthentication

Includes all client authentication information for VpcConnectivity.

PropertyTypeRequiredDescription
sasl

VpcConnectivitySasl

False

Details for VpcConnectivity ClientAuthentication using SASL.

tls

VpcConnectivityTls

False

Details for VpcConnectivity ClientAuthentication using TLS.

VpcConnectivityIAM

Details for SASL/IAM client authentication for VpcConnectivity.

PropertyTypeRequiredDescription
enabled

boolean

False

SASL/IAM authentication is enabled or not.

VpcConnectivitySasl

Details for client authentication using SASL for VpcConnectivity.

PropertyTypeRequiredDescription
iam

VpcConnectivityIAM

False

Details for ClientAuthentication using IAM for VpcConnectivity.

scram

VpcConnectivityScram

False

Details for SASL/SCRAM client authentication for VpcConnectivity.

VpcConnectivityScram

Details for SASL/SCRAM client authentication for vpcConnectivity.

PropertyTypeRequiredDescription
enabled

boolean

False

SASL/SCRAM authentication is enabled or not.

VpcConnectivityTls

Details for client authentication using TLS for vpcConnectivity.

PropertyTypeRequiredDescription
enabled

boolean

False

TLS authentication is enabled or not.

See also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

ListClustersV2

CreateClusterV2