Cluster - Amazon Managed Streaming for Apache Kafka

Cluster

Represents an Amazon MSK cluster.

URI

/v1/clusters/clusterArn

HTTP Methods

GET

Operation ID: DescribeCluster

Returns a description of the MSK cluster whose Amazon Resource Name (ARN) is specified in the request. The following is a Python 3.6 example of how to use this operation. Before you run this Python script, replace the example cluster ARN with the ARN of the cluster you want to describe. If you don't know the ARN of the cluster, you can use the ListClusters operation to list all the clusters and see their ARNs and full descriptions.

import boto3 client = boto3.client('kafka') response = client.describe_cluster( ClusterArn='arn:aws:kafka:us-east-1:0123456789019:cluster/SalesCluster/abcd1234-abcd-cafe-abab-9876543210ab-4' ) print(response)
Path Parameters
Name Type Required Description
clusterArn String True

The Amazon Resource Name (ARN) that uniquely identifies the cluster.

Responses
Status Code Response Model Description
200 DescribeClusterResponse

Successful response.

400 Error

The request isn't valid because the input is incorrect. Correct your input and then submit it again.

401 Error

The request is not authorized. The provided credentials couldn't be validated.

403 Error

Access forbidden. Check your credentials and then retry your request.

404 Error

The resource could not be found due to incorrect input. Correct the input, then retry the request.

429 Error

429 response

500 Error

There was an unexpected internal server error. Retrying your request might resolve the issue.

503 Error

503 response

DELETE

Operation ID: DeleteCluster

Deletes the MSK cluster specified by the Amazon Resource Name (ARN) in the request.

Path Parameters
Name Type Required Description
clusterArn String True

The Amazon Resource Name (ARN) that uniquely identifies the cluster.

Query Parameters
Name Type Required Description
currentVersion String False

The current version of the MSK cluster.

Responses
Status Code Response Model Description
200 DeleteClusterResponse

Successful response.

400 Error

The request isn't valid because the input is incorrect. Correct your input and then submit it again.

401 Error

The request is not authorized. The provided credentials couldn't be validated.

403 Error

Access forbidden. Check your credentials and then retry your request.

404 Error

The resource could not be found due to incorrect input. Correct the input, then retry the request.

429 Error

429 response

500 Error

There was an unexpected internal server error. Retrying your request might resolve the issue.

503 Error

503 response

Schemas

Response Bodies

Example DescribeClusterResponse

{ "clusterInfo": { "encryptionInfo": { "encryptionInTransit": { "inCluster": boolean, "clientBroker": enum }, "encryptionAtRest": { "dataVolumeKMSKeyId": "string" } }, "zookeeperConnectString": "string", "creationTime": "string", "loggingInfo": { "brokerLogs": { "s3": { "bucket": "string", "prefix": "string", "enabled": boolean }, "firehose": { "deliveryStream": "string", "enabled": boolean }, "cloudWatchLogs": { "logGroup": "string", "enabled": boolean } } }, "currentVersion": "string", "tags": { }, "numberOfBrokerNodes": integer, "clusterArn": "string", "activeOperationArn": "string", "enhancedMonitoring": enum, "clusterName": "string", "stateInfo": { "code": "string", "message": "string" }, "clientAuthentication": { "tls": { "certificateAuthorityArnList": [ "string" ] } }, "state": enum, "brokerNodeGroupInfo": { "clientSubnets": [ "string" ], "instanceType": "string", "securityGroups": [ "string" ], "brokerAZDistribution": enum, "storageInfo": { "ebsStorageInfo": { "volumeSize": integer } } }, "openMonitoring": { "prometheus": { "nodeExporter": { "enabledInBroker": boolean }, "jmxExporter": { "enabledInBroker": boolean } } }, "currentBrokerSoftwareInfo": { "configurationRevision": integer, "kafkaVersion": "string", "configurationArn": "string" } } }

Example DeleteClusterResponse

{ "clusterArn": "string", "state": enum }

Example Error

{ "message": "string", "invalidParameter": "string" }

Properties

BrokerAZDistribution

The distribution of broker nodes across Availability Zones. This is an optional parameter. If you don't specify it, Amazon MSK gives it the value DEFAULT. You can also explicitly set this parameter to the value DEFAULT. No other values are currently allowed.

Amazon MSK distributes the broker nodes evenly across the Availability Zones that correspond to the subnets that you provide when you create the cluster.

To create a cluster, specify exactly two subnets if you are using one of the following Regions: South America (São Paulo), Canada (Central), or US West (N. California). For other Regions where Amazon MSK is available, you can specify either two or three subnets. The subnets that you specify must be in distinct Availability Zones.

Client subnets can't be in Availability Zone us-east-1e.

  • DEFAULT

BrokerLogs

The broker logs configuration for this MSK cluster.

Property Type Required Description
s3

S3

False

Details of the Amazon S3 destination for broker logs.

firehose

Firehose

False

Details of the Kinesis Data Firehose delivery stream that is the destination for broker logs.

cloudWatchLogs

CloudWatchLogs

False

Details of the CloudWatch Logs destination for broker logs.

BrokerNodeGroupInfo

Describes the setup to be used for the broker nodes in the cluster.

Property Type Required Description
clientSubnets

Array of type string

True

The list of subnets to connect to in the client virtual private cloud (VPC). AWS creates elastic network interfaces inside these subnets. Client applications use elastic network interfaces to produce and consume data.

Specify exactly two subnets if you are using the US West (N. California) Region. For other Regions where Amazon MSK is available, you can specify either two or three subnets. The subnets that you specify must be in distinct Availability Zones. When you create a cluster, Amazon MSK distributes the broker nodes evenly across the subnets that you specify.

Client subnets can't be in Availability Zone us-east-1e.

instanceType

string

MinLength: 5

MaxLength: 32

True

The type of Amazon EC2 instances to use for brokers. The following instance types are allowed: kafka.m5.large, kafka.m5.xlarge, kafka.m5.2xlarge, kafka.m5.4xlarge, kafka.m5.8xlarge, kafka.m5.12xlarge, kafka.m5.16xlarge, and kafka.m5.24xlarge.

securityGroups

Array of type string

False

The AWS security groups to associate with the elastic network interfaces in order to specify who can connect to and communicate with the Amazon MSK cluster. If you don't specify a security group, Amazon MSK uses the default security group associated with the VPC. If you specify security groups that were shared with you, you must ensure that you have permissions to them. Specifically, you need the ec2:DescribeSecurityGroups permission.

brokerAZDistribution

BrokerAZDistribution

False

The distribution of broker nodes across Availability Zones. This is an optional parameter. If you don't specify it, Amazon MSK gives it the value DEFAULT. You can also explicitly set this parameter to the value DEFAULT. No other values are currently allowed.

Amazon MSK distributes the broker nodes evenly across the Availability Zones that correspond to the subnets that you provide when you create the cluster.

To create a cluster, specify exactly two subnets if you are using one of the following Regions: South America (São Paulo), Canada (Central), or US West (N. California). For other Regions where Amazon MSK is available, you can specify either two or three subnets. The subnets that you specify must be in distinct Availability Zones.

Client subnets can't be in Availability Zone us-east-1e.

storageInfo

StorageInfo

False

Contains information about storage volumes attached to MSK broker nodes.

BrokerSoftwareInfo

Information about the current software installed on the cluster.

Property Type Required Description
configurationRevision

integer

Format: int64

False

The revision of the configuration to use. This field isn't visible in this preview release.

kafkaVersion

string

False

The version of Apache Kafka. You can use Amazon MSK to create clusters that use Apache Kafka versions 1.1.1 and 2.2.1.

configurationArn

string

False

The Amazon Resource Name (ARN) of the configuration used for the cluster. This field isn't visible in this preview release.

ClientAuthentication

Includes all client authentication information.

Property Type Required Description
tls

Tls

False

Details for ClientAuthentication using TLS.

ClientBroker

Client-broker encryption in transit setting.

  • TLS

  • TLS_PLAINTEXT

  • PLAINTEXT

CloudWatchLogs

Details of the CloudWatch Logs destination for broker logs.

Property Type Required Description
logGroup

string

False

The CloudWatch log group that is the destination for broker logs.

enabled

boolean

True

Specifies whether broker logs get sent to the specified CloudWatch Logs destination.

ClusterInfo

Returns information about a cluster.

Property Type Required Description
encryptionInfo

EncryptionInfo

False

Includes all encryption-related information.

zookeeperConnectString

string

False

The connection string to use to connect to the Apache ZooKeeper cluster.

creationTime

string

False

The time when the cluster was created.

loggingInfo

LoggingInfo

False

You can configure your MSK cluster to send broker logs to different destination types. This is a container for the configuration details related to broker logs.

currentVersion

string

False

The current version of the MSK cluster. Cluster versions aren't simple integers. You can obtain the current version by describing the cluster. An example version is KTVPDKIKX0DER.

tags

object

False

Tags attached to the cluster.

numberOfBrokerNodes

integer

False

The number of broker nodes in the cluster.

clusterArn

string

False

The Amazon Resource Name (ARN) that uniquely identifies the cluster.

activeOperationArn

string

False

Arn of active cluster operation.

enhancedMonitoring

EnhancedMonitoring

False

Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster. This property has three possible values: DEFAULT, PER_BROKER, and PER_TOPIC_PER_BROKER. For a list of the metrics associated with each of these three levels of monitoring, see Monitoring.

clusterName

string

False

The name of the cluster.

stateInfo

StateInfo

False

Includes information of the cluster state.

clientAuthentication

ClientAuthentication

False

Includes all client authentication information.

state

ClusterState

False

The state of the cluster. Amazon MSK automatically renews certificates on clusters every 13 months. It sets the state of the cluster to MAINTENANCE when it starts the certificate-update operation. It sets it back to ACTIVE when the update is done. While a cluster is in the MAINTENANCE state, you can continue to produce and consume data, but you can't perform any update operations on it. You can perform update operations on a cluster when it is in the ACTIVE state.

brokerNodeGroupInfo

BrokerNodeGroupInfo

False

Information about the broker nodes.

openMonitoring

OpenMonitoring

False

Settings for open monitoring using Prometheus.

currentBrokerSoftwareInfo

BrokerSoftwareInfo

False

Information about the version of software currently deployed on the brokers in the cluster.

ClusterState

The state of the cluster. Amazon MSK automatically renews certificates on clusters every 13 months. It sets the state of the cluster to MAINTENANCE when it starts the certificate-update operation. It sets it back to ACTIVE when the update is done. While a cluster is in the MAINTENANCE state, you can continue to produce and consume data, but you can't perform any update operations on it. You can perform update operations on a cluster when it is in the ACTIVE state.

  • ACTIVE

  • CREATING

  • UPDATING

  • DELETING

  • FAILED

  • MAINTENANCE

DeleteClusterResponse

Returns information about the deleted cluster.

Property Type Required Description
clusterArn

string

False

The Amazon Resource Name (ARN) of the cluster.

state

ClusterState

False

The state of the cluster. Amazon MSK automatically renews certificates on clusters every 13 months. It sets the state of the cluster to MAINTENANCE when it starts the certificate-update operation. It sets it back to ACTIVE when the update is done. While a cluster is in the MAINTENANCE state, you can continue to produce and consume data, but you can't perform any update operations on it. You can perform update operations on a cluster when it is in the ACTIVE state.

DescribeClusterResponse

Returns information about a cluster.

Property Type Required Description
clusterInfo

ClusterInfo

False

The cluster information.

EBSStorageInfo

Contains information about the EBS storage volumes attached to the broker nodes.

Property Type Required Description
volumeSize

integer

Minimum: 1

Maximum: 16384

False

The size in GiB of the EBS volume for the data drive on each broker node.

EncryptionAtRest

The data-volume encryption details.

Property Type Required Description
dataVolumeKMSKeyId

string

True

The ARN of the AWS KMS key for encrypting data at rest. If you don't specify a KMS key, MSK creates one for you and uses it.

EncryptionInTransit

The settings for encrypting data in transit.

Property Type Required Description
inCluster

boolean

False

When set to true, it indicates that data communication among the broker nodes of the cluster is encrypted. When set to false, the communication happens in plaintext.

The default value is true.

clientBroker

ClientBroker

False

Indicates the encryption setting for data in transit between clients and brokers. You must set it to one of the following values.

TLS means that client-broker communication is enabled with TLS only.

TLS_PLAINTEXT means that client-broker communication is enabled for both TLS-encrypted, as well as plaintext data.

PLAINTEXT means that client-broker communication is enabled in plaintext only.

The default value is TLS.

EncryptionInfo

Includes encryption-related information, such as the AWS KMS key used for encrypting data at rest and whether you want MSK to encrypt your data in transit.

Property Type Required Description
encryptionInTransit

EncryptionInTransit

False

The details for encryption in transit.

encryptionAtRest

EncryptionAtRest

False

The data-volume encryption details.

EnhancedMonitoring

Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster. This property has three possible values: DEFAULT, PER_BROKER, and PER_TOPIC_PER_BROKER. For a list of the metrics associated with each of these three levels of monitoring, see Monitoring.

  • DEFAULT

  • PER_BROKER

  • PER_TOPIC_PER_BROKER

Error

Returns information about an error.

Property Type Required Description
message

string

False

The description of the error.

invalidParameter

string

False

The parameter that caused the error.

Firehose

Firehose details for BrokerLogs.

Property Type Required Description
deliveryStream

string

False

The Kinesis Data Firehose delivery stream that is the destination for broker logs.

enabled

boolean

True

Specifies whether broker logs get send to the specified Kinesis Data Firehose delivery stream.

JmxExporter

Indicates whether you want to enable or disable the JMX Exporter.

Property Type Required Description
enabledInBroker

boolean

True

Indicates whether you want to enable or disable the JMX Exporter.

LoggingInfo

You can configure your MSK cluster to send broker logs to different destination types. This is a container for the configuration details related to broker logs.

Property Type Required Description
brokerLogs

BrokerLogs

True

You can configure your MSK cluster to send broker logs to different destination types. This configuration specifies the details of these destinations.

NodeExporter

Indicates whether you want to enable or disable the Node Exporter.

Property Type Required Description
enabledInBroker

boolean

True

Indicates whether you want to enable or disable the Node Exporter.

OpenMonitoring

JMX and Node monitoring for the MSK cluster.

Property Type Required Description
prometheus

Prometheus

True

Prometheus exporter settings.

Prometheus

Prometheus settings for open monitoring.

Property Type Required Description
nodeExporter

NodeExporter

False

Indicates whether you want to enable or disable the Node Exporter.

jmxExporter

JmxExporter

False

Indicates whether you want to enable or disable the JMX Exporter.

S3

The details of the Amazon S3 destination for broker logs.

Property Type Required Description
bucket

string

False

The name of the S3 bucket that is the destination for broker logs.

prefix

string

False

The S3 prefix that is the destination for broker logs.

enabled

boolean

True

Specifies whether broker logs get sent to the specified Amazon S3 destination.

StateInfo

Includes information about the state of the cluster.

Property Type Required Description
code

string

False

If the cluster is in an unusable state, this field contains the code that describes the issue.

message

string

False

If the cluster is in an unusable state, this field contains a message that describes the issue.

StorageInfo

Contains information about storage volumes attached to MSK broker nodes.

Property Type Required Description
ebsStorageInfo

EBSStorageInfo

False

EBS volume information.

Tls

Details for client authentication using TLS.

Property Type Required Description
certificateAuthorityArnList

Array of type string

False

List of ACM Certificate Authority ARNs.

See Also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

DescribeCluster

DeleteCluster