Monitoring Properties - Amazon Managed Streaming for Apache Kafka

Monitoring Properties

The options for monitoring an Amazon MSK cluster. You can specify which Apache Kafka metrics you want Amazon MSK to gather and send to Amazon CloudWatch. You can also configure open monitoring to gather metrics with Prometheus or Prometheus-compatible tools.

URI

/v1/clusters/clusterArn/monitoring

HTTP Methods

PUT

Operation ID: UpdateMonitoring

Updates the monitoring settings for the cluster. You can use this operation to specify which Apache Kafka metrics you want Amazon MSK to send to Amazon CloudWatch. You can also specify settings for open monitoring with Prometheus. The following Python 3.6 example enables open monitoring with the Node Exporter. It also sets enhanced monitoring to PER_BROKER. For more information about monitoring, see Monitoring.

import boto3 import time client = boto3.client('kafka') update_monitoring_response = client.update_monitoring( ClusterArn='arn:aws:kafka:us-east-1:0123456789019:cluster/SalesCluster/abcd1234-abcd-cafe-abab-9876543210ab-4', CurrentVersion='K12V3IB1VIZHHY', EnhancedMonitoring='PER_BROKER', OpenMonitoring={"Prometheus":{"JmxExporter":{"EnabledInBroker":False},"NodeExporter":{"EnabledInBroker":True}}} ) operation_arn = update_monitoring_response['ClusterOperationArn'] print('The ARN of the update operation is ' + operation_arn) describe_cluster_operation_response = client.describe_cluster_operation(ClusterOperationArn=operation_arn) operation_state = describe_cluster_operation_response['ClusterOperationInfo']['OperationState'] print('The status of the update operation is ' + operation_state) updated = False while not updated: print('Sleeping for 15 seconds before checking to see if the monitoring update is done...') time.sleep(15) describe_cluster_operation_response = client.describe_cluster_operation(ClusterOperationArn=operation_arn) operation_state = describe_cluster_operation_response['ClusterOperationInfo']['OperationState'] if 'UPDATE_COMPLETE' == operation_state: updated = True print('You have successfully updated the monitoring settings.')
Path Parameters
Name Type Required Description
clusterArn String True

The Amazon Resource Name (ARN) that uniquely identifies the cluster.

Responses
Status Code Response Model Description
200 UpdateMonitoringRequest

Successful response.

400 Error

The request isn't valid because the input is incorrect. Correct your input and then submit it again.

401 Error

The request is not authorized. The provided credentials couldn't be validated.

403 Error

Access forbidden. Check your credentials and then retry your request.

404 Error

The resource could not be found due to incorrect input. Correct the input, then retry the request.

429 Error

429 response

500 Error

There was an unexpected internal server error. Retrying your request might resolve the issue.

503 Error

503 response

Schemas

Request Bodies

Example PUT

{ "enhancedMonitoring": enum, "loggingInfo": { "brokerLogs": { "s3": { "bucket": "string", "prefix": "string", "enabled": boolean }, "firehose": { "deliveryStream": "string", "enabled": boolean }, "cloudWatchLogs": { "logGroup": "string", "enabled": boolean } } }, "openMonitoring": { "prometheus": { "nodeExporter": { "enabledInBroker": boolean }, "jmxExporter": { "enabledInBroker": boolean } } }, "currentVersion": "string" }

Response Bodies

Example UpdateMonitoringRequest

{ "clusterArn": "string", "clusterOperationArn": "string" }

Example Error

{ "message": "string", "invalidParameter": "string" }

Properties

BrokerLogs

The broker logs configuration for this MSK cluster.

Property Type Required Description
s3

S3

False

Details of the Amazon S3 destination for broker logs.

firehose

Firehose

False

Details of the Kinesis Data Firehose delivery stream that is the destination for broker logs.

cloudWatchLogs

CloudWatchLogs

False

Details of the CloudWatch Logs destination for broker logs.

CloudWatchLogs

Details of the CloudWatch Logs destination for broker logs.

Property Type Required Description
logGroup

string

False

The CloudWatch log group that is the destination for broker logs.

enabled

boolean

True

Specifies whether broker logs get sent to the specified CloudWatch Logs destination.

EnhancedMonitoring

Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster. This property has three possible values: DEFAULT, PER_BROKER, and PER_TOPIC_PER_BROKER. For a list of the metrics associated with each of these three levels of monitoring, see Monitoring.

  • DEFAULT

  • PER_BROKER

  • PER_TOPIC_PER_BROKER

Error

Returns information about an error.

Property Type Required Description
message

string

False

The description of the error.

invalidParameter

string

False

The parameter that caused the error.

Firehose

Firehose details for BrokerLogs.

Property Type Required Description
deliveryStream

string

False

The Kinesis Data Firehose delivery stream that is the destination for broker logs.

enabled

boolean

True

Specifies whether broker logs get send to the specified Kinesis Data Firehose delivery stream.

JmxExporterInfo

JMX Exporter details.

Property Type Required Description
enabledInBroker

boolean

True

JMX Exporter being enabled in broker.

LoggingInfo

You can configure your MSK cluster to send broker logs to different destination types. This is a container for the configuration details related to broker logs.

Property Type Required Description
brokerLogs

BrokerLogs

True

You can configure your MSK cluster to send broker logs to different destination types. This configuration specifies the details of these destinations.

NodeExporterInfo

Node Exporter details.

Property Type Required Description
enabledInBroker

boolean

True

Node Exporter being enabled in broker.

OpenMonitoringInfo

JMX and Node monitoring for cluster.

Property Type Required Description
prometheus

PrometheusInfo

True

Prometheus details.

PrometheusInfo

Prometheus details.

Property Type Required Description
nodeExporter

NodeExporterInfo

False

Node Exporter details.

jmxExporter

JmxExporterInfo

False

JMX Exporter details.

S3

The details of the Amazon S3 destination for broker logs.

Property Type Required Description
bucket

string

False

The name of the S3 bucket that is the destination for broker logs.

prefix

string

False

The S3 prefix that is the destination for broker logs.

enabled

boolean

True

Specifies whether broker logs get sent to the specified Amazon S3 destination.

UpdateMonitoringRequest

Request body for UpdateMonitoring.

Property Type Required Description
enhancedMonitoring

EnhancedMonitoring

False

Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster.

loggingInfo

LoggingInfo

False

LoggingInfo details.

openMonitoring

OpenMonitoringInfo

False

The settings for open monitoring.

currentVersion

string

True

The version of the MSK cluster to update. Cluster versions aren't simple numbers. You can describe an MSK cluster to find its version. When this update operation is successful, it generates a new cluster version.

UpdateMonitoringResponse

Request body for UpdateMonitoring.

Property Type Required Description
clusterArn

string

False

The Amazon Resource Name (ARN) of the cluster.

clusterOperationArn

string

False

The Amazon Resource Name (ARN) of the cluster operation.

See Also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

UpdateMonitoring