Namespace Amazon.CDK.AWS.MSK
Amazon Managed Streaming for Apache Kafka Construct Library
---AWS CDK v1 has reached End-of-Support on 2023-06-01.
This package is no longer being updated, and users should migrate to AWS CDK v2.
For more information on how to migrate, see the Migrating to AWS CDK v2 guide.
Amazon MSK is a fully managed service that makes it easy for you to build and run applications that use Apache Kafka to process streaming data.
The following example creates an MSK Cluster.
Vpc vpc;
var cluster = new Cluster(this, "Cluster", new ClusterProps {
ClusterName = "myCluster",
KafkaVersion = KafkaVersion.V2_8_1,
Vpc = vpc
});
Allowing Connections
To control who can access the Cluster, use the .connections
attribute. For a list of ports used by MSK, refer to the MSK documentation.
Vpc vpc;
var cluster = new Cluster(this, "Cluster", new ClusterProps {
ClusterName = "myCluster",
KafkaVersion = KafkaVersion.V2_8_1,
Vpc = vpc
});
cluster.Connections.AllowFrom(Peer.Ipv4("1.2.3.4/8"), Port.Tcp(2181));
cluster.Connections.AllowFrom(Peer.Ipv4("1.2.3.4/8"), Port.Tcp(9094));
Cluster Endpoints
You can use the following attributes to get a list of the Kafka broker or ZooKeeper node endpoints
Cluster cluster;
new CfnOutput(this, "BootstrapBrokers", new CfnOutputProps { Value = cluster.BootstrapBrokers });
new CfnOutput(this, "BootstrapBrokersTls", new CfnOutputProps { Value = cluster.BootstrapBrokersTls });
new CfnOutput(this, "BootstrapBrokersSaslScram", new CfnOutputProps { Value = cluster.BootstrapBrokersSaslScram });
new CfnOutput(this, "ZookeeperConnection", new CfnOutputProps { Value = cluster.ZookeeperConnectionString });
new CfnOutput(this, "ZookeeperConnectionTls", new CfnOutputProps { Value = cluster.ZookeeperConnectionStringTls });
Importing an existing Cluster
To import an existing MSK cluster into your CDK app use the .fromClusterArn()
method.
var cluster = Cluster.FromClusterArn(this, "Cluster", "arn:aws:kafka:us-west-2:1234567890:cluster/a-cluster/11111111-1111-1111-1111-111111111111-1");
Client Authentication
MSK supports the following authentication mechanisms.
Only one authentication method can be enabled.
TLS
To enable client authentication with TLS set the certificateAuthorityArns
property to reference your ACM Private CA. More info on Private CAs.
using Amazon.CDK.AWS.ACMPCA;
Vpc vpc;
var cluster = new Cluster(this, "Cluster", new ClusterProps {
ClusterName = "myCluster",
KafkaVersion = KafkaVersion.V2_8_1,
Vpc = vpc,
EncryptionInTransit = new EncryptionInTransitConfig {
ClientBroker = ClientBrokerEncryption.TLS
},
ClientAuthentication = ClientAuthentication.Tls(new TlsAuthProps {
CertificateAuthorities = new [] { CertificateAuthority.FromCertificateAuthorityArn(this, "CertificateAuthority", "arn:aws:acm-pca:us-west-2:1234567890:certificate-authority/11111111-1111-1111-1111-111111111111") }
})
});
SASL/SCRAM
Enable client authentication with SASL/SCRAM:
Vpc vpc;
var cluster = new Cluster(this, "cluster", new ClusterProps {
ClusterName = "myCluster",
KafkaVersion = KafkaVersion.V2_8_1,
Vpc = vpc,
EncryptionInTransit = new EncryptionInTransitConfig {
ClientBroker = ClientBrokerEncryption.TLS
},
ClientAuthentication = ClientAuthentication.Sasl(new SaslAuthProps {
Scram = true
})
});
SASL/IAM
Enable client authentication with IAM:
Vpc vpc;
var cluster = new Cluster(this, "cluster", new ClusterProps {
ClusterName = "myCluster",
KafkaVersion = KafkaVersion.V2_8_1,
Vpc = vpc,
EncryptionInTransit = new EncryptionInTransitConfig {
ClientBroker = ClientBrokerEncryption.TLS
},
ClientAuthentication = ClientAuthentication.Sasl(new SaslAuthProps {
Iam = true
})
});
Classes
BrokerLogging | (experimental) Configuration details related to broker logs. |
CfnBatchScramSecret | A CloudFormation |
CfnBatchScramSecretProps | Properties for defining a |
CfnCluster | A CloudFormation |
CfnCluster.BrokerLogsProperty | The broker logs configuration for this MSK cluster. |
CfnCluster.BrokerNodeGroupInfoProperty | Describes the setup to be used for the broker nodes in the cluster. |
CfnCluster.ClientAuthenticationProperty | Includes all client authentication information. |
CfnCluster.CloudWatchLogsProperty | Details of the CloudWatch Logs destination for broker logs. |
CfnCluster.ConfigurationInfoProperty | Specifies the configuration to use for the brokers. |
CfnCluster.ConnectivityInfoProperty | Broker access controls. |
CfnCluster.EBSStorageInfoProperty | Contains information about the EBS storage volumes attached to the broker nodes. |
CfnCluster.EncryptionAtRestProperty | The data-volume encryption details. |
CfnCluster.EncryptionInfoProperty | Includes encryption-related information, such as the Amazon KMS key used for encrypting data at rest and whether you want MSK to encrypt your data in transit. |
CfnCluster.EncryptionInTransitProperty | The settings for encrypting data in transit. |
CfnCluster.FirehoseProperty | Firehose details for BrokerLogs. |
CfnCluster.IamProperty | Details for SASL/IAM client authentication. |
CfnCluster.JmxExporterProperty | Indicates whether you want to enable or disable the JMX Exporter. |
CfnCluster.LoggingInfoProperty | You can configure your MSK cluster to send broker logs to different destination types. |
CfnCluster.NodeExporterProperty | Indicates whether you want to enable or disable the Node Exporter. |
CfnCluster.OpenMonitoringProperty | JMX and Node monitoring for the MSK cluster. |
CfnCluster.PrometheusProperty | Prometheus settings for open monitoring. |
CfnCluster.ProvisionedThroughputProperty | Contains information about provisioned throughput for EBS storage volumes attached to kafka broker nodes. |
CfnCluster.PublicAccessProperty | Broker access controls. |
CfnCluster.S3Property | The details of the Amazon S3 destination for broker logs. |
CfnCluster.SaslProperty | Details for client authentication using SASL. |
CfnCluster.ScramProperty | Details for SASL/SCRAM client authentication. |
CfnCluster.StorageInfoProperty | Contains information about storage volumes attached to Amazon MSK broker nodes. |
CfnCluster.TlsProperty | Details for client authentication using TLS. |
CfnCluster.UnauthenticatedProperty | Details for allowing no client authentication. |
CfnCluster.VpcConnectivityClientAuthenticationProperty | Includes all client authentication information for VpcConnectivity. |
CfnCluster.VpcConnectivityIamProperty | Details for SASL/IAM client authentication for VpcConnectivity. |
CfnCluster.VpcConnectivityProperty | VPC connection control settings for brokers. |
CfnCluster.VpcConnectivitySaslProperty | Details for client authentication using SASL for VpcConnectivity. |
CfnCluster.VpcConnectivityScramProperty | Details for SASL/SCRAM client authentication for vpcConnectivity. |
CfnCluster.VpcConnectivityTlsProperty | Details for client authentication using TLS for vpcConnectivity. |
CfnClusterPolicy | A CloudFormation |
CfnClusterPolicyProps | Properties for defining a |
CfnClusterProps | Properties for defining a |
CfnConfiguration | A CloudFormation |
CfnConfigurationProps | Properties for defining a |
CfnServerlessCluster | A CloudFormation |
CfnServerlessCluster.ClientAuthenticationProperty | Includes all client authentication information. |
CfnServerlessCluster.IamProperty | Details for SASL/IAM client authentication. |
CfnServerlessCluster.SaslProperty | Details for client authentication using SASL. |
CfnServerlessCluster.VpcConfigProperty | |
CfnServerlessClusterProps | Properties for defining a |
CfnVpcConnection | A CloudFormation |
CfnVpcConnectionProps | Properties for defining a |
ClientAuthentication | (experimental) Configuration properties for client authentication. |
ClientBrokerEncryption | (experimental) Indicates the encryption setting for data in transit between clients and brokers. |
Cluster | (experimental) Create a MSK Cluster. |
ClusterConfigurationInfo | (experimental) The Amazon MSK configuration to use for the cluster. |
ClusterMonitoringLevel | (experimental) The level of monitoring for the MSK cluster. |
ClusterProps | (experimental) Properties for a MSK Cluster. |
EbsStorageInfo | (experimental) EBS volume information. |
EncryptionInTransitConfig | (experimental) The settings for encrypting data in transit. |
KafkaVersion | (experimental) Kafka cluster version. |
MonitoringConfiguration | (experimental) Monitoring Configuration. |
S3LoggingConfiguration | (experimental) Details of the Amazon S3 destination for broker logs. |
SaslAuthProps | (experimental) SASL authentication properties. |
TlsAuthProps | (experimental) TLS authentication properties. |
Interfaces
CfnCluster.IBrokerLogsProperty | The broker logs configuration for this MSK cluster. |
CfnCluster.IBrokerNodeGroupInfoProperty | Describes the setup to be used for the broker nodes in the cluster. |
CfnCluster.IClientAuthenticationProperty | Includes all client authentication information. |
CfnCluster.ICloudWatchLogsProperty | Details of the CloudWatch Logs destination for broker logs. |
CfnCluster.IConfigurationInfoProperty | Specifies the configuration to use for the brokers. |
CfnCluster.IConnectivityInfoProperty | Broker access controls. |
CfnCluster.IEBSStorageInfoProperty | Contains information about the EBS storage volumes attached to the broker nodes. |
CfnCluster.IEncryptionAtRestProperty | The data-volume encryption details. |
CfnCluster.IEncryptionInfoProperty | Includes encryption-related information, such as the Amazon KMS key used for encrypting data at rest and whether you want MSK to encrypt your data in transit. |
CfnCluster.IEncryptionInTransitProperty | The settings for encrypting data in transit. |
CfnCluster.IFirehoseProperty | Firehose details for BrokerLogs. |
CfnCluster.IIamProperty | Details for SASL/IAM client authentication. |
CfnCluster.IJmxExporterProperty | Indicates whether you want to enable or disable the JMX Exporter. |
CfnCluster.ILoggingInfoProperty | You can configure your MSK cluster to send broker logs to different destination types. |
CfnCluster.INodeExporterProperty | Indicates whether you want to enable or disable the Node Exporter. |
CfnCluster.IOpenMonitoringProperty | JMX and Node monitoring for the MSK cluster. |
CfnCluster.IPrometheusProperty | Prometheus settings for open monitoring. |
CfnCluster.IProvisionedThroughputProperty | Contains information about provisioned throughput for EBS storage volumes attached to kafka broker nodes. |
CfnCluster.IPublicAccessProperty | Broker access controls. |
CfnCluster.IS3Property | The details of the Amazon S3 destination for broker logs. |
CfnCluster.ISaslProperty | Details for client authentication using SASL. |
CfnCluster.IScramProperty | Details for SASL/SCRAM client authentication. |
CfnCluster.IStorageInfoProperty | Contains information about storage volumes attached to Amazon MSK broker nodes. |
CfnCluster.ITlsProperty | Details for client authentication using TLS. |
CfnCluster.IUnauthenticatedProperty | Details for allowing no client authentication. |
CfnCluster.IVpcConnectivityClientAuthenticationProperty | Includes all client authentication information for VpcConnectivity. |
CfnCluster.IVpcConnectivityIamProperty | Details for SASL/IAM client authentication for VpcConnectivity. |
CfnCluster.IVpcConnectivityProperty | VPC connection control settings for brokers. |
CfnCluster.IVpcConnectivitySaslProperty | Details for client authentication using SASL for VpcConnectivity. |
CfnCluster.IVpcConnectivityScramProperty | Details for SASL/SCRAM client authentication for vpcConnectivity. |
CfnCluster.IVpcConnectivityTlsProperty | Details for client authentication using TLS for vpcConnectivity. |
CfnServerlessCluster.IClientAuthenticationProperty | Includes all client authentication information. |
CfnServerlessCluster.IIamProperty | Details for SASL/IAM client authentication. |
CfnServerlessCluster.ISaslProperty | Details for client authentication using SASL. |
CfnServerlessCluster.IVpcConfigProperty | |
IBrokerLogging | (experimental) Configuration details related to broker logs. |
ICfnBatchScramSecretProps | Properties for defining a |
ICfnClusterPolicyProps | Properties for defining a |
ICfnClusterProps | Properties for defining a |
ICfnConfigurationProps | Properties for defining a |
ICfnServerlessClusterProps | Properties for defining a |
ICfnVpcConnectionProps | Properties for defining a |
ICluster | (experimental) Represents a MSK Cluster. |
IClusterConfigurationInfo | (experimental) The Amazon MSK configuration to use for the cluster. |
IClusterProps | (experimental) Properties for a MSK Cluster. |
IEbsStorageInfo | (experimental) EBS volume information. |
IEncryptionInTransitConfig | (experimental) The settings for encrypting data in transit. |
IMonitoringConfiguration | (experimental) Monitoring Configuration. |
IS3LoggingConfiguration | (experimental) Details of the Amazon S3 destination for broker logs. |
ISaslAuthProps | (experimental) SASL authentication properties. |
ITlsAuthProps | (experimental) TLS authentication properties. |