Step 6: Produce and Consume Data
In this step of Getting Started Using Amazon MSK, you produce and consume data.
To produce and consume messages
-
In this example we use the JVM truststore to talk to the MSK cluster. To do this, first create a folder named
/tmp
on the client machine. Then, go to thebin
folder of the Apache Kafka installation and run the following command, replacingJDKFolder
with the name of your JDK folder. For example, the name of the JDK folder on your instance might bejava-1.8.0-openjdk-1.8.0.201.b09-0.amzn2.x86_64
.cp /usr/lib/jvm/
JDKFolder
/jre/lib/security/cacerts /tmp/kafka.client.truststore.jks -
While still in the
bin
folder of the Apache Kafka installation on the client machine, create a text file namedclient.properties
with the following contents.security.protocol=SSL ssl.truststore.location=/tmp/kafka.client.truststore.jks
-
Run the following command, replacing
ClusterArn
with the Amazon Resource Name (ARN) that you obtained at the end of Step 3: Create an Amazon MSK Cluster.aws kafka get-bootstrap-brokers --region us-east-1 --cluster-arn
ClusterArn
From the JSON result of the command, save the value associated with the string named
"BootstrapBrokerStringTls"
because you need it in the following commands. -
Run the following command in the
bin
folder, replacingBootstrapBrokerStringTls
with the value that you obtained when you ran the previous command../kafka-console-producer.sh --broker-list
BootstrapBrokerStringTls
--producer.config client.properties --topic AWSKafkaTutorialTopic -
Enter any message that you want, and press Enter. Repeat this step two or three times. Every time you enter a line and press Enter, that line is sent to your Apache Kafka cluster as a separate message.
-
Keep the connection to the client machine open, and then open a second, separate connection to that machine in a new window.
-
In the following command, replace
BootstrapBrokerStringTls
with the value that you saved earlier. Then, go to thebin
folder and run the command using your second connection to the client machine../kafka-console-consumer.sh --bootstrap-server
BootstrapBrokerStringTls
--consumer.config client.properties --topic AWSKafkaTutorialTopic --from-beginningYou start seeing the messages you entered earlier when you used the console producer command. These messages are TLS encrypted in transit.
-
Enter more messages in the producer window, and watch them appear in the consumer window.
Next Step
Step 7: Use Amazon CloudWatch to View Amazon MSK Metrics