Step 4: Create a topic - Amazon Managed Streaming for Apache Kafka

Step 4: Create a topic

In this step of Getting Started Using Amazon MSK, you install Apache Kafka client libraries and tools on the client machine, and then you create a topic.


Apache Kafka version numbers used in this tutorial are examples only. When you need to connect to your Amazon MSK cluster using the Apache Kafka client, to create or change a topic configuration for example, ensure that the Apache Kafka client version you're using matches your Amazon MSK cluster version. Be aware that using an Apache Kafka client version that is not the same as your MSK cluster version may lead to Apache Kafka data corruption, loss and down time.

To find the version of your MSK cluster
  1. Go to

  2. Select the MSK cluster.

  3. Note the version of Apache Kafka used on the cluster.

  4. Replace instances of Amazon MSK version numbers in this tutorial with the version obtained in Step 3.

To create a topic on the client machine
  1. Open the Amazon EC2 console at

  2. In the navigation pane, choose Instances. Then select the check box beside the name of the client machine that you created in Step 3: Create a client machine.

  3. Choose Actions, and then choose Connect. Follow the instructions in the console to connect to your client machine.

  4. Install Java on the client machine by running the following command:

    sudo yum -y install java-11
  5. Run the following command to download Apache Kafka.


    If you want to use a mirror site other than the one used in this command, you can choose a different one on the Apache website.

  6. Run the following command in the directory where you downloaded the TAR file in the previous step.

    tar -xzf kafka_2.12-2.8.1.tgz
  7. Go to the kafka_2.12-2.8.1/libs directory, then run the following command to download the Amazon MSK IAM JAR file. The Amazon MSK IAM JAR makes it possible for the client machine to access the cluster.

  8. Go to the kafka_2.12-2.8.1/bin directory. Copy the following property settings and paste them into a new file. Name the file and save it.

    security.protocol=SASL_SSL sasl.mechanism=AWS_MSK_IAM required;
  9. Open the Amazon MSK console at

  10. Wait for the status of your cluster to become Active. This might take several minutes. After the status becomes Active, choose the cluster name. This takes you to a page containing the cluster summary.

  11. Choose View client information.

  12. Copy the connection string for the private endpoint.

    You will get three endpoints for each of the brokers. You only need one broker endpoint for the following step.

  13. Run the following command, replacing BootstrapServerString with one of the broker endpoints that you obtained in the previous step.

    <path-to-your-kafka-installation>/bin/ --create --bootstrap-server BootstrapServerString --command-config --replication-factor 3 --partitions 1 --topic MSKTutorialTopic

    If the command succeeds, you see the following message: Created topic MSKTutorialTopic.

Next Step

Step 5: Produce and consume data