AWS Documentation » Amazon Managed Streaming for Apache Kafka » Developer Guide » Getting Started Using Amazon MSK » Step 5: Create a Topic

Step 5: Create a Topic

In this step of Getting Started Using Amazon MSK, you install Apache Kafka client libraries and tools on the client machine, and then you create a topic.

To create a topic on the client machine

1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

2. In the navigation pane, choose Instances, and then choose AWSKafkaTutorialClient by selecting the check box next to it.

3. Choose Actions, and then choose Connect. Follow the instructions to connect to the client machine AWSKafkaTutorialClient.

4. Install Java on the client machine by running the following command:

   sudo yum install java-1.8.0

5. Run the following command to download Apache Kafka.

   wget https://archive.apache.org/dist/kafka/2.2.1/kafka_2.12-2.2.1.tgz

   Note

   If you want to use a mirror site other than the one used in this command, you can choose a different one on the Apache website.

6. Run the following command in the directory where you downloaded the TAR file in the previous step.

   tar -xzf kafka_2.12-2.2.1.tgz

7. Go to the kafka_2.12-2.2.1 directory.

8. Cluster creation can take a few minutes. To find out whether the cluster you created is ready, run the following command, replacing ClusterArn with the Amazon Resource Name (ARN) that you obtained at the end of Step 3: Create an Amazon MSK Cluster.

   aws kafka describe-cluster --region us-east-1 --cluster-arn "ClusterArn"

   The result of running this command looks like the following JSON:

   {
       "ClusterInfo": {
           "BrokerNodeGroupInfo": {
               "BrokerAZDistribution": "DEFAULT",
               "ClientSubnets": [
                   "subnet-0d44a1567c2ce409a",
                   "subnet-051201cac65561565",
                   "subnet-08b4eceb2bd3bd8c2"
               ],
               "InstanceType": "kafka.m5.large",
               "SecurityGroups": [
                   "sg-041e78b0a8ba7f834"
               ],
               "StorageInfo": {
                   "EbsStorageInfo": {
                       "VolumeSize": 1000
                   }
               }
           },
           "ClusterArn": "...",
           "ClusterName": "AWSKafkaTutorialCluster",
           "CreationTime": "2018-11-06T01:36:57.451Z",
           "CurrentBrokerSoftwareInfo": {
               "KafkaVersion": "2.2.1"
           },
           "CurrentVersion": "K3UN6WX5RRO2AG",
           "EncryptionInfo": {
               "EncryptionAtRest": {
                   "DataVolumeKMSKeyId": "arn:aws:kms:us-east-1:012345678901:key/a7de6539-7d2e-4e71-a279-aaaa5555878"
               }
           },
           "EnhancedMonitoring": "DEFAULT",
           "NumberOfBrokerNodes": 3,
           "State": "CREATING"
       }
   }

   If the output of the command shows that the state of the cluster is still CREATING, wait a few minutes, and then run the command again. Keep running this command every few minutes until the state turns to ACTIVE. When the state is ACTIVE, the result of this describe-cluster command includes an additional key named ZooKeeperConnectionString. Copy the entire value associated with this key because you need it to create an Apache Kafka topic in the following command.

9. Run the following command, replacing ZooKeeperConnectionString with the value that you saved after you ran the describe-cluster command.

   bin/kafka-topics.sh --create --zookeeper ZooKeeperConnectionString --replication-factor 3 --partitions 1 --topic AWSKafkaTutorialTopic

   If the command succeeds, you see the following message: Created topic AWSKafkaTutorialTopic.

Next Step

Step 6: Produce and Consume Data