AWS Documentation » Amazon Managed Streaming for Apache Kafka » Developer Guide » Getting Started Using Amazon MSK » Step 3: Create an Amazon MSK Cluster

Step 3: Create an Amazon MSK Cluster

In this step of Getting Started Using Amazon MSK, you create an Amazon MSK cluster in the VPC.

To create an Amazon MSK cluster using the AWS CLI

1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

2. In the navigation pane, choose Subnets, and then copy the subnet IDs of the three subnets you created previously.

3. In the navigation pane, choose Security Groups. Then in the table of security groups, find the group for which the VPC ID column has the ID you saved for AWSKafkaTutorialVPC. Copy the ID of this security group because you need it in the next step.

4. Copy the following JSON and save it to a file. Name the file brokernodegroupinfo.json. Replace the three subnet IDs and the security group ID in the JSON with the values that you saved in previous steps. Then save the updated JSON file on the computer where you have the AWS CLI installed.

   {
     "InstanceType": "kafka.m5.large",
     "ClientSubnets": [
       "AWSKafkaTutorialSubnet-1 Subnet ID",
       "AWSKafkaTutorialSubnet-2 Subnet ID",
       "AWSKafkaTutorialSubnet-3 Subnet ID"
     ],
     "SecurityGroups": [
       "AWSKafkaTutorialVPC Security Group ID"
     ]
   }

5. Copy the following JSON and save it to a file. Name the file encryptioninfo.json. Replace your-CMK with a customer managed CMK. You can also remove EncryptionAtRest and let Amazon MSK create a CMK and use it on your behalf. Setting inCluster to true means that you want Amazon MSK to encrypt your data as it travels between brokers within the cluster. For ClientBroker you can choose one of the following settings: TLS, TLS_PLAINTEXT, or PLAINTEXT. In this exercise, we use TLS to indicate that we want data to be encrypted as it travels between clients and brokers. For more information about encryption settings, see Amazon MSK Encryption.

   {
      "EncryptionAtRest": {
          "DataVolumeKMSKeyId": "your-CMK"
       },
      "EncryptionInTransit": {
           "InCluster": true,
           "ClientBroker": "TLS"
       }
   }

6. Upgrade your AWS CLI to the latest version to ensure that it has support for Amazon MSK. For detailed instructions on how to upgrade the AWS CLI, see Installing the AWS Command Line Interface.

7. Run the following AWS CLI command in the directory where you saved the brokernodegroupinfo.json and encryptioninfo.json files.

   aws kafka create-cluster --cluster-name "AWSKafkaTutorialCluster" --broker-node-group-info file://brokernodegroupinfo.json --encryption-info file://encryptioninfo.json --kafka-version "2.2.1" --number-of-broker-nodes 3 --enhanced-monitoring PER_TOPIC_PER_BROKER --region us-east-1

   The output of the command looks like the following JSON:

   {
       "ClusterArn": "...",
       "ClusterName": "AWSKafkaTutorialCluster",
       "State": "CREATING"
   }

8. Save the value of the ClusterArn key because you need it later.

Important

Ensure that you saved ClusterArn before you proceed.

Next Step

Step 4: Create a Client Machine