Create a Channel

Using the AWS Management Console

1. Open the Amazon MSK console at https://console.aws.amazon.com/msk/home?region=us-east-1#/home/.

2. In the navigation pane, choose Clusters.

3. Choose the name of your Amazon MSK Provisioned cluster with Express brokers.

4. Choose the Channel tab, then choose Create Channel.

5. In the Source configuration step:

   • Channel name: enter the name of the Channel to create.

   • Topic: select an existing topic. Create a new topic if one doesn't exist yet.

6. Proceed to the Destination configuration step.

7. Under Destination type, select data delivery to general purpose S3.

   • Select the Record format (JSON, Byte array, or String).

   • Under Destination settings:

     • Select Current account or Cross account. If Cross account was selected, you may need to enter an additional Account ID.

     • Select or enter an S3 bucket URI for the delivery bucket.

     • Enter an optional S3 bucket prefix and Output key template.

     • Choose the Storage class and Compression type (NONE, GZIP, or ZSTD).

   • Set the Data freshness (5–15 minutes).

8. Proceed to the Additional configuration step.

9. Under Encryption configuration, optionally select a customer-managed key to customize your encryption settings.

10. Under Service Access, select or create an IAM role with the required permissions.

11. Under Dead-letter queue, select Current account or Cross account and specify the S3 bucket URI for unprocessable records. Optionally enter an error output prefix. If Cross account was selected, you may need to enter an additional Account ID.

12. (Optional) Under Log Delivery, select an Amazon CloudWatch log group, Amazon S3 bucket, or Amazon Data Firehose stream to send logs for monitoring and troubleshooting.

13. (Optional) Add tags.

14. Proceed to the Review step.

15. Review and choose Create Channel.

The Channel enters the Creating state and transitions to Active once the delivery pipeline is fully provisioned.

Using the AWS CLI

The source topic is set in TopicConfigurationList (exactly one topic). The destination is set in either IcebergDestinationConfiguration or S3DestinationConfiguration (mutually exclusive), and each destination configuration contains its own ServiceExecutionRoleArn, DataFreshnessInSeconds (300–900, default 600), and DeadLetterQueueS3.

To create a Channel with an Amazon S3 general purpose bucket destination, use the following command.

aws kafka create-channel \
    --cluster-arn "arn:aws:kafka:us-east-1:123456789012:cluster/my-express-cluster/abc123" \
    --channel-name "orders-s3-channel" \
    --topic-configuration-list '[
        {
            "TopicArn": "arn:aws:kafka:us-east-1:123456789012:topic/my-express-cluster/abc123/orders-topic",
            "RecordConverter": { "ValueConverter": "JSON" }
        }
    ]' \
    --s3-destination-configuration '{
        "ServiceExecutionRoleArn": "arn:aws:iam::123456789012:role/MSKChannelRole",
        "DataFreshnessInSeconds": 300,
        "DeadLetterQueueS3": {
            "BucketArn": "arn:aws:s3:::my-dlq-bucket",
            "ErrorOutputPrefix": "dlq/"
        },
        "Storage": {
            "BucketArn": "arn:aws:s3:::my-delivery-bucket",
            "OutputPrefix": "expresslink/",
            "OutputKeyTemplate": "!{channel-id}/!{topic-name}/year=!{yyyy}/month=!{MM}/day=!{dd}/hour=!{HH}/!{topic-name}+!{partition-id}+!{kafka-offset}",
            "StorageClass": "STANDARD",
            "CompressionType": "GZIP"
        }
    }'

For Amazon S3 general purpose buckets, RecordConverter.ValueConverter may be JSON, BYTE_ARRAY, or STRING (no schema registry required). In Storage, CompressionType may be NONE, GZIP, or ZSTD, and StorageClass may be STANDARD, INTELLIGENT_TIERING, or GLACIER_IR.

The response is the same for both destinations.

{
    "ChannelArn": "arn:aws:kafka:us-east-1:123456789012:channel/my-express-cluster/abc123/orders-s3-channel",
    "ClusterOperationArn": "arn:aws:kafka:us-east-1:123456789012:cluster-operation/my-express-cluster/abc123/..."
}

Note

Use the ClusterOperationArn to track the asynchronous operation's status and any error message. The Channel starts in CREATING and transitions to ACTIVE when provisioning completes.

API reference — see CreateChannel in the Amazon MSK API Reference.