AWS::Pipes::Pipe PipeSourceSelfManagedKafkaParameters

The parameters for using a self-managed Apache Kafka stream as a source.

A self managed cluster refers to any Apache Kafka cluster not hosted by AWS. This includes both clusters you manage yourself, as well as those hosted by a third-party provider, such as Confluent Cloud, CloudKarafka, or Redpanda. For more information, see Apache Kafka streams as a source in the Amazon EventBridge User Guide.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON

{
  "AdditionalBootstrapServers" : [ String, ... ],
  "BatchSize" : Integer,
  "ConsumerGroupID" : String,
  "Credentials" : SelfManagedKafkaAccessConfigurationCredentials,
  "MaximumBatchingWindowInSeconds" : Integer,
  "ServerRootCaCertificate" : String,
  "StartingPosition" : String,
  "TopicName" : String,
  "Vpc" : SelfManagedKafkaAccessConfigurationVpc
}

YAML

  AdditionalBootstrapServers: 
    - String
  BatchSize: Integer
  ConsumerGroupID: String
  Credentials: 
    SelfManagedKafkaAccessConfigurationCredentials
  MaximumBatchingWindowInSeconds: Integer
  ServerRootCaCertificate: String
  StartingPosition: String
  TopicName: String
  Vpc: 
    SelfManagedKafkaAccessConfigurationVpc

Properties

AdditionalBootstrapServers

An array of server URLs.

Required: No

Type: Array of String

Minimum: 1 | 0

Maximum: 300 | 2

Update requires: No interruption

BatchSize

The maximum number of records to include in each batch.

Required: No

Type: Integer

Minimum: 1

Maximum: 10000

Update requires: No interruption

ConsumerGroupID

The name of the destination queue to consume.

Required: No

Type: String

Pattern: ^[a-zA-Z0-9-\/*:_+=.@-]*$

Minimum: 1

Maximum: 200

Update requires: No interruption

Credentials

The credentials needed to access the resource.

Required: No

Type: SelfManagedKafkaAccessConfigurationCredentials

Update requires: No interruption

MaximumBatchingWindowInSeconds

The maximum length of a time to wait for events.

Required: No

Type: Integer

Minimum: 0

Maximum: 300

Update requires: No interruption

ServerRootCaCertificate

The ARN of the Secrets Manager secret used for certification.

Required: No

Type: String

Pattern: ^(^arn:aws([a-z]|\-)*:secretsmanager:([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1}):(\d{12}):secret:.+)$

Minimum: 1

Maximum: 1600

Update requires: No interruption

StartingPosition

(Streams only) The position in a stream from which to start reading.

Required: No

Type: String

Allowed values: TRIM_HORIZON | LATEST

Update requires: No interruption

TopicName

The name of the topic that the pipe will read from.

Required: Yes

Type: String

Pattern: ^[^.]([a-zA-Z0-9\-_.]+)$

Minimum: 1

Maximum: 249

Update requires: No interruption

Vpc

This structure specifies the VPC subnets and security groups for the stream, and whether a public IP address is to be used.

Required: No

Type: SelfManagedKafkaAccessConfigurationVpc

Update requires: No interruption