SelfManagedKafkaEventSource

class aws_cdk.aws_lambda_event_sources.SelfManagedKafkaEventSource(*, bootstrap_servers, authentication_method=None, security_group=None, vpc=None, vpc_subnets=None, secret=None, topic, batch_size=None, bisect_batch_on_error=None, enabled=None, max_batching_window=None, max_record_age=None, on_failure=None, parallelization_factor=None, report_batch_item_failures=None, retry_attempts=None, starting_position, tumbling_window=None)

Bases: aws_cdk.aws_lambda_event_sources.StreamEventSource

Use a self hosted Kafka installation as a streaming source for AWS Lambda.

Example:

from aws_cdk.aws_secretsmanager import Secret
from aws_cdk.aws_lambda_event_sources import SelfManagedKafkaEventSource

# The secret that allows access to your self hosted Kafka cluster
# secret is of type Secret

# my_function is of type Function


# The list of Kafka brokers
bootstrap_servers = ["kafka-broker:9092"]

# The Kafka topic you want to subscribe to
topic = "some-cool-topic"
my_function.add_event_source(SelfManagedKafkaEventSource(
    bootstrap_servers=bootstrap_servers,
    topic=topic,
    secret=secret,
    batch_size=100,  # default
    starting_position=lambda_.StartingPosition.TRIM_HORIZON
))
Parameters
  • bootstrap_servers (Sequence[str]) – The list of host and port pairs that are the addresses of the Kafka brokers in a “bootstrap” Kafka cluster that a Kafka client connects to initially to bootstrap itself. They are in the format abc.xyz.com:xxxx.

  • authentication_method (Optional[AuthenticationMethod]) – The authentication method for your Kafka cluster. Default: AuthenticationMethod.SASL_SCRAM_512_AUTH

  • security_group (Optional[ISecurityGroup]) – If your Kafka brokers are only reachable via VPC, provide the security group here. Default: - none, required if setting vpc

  • vpc (Optional[IVpc]) – If your Kafka brokers are only reachable via VPC provide the VPC here. Default: none

  • vpc_subnets (Optional[SubnetSelection]) – If your Kafka brokers are only reachable via VPC, provide the subnets selection here. Default: - none, required if setting vpc

  • secret (Optional[ISecret]) – The secret with the Kafka credentials, see https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html for details This field is required if your Kafka brokers are accessed over the Internet. Default: none

  • topic (str) – The Kafka topic to subscribe to.

  • batch_size (Union[int, float, None]) – The largest number of records that AWS Lambda will retrieve from your event source at the time of invoking your function. Your function receives an event with all the retrieved records. Valid Range: - Minimum value of 1 - Maximum value of: - 1000 for {@link DynamoEventSource} - 10000 for {@link KinesisEventSource} Default: 100

  • bisect_batch_on_error (Optional[bool]) – If the function returns an error, split the batch in two and retry. Default: false

  • enabled (Optional[bool]) – If the stream event source mapping should be enabled. Default: true

  • max_batching_window (Optional[Duration]) – The maximum amount of time to gather records before invoking the function. Maximum of Duration.minutes(5) Default: Duration.seconds(0)

  • max_record_age (Optional[Duration]) – The maximum age of a record that Lambda sends to a function for processing. Valid Range: - Minimum value of 60 seconds - Maximum value of 7 days Default: - the retention period configured on the stream

  • on_failure (Optional[IEventSourceDlq]) – An Amazon SQS queue or Amazon SNS topic destination for discarded records. Default: discarded records are ignored

  • parallelization_factor (Union[int, float, None]) – The number of batches to process from each shard concurrently. Valid Range: - Minimum value of 1 - Maximum value of 10 Default: 1

  • report_batch_item_failures (Optional[bool]) – Allow functions to return partially successful responses for a batch of records. Default: false

  • retry_attempts (Union[int, float, None]) – Maximum number of retry attempts Valid Range: * Minimum value of 0 * Maximum value of 10000. Default: - retry until the record expires

  • starting_position (StartingPosition) – Where to begin consuming the stream.

  • tumbling_window (Optional[Duration]) – The size of the tumbling windows to group records sent to DynamoDB or Kinesis Valid Range: 0 - 15 minutes. Default: - None

Methods

bind(target)

Called by lambda.addEventSource to allow the event source to bind to this function.

Parameters

target (IFunction) –

Return type

None