Kafka / Client / create_replicator

create_replicator

Kafka.Client.create_replicator(**kwargs)

Creates the replicator.

See also: AWS API Documentation

Request Syntax

response = client.create_replicator(
    Description='string',
    KafkaClusters=[
        {
            'AmazonMskCluster': {
                'MskClusterArn': 'string'
            },
            'ApacheKafkaCluster': {
                'ApacheKafkaClusterId': 'string',
                'BootstrapBrokerString': 'string'
            },
            'VpcConfig': {
                'SecurityGroupIds': [
                    'string',
                ],
                'SubnetIds': [
                    'string',
                ]
            },
            'ClientAuthentication': {
                'SaslScram': {
                    'Mechanism': 'SHA256'|'SHA512',
                    'SecretArn': 'string'
                }
            },
            'EncryptionInTransit': {
                'EncryptionType': 'TLS',
                'RootCaCertificate': 'string'
            }
        },
    ],
    ReplicationInfoList=[
        {
            'ConsumerGroupReplication': {
                'ConsumerGroupsToExclude': [
                    'string',
                ],
                'ConsumerGroupsToReplicate': [
                    'string',
                ],
                'DetectAndCopyNewConsumerGroups': True|False,
                'SynchroniseConsumerGroupOffsets': True|False,
                'ConsumerGroupOffsetSyncMode': 'LEGACY'|'ENHANCED'
            },
            'SourceKafkaClusterArn': 'string',
            'SourceKafkaClusterId': 'string',
            'TargetCompressionType': 'NONE'|'GZIP'|'SNAPPY'|'LZ4'|'ZSTD',
            'TargetKafkaClusterArn': 'string',
            'TargetKafkaClusterId': 'string',
            'TopicReplication': {
                'CopyAccessControlListsForTopics': True|False,
                'CopyTopicConfigurations': True|False,
                'DetectAndCopyNewTopics': True|False,
                'StartingPosition': {
                    'Type': 'LATEST'|'EARLIEST'
                },
                'TopicNameConfiguration': {
                    'Type': 'PREFIXED_WITH_SOURCE_CLUSTER_ALIAS'|'IDENTICAL'
                },
                'TopicsToExclude': [
                    'string',
                ],
                'TopicsToReplicate': [
                    'string',
                ]
            }
        },
    ],
    ReplicatorName='string',
    ServiceExecutionRoleArn='string',
    Tags={
        'string': 'string'
    },
    LogDelivery={
        'ReplicatorLogDelivery': {
            'CloudWatchLogs': {
                'Enabled': True|False,
                'LogGroup': 'string'
            },
            'Firehose': {
                'Enabled': True|False,
                'DeliveryStream': 'string'
            },
            'S3': {
                'Enabled': True|False,
                'Bucket': 'string',
                'Prefix': 'string'
            }
        }
    }
)

Parameters:

• Description (string) – A summary description of the replicator.

• KafkaClusters (list) –

  [REQUIRED]

  Kafka Clusters to use in setting up sources / targets for replication.

  • (dict) –

    Information about Kafka Cluster to be used as source / target for replication.

    • AmazonMskCluster (dict) –

      Details of an Amazon MSK Cluster.

      • MskClusterArn (string) – [REQUIRED]

        The Amazon Resource Name (ARN) of an Amazon MSK cluster.

    • ApacheKafkaCluster (dict) –

      Details of an Apache Kafka Cluster.

      • ApacheKafkaClusterId (string) – [REQUIRED]

        The ID of the Apache Kafka cluster.

      • BootstrapBrokerString (string) – [REQUIRED]

        The bootstrap broker string of the Apache Kafka cluster.

    • VpcConfig (dict) –

      Details of an Amazon VPC which has network connectivity to the Apache Kafka cluster.

      • SecurityGroupIds (list) –

        The security groups to attach to the ENIs for the broker nodes.

        • (string) –

      • SubnetIds (list) – [REQUIRED]

        The list of subnets in the client VPC to connect to.

        • (string) –

    • ClientAuthentication (dict) –

      Details of the client authentication used by the Apache Kafka cluster.

      • SaslScram (dict) – [REQUIRED]

        Details for SASL/SCRAM client authentication.

        • Mechanism (string) – [REQUIRED]

          The SASL/SCRAM authentication mechanism.

        • SecretArn (string) – [REQUIRED]

          The Amazon Resource Name (ARN) of the Secrets Manager secret.

    • EncryptionInTransit (dict) –

      Details of encryption in transit to the Apache Kafka cluster.

      • EncryptionType (string) – [REQUIRED]

        The type of encryption in transit to the Apache Kafka cluster.

      • RootCaCertificate (string) –

        The root CA certificate.

• ReplicationInfoList (list) –

  [REQUIRED]

  A list of replication configurations, where each configuration targets a given source cluster to target cluster replication flow.

  • (dict) –

    Specifies configuration for replication between a source and target Kafka cluster.

    • ConsumerGroupReplication (dict) – [REQUIRED]

      Configuration relating to consumer group replication.

      • ConsumerGroupsToExclude (list) –

        List of regular expression patterns indicating the consumer groups that should not be replicated.

        • (string) –

      • ConsumerGroupsToReplicate (list) – [REQUIRED]

        List of regular expression patterns indicating the consumer groups to copy.

        • (string) –

      • DetectAndCopyNewConsumerGroups (boolean) –

        Enables synchronization of consumer groups to target cluster.

      • SynchroniseConsumerGroupOffsets (boolean) –

        Enables synchronization of consumer group offsets to target cluster. The translated offsets will be written to topic __consumer_offsets.

      • ConsumerGroupOffsetSyncMode (string) –

        The consumer group offset synchronization mode. With LEGACY, offsets are synchronized when producers write to the source cluster. With ENHANCED, consumer offsets are synchronized regardless of producer location. ENHANCED requires a corresponding replicator that replicates data from the target cluster to the source cluster.

    • SourceKafkaClusterArn (string) –

      The ARN of the source Kafka cluster.

    • SourceKafkaClusterId (string) –

      The ID of the source Kafka cluster.

    • TargetCompressionType (string) – [REQUIRED]

      The compression type to use when producing records to target cluster.

    • TargetKafkaClusterArn (string) –

      The ARN of the target Kafka cluster.

    • TargetKafkaClusterId (string) –

      The ID of the target Kafka cluster.

    • TopicReplication (dict) – [REQUIRED]

      Configuration relating to topic replication.

      • CopyAccessControlListsForTopics (boolean) –

        Whether to periodically configure remote topic ACLs to match their corresponding upstream topics.

      • CopyTopicConfigurations (boolean) –

        Whether to periodically configure remote topics to match their corresponding upstream topics.

      • DetectAndCopyNewTopics (boolean) –

        Whether to periodically check for new topics and partitions.

      • StartingPosition (dict) –

        Configuration for specifying the position in the topics to start replicating from.

        • Type (string) –

          The type of replication starting position.

      • TopicNameConfiguration (dict) –

        Configuration for specifying replicated topic names should be the same as their corresponding upstream topics or prefixed with source cluster alias.

        • Type (string) –

          The type of replicated topic name.

      • TopicsToExclude (list) –

        List of regular expression patterns indicating the topics that should not be replicated.

        • (string) –

      • TopicsToReplicate (list) – [REQUIRED]

        List of regular expression patterns indicating the topics to copy.

        • (string) –

• ReplicatorName (string) –

  [REQUIRED]

  The name of the replicator. Alpha-numeric characters with ‘-’ are allowed.

• ServiceExecutionRoleArn (string) –

  [REQUIRED]

  The ARN of the IAM role used by the replicator to access resources in the customer’s account (e.g source and target clusters)

• Tags (dict) –

  List of tags to attach to created Replicator.

  • (string) –

    • (string) –

• LogDelivery (dict) –

  Configuration for delivering replicator logs to customer destinations.

  • ReplicatorLogDelivery (dict) –

    Configuration for replicator log delivery.

    • CloudWatchLogs (dict) –

      Configuration for CloudWatch Logs delivery.

      • Enabled (boolean) – [REQUIRED]

        Whether log delivery to CloudWatch Logs is enabled.

      • LogGroup (string) –

        The CloudWatch log group that is the destination for log delivery.

    • Firehose (dict) –

      Configuration for Firehose delivery.

      • Enabled (boolean) – [REQUIRED]

        Whether log delivery to Firehose is enabled.

      • DeliveryStream (string) –

        The Firehose delivery stream that is the destination for log delivery.

    • S3 (dict) –

      Configuration for S3 delivery.

      • Enabled (boolean) – [REQUIRED]

        Whether log delivery to S3 is enabled.

      • Bucket (string) –

        The S3 bucket that is the destination for log delivery.

      • Prefix (string) –

        The S3 prefix that is the destination for log delivery.

Return type:

dict

Returns:

Response Syntax

{
    'ReplicatorArn': 'string',
    'ReplicatorName': 'string',
    'ReplicatorState': 'RUNNING'|'CREATING'|'UPDATING'|'DELETING'|'FAILED'
}

Response Structure

• (dict) –

  HTTP Status Code 200: OK.

  • ReplicatorArn (string) –

    The Amazon Resource Name (ARN) of the replicator.

  • ReplicatorName (string) –

    Name of the replicator provided by the customer.

  • ReplicatorState (string) –

    State of the replicator.

Exceptions

• Kafka.Client.exceptions.BadRequestException

• Kafka.Client.exceptions.UnauthorizedException

• Kafka.Client.exceptions.InternalServerErrorException

• Kafka.Client.exceptions.ForbiddenException

• Kafka.Client.exceptions.NotFoundException

• Kafka.Client.exceptions.ServiceUnavailableException

• Kafka.Client.exceptions.TooManyRequestsException

• Kafka.Client.exceptions.ConflictException