StreamSourceParameters

class aws_cdk.aws_pipes_sources_alpha.StreamSourceParameters(*, batch_size=None, dead_letter_target=None, maximum_batching_window=None, maximum_record_age=None, maximum_retry_attempts=None, on_partial_batch_item_failure=None, parallelization_factor=None)

Bases: object

(experimental) Base parameters for streaming sources.

Parameters:

  • batch_size (Union[int, float, None]) – (experimental) The maximum number of records to include in each batch. Default: 1

  • dead_letter_target (Union[IQueue, ITopic, None]) – (experimental) Define the target to send dead-letter queue events to. The dead-letter queue stores any events that are not successfully delivered to a Pipes target after all retry attempts are exhausted. You can then resolve the issue that caused the failed invocations and replay the events at a later time. In some cases, such as when access is denied to a resource, events are sent directly to the dead-letter queue and are not retried. Default: - no dead-letter queue or topic

  • maximum_batching_window (Optional[Duration]) – (experimental) The maximum length of a time to wait for events. Default: - the events will be handled immediately

  • maximum_record_age (Optional[Duration]) – (experimental) Discard records older than the specified age. The default value is -1, which sets the maximum age to infinite. When the value is set to infinite, EventBridge never discards old records. Default: -1 - EventBridge won’t discard old records

  • maximum_retry_attempts (Union[int, float, None]) – (experimental) Discard records after the specified number of retries. The default value is -1, which sets the maximum number of retries to infinite. When MaximumRetryAttempts is infinite, EventBridge retries failed records until the record expires in the event source. Default: -1 - EventBridge will retry failed records until the record expires in the event source

  • on_partial_batch_item_failure (Optional[OnPartialBatchItemFailure]) – (experimental) Define how to handle item process failures. {@link OnPartialBatchItemFailure.AUTOMATIC_BISECT} halves each batch and will retry each half until all the records are processed or there is one failed message left in the batch. Default: off - EventBridge will retry the entire batch

  • parallelization_factor (Union[int, float, None]) – (experimental) The number of batches to process concurrently from each shard. Default: 1

Stability:

experimental

ExampleMetadata:

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_pipes_sources_alpha as pipes_sources_alpha
import aws_cdk as cdk
from aws_cdk import aws_sqs as sqs

# queue: sqs.Queue

stream_source_parameters = pipes_sources_alpha.StreamSourceParameters(
    batch_size=123,
    dead_letter_target=queue,
    maximum_batching_window=cdk.Duration.minutes(30),
    maximum_record_age=cdk.Duration.minutes(30),
    maximum_retry_attempts=123,
    on_partial_batch_item_failure=pipes_sources_alpha.OnPartialBatchItemFailure.AUTOMATIC_BISECT,
    parallelization_factor=123
)

Attributes

batch_size

(experimental) The maximum number of records to include in each batch.

Default:

1

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-batchsize

Stability:

experimental

dead_letter_target

(experimental) Define the target to send dead-letter queue events to.

The dead-letter queue stores any events that are not successfully delivered to a Pipes target after all retry attempts are exhausted. You can then resolve the issue that caused the failed invocations and replay the events at a later time. In some cases, such as when access is denied to a resource, events are sent directly to the dead-letter queue and are not retried.

Default:

  • no dead-letter queue or topic

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-deadletterconfig

Stability:

experimental

maximum_batching_window

(experimental) The maximum length of a time to wait for events.

Default:

  • the events will be handled immediately

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-maximumbatchingwindowinseconds

Stability:

experimental

maximum_record_age

(experimental) Discard records older than the specified age.

The default value is -1, which sets the maximum age to infinite. When the value is set to infinite, EventBridge never discards old records.

Default:

-1 - EventBridge won’t discard old records

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-maximumrecordageinseconds

Stability:

experimental

maximum_retry_attempts

(experimental) Discard records after the specified number of retries.

The default value is -1, which sets the maximum number of retries to infinite. When MaximumRetryAttempts is infinite, EventBridge retries failed records until the record expires in the event source.

Default:

-1 - EventBridge will retry failed records until the record expires in the event source

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-maximumretryattempts

Stability:

experimental

on_partial_batch_item_failure

(experimental) Define how to handle item process failures.

{@link OnPartialBatchItemFailure.AUTOMATIC_BISECT} halves each batch and will retry each half until all the records are processed or there is one failed message left in the batch.

Default:

off - EventBridge will retry the entire batch

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-onpartialbatchitemfailure

Stability:

experimental

parallelization_factor

(experimental) The number of batches to process concurrently from each shard.

Default:

1

See:

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-pipes-pipe-pipesourcekinesisstreamparameters.html#cfn-pipes-pipe-pipesourcekinesisstreamparameters-parallelizationfactor

Stability:

experimental