AWS::Pipes::Pipe - AWS CloudFormation

AWS::Pipes::Pipe

Specifies a pipe. Amazon EventBridge Pipes connect event sources to targets and reduces the need for specialized knowledge and integration code.

Note

As an aid to help you jumpstart developing CloudFormation templates, the EventBridge console enables you to create templates from the existing pipes in your account. For more information, see Generate an CloudFormation template from EventBridge Pipes in the Amazon EventBridge User Guide.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON

{ "Type" : "AWS::Pipes::Pipe", "Properties" : { "Description" : String, "DesiredState" : String, "Enrichment" : String, "EnrichmentParameters" : PipeEnrichmentParameters, "KmsKeyIdentifier" : String, "LogConfiguration" : PipeLogConfiguration, "Name" : String, "RoleArn" : String, "Source" : String, "SourceParameters" : PipeSourceParameters, "Tags" : {Key: Value, ...}, "Target" : String, "TargetParameters" : PipeTargetParameters } }

Properties

Description

A description of the pipe.

Required: No

Type: String

Pattern: ^.*$

Minimum: 0

Maximum: 512

Update requires: No interruption

DesiredState

The state the pipe should be in.

Required: No

Type: String

Allowed values: RUNNING | STOPPED

Update requires: No interruption

Enrichment

The ARN of the enrichment resource.

Required: No

Type: String

Pattern: ^$|arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-]+):([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.+)$

Minimum: 0

Maximum: 1600

Update requires: No interruption

EnrichmentParameters

The parameters required to set up enrichment on your pipe.

Required: No

Type: PipeEnrichmentParameters

Update requires: No interruption

KmsKeyIdentifier

The identifier of the AWS KMS customer managed key for EventBridge to use, if you choose to use a customer managed key to encrypt pipe data. The identifier can be the key Amazon Resource Name (ARN), KeyId, key alias, or key alias ARN.

To update a pipe that is using the default AWS owned key to use a customer managed key instead, or update a pipe that is using a customer managed key to use a different customer managed key, specify a customer managed key identifier.

To update a pipe that is using a customer managed key to use the default AWS owned key, specify an empty string.

For more information, see Managing keys in the AWS Key Management Service Developer Guide.

Required: No

Type: String

Minimum: 0

Maximum: 2048

Update requires: No interruption

LogConfiguration

The logging configuration settings for the pipe.

Required: No

Type: PipeLogConfiguration

Update requires: No interruption

Name

The name of the pipe.

Required: No

Type: String

Pattern: ^[\.\-_A-Za-z0-9]+$

Minimum: 1

Maximum: 64

Update requires: Replacement

RoleArn

The ARN of the role that allows the pipe to send data to the target.

Required: Yes

Type: String

Pattern: ^arn:(aws[a-zA-Z-]*)?:iam::\d{12}:role/?[a-zA-Z0-9+=,.@\-_/]+$

Minimum: 1

Maximum: 1600

Update requires: No interruption

Source

The ARN of the source resource.

Required: Yes

Type: String

Pattern: ^smk://(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9]):[0-9]{1,5}|arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-]+):([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.+)$

Minimum: 1

Maximum: 1600

Update requires: Replacement

SourceParameters

The parameters required to set up a source for your pipe.

Required: No

Type: PipeSourceParameters

Update requires: No interruption

Tags

The list of key-value pairs to associate with the pipe.

Required: No

Type: Object of String

Pattern: .+

Minimum: 0

Maximum: 256

Update requires: No interruption

Target

The ARN of the target resource.

Required: Yes

Type: String

Pattern: ^arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\-]+):([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1})?:(\d{12})?:(.+)$

Minimum: 1

Maximum: 1600

Update requires: No interruption

TargetParameters

The parameters required to set up a target for your pipe.

For more information about pipe target parameters, including how to use dynamic path parameters, see Target parameters in the Amazon EventBridge User Guide.

Required: No

Type: PipeTargetParameters

Update requires: No interruption

Return values

Ref

When you pass the logical ID of this resource to the intrinsic Ref function, Ref returns the name of the pipe that was created by the request.

Fn::GetAtt

The Fn::GetAtt intrinsic function returns a value for a specified attribute of this type. The following are the available attributes and sample return values.

For more information about using the Fn::GetAtt intrinsic function, see Fn::GetAtt.

Arn

The ARN of the pipe.

CreationTime

The time the pipe was created.

CurrentState

The state the pipe is in.

LastModifiedTime

When the pipe was last updated, in ISO-8601 format (YYYY-MM-DDThh:mm:ss.sTZD).

StateReason

The reason the pipe is in its current state.

Examples

Create a Pipe with an enrichment

Create a Pipe with an Amazon SQS source, an API Gateway enrichment, and a Step Functions state machine target.

JSON

{ "AWSTemplateFormatVersion": "2010-09-09", "Resources": { "TestPipe": { "Type": "AWS::Pipes::Pipe", "Properties": { "Name": "PipeCfnExample", "RoleArn": "arn:aws:iam::123456789123:role/Pipe-Dev-All-Targets-Dummy-Execution-Role", "Source": "arn:aws:sqs:us-east-1:123456789123:pipeDemoSource", "Enrichment": "arn:aws:execute-api:us-east-1:123456789123:53eo2i89p9/*/POST/pets", "Target": "arn:aws:states:us-east-1:123456789123:stateMachine:PipeTargetStateMachine" } } } }

YAML

AWSTemplateFormatVersion: 2010-09-09 Resources: TestPipe: Type: AWS::Pipes::Pipe Properties: Name: PipeCfnExample RoleArn: arn:aws:iam::123456789123:role/Pipe-Dev-All-Targets-Dummy-Execution-Role Source: arn:aws:sqs:us-east-1:123456789123:pipeDemoSource Enrichment: arn:aws:execute-api:us-east-1:123456789123:53eo2i89p9/*/POST/pets Target: arn:aws:states:us-east-1:123456789123:stateMachine:PipeTargetStateMachine

Create a pipe with an event filter

The following example:

  • Provisions a DynamoDB table and associated data stream to act as the pipe source, and a Amazon SQS queue for the pipe target.

  • Provisions an IAM execution role for the pipe that defines the necessary permissions to access both the source and target.

  • Creates a pipe that connects the DynamoDB stream source to the Amazon SQS queue target.

  • Within the pipe, defines an event filter with an event pattern that selects events where eventname is INSERT or MODIFY.

Note

Be aware that you will be billed for the AWS resources used if you create a stack from this template.

JSON

{ "AWSTemplateFormatVersion": "2010-09-09", "Description" : "EventBridge Pipe template example. Provisions a pipe, along with a DynamoDB stream as the pipe source and an SQS queue as the pipe target. Also provisions an execution role that contains the necessary permissions to access both the source and target. Once provisioned, the pipe receives events from the DynamoDB data stream, applies a filter, and sends matching events on to an SQS Queue. You will be billed for the Amazon resources used if you create a stack from this template.", "Parameters" : { "SourceTableName" : { "Type" : "String", "Default" : "pipe-example-source", "Description" : "Specify the name of the table to provision as the pipe source, or accept the default." }, "TargetQueueName" : { "Type" : "String", "Default" : "pipe-example-target", "Description" : "Specify the name of the queue to provision as the pipe target, or accept the default." }, "PipeName" : { "Type" : "String", "Default" : "pipe-with-filtering-example", "Description" : "Specify the name of the table to provision as the pipe source, or accept the default." } }, "Resources": { "PipeSourceDynamoDBTable": { "Type": "AWS::DynamoDB::Table", "Properties": { "AttributeDefinitions": [{ "AttributeName": "Album", "AttributeType": "S" }, { "AttributeName": "Artist", "AttributeType": "S" } ], "KeySchema": [{ "AttributeName": "Album", "KeyType": "HASH" }, { "AttributeName": "Artist", "KeyType": "RANGE" } ], "ProvisionedThroughput": { "ReadCapacityUnits": 10, "WriteCapacityUnits": 10 }, "StreamSpecification": { "StreamViewType": "NEW_AND_OLD_IMAGES" }, "TableName": { "Ref" : "SourceTableName" } } }, "PipeTargetQueue": { "Type": "AWS::SQS::Queue", "Properties": { "QueueName": { "Ref" : "TargetQueueName" } } }, "PipeTutorialPipeRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": { "Service": "pipes.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringLike": { "aws:SourceArn": { "Fn::Join": [ "", [ "arn:", { "Ref": "AWS::Partition" }, ":pipes:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":pipe/", { "Ref": "PipeName" } ] ] }, "aws:SourceAccount": { "Ref" : "AWS::AccountId" } } } }] }, "Description" : "EventBridge Pipe template example. Execution role that grants the pipe the permissions necessary to send events to the specified pipe.", "Path": "/", "Policies": [{ "PolicyName": "SourcePermissions", "PolicyDocument": { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Action": [ "dynamodb:DescribeStream", "dynamodb:GetRecords", "dynamodb:GetShardIterator", "dynamodb:ListStreams" ], "Resource": [ { "Fn::GetAtt" : [ "PipeSourceDynamoDBTable", "StreamArn" ] } ] }] } }, { "PolicyName": "TargetPermissions", "PolicyDocument": { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Action": [ "sqs:SendMessage" ], "Resource": [ { "Fn::GetAtt" : [ "PipeTargetQueue", "Arn" ] } ] }] } } ] } }, "PipeWithFiltering": { "Type": "AWS::Pipes::Pipe", "Properties": { "Description" : "EventBridge Pipe template example. Pipe that receives events from a DynamoDB stream, applies a filter, and sends matching events on to an SQS Queue.", "Name": { "Ref" : "PipeName" }, "RoleArn": {"Fn::GetAtt" : ["PipeTutorialPipeRole", "Arn"] }, "Source": { "Fn::GetAtt" : [ "PipeSourceDynamoDBTable", "StreamArn" ] }, "SourceParameters": { "DynamoDBStreamParameters" : { "StartingPosition" : "LATEST" }, "FilterCriteria" : { "Filters" : [ { "Pattern" : "{ \"eventName\": [\"INSERT\", \"MODIFY\"] }" }] } }, "Target": { "Fn::GetAtt" : [ "PipeTargetQueue", "Arn" ] } } } } }

YAML

AWSTemplateFormatVersion: 2010-09-09 Description: >- EventBridge Pipe template example. Provisions a pipe, along with a DynamoDB stream as the pipe source and an SQS queue as the pipe target. Also provisions an execution role that contains the necessary permissions to access both the source and target. Once provisioned, the pipe receives events from the DynamoDB data stream, applies a filter, and sends matching events on to an SQS Queue. You will be billed for the Amazon resources used if you create a stack from this template. Parameters: SourceTableName: Type: String Default: pipe-example-source Description: >- Specify the name of the table to provision as the pipe source, or accept the default. TargetQueueName: Type: String Default: pipe-example-target Description: >- Specify the name of the queue to provision as the pipe target, or accept the default. PipeName: Type: String Default: pipe-with-filtering-example Description: >- Specify the name of the table to provision as the pipe source, or accept the default. Resources: PipeSourceDynamoDBTable: Type: 'AWS::DynamoDB::Table' Properties: AttributeDefinitions: - AttributeName: Album AttributeType: S - AttributeName: Artist AttributeType: S KeySchema: - AttributeName: Album KeyType: HASH - AttributeName: Artist KeyType: RANGE ProvisionedThroughput: ReadCapacityUnits: 10 WriteCapacityUnits: 10 StreamSpecification: StreamViewType: NEW_AND_OLD_IMAGES TableName: Ref: SourceTableName PipeTargetQueue: Type: 'AWS::SQS::Queue' Properties: QueueName: Ref: TargetQueueName PipeTutorialPipeRole: Type: 'AWS::IAM::Role' Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: pipes.amazonaws.com Action: 'sts:AssumeRole' Condition: StringLike: 'aws:SourceArn': 'Fn::Join': - '' - - 'arn:' - Ref: 'AWS::Partition' - ':pipes:' - Ref: 'AWS::Region' - ':' - Ref: 'AWS::AccountId' - ':pipe/' - Ref: PipeName 'aws:SourceAccount': Ref: 'AWS::AccountId' Description: >- EventBridge Pipe template example. Execution role that grants the pipe the permissions necessary to send events to the specified pipe. Path: / Policies: - PolicyName: SourcePermissions PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - 'dynamodb:DescribeStream' - 'dynamodb:GetRecords' - 'dynamodb:GetShardIterator' - 'dynamodb:ListStreams' Resource: - 'Fn::GetAtt': - PipeSourceDynamoDBTable - StreamArn - PolicyName: TargetPermissions PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - 'sqs:SendMessage' Resource: - 'Fn::GetAtt': - PipeTargetQueue - Arn PipeWithFiltering: Type: 'AWS::Pipes::Pipe' Properties: Description: >- EventBridge Pipe template example. Pipe that receives events from a DynamoDB stream, applies a filter, and sends matching events on to an SQS Queue. Name: Ref: PipeName RoleArn: 'Fn::GetAtt': - PipeTutorialPipeRole - Arn Source: 'Fn::GetAtt': - PipeSourceDynamoDBTable - StreamArn SourceParameters: DynamoDBStreamParameters: StartingPosition: LATEST FilterCriteria: Filters: - Pattern: '{ "eventName": ["INSERT", "MODIFY"] }' Target: 'Fn::GetAtt': - PipeTargetQueue - Arn