AWS::Pipes::Pipe - AWS CloudFormation
SyntaxPropertiesReturn valuesExamples

AWS::Pipes::Pipe

Create a pipe. Amazon EventBridge Pipes connect event sources to targets and reduces the need for specialized knowledge and integration code.

Syntax

To declare this entity in your AWS CloudFormation template, use the following syntax:

JSON

{
  "Type" : "AWS::Pipes::Pipe",
  "Properties" : {
      "Description" : String,
      "DesiredState" : String,
      "Enrichment" : String,
      "EnrichmentParameters" : PipeEnrichmentParameters,
      "Name" : String,
      "RoleArn" : String,
      "Source" : String,
      "SourceParameters" : PipeSourceParameters,
      "Tags" : {Key: Value, ...},
      "Target" : String,
      "TargetParameters" : PipeTargetParameters
    }
}

YAML

Type: AWS::Pipes::Pipe
Properties: 
  Description: String
  DesiredState: String
  Enrichment: String
  EnrichmentParameters: 
    PipeEnrichmentParameters
  Name: String
  RoleArn: String
  Source: String
  SourceParameters: 
    PipeSourceParameters
  Tags: 
    Key: Value
  Target: String
  TargetParameters: 
    PipeTargetParameters

Properties

Description

A description of the pipe.

Required: No

Type: String

Update requires: No interruption

DesiredState

The state the pipe should be in.

Required: No

Type: String

Update requires: No interruption

Enrichment

The ARN of the enrichment resource.

Required: No

Type: String

Update requires: No interruption

EnrichmentParameters

The parameters required to set up enrichment on your pipe.

Required: No

Type: PipeEnrichmentParameters

Update requires: No interruption

Name

The name of the pipe.

Required: No

Type: String

Update requires: Replacement

RoleArn

The ARN of the role that allows the pipe to send data to the target.

Required: Yes

Type: String

Update requires: No interruption

Source

The ARN of the source resource.

Required: Yes

Type: String

Update requires: Replacement

SourceParameters

The parameters required to set up a source for your pipe.

Required: No

Type: PipeSourceParameters

Update requires: No interruption

Tags

The list of key-value pairs to associate with the pipe.

Required: No

Type: Map of String

Update requires: No interruption

Target

The ARN of the target resource.

Required: Yes

Type: String

Update requires: No interruption

TargetParameters

The parameters required to set up a target for your pipe.

For more information about pipe target parameters, including how to use dynamic path parameters, see Target parameters in the Amazon EventBridge User Guide.

Required: No

Type: PipeTargetParameters

Update requires: No interruption

Return values

Ref

When you pass the logical ID of this resource to the intrinsic Ref function, Ref returns the name of the pipe that was created by the request.

Fn::GetAtt

The Fn::GetAtt intrinsic function returns a value for a specified attribute of this type. The following are the available attributes and sample return values.

For more information about using the Fn::GetAtt intrinsic function, see Fn::GetAtt.

Arn

The ARN of the pipe.

CreationTime

The time the pipe was created.

CurrentState

The state the pipe is in.

LastModifiedTime

When the pipe was last updated, in ISO-8601 format (YYYY-MM-DDThh:mm:ss.sTZD).

StateReason

The reason the pipe is in its current state.

Examples

Create a Pipe with an enrichment

Create a Pipe with an Amazon SQS source, an API Gateway enrichment, and a Step Functions state machine target.

JSON

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Resources": {
    "TestPipe": {
      "Type": "AWS::Pipes::Pipe",
      "Properties": {
        "Name": "PipeCfnExample",
        "RoleArn": "arn:aws:iam::123456789123:role/Pipe-Dev-All-Targets-Dummy-Execution-Role",
        "Source": "arn:aws:sqs:us-east-1:123456789123:pipeDemoSource",
        "Enrichment": "arn:aws:execute-api:us-east-1:123456789123:53eo2i89p9/*/POST/pets",
        "Target": "arn:aws:states:us-east-1:123456789123:stateMachine:PipeTargetStateMachine"
      }
    }
  }
}

YAML

AWSTemplateFormatVersion: 2010-09-09
Resources:
  TestPipe:
    Type: AWS::Pipes::Pipe
    Properties:
      Name:  PipeCfnExample
      RoleArn: arn:aws:iam::123456789123:role/Pipe-Dev-All-Targets-Dummy-Execution-Role
      Source: arn:aws:sqs:us-east-1:123456789123:pipeDemoSource
      Enrichment: arn:aws:execute-api:us-east-1:123456789123:53eo2i89p9/*/POST/pets
      Target: arn:aws:states:us-east-1:123456789123:stateMachine:PipeTargetStateMachine

Create a pipe with an event filter

The following example:

  • Provisions a DynamoDB table and associated data stream to act as the pipe source, and a Amazon SQS queue for the pipe target.

  • Provisions an IAM execution role for the pipe that defines the necessary permissions to access both the source and target.

  • Creates a pipe that connects the DynamoDB stream source to the Amazon SQS queue target.

  • Within the pipe, defines an event filter with an event pattern that selects events where eventname is INSERT or MODIFY.

Note

Be aware that you will be billed for the AWS resources used if you create a stack from this template.

JSON


{
  "AWSTemplateFormatVersion": "2010-09-09",

 "Description" : "EventBridge Pipe template example. Provisions a pipe, along with a DynamoDB stream as the pipe source and an SQS queue as the pipe target. Also provisions an execution role that contains the necessary permissions to access both the source and target. Once provisioned, the pipe receives events from the DynamoDB data stream, applies a filter, and sends matching events on to an SQS Queue. You will be billed for the Amazon resources used if you create a stack from this template.",

  "Parameters" : {
    "SourceTableName" : {
      "Type" : "String",
      "Default" : "pipe-example-source",
      "Description" : "Specify the name of the table to provision as the pipe source, or accept the default."
    },
  "TargetQueueName" : {
    "Type" : "String",
    "Default" : "pipe-example-target",
    "Description" : "Specify the name of the queue to provision as the pipe target, or accept the default."
  },
    "PipeName" : {
      "Type" : "String",
      "Default" : "pipe-with-filtering-example",
      "Description" : "Specify the name of the table to provision as the pipe source, or accept the default."
    }
},
  "Resources": {
    "PipeSourceDynamoDBTable": {
      "Type": "AWS::DynamoDB::Table",
      "Properties": {
        "AttributeDefinitions": [{
            "AttributeName": "Album",
            "AttributeType": "S"
          },
          {
            "AttributeName": "Artist",
            "AttributeType": "S"
          }

        ],
        "KeySchema": [{
            "AttributeName": "Album",
            "KeyType": "HASH"

          },
          {
            "AttributeName": "Artist",
            "KeyType": "RANGE"
          }
        ],
        "ProvisionedThroughput": {
          "ReadCapacityUnits": 10,
          "WriteCapacityUnits": 10
        },
        "StreamSpecification": {
          "StreamViewType": "NEW_AND_OLD_IMAGES"
        },
        "TableName": { "Ref" : "SourceTableName" }
      }
    },
    "PipeTargetQueue": {
      "Type": "AWS::SQS::Queue",
      "Properties": {
        "QueueName": { "Ref" : "TargetQueueName" }
      }
    },
    "PipeTutorialPipeRole": {
      "Type": "AWS::IAM::Role",
      "Properties": {
        "AssumeRolePolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [{
            "Effect": "Allow",
            "Principal": {
              "Service": "pipes.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
              "StringLike": {
                "aws:SourceArn": {
	"Fn::Join": [
		"", 
                [
                  "arn:",
                  { "Ref": "AWS::Partition" },
                  ":pipes:",
                  { "Ref": "AWS::Region" },
                  ":",
                  { "Ref": "AWS::AccountId" },
                  ":pipe/",
                  { "Ref": "PipeName" }
               ]
	]
       },
                "aws:SourceAccount": { "Ref" : "AWS::AccountId" }
              }
            }
          }]
        },
        "Description" : "EventBridge Pipe template example. Execution role that grants the pipe the permissions necessary to send events to the specified pipe.",
        "Path": "/",
        "Policies": [{
            "PolicyName": "SourcePermissions",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [{
                "Effect": "Allow",
                "Action": [
                  "dynamodb:DescribeStream",
                  "dynamodb:GetRecords",
                  "dynamodb:GetShardIterator",
                  "dynamodb:ListStreams"
                ],
                "Resource": [
                  { "Fn::GetAtt" : [ "PipeSourceDynamoDBTable", "StreamArn" ] }
                ]
              }]
            }
          },
          {
            "PolicyName": "TargetPermissions",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [{
                "Effect": "Allow",
                "Action": [
                  "sqs:SendMessage"
                ],
                "Resource": [
                  { "Fn::GetAtt" : [ "PipeTargetQueue", "Arn" ] }
                ]
              }]
            }
          }
        ]
      }
  },
    "PipeWithFiltering": {
      "Type": "AWS::Pipes::Pipe",
      "Properties": {
        "Description" : "EventBridge Pipe template example. Pipe that receives events from a DynamoDB stream, applies a filter, and sends matching events on to an SQS Queue.",
        "Name": { "Ref" : "PipeName" },
        "RoleArn": {"Fn::GetAtt" : ["PipeTutorialPipeRole", "Arn"] },
        "Source": { "Fn::GetAtt" : [ "PipeSourceDynamoDBTable", "StreamArn" ] },
        "SourceParameters": {
          "DynamoDBStreamParameters" : {
            "StartingPosition" : "LATEST"
         },
        "FilterCriteria" : {
          "Filters" : [ {
            "Pattern" : "{ \"eventName\": [\"INSERT\", \"MODIFY\"] }"
         }]
        }
        },
        "Target": { "Fn::GetAtt" : [ "PipeTargetQueue", "Arn" ] }
      }
    }
  }
}

YAML

AWSTemplateFormatVersion: 2010-09-09
Description: >-
  EventBridge Pipe template example. Provisions a pipe, along with a DynamoDB
  stream as the pipe source and an SQS queue as the pipe target. Also provisions
  an execution role that contains the necessary permissions to access both the
  source and target. Once provisioned, the pipe receives events from the
  DynamoDB data stream, applies a filter, and sends matching events on to an SQS
  Queue. You will be billed for the Amazon resources used if you create a stack
  from this template.
Parameters:
  SourceTableName:
    Type: String
    Default: pipe-example-source
    Description: >-
      Specify the name of the table to provision as the pipe source, or accept
      the default.
  TargetQueueName:
    Type: String
    Default: pipe-example-target
    Description: >-
      Specify the name of the queue to provision as the pipe target, or accept
      the default.
  PipeName:
    Type: String
    Default: pipe-with-filtering-example
    Description: >-
      Specify the name of the table to provision as the pipe source, or accept
      the default.
Resources:
  PipeSourceDynamoDBTable:
    Type: 'AWS::DynamoDB::Table'
    Properties:
      AttributeDefinitions:
        - AttributeName: Album
          AttributeType: S
        - AttributeName: Artist
          AttributeType: S
      KeySchema:
        - AttributeName: Album
          KeyType: HASH
        - AttributeName: Artist
          KeyType: RANGE
      ProvisionedThroughput:
        ReadCapacityUnits: 10
        WriteCapacityUnits: 10
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES
      TableName:
        Ref: SourceTableName
  PipeTargetQueue:
    Type: 'AWS::SQS::Queue'
    Properties:
      QueueName:
        Ref: TargetQueueName
  PipeTutorialPipeRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: pipes.amazonaws.com
            Action: 'sts:AssumeRole'
            Condition:
              StringLike:
                'aws:SourceArn':
                  'Fn::Join':
                    - ''
                    - - 'arn:'
                      - Ref: 'AWS::Partition'
                      - ':pipes:'
                      - Ref: 'AWS::Region'
                      - ':'
                      - Ref: 'AWS::AccountId'
                      - ':pipe/'
                      - Ref: PipeName
                'aws:SourceAccount':
                  Ref: 'AWS::AccountId'
      Description: >-
        EventBridge Pipe template example. Execution role that grants the pipe
        the permissions necessary to send events to the specified pipe.
      Path: /
      Policies:
        - PolicyName: SourcePermissions
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'dynamodb:DescribeStream'
                  - 'dynamodb:GetRecords'
                  - 'dynamodb:GetShardIterator'
                  - 'dynamodb:ListStreams'
                Resource:
                  - 'Fn::GetAtt':
                      - PipeSourceDynamoDBTable
                      - StreamArn
        - PolicyName: TargetPermissions
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'sqs:SendMessage'
                Resource:
                  - 'Fn::GetAtt':
                      - PipeTargetQueue
                      - Arn
  PipeWithFiltering:
    Type: 'AWS::Pipes::Pipe'
    Properties:
      Description: >-
        EventBridge Pipe template example. Pipe that receives events from a
        DynamoDB stream, applies a filter, and sends matching events on to an
        SQS Queue.
      Name:
        Ref: PipeName
      RoleArn:
        'Fn::GetAtt':
          - PipeTutorialPipeRole
          - Arn
      Source:
        'Fn::GetAtt':
          - PipeSourceDynamoDBTable
          - StreamArn
      SourceParameters:
        DynamoDBStreamParameters:
          StartingPosition: LATEST
        FilterCriteria:
          Filters:
            - Pattern: '{ "eventName": ["INSERT", "MODIFY"] }'
      Target:
        'Fn::GetAtt':
          - PipeTargetQueue
          - Arn