Logging and monitoring Amazon EventBridge Pipes using AWS CloudTrail and Amazon CloudWatch Logs
You can log EventBridge Pipes invocations and using CloudTrail and monitor the health of your pipes using CloudWatch metrics.
CloudWatch metrics
EventBridge Pipes sends metrics to Amazon CloudWatch every minute for everything from a pipe executions being throttled to a target successfully being invoked.
Metric | Description |
---|---|
|
The number of concurrent executions of a pipe. Valid Dimensions: AwsAccountId Units: Count |
|
Length of time the pipe execution took. Valid Dimensions: PipeName Units: Milliseconds |
|
The number of events a pipe has processed. Valid Dimensions: PipeName Units: Count |
|
The size of the payload of the event that invoked the pipe. Valid Dimensions: PipeName Units: Bytes |
|
How many executions of a pipe were throttled. NoteThis value will be Valid Dimensions: AwsAccountId, PipeName Units: Count |
|
How many executions of a pipe timed out before completing execution. NoteThis value will be Valid Dimensions: PipeName Units: Count |
|
How many executions of a pipe failed. NoteThis value will be Valid Dimensions: PipeName Units: Count |
|
How many executions of a pipe partially failed. NoteThis value will be Valid Dimensions: PipeName Units: Count |
|
How long the enrichment stage took to complete. Valid Dimensions: PipeName Units: Milliseconds |
|
How many executions of a pipe's enrichment stage failed. NoteThis value will be Valid Dimensions: PipeName Units: Count |
|
Total number of invocations. Valid Dimensions: AwsAccountId, PipeName Units: Count or sum |
|
How long the target stage took to complete. Valid Dimensions: PipeName Units: Milliseconds |
|
How many executions of a pipe's target stage failed. NoteThis value will be Valid Dimensions: PipeName Units: Count |
Dimensions for CloudWatch metrics
CloudWatch metrics have dimensions, or sortable attributes, which are listed below.
Dimension | Description |
---|---|
AwsAccountId
|
Filters the available metrics by account ID. |
PipeName
|
Filters the available metrics by pipe name. |
CloudTrail logging
EventBridge Pipes works with AWS CloudTrail, a service that records actions from AWS services. CloudTrail captures API calls made by or on behalf of your AWS account from the EventBridge console and to EventBridge Pipes API operations.
Using the information collected by CloudTrail, you can determine what request was made to EventBridge, the IP address from which the request was made, who made the request, when it was made, and more.
For more information about CloudTrail, see the AWS CloudTrail User Guide.
CloudTrail is enabled on your AWS account when you create your account. When an event occurs in EventBridge, CloudTrail records the event in Event history. You can view, search, and download recent events in your AWS account. For more information, see Viewing Events with CloudTrail Event History.
For a record of events in your AWS account, including events for EventBridge, create a trail. A trail is a configuration that CloudTrail uses to deliver log files to an Amazon S3 bucket. By default, the trail logs events from all Regions in the AWS partition and then delivers the log files to an S3 bucket. You can configure other AWS services to analyze and act on the event data collected in CloudTrail logs. For more information, see the following:
You can log the following EventBridge Pipes actions as events in CloudTrail log files:
CloudTrail log entries for actions taken by EventBridge Pipes
EventBridge Pipes assumes the provided IAM role when reading events from sources, invoking enrichments, or invoking targets. For CloudTrail entries related to actions taken in your
account on all enrichments, targets, and Amazon SQS, Kinesis, and DynamoDB sources, the sourceIPAddress
and invokedBy
fields will include
pipes.amazonaws.com
.
Sample CloudTrail log entry for all enrichments, targets, and Amazon SQS, Kinesis, and DynamoDB sources
{ "eventVersion": "1.08", "userIdentity": { "type": "AssumedRole", "principalId": "...", "arn": "arn:aws:sts::111222333444:assumed-role/...", "accountId": "111222333444", "accessKeyId": "...", "sessionContext": { "sessionIssuer": { "type": "Role", "principalId": "...", "arn": "...", "accountId": "111222333444", "userName": "userName" }, "webIdFederationData": {}, "attributes": { "creationDate": "2022-09-22T21:41:15Z", "mfaAuthenticated": "false" } }, "invokedBy": "pipes.amazonaws.com" }, "eventTime": ",,,", "eventName": "...", "awsRegion": "us-west-2", "sourceIPAddress": "pipes.amazonaws.com", "userAgent": "pipes.amazonaws.com", "requestParameters": { ... }, "responseElements": null, "requestID": "...", "eventID": "...", "readOnly": true, "eventType": "AwsApiCall", "managementEvent": true, "recipientAccountId": "...", "eventCategory": "Management" }
For all other sources, the sourceIPAddress
field of the CloudTrail log entries will have a dynamic IP address and shouldn't be
relied upon for any integration or event categorization. In addition, these entries won't have the invokedBy
field.
Sample CloudTrail log entry for all other sources
{ "eventVersion": "1.08", "userIdentity": { ... }, "eventTime": ",,,", "eventName": "...", "awsRegion": "us-west-2", "sourceIPAddress": "127.0.0.1", "userAgent": "Python-httplib2/0.8 (gzip)", }