Logging and monitoring Amazon EventBridge Pipes using AWS CloudTrail and Amazon CloudWatch Logs - Amazon EventBridge
CloudWatch metricsDimensions for CloudWatch metricsCloudTrail logging

Logging and monitoring Amazon EventBridge Pipes using AWS CloudTrail and Amazon CloudWatch Logs

You can log EventBridge Pipes invocations and using CloudTrail and monitor the health of your pipes using CloudWatch metrics.

CloudWatch metrics

EventBridge Pipes sends metrics to Amazon CloudWatch every minute for everything from a pipe executions being throttled to a target successfully being invoked.

Metric Description

Concurrency

The number of concurrent executions of a pipe.

Valid Dimensions: AwsAccountId

Units: None

Duration

Length of time the pipe execution took.

Valid Dimensions: PipeName

Units: Milliseconds

EventCount

The number of events a pipe has processed.

Valid Dimensions: PipeName

Units: None

EventSize

The size of the payload of the event that invoked the pipe.

Valid Dimensions: PipeName

Units: Bytes

ExecutionThrottled

How many executions of a pipe were throttled.

Note

This value will be 0 if no executions were throttled.

Valid Dimensions: AwsAccountId, PipeName

Units: None

ExecutionTimeout

How many executions of a pipe timed out before completing execution.

Note

This value will be 0 if no executions timed out.

Valid Dimensions: PipeName

Units: None

ExecutionFailed

How many executions of a pipe failed.

Note

This value will be 0 if no executions failed.

Valid Dimensions: PipeName

Units: None

ExecutionPartiallyFailed

How many executions of a pipe partially failed.

Note

This value will be 0 if no executions partially failed.

Valid Dimensions: PipeName

Units: None

EnrichmentStageDuration

How long the enrichment stage took to complete.

Valid Dimensions: PipeName

Units: Milliseconds

EnrichmentStageFailed

How many executions of a pipe's enrichment stage failed.

Note

This value will be 0 if no executions failed.

Valid Dimensions: PipeName

Units: None

Invocations

Total number of invocations.

Valid Dimensions: AwsAccountId, PipeName

Units: None

TargetStageDuration

How long the target stage took to complete.

Valid Dimensions: PipeName

Units: Milliseconds

TargetStageFailed

How many executions of a pipe's target stage failed.

Note

This value will be 0 if no executions failed.

Valid Dimensions: PipeName

Units: None

TargetStagePartiallyFailed

How many executions of a pipe's target stage partially failed.

Note

This value will be 0 if no target stage executions partially failed.

Valid Dimensions: PipeName

Units: None

TargetStageSkipped

How many executions of a pipe's target stage were skipped (for example, due to the enrichment returning an empty payload).

Valid Dimensions: PipeName

Units: Count

Dimensions for CloudWatch metrics

CloudWatch metrics have dimensions, or sortable attributes, which are listed below.

Dimension Description
AwsAccountId

Filters the available metrics by account ID.
PipeName

Filters the available metrics by pipe name.

CloudTrail logging

EventBridge Pipes works with AWS CloudTrail, a service that records actions from AWS services. CloudTrail captures API calls made by or on behalf of your AWS account from the EventBridge console and to EventBridge Pipes API operations.

Using the information collected by CloudTrail, you can determine what request was made to EventBridge, the IP address from which the request was made, who made the request, when it was made, and more.

For more information about CloudTrail, see the AWS CloudTrail User Guide.

CloudTrail is enabled on your AWS account when you create your account. When an event occurs in EventBridge, CloudTrail records the event in Event history. You can view, search, and download recent events in your AWS account. For more information, see Viewing Events with CloudTrail Event History.

For a record of events in your AWS account, including events for EventBridge, create a trail. A trail is a configuration that CloudTrail uses to deliver log files to an Amazon S3 bucket. By default, the trail logs events from all Regions in the AWS partition and then delivers the log files to an S3 bucket. You can configure other AWS services to analyze and act on the event data collected in CloudTrail logs. For more information, see the following:

You can log the following EventBridge Pipes actions as events in CloudTrail log files:

CloudTrail log entries for actions taken by EventBridge Pipes

EventBridge Pipes assumes the provided IAM role when reading events from sources, invoking enrichments, or invoking targets. For CloudTrail entries related to actions taken in your account on all enrichments, targets, and Amazon SQS, Kinesis, and DynamoDB sources, the sourceIPAddress and invokedBy fields will include pipes.amazonaws.com.

Sample CloudTrail log entry for all enrichments, targets, and Amazon SQS, Kinesis, and DynamoDB sources

{
  "eventVersion": "1.08",
  "userIdentity": {
    "type": "AssumedRole",
    "principalId": "...",
    "arn": "arn:aws:sts::111222333444:assumed-role/...",
    "accountId": "111222333444",
    "accessKeyId": "...",
    "sessionContext": {
      "sessionIssuer": {
        "type": "Role",
        "principalId": "...",
        "arn": "...",
        "accountId": "111222333444",
        "userName": "userName"
      },
      "webIdFederationData": {},
      "attributes": {
        "creationDate": "2022-09-22T21:41:15Z",
        "mfaAuthenticated": "false"
      }
    },
    "invokedBy": "pipes.amazonaws.com"
  },
  "eventTime": ",,,",
  "eventName": "...",
  "awsRegion": "us-west-2",
  "sourceIPAddress": "pipes.amazonaws.com",
  "userAgent": "pipes.amazonaws.com",
  "requestParameters": {
    ...
  },
  "responseElements": null,
  "requestID": "...",
  "eventID": "...",
  "readOnly": true,
  "eventType": "AwsApiCall",
  "managementEvent": true,
  "recipientAccountId": "...",
  "eventCategory": "Management"
}

For all other sources, the sourceIPAddress field of the CloudTrail log entries will have a dynamic IP address and shouldn't be relied upon for any integration or event categorization. In addition, these entries won't have the invokedBy field.

Sample CloudTrail log entry for all other sources

{
  "eventVersion": "1.08",
  "userIdentity": {
    ...
  },
  "eventTime": ",,,",
  "eventName": "...",
  "awsRegion": "us-west-2",
  "sourceIPAddress": "127.0.0.1",
  "userAgent": "Python-httplib2/0.8 (gzip)",
}