Work with Transit Gateway Flow Logs - Amazon VPC

Work with Transit Gateway Flow Logs

You can work with Transit Gateway Flow Logs using the Amazon EC2, Amazon VPC, CloudWatch, and Amazon S3 consoles.

Control the use of flow logs

By default, users do not have permission to work with flow logs. You can create a user policy that grants users the permissions to create, describe, and delete flow logs. For more information, see Granting IAM Users Required Permissions for Amazon EC2 Resources in the Amazon EC2 API Reference.

The following is an example policy that grants users full permissions to create, describe, and delete flow logs.

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DeleteFlowLogs", "ec2:CreateFlowLogs", "ec2:DescribeFlowLogs" ], "Resource": "*" } ] }

Some additional IAM role and permission configuration is required, depending on whether you're publishing to CloudWatch Logs or Amazon S3. For more information, see Create a flow log that publishes to CloudWatch Logs and Create a flow log that publishes to Amazon S3.

Create a flow log

You can create flow logs for your transit gateways that can publish data to CloudWatch Logs, Amazon S3, or Firehose.

For more information, see the following:

View flow logs

You can view information about your flow logs in the Amazon VPC console by viewing the Flow Logs tab for a specific resource. When you select the resource, all of the flow logs for that resource are listed. The information displayed includes the ID of the flow log, the flow log configuration, and information about the status of the flow log.

To view information about flow logs for transit gateways
  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the navigation pane, choose Transit gateways or Transit gateway attachments.

  3. Select a transit gateway or transit gateway attachment and choose Flow Logs. Information about the flow logs is displayed on the tab. The Destination type column indicates the destination to which the flow logs are published.

Add or remove tags for flow logs

You can add or remove tags for a flow log in the Amazon EC2 and Amazon VPC consoles.

To add or remove tags for a transit gateway flow log
  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the navigation pane, choose Transit gateways or Transit gateway attachments.

  3. Select a transit gateway or transit gateway attachment

  4. Choose Manage tags for the required flow log.

  5. To add a new tag, choose Create Tag. To remove a tag, choose the delete button (x).

  6. Choose Save.

View flow log records

You can view your flow log records using the CloudWatch Logs console or Amazon S3 console, depending on the chosen destination type. It might take a few minutes after you've created your flow log for it to be visible in the console.

To view flow log records published to CloudWatch Logs
  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the navigation pane, choose Logs, and select the log group that contains your flow log. A list of log streams for each transit gateway is displayed.

  3. Select the log stream that contains the ID of the transit gateway that you want to view the flow log records for. For more information, see Transit Gateway Flow Log records.

To view flow log records published to Amazon S3
  1. Open the Amazon S3 console at https://console.aws.amazon.com/s3/.

  2. For Bucket name, select the bucket to which the flow logs are published.

  3. For Name, select the check box next to the log file. On the object overview panel, choose Download.

Search flow log records

You can search your flow log records that are published to CloudWatch Logs by using the CloudWatch Logs console. You can use metric filters to filter flow log records. Flow log records are space delimited.

To search flow log records using the CloudWatch Logs console
  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. In the navigation pane, choose Logs, and then choose Log groups.

  3. Select the log group that contains your flow log. A list of log streams for each transit gateway is displayed.

  4. Select the individual log stream if you know the transit gateway that you are searching for. Alternatively, choose Search Log Group to search the entire log group. This might take some time if there are many transit gateways in your log group, or depending on the time range that you select.

  5. For Filter events, enter the following string. This assumes that the flow log record uses the default format.

    [version, resource_type, account_id,tgw_id, tgw_attachment_id, tgw_src_vpc_account_id, tgw_dst_vpc_account_id, tgw_src_vpc_id, tgw_dst_vpc_id, tgw_src_subnet_id, tgw_dst_subnet_id, tgw_src_eni, tgw_dst_eni, tgw_src_az_id, tgw_dst_az_id, tgw_pair_attachment_id, srcaddr, dstaddr, srcport, dstport, protocol, packets, bytes,start,end, log_status, type,packets_lost_no_route, packets_lost_blackhole, packets_lost_mtu_exceeded, packets_lost_ttl_expired, tcp_flags,region, flow_direction, pkt_src_aws_service, pkt_dst_aws_service]
  6. Modify the filter as needed by specifying values for the fields. The following examples filter by specific source IP addresses.

    [version, resource_type, account_id,tgw_id, tgw_attachment_id, tgw_src_vpc_account_id, tgw_dst_vpc_account_id, tgw_src_vpc_id, tgw_dst_vpc_id, tgw_src_subnet_id, tgw_dst_subnet_id, tgw_src_eni, tgw_dst_eni, tgw_src_az_id, tgw_dst_az_id, tgw_pair_attachment_id, srcaddr= 10.0.0.1, dstaddr, srcport, dstport, protocol, packets, bytes,start,end, log_status, type,packets_lost_no_route, packets_lost_blackhole, packets_lost_mtu_exceeded, packets_lost_ttl_expired, tcp_flags,region, flow_direction, pkt_src_aws_service, pkt_dst_aws_service] [version, resource_type, account_id,tgw_id, tgw_attachment_id, tgw_src_vpc_account_id, tgw_dst_vpc_account_id, tgw_src_vpc_id, tgw_dst_vpc_id, tgw_src_subnet_id, tgw_dst_subnet_id, tgw_src_eni, tgw_dst_eni, tgw_src_az_id, tgw_dst_az_id, tgw_pair_attachment_id, srcaddr= 10.0.2.*, dstaddr, srcport, dstport, protocol, packets, bytes,start,end, log_status, type,packets_lost_no_route, packets_lost_blackhole, packets_lost_mtu_exceeded, packets_lost_ttl_expired, tcp_flags,region, flow_direction, pkt_src_aws_service, pkt_dst_aws_service]

    The following example filters by transit gateway ID tgw-123abc456bca, destination port, and number of bytes.

    [version, resource_type, account_id,tgw_id=tgw-123abc456bca, tgw_attachment_id, tgw_src_vpc_account_id, tgw_dst_vpc_account_id, tgw_src_vpc_id, tgw_dst_vpc_id, tgw_src_subnet_id, tgw_dst_subnet_id, tgw_src_eni, tgw_dst_eni, tgw_src_az_id, tgw_dst_az_id, tgw_pair_attachment_id, srcaddr, dstaddr, srcport, dstport = 80 || dstport = 8080, protocol, packets, bytes >= 500,start,end, log_status, type,packets_lost_no_route, packets_lost_blackhole, packets_lost_mtu_exceeded, packets_lost_ttl_expired, tcp_flags,region, flow_direction, pkt_src_aws_service, pkt_dst_aws_service]

Delete a flow log

You can delete a transit gateway flow log using the Amazon VPC console.

These procedures disable the flow log service for a resource. Deleting a flow log does not delete the existing log streams from CloudWatch Logs or log files from Amazon S3. Existing flow log data must be deleted using the respective service's console. In addition, deleting a flow log that publishes to Amazon S3 does not remove the bucket policies and log file access control lists (ACLs).

To delete a transit gateway flow log
  1. Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

  2. In the navigation pane, choose Transit gateways.

  3. Choose a Transit gateway ID.

  4. In the Flow logs section, choose the flow logs that you want to delete.

  5. Choose Actions, and then choose Delete flow logs.

  6. Confirm that you want to delete the flow by choosing Delete.

API and CLI overview and limitations

You can perform the tasks described on this page using the command line or API.

The following limitations apply when using the CreateFlowLogs API or the create-flow-logs CLI:

  • --resource-ids has a maximum constraint of 25 TransitGateway or TransitGatewayAttachment resource types.

  • --traffic-type is not a required field by default. An error is returned if you provide this for transit gateway resource types. This limit applies only to transit gateway resource types.

  • --max-aggregation-interval has a default value of 60, and is the only accepted value for transit gateway resource types. An error is returned if you try to pass any other value. This limit applies only to transit gateway resource types.

  • --resource-type supports two new resource types, TransitGateway and TransitGatewayAttachment.

  • --log-format includes all log fields for transit gateway resource types if you do not set which fields you want to include. This applies only to transit gateway resource types.

Create a flow log
Describe your flow logs
View your flow log records (log events)
Delete a flow log