IAM roles for cross account delivery Create a flow log that publishes to Kinesis Data Firehose Process flow log records in Kinesis Data Firehose

Publish flow logs to Kinesis Data Firehose

Flow logs can publish flow log data directly to Kinesis Data Firehose.

When publishing to Kinesis Data Firehose, flow log data is published to a Kinesis Data Firehose delivery stream, in plain text format.

Pricing

Standard ingestion and delivery charges apply. For more information, open Amazon CloudWatch Pricing, select Logs and find Vended Logs.

IAM roles for cross account delivery
Create a flow log that publishes to Kinesis Data Firehose
Process flow log records in Kinesis Data Firehose

IAM roles for cross account delivery

When you publish to Kinesis Data Firehose, you can choose a delivery stream that's in the same account as the resource to monitor (the source account), or in a different account (the destination account). To enable cross account delivery of flow logs to Kinesis Data Firehose, you must create an IAM role in the source account and an IAM role in the destination account.

Source account role

In the source account, create a role that grants the following permissions. In this example, the name of the role is mySourceRole, but you can choose a different name for this role. The last statement allows the role in the destination account to assume this role. The condition statements ensure that this role is passed only to the log delivery service, and only when monitoring the specified resource. When you create your policy, specify the VPCs, network interfaces, or subnets that you're monitoring with the condition key iam:AssociatedResourceARN.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "arn:aws:iam::source-account:role/mySourceRole",
      "Condition": {
          "StringEquals": {
              "iam:PassedToService": "delivery.logs.amazonaws.com"
          },
          "StringLike": {
              "iam:AssociatedResourceARN": [
                  "arn:aws:ec2:region:source-account:vpc/vpc-00112233344556677"
              ]
          }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
          "logs:CreateLogDelivery",
          "logs:DeleteLogDelivery",
          "logs:ListLogDeliveries",
          "logs:GetLogDelivery"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": "arn:aws:iam::destination-account:role/AWSLogDeliveryFirehoseCrossAccountRole"      
    }
  ]
}

Ensure that this role has the following trust policy, which allows the log delivery service to assume the role.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "delivery.logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

From the source account, use the following procedure to create the role.

To create the source account role

Open the IAM console at https://console.aws.amazon.com/iam/.
In the navigation pane, choose Policies.
Choose Create policy.
On the Create policy page, do the following:
1. Choose JSON.
2. Replace the contents of this window with the permissions policy at the start of this section.
3. Choose Next: Tags and Next: Review.
4. Enter a name for your policy and an optional description, and then choose Create policy.
In the navigation pane, choose Roles.
Choose Create role.
For Trusted entity type, choose Custom trust policy. For Custom trust policy, replace "Principal": {}, with the following, which specifies the log delivery service. Choose Next.
```
"Principal": {
   "Service": "delivery.logs.amazonaws.com"
},
```
On the Add permissions page, select the checkbox for the policy that you created earlier in this procedure, and then choose Next.
Enter a name for your role and optionally provide a description.
Choose Create role.

Destination account role

In the destination account, create a role with a name that starts with AWSLogDeliveryFirehoseCrossAccountRole. This role must grant the following permissions.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
          "iam:CreateServiceLinkedRole",
          "firehose:TagDeliveryStream"
      ],
      "Resource": "*"
    }
  ]
}

Ensure that this role has the following trust policy, which allows the role that you created in the source account to assume this role.


{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
          "AWS": "arn:aws:iam::source-account:role/mySourceRole"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

From the destination account, use the following procedure to create the role.

To create the destination account role

Open the IAM console at https://console.aws.amazon.com/iam/.
In the navigation pane, choose Policies.
Choose Create policy.
On the Create policy page, do the following:
1. Choose JSON.
2. Replace the contents of this window with the permissions policy at the start of this section.
3. Choose Next: Tags and Next: Review.
4. Enter a name for your policy that starts with AWSLogDeliveryFirehoseCrossAccountRole, and then choose Create policy.
In the navigation pane, choose Roles.
Choose Create role.
For Trusted entity type, choose Custom trust policy. For Custom trust policy, replace "Principal": {}, with the following, which specifies the source account role. Choose Next.
```
"Principal": {
   "AWS": "arn:aws:iam::source-account:role/mySourceRole"
},
```
On the Add permissions page, select the checkbox for the policy that you created earlier in this procedure, and then choose Next.
Enter a name for your role and optionally provide a description.
Choose Create role.

Create a flow log that publishes to Kinesis Data Firehose

You can create flow logs for your VPCs, subnets, or network interfaces.

Prerequisites

Create the destination Kinesis Data Firehose delivery stream. Use Direct Put as the source. For more information, see Creating an Kinesis Data Firehose delivery stream.
If you're publishing flow logs to a different account, create the required IAM roles, as described in IAM roles for cross account delivery.

To create a flow log that publishes to Kinesis Data Firehose

Do one of the following:
- Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/. In the navigation pane, choose Network Interfaces. Select the checkbox for the network interface.
- Open the Amazon VPC console at https://console.aws.amazon.com/vpc/. In the navigation pane, choose Your VPCs. Select the checkbox for the VPC.
- Open the Amazon VPC console at https://console.aws.amazon.com/vpc/. In the navigation pane, choose Subnets. Select the checkbox for the subnet.
Choose Actions, Create flow log.
For Filter, specify the type of traffic to log.
- Accept – Log only accepted traffic
- Reject – Log only rejected traffic
- All – Log accepted and rejected traffic
For Maximum aggregation interval, choose the maximum period of time during which a flow is captured and aggregated into one flow log record.
For Destination, choose either of the following options:
- Send to Kinesis Data Firehose in the same account – The delivery stream and the resource to monitor are in the same account.
- Send to Kinesis Data Firehose in a different account – The delivery stream and the resource to monitor are in different accounts.
For Kinesis Data Firehose delivery stream, choose the delivery stream that you created.
[Cross account delivery only] For IAM roles, specify the required roles (see IAM roles for cross account delivery).
(Optional) Choose Add new tag to apply tags to the flow log.
Choose Create flow log.

To create a flow log that publishes to Kinesis Data Firehose using a command line tool

Use one of the following commands:

create-flow-logs (AWS CLI)
New-EC2FlowLogs (AWS Tools for Windows PowerShell)

The following AWS CLI example creates a flow log that captures all traffic for the specified VPC and delivers the flow logs to the specified Kinesis Data Firehose delivery stream in the same account.


aws ec2 create-flow-logs --traffic-type ALL \
  --resource-type VPC \
  --resource-ids vpc-00112233344556677 \
  --log-destination-type kinesis-data-firehose \
  --log-destination arn:aws:firehose:us-east-1:123456789012:deliverystream:flowlogs_stream

The following AWS CLI example creates a flow log that captures all traffic for the specified VPC and delivers the flow logs to the specified Kinesis Data Firehose delivery stream in a different account.


aws ec2 create-flow-logs --traffic-type ALL \
  --resource-type VPC \
  --resource-ids vpc-00112233344556677 \
  --log-destination-type kinesis-data-firehose \
  --log-destination arn:aws:firehose:us-east-1:123456789012:deliverystream:flowlogs_stream \
  --deliver-logs-permission-arn arn:aws:iam::source-account:role/mySourceRole \ 
  --deliver-cross-account-role arn:aws:iam::destination-account:role/AWSLogDeliveryFirehoseCrossAccountRole

Process flow log records in Kinesis Data Firehose

You can get the flow log data from the destination that you configured for the delivery stream.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Publish to Amazon S3

Query using Athena