Step 3: Send the Data from Amazon CloudWatch to Kinesis Data Firehose
In this step of this Kinesis Data Firehose tutorial, you subscribe the delivery stream to the Amazon CloudWatch log group. This step causes the log data to flow from the log group to the delivery stream.
To send log data from CloudWatch Logs to your delivery stream
In this procedure, you use the AWS Command Line Interface
(AWS CLI)
-
Save the following trust policy to a local file, and name the file
VPCtoSplunkCWtoFHTrustPolicy.json
. Be sure to replace theyour-region
placeholder with your AWS Region code.{ "Statement": { "Effect": "Allow", "Principal": { "Service": "logs.
your-region
.amazonaws.com" }, "Action": "sts:AssumeRole" } } -
In a command window, go to the directory where you saved
VPCtoSplunkCWtoFHPolicy.json
, and run the following AWS CLI command.aws iam create-role --role-name VPCtoSplunkCWtoFHRole --assume-role-policy-document file://VPCtoSplunkCWtoFHTrustPolicy.json
-
Save the following access policy to a local file, and name the file
VPCtoSplunkCWtoFHAccessPolicy.json
. Be sure to replace theyour-region
andyour-aws-account-id
placeholders with your AWS Region code and account ID.{ "Statement":[ { "Effect":"Allow", "Action":["firehose:*"], "Resource":["arn:aws:firehose:
your-region
:your-aws-account-id
:deliverystream/VPCtoSplunkStream"] }, { "Effect":"Allow", "Action":["iam:PassRole"], "Resource":["arn:aws:iam::your-aws-account-id
:role/VPCtoSplunkCWtoFHRole"] } ] } -
In a command window, go to the directory where you saved
VPCtoSplunkCWtoFHAccessPolicy.json
, and run the following AWS CLI command.aws iam put-role-policy --role-name VPCtoSplunkCWtoFHRole --policy-name VPCtoSplunkCWtoFHAccessPolicy --policy-document file://VPCtoSplunkCWtoFHAccessPolicy.json
-
Replace the
your-region
andyour-aws-account-id
placeholders in the following AWS CLI command with your AWS Region code and account ID, and then run the command.aws logs put-subscription-filter --log-group-name "VPCtoSplunkLogGroup" --filter-name "Destination" --filter-pattern "" --destination-arn "arn:aws:firehose:
your-region
:your-aws-account-id
:deliverystream/VPCtoSplunkStream" --role-arn "arn:aws:iam::your-aws-account-id
:role/VPCtoSplunkCWtoFHRole"
Proceed to Step 4: Check the Results in Splunk and in Kinesis Data Firehose.