Manage IAM roles through Amazon Data Firehose console

Amazon Data Firehose is a fully managed service that delivers real-time streaming data to destinations. You can also configure Firehose to transform and convert the format of your data before delivery. To use these features, you must first provide IAM roles to grant permissions to Firehose when you create or edit a Firehose stream. Firehose uses this IAM role for all the permissions that the Firehose stream needs.

For example, consider a scenario where you create a Firehose stream that delivers data to Amazon S3, and this Firehose stream has Transform source records with AWS Lambda feature enabled. In this case, you must provide IAM roles to grant Firehose permissions to access the S3 bucket and invoke the Lambda function, as shown in the following.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "lambdaProcessing",
        "Effect": "Allow",
        "Action": ["lambda:InvokeFunction", "lambda:GetFunctionConfiguration"],
        "Resource": "arn:aws:lambda:us-east-1:<account id>:function:<lambda function name>:<lambda function version>"
    }, {
        "Sid": "s3Permissions",
        "Effect": "Allow",
        "Action": ["s3:AbortMultipartUpload", "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:PutObject"],
        "Resource": ["arn:aws:s3:::<bucket name>", "arn:aws:s3:::<bucket name>/*"]
    }]
}

Firehose console allows you to choose how you want to provide these roles. You can choose from one of the following options.

• Choose an existing IAM role

• Create a new IAM role from console

Choose an existing IAM role

You can choose from an existing IAM role. With this option, make sure that the IAM role you choose has a proper trust policy and permissions required for your source and destination. For more information, see Controlling access with Amazon Data Firehose.

Create a new IAM role from console

Alternatively, you could also use the Firehose console to create a new role on your behalf.

When Firehose creates an IAM role on your behalf, the role automatically includes all permission and trust policies that grant the required permissions based on the Firehose stream configuration.

For example, if you didn’t enable Transform source records with AWS Lambda feature then console generates the following statement in the permission policy.

{
  "Sid": "lambdaProcessing",
  "Effect": "Allow",
   "Action": [
     "lambda:InvokeFunction",
     "lambda:GetFunctionConfiguration"
   ],
   "Resource": "arn:aws:lambda:us-east-1:<account id>:function:%FIREHOSE_POLICY_TEMPLATE_PLACEHOLDER%"
}

Note

It's safe to ignore the policy statements that contain %FIREHOSE_POLICY_TEMPLATE_PLACEHOLDER% as they don't grant permissions on any resources.

The console create and edit Firehose stream workflows also create a trust policy and attach it to the IAM role. The trust policy allows Firehose to assume the IAM role. Following is a example of a trust policy.

{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "firehoseAssume",
        "Effect": "Allow",
        "Principal": {
            "Service": "firehose.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
    }]
}

Important

• You should avoid using the same console-managed IAM role for multiple Firehose streams. Otherwise, the IAM role could become overly permissive or result in errors.

• To use different policy statements within a permission policy from a console-managed IAM role, you can create your own IAM role, and copy the policy statements to a permission policy attached to the new role. To attach the role to the Firehose stream, select the Choose existing IAM role option in the Service access.

• Console manages any IAM role that contains the string service-role in its ARN. When you choose the existing IAM role option, make sure to select an IAM role without the service-role string in its ARN so that console doesn’t make any changes to it.

Steps to create an IAM role from console

1. Open the Firehose console at https://console.aws.amazon.com/firehose/.

2. Choose Create Firehose stream.

3. Choose a source and destination. For more information, see Create a Firehose stream.

4. Choose the destination settings. For more information, see Configure destination settings.

5. Under Advanced settings, for Service access, choose Create or update IAM role.

   Note

   This is a default option. To use an existing role, select the Choose existing IAM role option. Firehose console won’t make any changes to your own role.

6. Choose Create Firehose stream.

Edit IAM role from console

When you edit a Firehose stream, Firehose updates the corresponding permission policy accordingly to reflect the configuration and permission changes.

For example, when you edit the Firehose stream and enable Transform source records with AWS Lambda feature using the latest version of Lambda function as exampleLambdaFunction, you get the following policy statement in the permission policy.

{
  "Sid": "lambdaProcessing",
  "Effect": "Allow",
  "Action": [
    "lambda:InvokeFunction",
    "lambda:GetFunctionConfiguration"
  ],
  "Resource": "arn:aws:lambda:us-east-1:<account id>:function:exampleLambdaFunction:$LATEST"
}

Important

A console-managed IAM role is designed to be autonomous. We don't recommend that you modify the permission policy or trust policy outside of the console.

Steps to edit IAM role from console

1. Open the Firehose console at https://console.aws.amazon.com/firehose/.

2. Choose Firehose streams and choose the name of a Firehose stream you want to update.

3. On the Configuration tab, in the Server access section, choose Edit.

4. Update the IAM role option.

   Note

   By default, the console always updates an IAM role with the pattern service-role in its ARN. When you choose the existing IAM role option, make sure to select an IAM role without the service-role string in its ARN so that console doesn’t make any changes to it.

5. Choose Save changes.