Managing data access for Security Lake subscribers

Subscribers with data access to source data in Amazon Security Lake are notified of new objects for the source as the data is written to the S3 bucket. By default, subscribers are notified about new objects through an HTTPS endpoint that they provide. Alternatively, subscribers can be notified about new objects by polling an Amazon Simple Queue Service (Amazon SQS) queue.

Prerequisites for creating a subscriber with data access

You must complete the following prerequisites before you can create a subscriber with data access in Security Lake.

1. Create IAM role to invoke EventBridge API destinations

Before you can create a subscriber, you must create a role in AWS Identity and Access Management (IAM) that grants Amazon EventBridge permissions to invoke API destinations and send object notifications to the correct HTTPS endpoints.

Note

Security Lake creates this IAM role or uses an existing role on your behalf when you use the Security Lake console. However, you must create this role when using the Security Lake API or AWS CLI.

After creating this IAM role, you'll need the Amazon Resource Name (ARN) of the role in order to create the subscriber. This IAM role isn't necessary if the subscriber polls data from an Amazon Simple Queue Service (Amazon SQS) queue or directly queries data from AWS Lake Formation. For more information about this type of subscriber, see Managing query access for Security Lake subscribers.

Attach the following policy to your IAM role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowInvokeApiDestination",
            "Effect": "Allow",
            "Action": [
                "events:InvokeApiDestination"
            ],
            "Resource": [
                "arn:aws:events:{region}:{accountId}:api-destination/AmazonSecurityLake*/*"
            ]
        }
    ]
}

Attach the following trust policy to your role to permit EventBridge to assume the role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowEventBridgeToAssume",
            "Effect": "Allow",
            "Principal": {
                "Service": "events.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Security Lake automatically creates an IAM role that permits the subscriber to read data from the data lake (or poll events from an Amazon SQS queue if that's the preferred method of notification). This role is protected with an AWS managed policy called AmazonSecurityLakePermissionsBoundary.

2. Verify permissions

Before creating a subscriber with data access, verify that you have permission to perform the following actions:

• iam:CreateRole

• iam:DeleteRolePolicy

• iam:GetRole

• iam:PutRolePolicy

• lakeformation:GrantPermissions

• lakeformation:ListPermissions

• lakeformation:RegisterResource

• lakeformation:RevokePermissions

• ram:GetResourceShareAssociations

• ram:GetResourceShares

• ram:UpdateResourceShare

In addition to the preceding list, you also need permissions for the following actions to notify subscribers when new data is written to the data lake:

• events:CreateApiDestination

• events:CreateConnection

• events:DescribeRule

• events:ListApiDestinations

• events:ListConnections

• events:PutRule

• events:PutTargets

• s3:GetBucketNotification

• s3:PutBucketNotification

• sqs:CreateQueue

• sqs:DeleteQueue

• sqs:GetQueueAttributes

• sqs:GetQueueUrl

• sqs:SetQueueAttributes

To verify your permissions, use IAM to review the IAM policies that are attached to your IAM identity. Then, compare the information in those policies to the following list of actions that you must be allowed to perform to create a subscriber with data access.

3. Get the subscriber's external ID

To create a subscriber, you'll also need to get an external ID from the subscriber. The external ID is a unique identifier that the subscriber provides to you. Security Lake adds the external ID to the subscriber IAM role that it creates. You also provide the external ID when you create a subscriber in the Security Lake console, API, or AWS CLI.

For more information about external IDs, see How to use an external ID when granting access to your AWS resources to a third party in the IAM User Guide.

Creating a subscriber with data access

After creating your IAM role and getting the external ID from the subscriber, you can create a subscriber with data access by following these steps. These steps create a subscriber with access to data in the current AWS Region.

Console

1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

2. By using the AWS Region selector in the upper-right corner of the page, select the Region where you want to create the custom source.

3. In the navigation pane, choose Subscribers.

4. On the Add subscribers tab, choose Create custom subscriber.

5. For Subscriber details, enter the subscriber's name and (optional) description.

6. For Log and event sources, choose which sources the subscriber is authorized to consume.

7. For Data access method, choose S3 to set up data access for the subscriber.

8. For Subscriber credentials, provide the subscriber's AWS account ID and external ID.

9. For Notification details, select SQS queue if you want Security Lake to create an Amazon SQS queue that the subscriber can poll for object notifications. Select Subscription endpoint if you want Security Lake to send notifications through EventBridge to an HTTPS endpoint.

10. If you selected Subscription endpoint in the previous step, for Service Access, create a new IAM role or use an existing IAM role that gives EventBridge permission to invoke API destinations and send object notifications to the correct endpoints.

11. If you selected Subscription endpoint in step 9, provide the subscription endpoint. Optionally, you can also provide an HTTPS key name and key value.

12. Choose Create.

API

1. Run CreateSubscriber.

2. Provide the subscriber's accountID and externalId.

3. Populate sourceTypes with the sources that you want the subscriber to consume.

4. For accessTypes, provide S3 to create a subscriber with data access.

5. Run CreateSubscriptionNotificationConfiguration to notify the subscriber when new data is written to the data lake for the sources that the subscriber consumes.

6. To override the default method of notification (HTTPS endpoint) and create an Amazon SQS queue, set createSqs equal to true. If you prefer notification with an HTTPS endpoint, populate the subscriptionEndpoint field of the CreateSubscriptionNotificationConfiguration API. Populate the roleArn field with the ARN of the EventBridge API destinations IAM role that you created.

AWS CLI

1. Run the create-subscriber command to create a subscriber.

2. Provide the subscriber's AWS account ID and external-id.

3. Populate source-types with the sources that you want the subscriber to consume.

4. For access-types, provide S3 to create a subscriber with data access.

   
   aws securitylake create-subscriber --account-id account ID --external-id external ID --subscriber-name subscriber name --source-types source types --access-types access types

5. Run the create-subscription-notification-configuration command to notify the subscriber when new data is written to the data lake for the sources that the subscriber consumes. If you prefer notification with an HTTPS endpoint, populate the subscription-endpoint field of the create-subscription-notification-configuration command. Populate the role-arn field with the ARN of the EventBridge API destinations IAM role that you created.

   
   aws securitylake create-subscription-notification-configuration --subscription-endpoint HTTPS ENDPOINT --role-arn role ARN --subscription-id subscription ID
   

6. To override the default method of notification (HTTPS endpoint) and create an Amazon SQS queue, run the create-subscription-notification-configuration command. Set the create-sqs parameter equal to true.

   
   aws securitylake create-subscription-notification-configuration --create-sqs true --subscription-id subscription ID
   

To update the notification method for the subscription (SQS or HTTPs endpoint), run the UpdateSubscriptionNotificationConfiguration API or the update-subscription-notification-configuration command. You can also update the notification method on the Security Lake console by editing a subscriber on the Subscribers page.

Sample object notification message

{
  "source": "aws.s3",
  "time": "2021-11-12T00:00:00Z",
  "account": "123456789012",
  "region": "ca-central-1",
  "resources": [
    "arn:aws:s3:::example-bucket"
  ],
  "detail": {
    "bucket": {
      "name": "example-bucket"
    },
    "object": {
      "key": "example-key",
      "size": 5,
      "etag": "b57f9512698f4b09e608f4f2a65852e5"
    },
    "request-id": "N4N7GDK58NMKJ12R",
    "requester": "securitylake.amazonaws.com"
  }
}

Updating a subscriber

You can update a subscriber by changing the sources that the subscriber consumes data from. Choose your preferred access method, and follow these steps to define new sources for an existing subscription.

Console

1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

2. In the navigation pane, choose Subscribers.

3. Select a subscriber.

4. Choose Edit, and update the sources for the subscriber.

API

1. Run UpdateSubscriber.

2. Define the new sources for the subscription in sourceTypes.

AWS CLI

1. Run the update-subscriber command to update a subscriber.

2. Provide the subscription-id, and define the new sources for the subscription in source-types.

   
   aws securitylake update-subscriber--source-types sourceTypes --subscription-id subscriptionId
   

To get information about a specific subscriber, run GetSubscriber. This API returns the subscriber's account ID, name and description, notification method, and additional information. For a list of subscriptions associated with a specific organization or AWS account, run ListSubscribers. To update the notification to a subscriber by, for example, providing a new subscription endpoint or switching from an endpoint to an SQS queue, run UpdateSubscriptionNotificationConfiguration.

Removing a subscriber

If you no longer want a subscriber to consume data from Security Lake, you can remove the subscriber by following these steps.

Console

1. Open the Security Lake console at https://console.aws.amazon.com/securitylake/.

2. In the navigation pane, choose Subscribers.

3. Select a subscriber.

4. Choose Delete.

API

1. Run DeleteSubscriber to delete a subscription and all notification settings.

2. To retain the subscription but delete future notifications to the subscriber, run DeleteSubscriptionNotificationConfiguration.

AWS CLI

1. Run the delete-subscriber command to delete a subscription and all notification settings.

   aws securitylake delete-subscriber --subscription-id subscription ID

2. To retain the subscription but delete future notifications to the subscriber, run the delete-subscription-notification-configuration command.

   aws securitylake delete-subscription-notification-configuration --subscription-id subscription ID