Managing data access for Security Lake
subscribers
Subscribers with data access to source data in Amazon Security Lake are notified of new objects
for the source as the data is written to the S3 bucket. By default, subscribers are notified
about new objects through an HTTPS endpoint that they provide. Alternatively, subscribers
can be notified about new objects by polling an Amazon Simple Queue Service (Amazon SQS) queue.
Prerequisites to creating a subscriber
with data access
You must complete the following prerequisites before you can create a subscriber with
data access in Security Lake.
Verify
permissions
To verify your permissions, use IAM to review the IAM policies that are
attached to your IAM identity. Then, compare the information in those policies to
the following list of (permissions) actions that you must have to notify subscribers when new
data is written to the data lake.
You will need permission to perform the following actions:
-
iam:CreateRole
-
iam:DeleteRolePolicy
-
iam:GetRole
-
iam:PutRolePolicy
-
lakeformation:GrantPermissions
-
lakeformation:ListPermissions
-
lakeformation:RegisterResource
-
lakeformation:RevokePermissions
-
ram:GetResourceShareAssociations
-
ram:GetResourceShares
-
ram:UpdateResourceShare
In addition to the preceding list, you also need permission to perform the following actions:
Get the subscriber's external ID
To create a subscriber, apart from the subscriber's AWS account ID, you will also need to get
their external
ID. The external ID is a unique identifier that
the subscriber provides to you. Security Lake adds the external ID to the subscriber
IAM role that it creates. You use the external ID when you create a subscriber in the Security Lake console,
through the API, or AWS CLI.
For more information about external IDs, see How to use
an external ID when granting access to your AWS resources to a third
party in the IAM User Guide.
If you plan to use the Security Lake console to add a subscriber, you can
skip the next step and proceed to Creating a subscriber with data
access. The Security Lake console offers a
streamlined process for getting started, and creates all necessary IAM roles or uses
existing roles on your behalf.
If you plan to use Security Lake API or AWS CLI to add a subscriber, continue with
the next step to create an IAM role to invoke EventBridge API destinations.
Create IAM role to invoke EventBridge API
destinations (API and AWS CLI-only step)
If you're using Security Lake through API or AWS CLI, create a role in AWS Identity and Access Management (IAM)
that grants Amazon EventBridge permissions to invoke API destinations and send object
notifications to the correct HTTPS endpoints.
After creating this IAM role, you'll need the Amazon Resource Name (ARN) of the
role in order to create the subscriber. This IAM role isn't necessary if the
subscriber polls data from an Amazon Simple Queue Service (Amazon SQS) queue or directly queries data from
AWS Lake Formation. For more information about this type of data access method (access type), see Managing query access for Security Lake
subscribers.
Attach the following policy to your IAM role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowInvokeApiDestination",
"Effect": "Allow",
"Action": [
"events:InvokeApiDestination"
],
"Resource": [
"arn:aws:events:{us-west-2
}:{123456789012
}:api-destination/AmazonSecurityLake*/*"
]
}
]
}
Attach the following trust policy to your IAM role to permit EventBridge to assume the
role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowEventBridgeToAssume",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Security Lake automatically creates an IAM role that permits the subscriber to read
data from the data lake (or poll events from an Amazon SQS queue if that's the preferred
method of notification). This role is protected with an AWS managed policy called
AmazonSecurityLakePermissionsBoundary.
Creating a subscriber with data
access
Choose one of the following access methods to create a subscriber with access to data
in the current AWS Region.
- Console
-
Open the Security Lake console at https://console.aws.amazon.com/securitylake/.
-
By using the AWS Region selector in the upper-right corner of
the page, select the Region where you want to create the
subscriber.
-
In the navigation pane, choose
Subscribers.
-
On the Subscribers page, choose
Create subscriber.
-
For Subscriber details, enter
Subscriber name and an optional
Description.
The Region is auto-populated as your
currently selected AWS Region and can't be modified.
-
For Log and event sources, choose which
sources the subscriber is authorized to consume.
-
For Data access method, choose
S3 to set up data access for the
subscriber.
-
For Subscriber credentials, provide the
subscriber's AWS account ID and external ID.
-
(Optional) For Notification details, if you
want Security Lake to create an Amazon SQS queue that the subscriber can poll
for object notifications, select SQS queue. If
you want Security Lake to send notifications through EventBridge to an HTTPS
endpoint, select Subscription endpoint.
If you select Subscription endpoint, also do
the following:
-
Enter the Subscription endpoint.
Examples of valid endpoint formats include
http://example.com
. Optionally,
you can also provide an HTTPS key name
and HTTPS key value.
-
For Service Access, create a new
IAM role or use an existing IAM role that gives EventBridge
permission to invoke API destinations and send object
notifications to the correct endpoints.
For information about creating a new IAM role, see
Create IAM role to invoke EventBridge API
destinations.
-
(Optional) For Tags, enter as many as 50 tags
to assign to the subscriber.
A tag is a label that you can
define and assign to certain types of AWS resources. Each tag
consists of a required tag key and an optional tag value. Tags can
help you identify, categorize, and manage resources in different
ways. To learn more, see Tagging Amazon Security Lake resources.
-
Choose Create.
- API
-
To create a subscriber with data access programmatically, use the CreateSubscriber operation of the Security Lake API. If you're using the AWS Command Line Interface (AWS CLI), run
the create-subscriber command.
In your
request, use these parameters to specify the following settings for
the subscriber:
-
For sources
, specify each source that you want the
subscriber to access.
-
For subscriberIdentity
, specify the AWS account ID
and external ID that the subscriber will use to access source
data.
-
For subscriber-name
, specify the name of the subscriber.
-
For accessTypes
, specify S3
.
Example 1
The following example creates a subscriber with access to data in the
current AWS Region for the specified subscriber identity for an AWS
source.
$
aws securitylake create-subscriber \
--subscriber-identity {"accountID": 1293456789123
,"externalId": 123456789012
} \
--sources [{awsLogSource: {sourceName: VPC_FLOW
, sourceVersion: 1.0
}}] \
--subscriber-name subscriber name
\
--access-types S3
Example 2
The following example creates a subscriber with access to data in the
current AWS Region for the specified subscriber identity for a custom
source.
$
aws securitylake create-subscriber \
--subscriber-identity {"accountID": 1293456789123
,"externalId": 123456789012
} \
--sources [{customLogSource: {sourceName: custom-source-name
, sourceVersion: 1.0
}}] \
--subscriber-name subscriber name
--access-types S3
The preceding examples are formatted for Linux, macOS, or Unix, and they use the
backslash (\) line-continuation character to improve readability.
(Optional) After you create a subscriber, use the CreateSubscriberNotification operation to specify how to notify
the subscriber when new data is written to the data lake for the sources
that you want the subscriber to access. If you're using the AWS Command Line Interface (AWS CLI), run
the create-subscriber-notification command.
-
To override the default notification
method (HTTPS endpoint) and create an Amazon SQS queue, specify values for the
sqsNotificationConfiguration
parameters.
-
If you prefer notification with an HTTPS endpoint, specify values for the
httpsNotificationConfiguration
parameters.
-
For the targetRoleArn
field, specify the ARN of the IAM role that
you created to invoke EventBridge API destinations.
$
aws securitylake create-subscriber-notification \
--subscriber-id "12345ab8-1a34-1c34-1bd4-12345ab9012
" \
--configuration httpsNotificationConfiguration={"targetRoleArn"="arn:aws:iam::XXX:role/service-role/RoleName
", "endpoint"="https://account-management.$3.$2.securitylake.aws.dev/v1/datalake
"}
To get the subscriberID
, use the ListSubscribers operation of the Security Lake API. If you're using the AWS Command Line Interface (AWS CLI), run
the list-subscriber command.
$
aws securitylake list-subscribers
To subsequently change the notification method (Amazon SQS queue or HTTPS endpoint) for the
subscriber, use the UpdateSubscriberNotification operation or, if you're using the AWS CLI, run
the update-subscriber-notification command. You can also change the
notification method by using the Security Lake console: select the subscriber on the
Subscribers page, and then choose
Edit.
Sample object notification message
{
"source": "aws.s3",
"time": "2021-11-12T00:00:00Z",
"account": "123456789012",
"region": "ca-central-1",
"resources": [
"arn:aws:s3:::example-bucket"
],
"detail": {
"bucket": {
"name": "example-bucket"
},
"object": {
"key": "example-key",
"size": 5,
"etag": "b57f9512698f4b09e608f4f2a65852e5"
},
"request-id": "N4N7GDK58NMKJ12R",
"requester": "securitylake.amazonaws.com"
}
}
Updating a data subscriber
You can update a subscriber by changing the sources from which the subscriber
consumes. You can also assign or edit the tags for a subscriber. A tag is a label that you can define and assign to certain
types of AWS resources, including subscribers. To learn more, see Tagging Amazon Security Lake resources.
Choose one of the access methods, and follow these steps to define new sources for an
existing subscription.
- Console
-
Open the Security Lake console at https://console.aws.amazon.com/securitylake/.
-
In the navigation pane, choose
Subscribers.
-
Select the subscriber.
-
Choose Edit, and then do any of the
following:
-
To update the sources for the subscriber, enter the new
settings in the Log and event sources
section.
-
To assign or edit tags for the subscriber, change the tags
as necessary in the Tags
section.
-
When you finish, choose Save.
- API
-
To update data access sources for a subscriber programmatically, use the
UpdateSubscriber operation of the Security Lake API. If you're
using the AWS Command Line Interface (AWS CLI), run the update-subscriber command. In your request, use the
sources
parameters to specify each source that you want the
subscriber to access.
$
aws securitylake update-subscriber --subscriber-id subscriber ID
For a list of subscribers associated with a specific AWS account or
organization, use the ListSubscribers operation. If you're using the AWS Command Line Interface (AWS CLI), run the list-subscribers command.
$
aws securitylake list-subscribers
To review the current settings for a particular subscriber, use the
GetSubscriber operation. run the get-subscriber command. Security Lake then returns the subscriber's
name and description, external ID, and additional information. If you're
using the AWS Command Line Interface (AWS CLI), run the get-subscriber command.
To update the notification method for a subscriber, use the UpdateSubscriberNotification operation. If you're using the
AWS Command Line Interface (AWS CLI), run the update-subscriber-notification command. For example, you can
specify a new HTTPS endpoint for the subscriber or switch from an HTTPS
endpoint to an Amazon SQS queue.
Removing a data subscriber
If you no longer want a subscriber to consume data from Security Lake, you can remove the
subscriber by following these steps.
- Console
-
Open the Security Lake console at https://console.aws.amazon.com/securitylake/.
-
In the navigation pane, choose
Subscribers.
-
Select the subscriber that you want to remove.
-
Choose Delete and confirm the action. This will delete the subscriber
and all the associated notification settings.
- API
Based on your scenario, do one of the following:
-
To delete the subscriber and all associated notification settings,
use the DeleteSubscriber operation of the Security Lake API. If you're using the
AWS Command Line Interface (AWS CLI), run the delete-subscriber command.
-
To retain the subscriber but stop future notifications to the
subscriber, use the DeleteSubscriberNotification operation of the Security Lake
API. If you're using the
AWS Command Line Interface (AWS CLI), run the run the delete-subscriber-notification command.