Managing Cross-Account Permissions Using Both AWS Glue and Lake Formation - AWS Lake Formation

Managing Cross-Account Permissions Using Both AWS Glue and Lake Formation

It's possible to grant cross-account access to Data Catalog resources and underlying data by using either AWS Glue or AWS Lake Formation.

In AWS Glue, you grant cross-account permissions by creating a Data Catalog resource policy. In Lake Formation, you grant cross-account permissions by using the Lake Formation GRANT/REVOKE permissions model and the GrantPermissions API operation.

Tip

We recommend that you remove the Data Catalog resource policy and rely solely on Lake Formation permissions to secure your data lake.

You can view Lake Formation cross-account grants by using the Lake Formation console or the AWS Resource Access Manager (AWS RAM) console. However, those console pages don't show cross-account permissions granted by the AWS Glue Data Catalog resource policy. Similarly, you can view the cross-account grants in the Data Catalog resource policy using the Settings page of the AWS Glue console, but that page doesn't show the cross-account permissions granted using Lake Formation.

To ensure that you don't miss any grants when viewing and managing cross-account permissions, Lake Formation and AWS Glue require you to perform the following actions to indicate that you are aware of and are permitting cross-account grants by both Lake Formation and AWS Glue.

When Granting Cross-Account Permissions Using AWS Glue

If no Lake Formation cross-account grants have been made from your account, you can save a Data Catalog resource policy as usual in AWS Glue. However, if Lake Formation grants have already been made, you must do one of the following to ensure that saving the resource policy succeeds:

  • When you save the resource policy on the Settings page of the AWS Glue console, the console issues an alert stating that the permissions in the policy will be in addition to any permissions granted using Lake Formation. You must choose Proceed to save the policy.

  • When you save the resource policy using the glue:PutResourcePolicy API operation, you must set the EnableHybrid member of the PutResourcePolicyRequest structure to 'TRUE' (type = string). The following code example shows how to do this in Python.

    import boto3 import json REGION = 'us-east-2' PRODUCER_ACCOUNT_ID = '123456789012' CONSUMER_ACCOUNT_IDs = ['111122223333'] glue = glue_client = boto3.client('glue') policy = { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowConsumerFullCatalogAccess", "Effect": "Allow", "Action": [ "glue:*" ], "Principal": { "AWS": CONSUMER_ACCOUNT_IDs }, "Resource": [ f"arn:aws:glue:{REGION}:{PRODUCER_ACCOUNT_ID}:catalog", f"arn:aws:glue:{REGION}:{PRODUCER_ACCOUNT_ID}:database/*", f"arn:aws:glue:{REGION}:{PRODUCER_ACCOUNT_ID}:table/*/*" ] } ] } policy_str = json.dumps(policy) glue.put_resource_policy(PolicyInJson=policy_str, EnableHybrid='TRUE')

    For more information, see PutResourcePolicy Action (Python: put_resource_policy) in the AWS Glue Developer Guide.

When Granting Cross-Account Permissions Using Lake Formation

If there is no Data Catalog resource policy in your account, Lake Formation cross-account grants that you make proceed as usual. However, if a Data Catalog resource policy exists, you must add the following statement to it to permit your Lake Formation cross-account grants to succeed. Replace <region> with a valid Region name and <account-id> with your AWS account ID.

{ "Effect": "Allow", "Action": [ "glue:ShareResource" ], "Principal": {"Service": [ "ram.amazonaws.com" ]}, "Resource": [ "arn:aws:glue:<region>:<account-id>:table/*/*", "arn:aws:glue:<region>:<account-id>:database/*", "arn:aws:glue:<region>:<account-id>:catalog" ] }

Without this additional statement, the Lake Formation grant succeeds, but becomes blocked in AWS RAM, and the recipient account can't access the granted resource.

See Also: