Lake Formation Permissions Reference - AWS Lake Formation

Lake Formation Permissions Reference

To perform AWS Lake Formation operations, principals need both Lake Formation permissions and AWS Identity and Access Management (IAM) permissions. You typically grant IAM permissions using coarse-grained access control policies, as described in Lake Formation Access Control Overview. You can grant Lake Formation permissions by using the console, the API, or the AWS Command Line Interface (AWS CLI).

To learn how to grant or revoke Lake Formation permissions, see Granting and Revoking Data Catalog Permissions and Granting Data Location Permissions.

Note

The examples in this section show how to grant permissions to principals in the same AWS account. For examples of cross-account grants, see Granting Lake Formation Permissions.

Lake Formation Grant and Revoke AWS CLI Commands

Each permission description in this section includes examples of granting the permission using an AWS CLI command. The following are the synopses of the Lake Formation grant-permissions and revoke-permissions AWS CLI commands.

grant-permissions [--catalog-id <value>] --principal <value> --resource <value> --permissions <value> [--permissions-with-grant-option <value>] [--cli-input-json <value>] [--generate-cli-skeleton <value>]
revoke-permissions [--catalog-id <value>] --principal <value> --resource <value> --permissions <value> [--permissions-with-grant-option <value>] [--cli-input-json <value>] [--generate-cli-skeleton <value>]

For detailed descriptions of these commands, see grant-permissions and revoke-permissions in the AWS CLI Command Reference. This section provides additional information on the --principal option.

The value of the --principal option is one of the following:

  • Amazon Resource Name (ARN) for an AWS Identity and Access Management (IAM) user or role

  • ARN for an Active Directory user or group

  • ARN for an Amazon QuickSight user or group

  • For cross-account permissions, the ARN for an AWS account ID, organization ID, or organizational unit ID

The following are syntax and examples for all --principal types.

Principal is an IAM user

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:iam::<account-id>:user/<user-name>

Example:

--principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1
Principal is an IAM role

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:iam::<account-id>:role/<role-name>

Example:

--principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:role/workflowrole
Principal is an Active Directory user

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:iam::<account-id>:saml-provider/<SAMLproviderName>:user/<user-name>

Example:

--principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:saml-provider/idp1:user/datalake_user1
Principal is an Active Directory group

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:iam::<account-id>:saml-provider/<SAMLproviderName>:group/<group-name>

Example:

--principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:saml-provider/idp1:group/data-scientists
Principal is an Amazon QuickSight Enterprise Edition user

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:quicksight:<region>:<account-id>:user/<namespace>/<user-name>
Note

For <namespace>, you must specify default.

Example:

--principal DataLakePrincipalIdentifier=arn:aws:quicksight:us-east-1:111122223333:user/default/bi_user1
Principal is an Amazon QuickSight Enterprise Edition group

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:quicksight:<region>:<account-id>:group/<namespace>/<group-name>
Note

For <namespace>, you must specify default.

Example:

--principal DataLakePrincipalIdentifier=arn:aws:quicksight:us-east-1:111122223333:group/default/data_scientists
Principal is an AWS account

Syntax:

--principal DataLakePrincipalIdentifier=<account-id>

Example:

--principal DataLakePrincipalIdentifier=111122223333
Principal is an organization

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:organizations::<account-id>:organization/<organization-id>

Example:

--principal DataLakePrincipalIdentifier=arn:aws:organizations::111122223333:organization/o-abcdefghijkl
Principal is an organizational unit

Syntax:

--principal DataLakePrincipalIdentifier=arn:aws:organizations::<account-id>:ou/<organization-id>/<organizational-unit-id>

Example:

--principal DataLakePrincipalIdentifier=arn:aws:organizations::111122223333:ou/o-abcdefghijkl/ou-ab00-cdefghij

ALTER

Permission Granted on This Resource Grantee Also Needs
ALTER DATABASE glue:UpdateDatabase
ALTER TABLE glue:UpdateTable

A principal with this permission can alter metadata for a database or table in the Data Catalog. For tables, you can change the column schema and add column parameters. You cannot alter columns in the underlying data that a metadata table points to.

If the property that is being altered is a registered Amazon Simple Storage Service (Amazon S3) location, the principal must have data location permissions on the new location.

The following example grants the ALTER permission to user datalake_user1 on the database retail in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "ALTER" --resource '{ "Database": {"Name":"retail"}}'

The following example grants ALTER to user datalake_user1 on the table inventory in the database retail.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "ALTER" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory"}}'

CREATE_DATABASE

Permission Granted on This Resource Grantee Also Needs
CREATE_DATABASE Data Catalog glue:CreateDatabase

A principal with this permission can create a metadata database or resource link in the Data Catalog. The principal can also create tables in the database.

The following example grants CREATE_DATABASE to user datalake_user1 in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "CREATE_DATABASE" --resource '{ "Catalog": {}}'

When a principal creates a database in the Data Catalog, no permissions to underlying data are granted. The following additional metadata permissions are granted (along with the ability to grant these permissions to others):

  • CREATE_TABLE in the database

  • ALTER database

  • DROP database

When creating a database, the principal can optionally specify an Amazon S3 location. Depending on whether the principal has data location permissions, the CREATE_DATABASE permission might not be sufficient to create databases in all cases. It is important to keep the following three cases in mind.

Create Database Use Case Permissions Needed
The location property is unspecified. CREATE_DATABASE is sufficient.
The location property is specified, and the location is not managed by Lake Formation (is not registered). CREATE_DATABASE is sufficient.
The location property is specified, and the location is managed by Lake Formation (is registered). CREATE_DATABASE is required plus data location permissions on the specified location.

CREATE_TABLE

Permission Granted on This Resource Grantee Also Needs
CREATE_TABLE DATABASE glue:CreateTable

A principal with this permission can create a metadata table or resource link in the Data Catalog within the specified database.

The following example grants the user datalake_user1 permission to create tables in the retail database in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "CREATE_TABLE" --resource '{ "Database": {"Name":"retail"}}'

When a principal creates a table in the Data Catalog, both metadata permissions and permissions to underlying data are granted. The following additional permissions are granted to the table creator, along with the ability to grant these permissions to others:

  • ALTER table

  • DROP table

  • SELECT (on all columns) data

  • INSERT data

  • DELETE data

When you create a table that points to an Amazon S3 location, depending on whether the principal has data location permissions, the CREATE_TABLE permission might not be sufficient to create tables in all cases. It's important to keep the following three cases in mind.

Create Table Use Case Permissions Needed
The specified location is not managed by Lake Formation (is not registered). CREATE_TABLE is sufficient.
The specified location is managed by Lake Formation (is registered), and the containing database has no location property or has a location property that is not an Amazon S3 prefix of the table location. CREATE_TABLE is required plus data location permissions on the specified location.
The specified location is managed by Lake Formation (is registered), and the containing database has a location property that points to a location that is registered and is an Amazon S3 prefix of the table location. CREATE_TABLE is sufficient.

DATA_LOCATION_ACCESS

Permission Granted on This Resource Grantee Also Needs
DATA_LOCATION_ACCESS Amazon S3 location (Amazon S3 permissions on the location, which must be specified by the role used to register the location.)

This is the only data location permission. A principal with this permission can create a metadata database or table that points to the specified Amazon S3 location. The location must be registered. A principal who has data location permissions on a location also has location permissions on child locations.

The following example grants data location permissions on s3://products/retail to user datalake_user1 in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DATA_LOCATION_ACCESS" --resource '{ "DataLocation": {"S3Location":"s3://products/retail"}}'

DATA_LOCATION_ACCESS is not needed to query or update underlying data. This permission applies only to creating Data Catalog resources.

For more information about data location permissions, see Underlying Data Access Control.

DELETE

Permission Granted on This Resource Grantee Also Needs
DELETE TABLE (No additional IAM permissions are needed if the location is registered.)

A principal with this permission can delete underlying data at the Amazon S3 location specified by the table. The principal can also view the table on the Lake Formation console and retrieve information about the table with the AWS Glue API.

The following example grants the DELETE permission to the user datalake_user1 on the table inventory in the database retail in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DELETE" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory"}}'

This permission applies only to data in Amazon S3, and not to data in other data stores such as Amazon Relational Database Service (Amazon RDS).

DESCRIBE

Permission Granted on This Resource Grantee Also Needs
DESCRIBE Resource link glue:GetTable to grant DESCRIBE on a table resource link, and glue:GetDatabase to grant DESCRIBE on a database resource link.

A principal with this permission can view and access the specified database or table resource link.

The following example grants the DESCRIBE permission to the user datalake_user1 on the table resource link inventory-link in the database retail in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DESCRIBE" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory-link"}}'

DROP

Permission Granted on This Resource Grantee Also Needs
DROP DATABASE glue:DeleteDatabase
DROP TABLE glue:DeleteTable
DROP Resource link glue:DeleteDatabase to drop a database resource link, and glue:DeleteTable to drop a table resource link.

A principal with this permission can drop a database, table, or resource link in the Data Catalog.

The following example grants the DROP permission to the user datalake_user1 on the database retail in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DROP" --resource '{ "Database": {"Name":"retail"}}'

The following example grants DROP to the user datalake_user1 on the table inventory in the database retail.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DROP" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory"}}'

The following example grants DROP to the user datalake_user1 on the table resource link inventory-link in the database retail.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "DROP" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory-link"}}'

INSERT

Permission Granted on This Resource Grantee Also Needs
INSERT TABLE (No additional IAM permissions are needed if the location is registered.)

A principal with this permission can insert and update underlying data at the Amazon S3 location specified by the table. The principal can also view the table in the Lake Formation console and retrieve information about the table with the AWS Glue API.

The following example grants the INSERT permission to the user datalake_user1 on the table inventory in the database retail in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "INSERT" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory"}}'

This permission applies only to data in Amazon S3, and not to data in other data stores such as Amazon RDS.

SELECT

Permission Granted on This Resource Grantee Also Needs
SELECT
  • TABLE

  • COLUMN

(No additional IAM permissions are needed if the location is registered.)

A principal with this permission can select metadata from columns of a table in the Data Catalog, and can query the underlying data in Amazon S3 at the location specified by the table. The principal can also view the table in the Lake Formation console and retrieve information about the table with the AWS Glue API.

The following example grants the SELECT permission to the user datalake_user1 on all columns in the table inventory in the database retail in AWS account 1111-2222-3333.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" --resource '{ "Table": {"DatabaseName":"retail", "Name":"inventory"}}'

This permission applies only to data in Amazon S3, and not to data in other data stores such as Amazon RDS.

You can limit the access to specific columns with an optional inclusion list or an exclusion list. An inclusion list specifies the columns that are granted access. An exclusion list specifies the columns that are not granted access. In the absence of an inclusion or exclusion list, access is granted to all columns in a table.

The results of glue:GetTable return only the columns that the caller has permission to view. Integrated services such as Amazon Athena and Amazon Redshift honor column inclusion and exclusion lists.

The following example grants SELECT to the user datalake_user1 on a subset of columns using an inclusion list.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" --resource '{ "TableWithColumns": {"DatabaseName":"retail", "Name":"inventory", "ColumnNames": ["prodcode","location","period","withdrawals"]}}'

This next example grants SELECT on a subset of columns using an exclusion list.

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT" --resource '{ "TableWithColumns": {"DatabaseName":"retail", "Name":"inventory", "ColumnWildcard": {"ExcludedColumnNames": ["intkey", "prodcode"]}}}'

The following restrictions apply to the SELECT permission:

  • When granting SELECT, you can include the grant option only when granting SELECT on all columns.

  • You cannot limit access control on columns that are partition keys.

  • A principal with the SELECT permission on a subset of columns in a table cannot be granted the ALTER, DROP, DELETE, or INSERT permission on that table. Similarly, a principal with the ALTER, DROP, DELETE, or INSERT permission on a table cannot be granted SELECT permission on a subset of columns in the table.

The SELECT permission always appears on the Data permissions page of the Lake Formation console as a separate row. This following image shows that SELECT is granted to the users datalake_user2 and datalake_user3 on all columns in the inventory table.


            The Data permissions page shows two rows for user datalake_user1 and table
              inventory. The first row lists the Delete and Insert permissions with resource type
              Table, and the second row lists the Select permission with resource type Column, and
              with the resource shown as retail.inventory.*.

Super

Permission Granted on This Resource Grantee Also Needs
Super DATABASE glue:*Database*
Super TABLE glue:*Table*, glue:*Partition*

This permission allows a principal to perform every supported Lake Formation operation on the database or table. This permission can coexist with the other Lake Formation permissions. For example, you can grant the Super, SELECT, and INSERT permissions on a metadata table. The principal can then perform all supported operations on the table. When you revoke Super, the SELECT and INSERT permissions remain, and the principal can perform only select and insert operations.

Instead of granting Super to an individual principal, you can grant it to the group IAMAllowedPrincipals. The IAMAllowedPrincipals group is automatically created and includes all IAM users and roles that are permitted access to your Data Catalog resources by your IAM policies. When Super is granted to IAMAllowedPrincipals for a Data Catalog resource, access to the resource is effectively controlled solely by IAM policies.

You can cause the Super permission to be automatically granted to IAMAllowedPrincipals for new catalog resources by taking advantage of options on the Settings page of the Lake Formation console.


            The Data catalog settings dialog box has the subtitle "Default permissions for
              newly created databases and tables," and has two check boxes, which are described in
              the text.
  • To grant Super to IAMAllowedPrincipals for all new databases, select Use only IAM access control for new databases.

  • To grant Super to IAMAllowedPrincipals for all new tables in new databases, select Use only IAM access control for new tables in new databases.

    Note

    This option causes the check box Use only IAM access control for new tables in this database in the Create database dialog box to be selected by default. It does nothing more than that. It is the check box in the Create database dialog box that enables the grant of Super to IAMAllowedPrincipals.

These Settings page options are enabled by default. For more information, see the following: