About upgrading to the Lake Formation permissions model - AWS Lake Formation

About upgrading to the Lake Formation permissions model

To maintain backward compatibility with AWS Glue, by default, AWS Lake Formation grants the Super permission to the IAMAllowedPrincipals group on all existing AWS Glue Data Catalog resources, and grants the Super permission on new Data Catalog resources if the Use only IAM access control settings are enabled. This effectively causes access to Data Catalog resources and Amazon S3 locations to be controlled solely by AWS Identity and Access Management (IAM) policies. The IAMAllowedPrincipals group includes any IAM users and roles that are allowed access to your Data Catalog objects by your IAM policies. The Super permission enables a principal to perform every supported Lake Formation operation on the database or table on which it is granted.

You can start using Lake Formation to manage access to your data by registering the locations of existing Data Catalog resources in Lake Formation or by using hybrid access mode. When you register Amazon S3 location in hybrid access mode, you can enable Lake Formation permissions by opting in principals for databases and tables under that location.

To ease the transition of data lake permissions from an IAM and Amazon S3 model to Lake Formation permissions, we recommend you to use hybrid access mode for Data Catalog. With the hybrid access mode, you have an incremental path where you can enable Lake Formation permissions for a specific set of users without interrupting other existing users or workloads.

For more information, see Hybrid access mode.

Disable the default Data Catalog settings to move all existing users of a table to Lake Formation in a single step.

To start using Lake Formation permissions with your existing AWS Glue Data Catalog databases and tables, you must do the following:

  1. Determine your users’ existing IAM permissions for each database and table.

  2. Replicate these permissions in Lake Formation.

  3. For each Amazon S3 location that contains data:

    1. Revoke the Super permission from the IAMAllowedPrincipals group on each Data Catalog resource that references that location.

    2. Register the location with Lake Formation.

  4. Clean up existing fine-grained access control IAM policies.

Important

To add new users while in the process of transitioning your Data Catalog, you must set up granular AWS Glue permissions in IAM as before. You also must replicate those permissions in Lake Formation as described in this section. If new users have the coarse-grained IAM policies that are described in this guide, they can list any databases or tables that have the Super permission granted to IAMAllowedPrincipals. They can also view the metadata for those resources.

Follow the steps in this section to upgrade to the Lake Formation permissions model. Start with Step 1: List users' and roles' existing permissions.