AWS Lake Formation
Developer Guide

What Is AWS Lake Formation?

Welcome to the AWS Lake Formation Developer Guide.

AWS Lake Formation is a fully managed service that makes it easier for you to build, secure, and manage data lakes. Lake Formation simplifies and automates many of the complex manual steps that are usually required to create data lakes. These steps include collecting, cleansing, moving, and cataloging data, and securely making that data available for analytics and machine learning. You point Lake Formation at your data sources, and Lake Formation crawls those sources and moves the data into your new Amazon Simple Storage Service (Amazon S3) data lake.

Lake Formation provides its own permissions model that augments the AWS Identity and Access Management (IAM) permissions model. This centrally defined permissions model enables fine-grained access to data stored in data lakes through a simple grant/revoke mechanism.

Lake Formation permissions are enforced at the table and column level across the full portfolio of AWS analytics and machine learning services, including Amazon Athena and Amazon Redshift.

AWS Service Integrations with Lake Formation

The following AWS services integrate with Lake Formation and honor Lake Formation permissions.

AWS Service How Integrated
AWS Glue AWS Glue and Lake Formation share the same Data Catalog. For console operations (such as viewing a list of tables) and all API operations, AWS Glue users can access only the databases and tables on which they have Lake Formation permissions. AWS Glue does not support Lake Formation column permissions.
Amazon Athena When Amazon Athena users select the AWS Glue catalog in the query editor, they can query only the databases, tables, and columns on which they have Lake Formation permissions.
Amazon Redshift Spectrum When Amazon Redshift users create an external schema on a database in the AWS Glue catalog, they can query only the tables and columns in that schema on which they have Lake Formation permissions.
Amazon EMR (public beta) Lake Formation permissions are enforced when Apache Spark applications are submitted using Apache Zeppelin or EMR Notebooks.

Getting Started with Lake Formation

We recommend that you start with the following sections: