Step 1: Provision your resources - AWS Lake Formation

Step 1: Provision your resources

This section shows you how to set up the AWS resources using an AWS CloudFormation template.

To create your resources using AWS CloudFormation template

  1. Sign into the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation as an IAM administrator in the US East (N. Virginia) Region.

  2. Choose Launch Stack.

  3. Choose Next on the Create stack screen.

  4. Enter a Stack name.

  5. Choose Next.

  6. On the next page, choose Next.

  7. Review the details on the final page and select I acknowledge that AWS CloudFormation might create IAM resources.

  8. Choose Create.

    The stack creation can take up to two minutes.

Launching the cloud formation stack creates the following resources:

  • lf-otf-datalake-123456789012 – Amazon S3 bucket to store data

    Note

    The account id appended to the Amazon S3 bucket name is replaced with your account id.

  • lf-otf-tutorial-123456789012 – Amazon S3 bucket to store query results and AWS Glue job scripts

  • lficebergdb – AWS Glue Iceberg database

  • lfhudidb – AWS Glue Hudi database

  • lfdeltadb – AWS Glue Delta database

  • native-iceberg-create – AWS Glue job that creates an Iceberg table in the Data Catalog

  • native-hudi-create – AWS Glue job that creates a Hudi table in the Data Catalog

  • native-delta-create – AWS Glue job that creates a Delta table in the Data Catalog

  • LF-OTF-GlueServiceRole – IAM role that you pass to AWS Glue to run the jobs. This role has the required policies attached to access the resources like Data Catalog, Amazon S3 bucket etc.

  • LF-OTF-RegisterRole – IAM role to register the Amazon S3 location with Lake Formation. This role has LF-Data-Lake-Storage-Policy attached to the role.

  • lf-consumer-analystuser – IAM user to query the data using Athena

  • lf-consumer-analystuser-credentials – Password for the data analyst user stored in AWS Secrets Manager

After the stack creations is complete, navigate to the output tab and note down the values for:

  • AthenaQueryResultLocation – Amazon S3 location for Athena query output

  • BusinessAnalystUserCredentials – Password for the data analyst user

    To retrieve the password value:

    1. Choose the lf-consumer-analystuser-credentials value by navigating to the Secrets Manager console.

    2. In the Secret value section, choose Retrieve secret value.

    3. Note down the secret value for the password.