Performing Complex ETL Activities Using Blueprints and Workflows in AWS Glue - AWS Glue

Performing Complex ETL Activities Using Blueprints and Workflows in AWS Glue

The AWS Glue blueprints feature is in preview release for AWS Glue and is subject to change.

Some of your organization's complex extract, transform, and load (ETL) processes might best be implemented by using multiple, dependent AWS Glue jobs and crawlers. Using AWS Glue workflows, you can design a complex multi-job, multi-crawler ETL process that AWS Glue can execute and track as single entity. After you create a workflow and specify the jobs, crawlers, and triggers in the workflow, you can run the workflow on demand or on a schedule.

Your organization might have a set of similar ETL use cases that could benefit from being able to parameterize a single workflow to handle them all. To address this need, AWS Glue enables you to define blueprints, which you can use to generate workflows. A blueprint accepts parameters, so that from a single blueprint, a data analyst can create different workflows to handle similar ETL use cases. After you create a blueprint, you can reuse it for different departments, teams, and projects.