此頁面尚未翻譯為您的語言。 請求翻譯

MLREL-03: Use a data catalog

PDFRSS

Process data across multiple data stores using data catalog technology. An advanced data catalog service can enable ETL process integration. This approach enables more reliability and efficiency.

Implementation plan

• Use AWS Glue Data Catalog - The AWS Glue Data Catalog provides a way to track the data assets that have been loaded into your ML workload. Data catalogs also describe how data is transformed as it is loaded into the data lake and data warehouse. AWS Glue is a fully managed ETL (extract, transform, and load) service. It enables a simple and cost-effective approach to categorize your data, clean it, enrich it, and move it reliably between various data stores and data streams. AWS Glue consists of a central metadata repository known as the AWS Glue Data Catalog. It also has an ETL engine that automatically generates Python or Scala code. With a flexible scheduler, AWS Glue handles dependency resolution, job monitoring, and retries.

Documents

• Data Cataloging

• Populating the AWSAWS Glue Data Catalog

Blogs

• Moving from notebooks to automated ML pipelines using Amazon SageMaker AI and AWS Glue

• How Genworth built a serverless ML pipeline on AWS using Amazon SageMaker AI and AWS Glue

Videos

• Getting Started with AWS Glue Data Catalog

• AWS re:Invent 2018: How Bill.com Uses Amazon SageMaker AI & AWS Glue to Enable Machine Learning - STP10

• AWS re:Invent 2018: Build and Govern Your Data Lakes with AWS Glue (ANT309)

Examples

• Explaining Credit Decisions with Amazon SageMaker AI

• How to build an end-to-end Machine Learning pipeline using AWS Glue, Amazon S3, Amazon SageMaker AI and Amazon Athena.