MLREL-07: Ensure feature consistency across training and inference

Ensure consistent, scalable, and highly available features between training and inference using a feature storage. This results in reducing the training-serving skew by keeping feature consistency between training and inference.

Implementation plan

• Use Amazon SageMaker Feature Store -Create, share, and manage features for ML development using SageMaker Feature Store. The Feature Store is a centralized store for features and associated metadata so features can be easily discovered and reused. The online store is used for low latency, real-time inference use cases. The offline store is used for training and batch inference. The Feature Store reduces the repetitive data processing and curation work required to convert raw data into features for training an ML algorithm. Features generated will be used for both training and inference, reducing the training-serving skew. The Feature Store enables feature consistency, feature standardization, and the ability to integrate with Amazon SageMaker Pipelines.

Documents

• Get started with Amazon SageMakerFeature Store

Blogs

• Store, Discover, and Share Machine Learning Features with Amazon SageMakerFeature Store

• Using streaming ingestion with Amazon SageMakerFeature Store to make ML-backed decisions in near-real time

Videos

• AWS re:Invent 2020: Amazon SageMakerFeature Store: Store, discover, & share features for ML apps

• Introducing Amazon SageMaker Feature Store - AWS re:Invent2020

Examples

• Amazon SageMaker Feature Store: Introduction to Feature Store

• Amazon SageMaker Feature Store

• Amazon SageMaker Feature Store Introduction

• Amazon SageMaker Feature Store: Streaming Aggregation