Create Feature Groups
A FeatureGroup is the main Feature Store resource that contains the metadata for all
the data stored in Amazon SageMaker Feature Store. A feature group is a logical grouping of features, defined in the
feature store, to describe records. A feature group’s definition is composed of a list of
feature definitions, a record identifier name, and configurations for its online and offline
store. The example code in this topic uses the SageMaker Python SDK. The underlying APIs are
available for developers using other languages.
Prior to using a feature store you typically load your dataset, run transformations, and
set up your features for ingestion. This process has a lot of variation and is highly dependent
on your data. The example code in the following topics refer to the Introduction to Feature Store
Feature Store supports the following data types: String,
Fractional (IEEE 64-bit floating point value), and
Integral (Int64 - 64 bit signed integral value). The default type is set to String. This means that, if a
column in your dataset is not a float or long type, it defaults to
String in your feature store.
You may use a schema to describe your data’s columns and data types. You pass this schema
into FeatureDefinitions, a required parameter for a FeatureGroup. You
can use the SageMaker Python SDK, which has automatic data type detection when you use the
load_feature_definitions function.
The default behavior when a new feature record is added with an already existing record ID is as follows. In the offline store, the new record will be appended. In the online store, if the event time of the new record is less than the existing event time than nothing will happen, however if the event time of the new record is greater than or equal to the existing event time, the record will be over written.
When you create a new feature group you can choose one of the following table formats:
-
AWS Glue (Default)
-
Apache Iceberg
Ingesting data, especially when streaming, can result in a large number of small files deposited into the offline store. This can negatively impact query performance due the higher number of file operations required. To avoid potential performance issues, use the Apache Iceberg table format when creating new feature groups. With Iceberg you can compact the small data files into fewer large files in the partition, resulting in significantly faster queries. This compaction operation is concurrent and does not affect ongoing read and write operations on the feature group. If you choose the Iceberg option when creating new feature groups, Amazon SageMaker Feature Store will create the Iceberg tables using Parquet file format, and register the tables with the AWS Glue Data Catalog.
Note that for feature groups in Iceberg table format, you must specify String as the value for the event time. If you specify any other type, you can't create the feature group successfully.
