Glue Schema Registry to Iceberg schema type mapping
When you register a schema in the AWS Glue Schema Registry, the Channel maps the JSON Schema types to Iceberg column types as follows.
| JSON Schema type | Condition | Iceberg column type |
|---|---|---|
| (plain) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| fits in 32-bit range |
|
| exceeds 32-bit range |
|
| with |
|
| plain floating point |
|
| — |
|
| with named properties |
|
| with |
|
| — |
|
| — |
|
Required columns — A column is marked required (non-nullable) when it appears in the "required" array of the parent object in your JSON Schema.
Partition key column — Your table must include a timestamptz column that can be used for time-based partitioning (hour). The source column referenced by the partition is automatically treated as required, even if your schema does not list it in the "required" array. You can enable an S3 Tables record-expiration job based on the table's partition column — see Managing S3 Tables record expiration.
The Channel handles source fields as follows:
Extra fields in source data — Fields present in your source records but not defined in the table schema are silently dropped; they are not written to the Iceberg table.
Missing optional fields — If a source record omits a field defined as optional in the table schema, the column is written as
null.Missing required fields — If a source record omits a field defined as required, the record fails validation and is sent to the dead-letter queue (DLQ) with an error indicating the missing required field.
Nesting limit — Schemas with more than 16 levels of nesting are not supported.