Using Iceberg tables

Athena supports read, time travel, write, and DDL queries for Apache Iceberg tables that use the Apache Parquet format for data and the AWS Glue catalog for their metastore.

Apache Iceberg is an open table format for very large analytic datasets. Iceberg manages large collections of files as tables, and it supports modern analytical data lake operations such as record-level insert, update, delete, and time travel queries. The Iceberg specification allows seamless table evolution such as schema and partition evolution and is designed for optimized usage on Amazon S3. Iceberg also helps guarantee data correctness under concurrent write scenarios.

For more information about Apache Iceberg, see https://iceberg.apache.org/.

Considerations and limitations

Athena support for Iceberg tables has the following limitations:

• Tables with AWS Glue catalog only – Only Iceberg tables created against the AWS Glue catalog based on specifications defined by the open source glue catalog implementation are supported from Athena.

• Table locking support by AWS Glue only – Unlike the open source Glue catalog implementation, which supports plug-in custom locking, Athena supports AWS Glue optimistic locking only. Using Athena to modify an Iceberg table with any other lock implementation will cause potential data loss and break transactions.

• Supported file formats – Iceberg file format support in Athena depends on the Athena engine version, as shown in the following table.

  Athena engine version	Parquet	ORC	Avro
  2	Yes	No	No
  3	Yes	Yes	Yes

• Iceberg v2 tables – Athena only creates and operates on Iceberg v2 tables. For the difference between v1 and v2 tables, see Format version changes in the Apache Iceberg documentation.

• Display of time types without time zone – The time and timestamp without time zone types are displayed in UTC. If the time zone is unspecified in a filter expression on a time column, UTC is used.

• Timestamp related data precision – Iceberg supports microsecond precision for the timestamp data type. However, timestamp precision is handled differently depending on the Athena engine version.

  • In Athena engine version 2, millisecond precision is supported for timestamps for both read and write operations. Athena retains millisecond precision in time related columns only for data that is rewritten during manual compaction operations.

  • In Athena engine version 3, microsecond precision is supported for timestamps for both read and write operations, as described by Schemas and data types in the Apache Iceberg documentation.

• Unsupported operations – The following Athena operations are not supported for Iceberg tables.

  • ALTER TABLE SET LOCATION

• Views – Use CREATE VIEW to create Athena views as described in Working with views. If you are interested in using the Iceberg view specification to create views, contact [e-mail].

• OTF write management not supported in AWS Lake Formation – Although you can use Lake Formation to manage read access permissions for Open Table Formats (OTFs) like Apache Iceberg, Apache Hudi, and Delta Lake, you cannot use Lake Formation to manage write operations like VACUUM, MERGE, UPDATE or OPTIMIZE on these datasets. You also cannot use Lake Formation to update user write permissions on Iceberg tables. For more information about Lake Formation integration with Athena, see Using AWS Lake Formation with Amazon Athena in the AWS Lake Formation Developer Guide.

If you would like Athena to support a particular feature, send feedback to [e-mail].

Topics

• Creating Iceberg tables
• Managing Iceberg tables
• Querying Iceberg table metadata
• Evolving Iceberg table schema
• Querying Iceberg table data and performing time travel
• Updating Iceberg table data
• Optimizing Iceberg tables
• Supported data types
• Other Athena operations on Iceberg tables