Query Apache Iceberg tables - Amazon Athena
Considerations and limitations

Query Apache Iceberg tables

You can use Athena to perform read, time travel, write, and DDL queries on Apache Iceberg tables. The Iceberg tables must use the Apache Parquet format for data and the AWS Glue catalog for their metastore.

Apache Iceberg is an open table format for very large analytic datasets. Iceberg manages large collections of files as tables, and it supports modern analytical data lake operations such as record-level insert, update, delete, and time travel queries. The Iceberg specification allows seamless table evolution such as schema and partition evolution and is designed for optimized usage on Amazon S3. Iceberg also helps guarantee data correctness under concurrent write scenarios.

For more information about Apache Iceberg, see https://iceberg.apache.org/.

Considerations and limitations

Athena support for Iceberg tables has the following considerations and limitations:

  • Iceberg version support – Athena supports Apache Iceberg version 1.4.2.

  • Tables with AWS Glue catalog only – Only Iceberg tables created against the AWS Glue catalog based on specifications defined by the open source glue catalog implementation are supported from Athena.

  • Table locking support by AWS Glue only – Unlike the open source Glue catalog implementation, which supports plug-in custom locking, Athena supports AWS Glue optimistic locking only. Using Athena to modify an Iceberg table with any other lock implementation will cause potential data loss and break transactions.

  • Supported file formats – Iceberg file format support in Athena depends on the Athena engine version, as shown in the following table.

    Athena engine version Parquet ORC Avro
    2 Yes No No
    3 Yes Yes Yes

  • Iceberg v2 tables – Athena only creates and operates on Iceberg v2 tables. For the difference between v1 and v2 tables, see Format version changes in the Apache Iceberg documentation.

  • Display of time types without time zone – The time and timestamp without time zone types are displayed in UTC. If the time zone is unspecified in a filter expression on a time column, UTC is used.

  • Timestamp related data precision – Although Iceberg supports microsecond precision for the timestamp data type, Athena supports only millisecond precision for timestamps in both reads and writes. For data in time related columns that is rewritten during manual compaction operations, Athena retains only millisecond precision.

  • Unsupported operations – The following Athena operations are not supported for Iceberg tables.

  • Views – Use CREATE VIEW to create Athena views as described in Work with views. If you are interested in using the Iceberg view specification to create views, contact athena-feedback@amazon.com.

  • TTF management commands not supported in AWS Lake Formation – Although you can use Lake Formation to manage read access permissions for TransactionTable Formats (TTFs) like Apache Iceberg, Apache Hudi, and Linux Foundation Delta Lake, you cannot use Lake Formation to manage permissions for operations like VACUUM, MERGE, UPDATE or OPTIMIZE with these table formats. For more information about Lake Formation integration with Athena, see Using AWS Lake Formation with Amazon Athena in the AWS Lake Formation Developer Guide.

  • Partitioning by nested fields – Partitioning by nested fields is not supported. Attempting to do so produces the message NOT_SUPPORTED: Partitioning by nested field is unsupported: column_name.nested_field_name.

  • Skipping S3 Glacier objects not supported – If objects in the Apache Iceberg table are in an Amazon S3 Glacier storage class, setting the read_restored_glacier_objects table property to false has no effect.

    For example, suppose you issue the following command:

    ALTER TABLE table_name SET TBLPROPERTIES ('read_restored_glacier_objects' = 'false')

    For Iceberg and Delta Lake tables, the command produces the error Unsupported table property key: read_restored_glacier_objects. For Hudi tables, the ALTER TABLE command does not produce an error, but Amazon S3 Glacier objects are still not skipped. Running SELECT queries after the ALTER TABLE command continues to return all objects.

If you would like Athena to support a particular feature, send feedback to athena-feedback@amazon.com.

Topics