AWS::Glue::TableOptimizer

An AWS Glue resource for enabling table optimizers to improve read performance for open table formats.

Syntax

To declare this entity in your CloudFormation template, use the following syntax:

JSON

{
  "Type" : "AWS::Glue::TableOptimizer",
  "Properties" : {
      "CatalogId" : String,
      "DatabaseName" : String,
      "TableName" : String,
      "TableOptimizerConfiguration" : TableOptimizerConfiguration,
      "Type" : String
    }
}

YAML

Type: AWS::Glue::TableOptimizer
Properties:
  CatalogId: String
  DatabaseName: String
  TableName: String
  TableOptimizerConfiguration: 
    TableOptimizerConfiguration
  Type: String

Properties

CatalogId

The catalog ID of the table.

Required: Yes

Type: String

Update requires: Replacement

DatabaseName

The name of the database. For Hive compatibility, this is folded to lowercase when it is stored.

Required: Yes

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*

Minimum: 1

Maximum: 255

Update requires: Replacement

TableName

The table name. For Hive compatibility, this must be entirely lowercase.

Required: Yes

Type: String

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*

Minimum: 1

Maximum: 255

Update requires: Replacement

TableOptimizerConfiguration

Specifies configuration details of a table optimizer.

Required: Yes

Type: TableOptimizerConfiguration

Update requires: No interruption

Type

The type of table optimizer. The valid values are:

• compaction - for managing compaction with a table optimizer.

• retention - for managing the retention of snapshot with a table optimizer.

• orphan_file_deletion - for managing the deletion of orphan files with a table optimizer.

Required: Yes

Type: String

Update requires: Replacement

Return values

Ref

Fn::GetAtt

Examples

Compaction table optimizer example

JSON

{
  "GlueTableOptimizer": {
    "Type": "AWS::Glue::TableOptimizer",
    "Properties": {
      "CatalogId": {"Ref": "AWS::AccountId"},
      "DatabaseName": {"Ref": "GlueDatabase"},
      "TableName": {"Ref": "GlueTable"},
      "Type": "compaction",
      "TableOptimizerConfiguration": {
        "RoleArn": {"Fn:: GetAtt": ["CompactionRole", "Arn"]},
        "Enabled": true
      }
    }
  }
}

YAML

GlueTableOptimizer:
  Type: AWS::Glue::TableOptimizer
  Properties:
    CatalogId: !Ref AWS::AccountId
    DatabaseName: !Ref GlueDatabase
    TableName: !Ref GlueTable
    Type: "compaction"
    TableOptimizerConfiguration:
      RoleArn: !GetAtt CompactionRole.Arn
      Enabled: True

Snapshot retention table optimizer example

JSON

{
  "GlueTableOptimizer": {
    "Type": "AWS::Glue::TableOptimizer",
    "Properties": {
      "CatalogId": {"Ref": "AWS::AccountId"},
      "DatabaseName": {"Ref": "GlueDatabase"},
      "TableName": {"Ref": "GlueTable"},
      "Type": "retention",
      "TableOptimizerConfiguration": {
        "RoleArn": {"Fn::GetAtt": ["RetentionRole", "Arn"]},
        "Enabled": true,
        "RetentionConfiguration": {
          "icebergConfiguration": {
            "snapshotRetentionPeriodInDays": 7,
            "numberOfSnapshotsToRetain": 2,
            "cleanExpiredFiles": true
          }
        }
      }
    }
  }
}

YAML

GlueTableOptimizer:
    Type: AWS::Glue::TableOptimizer
    Properties:
        CatalogId: !Ref AWS::AccountId
        DatabaseName: !Ref GlueDatabase
        TableName: !Ref GlueTable
        Type: "retention"
        TableOptimizerConfiguration:
            RoleArn: !GetAtt TableOptimizerRole.Arn
            Enabled: False
            RetentionConfiguration:
              IcebergConfiguration:
                SnapshotRetentionPeriodInDays: 7
                NumberOfSnapshotsToRetain: 5
                CleanExpiredFiles: True

Orphan file deletion table optimizer example

JSON

{
  "OrphanFileDeletionOptimizer": {
    "Type": "AWS::Glue::TableOptimizer",
    "Properties": {
      "CatalogId": {"Ref": "AWS::AccountId"},
      "DatabaseName": {"Ref": "GlueDatabase"},
      "TableName": {"Ref": "GlueTable"},
      "Type": "orphan_file_deletion",
      "TableOptimizerConfiguration": {
        "RoleArn": {"Fn::GetAtt": ["OrphanFileDeletionRole", "Arn"]},
        "Enabled": true,
        "OrphanFileDeletionConfiguration": {
          "icebergConfiguration": {
            "orphanFileRetentionPeriodInDays": 5,
            "location": "s3://my-bucket/table-location/"
          }
        }
      }
    }
  }
}

YAML

GlueTableOptimizer:
    Type: AWS::Glue::TableOptimizer
    Properties:
      CatalogId: !Ref AWS::AccountId
      DatabaseName: !Ref GlueDatabase
      TableName: !Ref GlueTable
      Type: "orphan_file_deletion"
      TableOptimizerConfiguration:
        RoleArn: !GetAtt TableOptimizerRole.Arn
        Enabled: False
        OrphanFileDeletionConfiguration:
          IcebergConfiguration:
            OrphanFileRetentionPeriodInDays: 10
            Location: !Join ['', ['s3://', !ImportValue TestIcebergS3Bucket, '/orphan-files/']]