Interface DataQualityEvaluationConfigGlueTables.Builder

  • Method Details

    • catalogId

      A unique identifier for the Glue Data Catalog.

      Parameters:
      catalogId - A unique identifier for the Glue Data Catalog.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • databaseName

      A database name in the Glue Data Catalog.

      Parameters:
      databaseName - A database name in the Glue Data Catalog.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • tableNamePattern

      DataQualityEvaluationConfigGlueTables.Builder tableNamePattern(String tableNamePattern)

      For the CATALOG and DATABASE levels, a glob pattern, such as orders_*, that selects tables. If you don't specify a pattern, every table at the level is in scope. For the TABLE level, the name of the table.

      Parameters:
      tableNamePattern - For the CATALOG and DATABASE levels, a glob pattern, such as orders_*, that selects tables. If you don't specify a pattern, every table at the level is in scope. For the TABLE level, the name of the table.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • preProcessingQuery

      DataQualityEvaluationConfigGlueTables.Builder preProcessingQuery(String preProcessingQuery)

      SQL Query of SparkSQL format that can be used to pre-process the data for the table in Glue Data Catalog, before running the Data Quality Operation.

      For the CATALOG and DATABASE levels, use the {{DATABASE_NAME}} and {{TABLE_NAME}} placeholders, which are filled in for each table.

      Parameters:
      preProcessingQuery - SQL Query of SparkSQL format that can be used to pre-process the data for the table in Glue Data Catalog, before running the Data Quality Operation.

      For the CATALOG and DATABASE levels, use the {{DATABASE_NAME}} and {{TABLE_NAME}} placeholders, which are filled in for each table.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • connectionName

      DataQualityEvaluationConfigGlueTables.Builder connectionName(String connectionName)

      The name of the connection to the Glue Data Catalog.

      Parameters:
      connectionName - The name of the connection to the Glue Data Catalog.
      Returns:
      Returns a reference to this object so that method calls can be chained together.
    • additionalOptions

      DataQualityEvaluationConfigGlueTables.Builder additionalOptions(Map<String,String> additionalOptions)

      Options for reading the table. The following keys are supported:

      • pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset.

      • catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.

      Parameters:
      additionalOptions - Options for reading the table. The following keys are supported:

      • pushDownPredicate: to filter on partitions without having to list and read all the files in your dataset.

      • catalogPartitionPredicate: to use server-side partition pruning using partition indexes in the Glue Data Catalog.

      Returns:
      Returns a reference to this object so that method calls can be chained together.