Uniqueness - AWS Glue

Uniqueness

Checks the percentage of unique values in a column against a given expression. Unique values occur exactly once.

Syntax

Uniqueness <COL_NAME> <EXPRESSION>

  • COL_NAME – The name of the column that you want to evaluate the data quality rule against.

    Supported column types: Any column type

  • EXPRESSION – An expression to run against the rule type response in order to produce a Boolean value. For more information, see Expressions.

Example: Uniqueness percentage

The following example rule checks whether the percentage of unique values in a column matches certain numeric criteria.

Uniqueness "email" = 1.0
Uniqueness "Customer_ID" != 1.0 where "Customer_ID < 10"

Sample dynamic rules

  • Uniqueness "colA" between min(last(10)) and max(last(10))

  • Uniqueness "colA" >= avg(last(10))