Data quality and data profiling notifications
Data quality and data profiling notifications tell you about the outcome of data quality evaluations and data profile runs on tables in your catalog. For example, you can be notified when an evaluation score drops below a threshold that you set, or when a profile run detects anomalies.
In the schedule of a data quality ruleset or a data profile, any project member can choose which outcomes send notifications. These settings are saved on the ruleset.
Who receives data quality notifications
When a ruleset has notifications turned on, notifications go to all project members of the project where the notification settings were saved. This is the project in which you schedule the ruleset or data profile.
This includes users who are project members through a group.
The same recipients receive notifications for a profiling configuration for a catalog or database and for scheduled runs.
Events that send notifications
Data quality rulesets
| Option in the schedule | When it notifies | Notification title |
|---|---|---|
| Evaluation run fails to execute | The evaluation run could not complete. | Data quality evaluation job execution error |
| Evaluation passes | All rules passed, with a score of 100%. | Data quality ruleset evaluation passed |
| Evaluation fails when the score is below [N]% | The evaluation score is lower than the threshold. A score equal to the threshold doesn't send a notification. | Data quality ruleset evaluation failed |
Data profiles
| Option in the schedule | When it notifies | Notification title |
|---|---|---|
| Profiling run completes | A profile run finished. | Data profiling run completed |
| Profiling run fails to execute | A profile run could not complete. | Data profiling run failed (event type Data Quality Ruleset Evaluation Error; see Notification types) |
| Profiling run detects anomalies | A profile run found values that deviate from the expected pattern. | Anomalies detected in data profile |
These notification options apply to every run of the ruleset, including runs that you start manually.
Configure notifications for a data quality ruleset
To turn on notifications for a data quality ruleset
-
In the catalog, open the table and choose the Data quality tab.
-
Choose the Actions menu on your ruleset and select Schedule run. If the ruleset already has a schedule, open the schedule to edit it.
-
Configure the schedule. For details, see Schedule evaluations.
-
In the Notifications section, select the outcomes that you want to be notified about. By default, a new schedule has Evaluation run fails to execute and Evaluation fails when the score is below [N]% selected, with a threshold of 100%. If you select Evaluation fails when the score is below, you can change the threshold to any whole number from 1 to 100.
-
Choose Create schedule (or Save when you edit).
To turn on notifications for a data profile
-
In the catalog, open the table and choose the Data profile tab.
-
Choose the calendar icon or Edit Scheduled Run.
The Run data profile panel opens.
-
In the Schedule section, under Notifications, select one or more of Profiling run completes, Profiling run fails to execute, and Profiling run detects anomalies. By default, only Profiling run detects anomalies is selected for a new data profile schedule.
-
Choose Save.
Notifications for a profiling configuration for a catalog or database
A profiling configuration for a catalog or database profiles every table in the catalog or database, including tables added later. You choose its notifications in the Schedule section under Notifications. The options apply to the profiling runs of every table that the configuration covers, and each notification names the table that it is about.
| Option | When it notifies |
|---|---|
| Profiling run completes | A profiling run of a covered table finished. |
| Profiling run fails to execute | A profiling run of a covered table could not complete. |
| Profiling run detects anomalies | A profiling run of a covered table found values that deviate from the expected pattern. |
To turn on notifications for a profiling configuration for a catalog or database
-
In your project, choose Data in the left navigation pane to open the data explorer. On the Catalogs tab, select a catalog or a database in the tree.
-
Choose the Data profile tab.
-
Choose Create profiling configuration.
The Create profiling configuration panel opens. Its subtitle shows Catalog: or Database: and the name. The panel is the same for a catalog and a database.
-
In the Schedule section, under Notifications, select one or more of Profiling run completes, Profiling run fails to execute, and Profiling run detects anomalies. By default, only Profiling run detects anomalies is selected.
-
Choose Create.
To change the notifications of an existing profiling configuration, on the Data profile tab choose Edit profiling configuration, change the options under Notifications, and then choose Save changes.