Epistemic uncertainty

Epistemic uncertainty refers to the uncertainty of the model (epistemology is the study of knowledge) and is often due to a lack of training data. Examples of epistemic uncertainty include underrepresented minority groups in a facial recognition dataset or the presence of rare words in a language modeling context.

The epistemic uncertainty is found by the definition of variance:

where .

Epistemic uncertainty of a trained model will decrease as the size of training data increases. might also be affected by the suitability of model architecture. As such, the measure of epistemic uncertainty is of great value to the machine learning engineer. This is because large measures of epistemic uncertainty might suggest that inference is being made on data that the model has less experience with. Therefore, this epistemic uncertainty might correspond to erroneous predictions or outlier data.