What Is Fairness and Model Explainability for Machine Learning Predictions?

Amazon SageMaker Clarify helps improve your machine learning (ML) models by detecting potential bias and helping explain the predictions that models make. It helps you identify various types of bias in pretraining data and in posttraining that can emerge during model training or when the model is in production. SageMaker Clarify helps explain how these models make predictions using a feature attribution approach. It also monitors inferences models make in production for bias or feature attribution drift. The fairness and explainability functionality provided by SageMaker Clarify provides components that help AWS customers build less biased and more understandable machine learning models. It also provides tools to help you generate model governance reports that you can use to inform risk and compliance teams, and external regulators.

Machine learning models and data-driven systems are being increasingly used to help make decisions across domains such as financial services, healthcare, education, and human resources. Machine learning applications provide benefits such as improved accuracy, increased productivity, and cost savings to help meet regulatory requirements, improve business decisions, and provide better insights into data science procedures.

• Regulatory – In many situations, it is important to understand why an ML model made a specific prediction and also whether the prediction it made was impacted by any bias, either during training or at inference. Recently, policymakers, regulators, and advocates have raised awareness about the ethical and policy challenges posed by ML and data-driven systems. In particular, they have expressed concerns about the potentially discriminatory impact of such systems (for example, inadvertently encoding of bias into automated decisions).

• Business – The adoption of AI systems in regulated domains requires trust, which can be built by providing reliable explanations of the behavior of trained models and how the deployed models make predictions. Model explainability may be particularly important to certain industries with reliability, safety, and compliance requirements, such as financial services, human resources, healthcare, and automated transportation. To take a common financial example, lending applications that incorporate the use of ML models might need to provide explanations about how those models made certain predictions to internal teams of loan officers, customer service representatives, and forecasters, in addition to end users/customers.

• Data Science – Data scientists and ML engineers need tools to generate the insights required to debug and improve ML models through better feature engineering, to determine whether a model is making inferences based on noisy or irrelevant features, and to understand the limitations of their models and failure modes their models may encounter.

Best Practices for Evaluating Fairness and Explainability in the ML Lifecycle

Fairness as a Process – The notions of bias and fairness are highly dependent on the application and the choice of attributes for which bias is to be measured, in addition to the choice of bias metrics. These choices should be guided by ethical, business, and regulatory considerations. Building consensus and achieving collaboration across key stakeholders (such as product, policy, legal, public relations (PR), engineering, AI/ML teams, end users, and communities) is important for the successful adoption of fair and transparent ML applications.

Fairness and Explainability by Design in the ML Lifecycle – You should consider fairness and explainability during each stage of the ML lifecycle: problem formation, dataset construction, algorithm selection, model training process, testing process, deployment, and monitoring/feedback. It is important to have the right tools to do this analysis. To encourage engaging with these considerations, here are a few example questions we recommend you ask during each of these stages.

Sample Notebooks

Amazon SageMaker Clarify provides the following sample notebooks:

• Explainability and bias detection with Amazon SageMaker Clarify – Use SageMaker Clarify to create a processing job for the detecting bias and explaining model predictions with feature attributions.

• Monitoring bias drift and feature attribution drift Amazon SageMaker Clarify – Use Amazon SageMaker Model Monitor to monitor bias drift and feature attribution drift over time.

These notebooks have been verified to run in Amazon SageMaker Studio only. If you need instructions on how to open a notebook in Amazon SageMaker Studio, see Create or Open an Amazon SageMaker Studio Notebook. If you're prompted to choose a kernel, choose Python 3 (Data Science).

Guide to the SageMaker Clarify Documentation

Bias can occur and be measured in the data at each stage of the machine learning lifecycle: before training a model and after model training. SageMaker Clarify can provide feature attribution explanations of model predictions for trained models and for models deployed to production, where models can be monitored for any drift from their baseline explanatory attributions. The documentation for SageMaker Clarify is embedding throughout the larger SageMaker documentation set at the relevant ML stages as follows:

• For further information on detecting bias in preprocessing data before it's used to train a model, see Detect Pretraining Data Bias.

• For further information on detecting posttraining data and model bias, see Detect Posttraining Data and Model Bias.

• For further information on the model-agnostic feature attribution approach to explain model predictions after training, see Model Explainability.

• For further information on monitoring for bias in production model inferences due to the drift of data away from the baseline used to train the model, see Monitor Bias Drift for Models in Production.

• For further information on monitoring for the drift of features' contributions away from the baseline that was established during model training, see Monitor Feature Attribution Drift for Models in Production.