What Is Fairness and Model Explainability for Machine Learning Predictions?
Amazon SageMaker Clarify helps improve your machine learning (ML) models by detecting potential bias and helping explain the predictions that models make. It helps you identify various types of bias in pretraining data and in posttraining that can emerge during model training or when the model is in production. SageMaker Clarify helps explain how these models make predictions using a feature attribution approach. It also monitors inferences models make in production for bias or feature attribution drift. The fairness and explainability functionality provided by SageMaker Clarify provides components that help AWS customers build less biased and more understandable machine learning models. It also provides tools to help you generate model governance reports that you can use to inform risk and compliance teams, and external regulators.
Machine learning models and data-driven systems are being increasingly used to help make decisions across domains such as financial services, healthcare, education, and human resources. Machine learning applications provide benefits such as improved accuracy, increased productivity, and cost savings to help meet regulatory requirements, improve business decisions, and provide better insights into data science procedures.
-
Regulatory – In many situations, it is important to understand why an ML model made a specific prediction and also whether the prediction it made was impacted by any bias, either during training or at inference. Recently, policymakers, regulators, and advocates have raised awareness about the ethical and policy challenges posed by ML and data-driven systems. In particular, they have expressed concerns about the potentially discriminatory impact of such systems (for example, inadvertently encoding of bias into automated decisions).
-
Business – The adoption of AI systems in regulated domains requires trust, which can be built by providing reliable explanations of the behavior of trained models and how the deployed models make predictions. Model explainability may be particularly important to certain industries with reliability, safety, and compliance requirements, such as financial services, human resources, healthcare, and automated transportation. To take a common financial example, lending applications that incorporate the use of ML models might need to provide explanations about how those models made certain predictions to internal teams of loan officers, customer service representatives, and forecasters, in addition to end users/customers.
-
Data Science – Data scientists and ML engineers need tools to generate the insights required to debug and improve ML models through better feature engineering, to determine whether a model is making inferences based on noisy or irrelevant features, and to understand the limitations of their models and failure modes their models may encounter.
For a blog that shows how to architect and build a complete machine learning use case
involving fraudulent automobile claims that integrates SageMaker Clarify into a SageMaker
pipeline, see the
Architect and build the full machine learning lifecycle with AWS: An end-to-end
Amazon SageMaker
Best Practices for Evaluating Fairness and Explainability in the ML Lifecycle
Fairness as a Process – The notions of bias and fairness are highly dependent on the application. Further, the choice of the attributes for which bias is to be measured, as well as the choice of the bias metrics, may need to be guided by social, legal, and other non-technical considerations. Building consensus and achieving collaboration across key stakeholders (such as product, policy, legal, engineering, and AI/ML teams, as well as end users and communities) is a prerequisite for the successful adoption of fairness-aware ML approaches in practice.
Fairness and Explainability by Design in the ML Lifecycle – You should consider fairness and explainability during each stage of the ML lifecycle: problem formation, dataset construction, algorithm selection, model training process, testing process, deployment, and monitoring/feedback. It is important to have the right tools to do this analysis. To encourage engaging with these considerations, here are a few example questions we recommend you ask during each of these stages.
Sample Notebooks
Amazon SageMaker Clarify provides the following sample notebooks:
-
Explainability and bias detection with Amazon SageMaker Clarify
– Use SageMaker Clarify to create a processing job for the detecting bias and explaining model predictions with feature attributions. -
Monitoring bias drift and feature attribution drift Amazon SageMaker Clarify
– Use Amazon SageMaker Model Monitor to monitor bias drift and feature attribution drift over time. -
Fairness and Explainability with SageMaker Clarify (Bring Your Own Container)
– This sample notebook introduces key terms and concepts needed to understand SageMaker Clarify, and it walks you through an end-to-end data science workflow demonstrating how to build your own model and container that can work seamlessly with your Clarify jobs, use the model and SageMaker Clarify to measure bias, explain the importance of the various input features on the model's decision and then access the reports through SageMaker Studio if you have an instance set up. -
Fairness and Explainability with SageMaker Clarify - Spark Distributed Processing
– This sample notebook walks you through key terms and concepts needed to understand SageMaker Clarify, measures the pre-training bias of a dataset and post-training bias of a model, explains the importance of the various input features on the model's decision, and accesses the reports through SageMaker Studio if you have an instance set up. -
Mitigate Bias, Train another unbiased Model and Put in the Model Registry
– This notebook describes how to detect bias using SageMaker Clarify, mitigate it with Synthetic Minority Over-sampling Technique (SMOTE) , train another model, then put it in the Model Registry along with all the lineage of the artifacts created along the way: data, code and model metadata. This notebook forms part of a series that shows how to integrate SageMaker Clarify into a SageMaker Pipeline that is described in the Architect and build the full machine learning lifecycle with AWS blog.
These notebooks have been verified to run in Amazon SageMaker Studio only. If you need instructions on how to open a notebook in Amazon SageMaker Studio, see Create or Open an Amazon SageMaker Studio Notebook. If you're prompted to choose a kernel, choose Python 3 (Data Science).
Guide to the SageMaker Clarify Documentation
Bias can occur and be measured in the data at each stage of the machine learning lifecycle: before training a model and after model training. SageMaker Clarify can provide feature attribution explanations of model predictions for trained models and for models deployed to production, where models can be monitored for any drift from their baseline explanatory attributions. Clarify calculates baselines when needed. The documentation for SageMaker Clarify is embedded throughout the larger SageMaker documentation set at the relevant ML stages as follows:
-
For further information on detecting bias in preprocessing data before it's used to train a model, see Detect Pretraining Data Bias.
-
For further information on detecting posttraining data and model bias, see Detect Posttraining Data and Model Bias.
-
For further information on the model-agnostic feature attribution approach to explain model predictions after training, see Model Explainability.
-
For further information on monitoring for bias in production model inferences due to the drift of data away from the baseline used to train the model, see Monitor Bias Drift for Models in Production.
-
For further information on monitoring for the drift of features' contributions away from the baseline that was established during model training, see Monitor Feature Attribution Drift for Models in Production.
