What Is Fairness and Model Explainability for Machine Learning Predictions? - Amazon SageMaker

What Is Fairness and Model Explainability for Machine Learning Predictions?

Amazon SageMaker Clarify helps improve your machine learning (ML) models by detecting potential bias and helping explain the predictions that models make. It helps you identify various types of bias in pretraining data and in posttraining that can emerge during model training or when the model is in production. SageMaker Clarify helps explain how these models make predictions using a feature attribution approach. It also monitors inferences models make in production for bias or feature attribution drift. The fairness and explainability functionality provided by SageMaker Clarify provides components that help AWS customers build less biased and more understandable machine learning models. It also provides tools to help you generate model governance reports that you can use to inform risk and compliance teams, and external regulators.

Machine learning models and data-driven systems are being increasingly used to help make decisions across domains such as financial services, healthcare, education, and human resources. Machine learning applications provide benefits such as improved accuracy, increased productivity, and cost savings to help meet regulatory requirements, improve business decisions, and provide better insights into data science procedures.

  • Regulatory – In many situations, it is important to understand why an ML model made a specific prediction and also whether the prediction it made was impacted by any bias, either during training or at inference. Recently, policymakers, regulators, and advocates have raised awareness about the ethical and policy challenges posed by ML and data-driven systems. In particular, they have expressed concerns about the potentially discriminatory impact of such systems (for example, inadvertently encoding of bias into automated decisions).

  • Business – The adoption of AI systems in regulated domains requires trust, which can be built by providing reliable explanations of the behavior of trained models and how the deployed models make predictions. Model explainability may be particularly important to certain industries with reliability, safety, and compliance requirements, such as financial services, human resources, healthcare, and automated transportation. To take a common financial example, lending applications that incorporate the use of ML models might need to provide explanations about how those models made certain predictions to internal teams of loan officers, customer service representatives, and forecasters, in addition to end users/customers.

  • Data Science – Data scientists and ML engineers need tools to generate the insights required to debug and improve ML models through better feature engineering, to determine whether a model is making inferences based on noisy or irrelevant features, and to understand the limitations of their models and failure modes their models may encounter.

For a blog that shows how to architect and build a complete machine learning use case involving fraudulent automobile claims that integrates SageMaker Clarify into a SageMaker pipeline, see the Architect and build the full machine learning lifecycle with AWS: An end-to-end Amazon SageMaker demo. This blog discusses how to assess pre and post training bias, how to mitigate the bias, and how the data features impact the prediction. There are links to the relevent code for each task in the ML lifecycle, including the creation of an automated workflow that integrates the fairness and explainablity functionality of SageMaker Clarify into a SageMaker Pipeline.

Best Practices for Evaluating Fairness and Explainability in the ML Lifecycle

Fairness as a Process – The notions of bias and fairness are highly dependent on the application. Further, the choice of the attributes for which bias is to be measured, as well as the choice of the bias metrics, may need to be guided by social, legal, and other non-technical considerations. Building consensus and achieving collaboration across key stakeholders (such as product, policy, legal, engineering, and AI/ML teams, as well as end users and communities) is a prerequisite for the successful adoption of fairness-aware ML approaches in practice.

Fairness and Explainability by Design in the ML Lifecycle – You should consider fairness and explainability during each stage of the ML lifecycle: problem formation, dataset construction, algorithm selection, model training process, testing process, deployment, and monitoring/feedback. It is important to have the right tools to do this analysis. To encourage engaging with these considerations, here are a few example questions we recommend you ask during each of these stages.

                Best practices for the process of evaluating fairness and model

Sample Notebooks

Amazon SageMaker Clarify provides the following sample notebooks:

These notebooks have been verified to run in Amazon SageMaker Studio only. If you need instructions on how to open a notebook in Amazon SageMaker Studio, see Create or Open an Amazon SageMaker Studio Notebook. If you're prompted to choose a kernel, choose Python 3 (Data Science).

Guide to the SageMaker Clarify Documentation

Bias can occur and be measured in the data at each stage of the machine learning lifecycle: before training a model and after model training. SageMaker Clarify can provide feature attribution explanations of model predictions for trained models and for models deployed to production, where models can be monitored for any drift from their baseline explanatory attributions. Clarify calculates baselines when needed. The documentation for SageMaker Clarify is embedded throughout the larger SageMaker documentation set at the relevant ML stages as follows: