Study notes
As machine learning becomes increasingly integral to decisions that affect health, safety, economic wellbeing, and other aspects of people's lives, it's important to be able to understand how models make predictions; and to be able to explain the rationale for machine learning based decisions.
Model explainers use statistical techniques to calculate feature importance.
This enables you to quantify the relative influence each feature in the training dataset has on label prediction.
Explainerswork by evaluating a test data set of feature cases and the labels the model predicts for them.
- Global feature importance
Quantifies the relative importance of each feature in the test dataset as a whole.
Provides a general comparison of the extent to which each feature in the dataset influences prediction. - Local feature importance
Measures the influence of each feature value for a specific individual prediction.
- MimicExplainer
Creates a global surrogate model that approximates your trained modeland can be used to generate explanations. This explainable model must have the same kind of architecture as your trained model (for example, linear or tree-based). - TabularExplainer
Acts as a wrapper around various SHAP explainer algorithms, automatically choosing the one that is most appropriatefor your model architecture. - PFIExplainer
Permutation Feature Importance explainerthat analyzes feature importance by shuffling feature values and measuring the impact on prediction performance.
# MimicExplainer
from interpret.ext.blackbox import MimicExplainer
from interpret.ext.glassbox import DecisionTreeExplainableModel
mim_explainer = MimicExplainer(model=loan_model,
initialization_examples=X_test,
explainable_model = DecisionTreeExplainableModel,
features=['loan_amount','income','age','marital_status'],
classes=['reject', 'approve'])
# TabularExplainer
from interpret.ext.blackbox import TabularExplainer
tab_explainer = TabularExplainer(model=loan_model,
initialization_examples=X_test,
features=['loan_amount','income','age','marital_status'],
classes=['reject', 'approve'])
# PFIExplainer
from interpret.ext.blackbox import PFIExplainer
pfi_explainer = PFIExplainer(model = loan_model,
features=['loan_amount','income','age','marital_status'],
classes=['reject', 'approve'])
from interpret.ext.blackbox import MimicExplainer
from interpret.ext.glassbox import DecisionTreeExplainableModel
mim_explainer = MimicExplainer(model=loan_model,
initialization_examples=X_test,
explainable_model = DecisionTreeExplainableModel,
features=['loan_amount','income','age','marital_status'],
classes=['reject', 'approve'])
# TabularExplainer
from interpret.ext.blackbox import TabularExplainer
tab_explainer = TabularExplainer(model=loan_model,
initialization_examples=X_test,
features=['loan_amount','income','age','marital_status'],
classes=['reject', 'approve'])
# PFIExplainer
from interpret.ext.blackbox import PFIExplainer
pfi_explainer = PFIExplainer(model = loan_model,
features=['loan_amount','income','age','marital_status'],
classes=['reject', 'approve'])
Explaining global feature importance
- Call the explain_global() method of your explainer to get a global explanation
- Use the get_feature_importance_dict()method to get a dictionary of the feature importance values
Explaining local feature importance
- MimicExplainer or a TabularExplainer
Call the explain_local()method of your explainer, specifying the subset of cases you want to explain. - Use the get_ranked_local_names() and get_ranked_local_values() methods to retrieve dictionaries of the feature names and importance values, ranked by importance.
Install
- azureml-interpret
- azureml-contrib-interpret
Train a Decision Tree Classification Model
import pandas as pd
import numpy as np
import joblib
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve
# load the diabetes dataset
print("Loading Data...")
data = pd.read_csv('data/diabetes.csv')
# Separate features and labels
features = ['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']
labels = ['not-diabetic', 'diabetic']
X, y = data[features].values, data['Diabetic'].values
# Split data into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
# Train a decision tree model
print('Training a decision tree model')
model = DecisionTreeClassifier().fit(X_train, y_train)
# calculate accuracy
y_hat = model.predict(X_test)
acc = np.average(y_hat == y_test)
print('Accuracy:', acc)
# calculate AUC
y_scores = model.predict_proba(X_test)
auc = roc_auc_score(y_test,y_scores[:,1])
print('AUC: ' + str(auc))
print('Model trained.')
import numpy as np
import joblib
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import roc_auc_score
from sklearn.metrics import roc_curve
# load the diabetes dataset
print("Loading Data...")
data = pd.read_csv('data/diabetes.csv')
# Separate features and labels
features = ['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']
labels = ['not-diabetic', 'diabetic']
X, y = data[features].values, data['Diabetic'].values
# Split data into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)
# Train a decision tree model
print('Training a decision tree model')
model = DecisionTreeClassifier().fit(X_train, y_train)
# calculate accuracy
y_hat = model.predict(X_test)
acc = np.average(y_hat == y_test)
print('Accuracy:', acc)
# calculate AUC
y_scores = model.predict_proba(X_test)
auc = roc_auc_score(y_test,y_scores[:,1])
print('AUC: ' + str(auc))
print('Model trained.')
Get an explainer for the model.
We use a Tabular Explainer, which is a "black box" explainer that can be used to explain many kinds of model by invoking an appropriate SHAP model explainer.
from interpret.ext.blackbox import TabularExplainer
# "features" and "classes" fields are optional
tab_explainer = TabularExplainer(model,
X_train,
features=features,
classes=labels)
print(tab_explainer, "ready!")
# "features" and "classes" fields are optional
tab_explainer = TabularExplainer(model,
X_train,
features=features,
classes=labels)
print(tab_explainer, "ready!")
Get global feature importance
# you can use the training data or the test data here
global_tab_explanation = tab_explainer.explain_global(X_train)
# Get the top features by importance
global_tab_feature_importance = global_tab_explanation.get_feature_importance_dict()
for feature, importance in global_tab_feature_importance.items():
print(feature,":", importance)
Result:
Pregnancies : 0.2178177861691587
Age : 0.10665297671149365
BMI : 0.09201304395979432
SerumInsulin : 0.06813355622104296
PlasmaGlucose : 0.04990318086726935
TricepsThickness : 0.021622183903810027
DiastolicBloodPressure : 0.01711999672376072
DiabetesPedigree : 0.014523035900659776
Get local feature importance
# Get the observations we want to explain (the first two)
X_explain = X_test[0:2]
# Get predictions
predictions = model.predict(X_explain)
# Get local explanations
local_tab_explanation = tab_explainer.explain_local(X_explain)
# Get feature names and importance for each possible label
local_tab_features = local_tab_explanation.get_ranked_local_names()
local_tab_importance = local_tab_explanation.get_ranked_local_values()
for l in range(len(local_tab_features)):
print('Support for', labels[l])
label = local_tab_features[l]
for o in range(len(label)):
print("Observation", o + 1)
feature_list = label[o]
total_support = 0
for f in range(len(feature_list)):
print("", feature_list[f], ':', local_tab_importance[l][o][f])
total_support += local_tab_importance[l][o][f]
print(" ---------- Total:", total_support, "Prediction:", labels[predictions[o]])
X_explain = X_test[0:2]
# Get predictions
predictions = model.predict(X_explain)
# Get local explanations
local_tab_explanation = tab_explainer.explain_local(X_explain)
# Get feature names and importance for each possible label
local_tab_features = local_tab_explanation.get_ranked_local_names()
local_tab_importance = local_tab_explanation.get_ranked_local_values()
for l in range(len(local_tab_features)):
print('Support for', labels[l])
label = local_tab_features[l]
for o in range(len(label)):
print("Observation", o + 1)
feature_list = label[o]
total_support = 0
for f in range(len(feature_list)):
print("", feature_list[f], ':', local_tab_importance[l][o][f])
total_support += local_tab_importance[l][o][f]
print(" ---------- Total:", total_support, "Prediction:", labels[predictions[o]])
bservation 1
SerumInsulin : 0.3644802693006629
Age : 0.24405984503381126
TricepsThickness : 0.02410192516176561
BMI : 0.011477583284268565
DiabetesPedigree : 0.0030616002272825494
DiastolicBloodPressure : -0.016593088620978302
PlasmaGlucose : -0.04130417162634759
Pregnancies : -0.25585539133189406
----------
Total: 0.33342857142857085 Prediction: not-diabetic
Observation 2
BMI : 0.3216748143859248
Age : 0.08550984065968997
..
..
Adding explainability to a model training experiment
When you use experiments to train and register models in your Azure Machine Learning workspace, you can generate model explanations and log them.
import azureml.core
from azureml.core import Workspace
# Load the workspace from the saved config file
ws = Workspace.from_config()
print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))
From here, create and train a model like usual and then apply above.
You find all logged in experiment - Azure ML Studio.
References:
Feature importance - Training | Microsoft Learn