Model Interpretation and Feature Selection

10/28/2022

Robert Utterback (based on slides by Andreas Muller)

$\renewcommand{\vec}[1]{\boldsymbol{#1}} \newcommand{\E}{\mathop{\boldsymbol{E}}} \newcommand{\var}{\boldsymbol{Var}} \newcommand{\norm}[1]{\lvert\lvert#1\rvert\rvert} \newcommand{\abs}[1]{\lvert#1\rvert} \newcommand{\ltwo}[1]{\norm{#1}_2} \newcommand{\lone}[1]{\norm{#1}_1} \newcommand{\sgn}[1]{\text{sign}\left( #1 \right)} \newcommand{\e}{\mathrm{e}} \newcommand{\minw}{\min_{\vec{w} \in \mathbb{R}^p, b\in\mathbb{R}}} \newcommand{\summ}{\sum_{i=1}^m} \newcommand{\sumn}{\sum_{i=1}^n} \newcommand{\logloss}{\ln{(\exp{(-y^{(i)}(\vec{w}^T\vec{x}^{(i)}+b))} + 1)}} \newcommand{\ai}{\alpha^{(i)}} \newcommand{\w}{\vec{w}} \newcommand{\wt}{\vec{w}^T} \newcommand{\xi}{\vec{x}^{(i)}} \newcommand{\xit}{(\vec{x}^{(i)})^T} \newcommand{\xip}{\xit \vec{w}} \newcommand{\tip}{(\phi(\xi)^T \phi(\vec{x}))}$

Model Interpretation (post-hoc?)

NOT causal inference

Does x cause y?
That's NOT what model interpretation is giving us!
Don't want to get too into 'causal inference', but just be careful
Can still inform how to better model the data or do better exploration

Drop Feature Importance

$I^{drop}_i = Acc(f,X,y) - Acc(f',X_{-i},y)$

def drop_feature_importance(est, X, y):
    base_score = np.mean(cross_val_score(est, X, y))
    scores = []
    for feature in range(X.shape[1]):
        mask = np.ones(X.shape[1], 'bool')
        mask[feature] = False
        X_new = X[:, mask]
        this_score = np.mean(cross_val_score(est, X_new, y))
        scores.append(base_score - this_score)
    return np.array(scores)

Doesn't really explain model (refits for each features
Can't deal with correlated features well
Very slow
Can be used for feature selection

Permutation Importance

Idea: measure marginal influence of one feature $I^{perm}_i = Acc(f,X,y) - E_{x_i}[Acc(f(x_i,X_{-i}),y)]$

def permutation_importance(est, X, y, n_repeat=100):
  baseline_score = estimator.score(X, y)
  for f_idx in range(X.shape[1]):
      for repeat in range(n_repeat):
          X_new = X.copy()
          X_new[:, f_idx] = np.random.shuffle(X[:, f_idx])
          feature_score = estimator.score(X_new, y)
          scores[f_idx, repeat] = baseline_score - feature_score

Applied on a validation set given trained estimator.
Also kind of slow.

Case Study

Partial Dependence plots

Marginal dependence of prediction on one (or two) features $f^{pdp}_i(x_i) = E_{X_{-i}}[f(x_i,x_{-i})]$
Idea: get marginal predictions given feature
How? "integrate out" other features using validation data
Fast methods available for tree-based models (doesn't require validation data)

Partial Dependence

from sklearn.ensemble.partial_dependence import plot_partial_dependence
boston = load_boston()
X_train, X_test, y_train, y_test = \
    train_test_split(boston.data, boston.target,
                     random_state=0)

gbrt = GradientBoostingRegressor().fit(X_train, y_train)

fig, axs = \
    plot_partial_dependence(gbrt, X_train,
                            np.argsort(gbrt.feature_importances_)[-6:],
                            feature_names=boston.feature_names, n_jobs=3,
                            grid_resolution=50)

But there's also something else that's quite interesting, which is called Partial Dependence Plots that's actually possible for all trees. Unfortunately, in scikit-learn, it’s available only for the gradient boosting. So the idea here is to not only see what parameters are important but how will they influence the target. And so after you fit your model, I'm using the Boston data set to the gradient boosting regressor, there's a thing called plot partial dependence, which gets the model and the training data set and then the features for which I want to make the partial dependence plots.

For trees, don't actually need validation set (but in general you do!)
Trees actually have a faster way that doesn't require the brute force
Plots shown backwards (RM is most important)
This is about what the model learned, not necessarily about the data

So here, I'm looking at the six most important features. The feature importance is important according to the trees. So I sort this and take the six most important ones.

And so this is what the partner dependence plots look like. So this is the most important feature, the second most important feature and so on. This is sort of the marginal contribution of each feature. Basically, in each tree, you're summing out the contribution of all the other features, and you look only at what is the contribution of this particular feature. In general, this would be hard to compute. For tree-based models you can compute is a very efficient way because you can basically just sum up over the whole space by traversing the tree.

Here with increased room size, the price increases and you can see how it increases and you can see that there's like a big step function here. And you can see that for increased LSTAT, the price decreases. For the others, there aren’t any drastic effect. There seems to be some threshold for NOX. And something funky going on DIS.

The question is, so I said for random forest, you can't find out the direction of which way the feature influences. But that was for the eature importance. So the feature importance tells the impurity decrease. So here I also look at the feature importance and they're just numbers so they don't give you the direction. For both gradient boosting and random forest, you can look at partial dependency plot, and they will give you a non-parametric model of how a single feature influences it. Unfortunately, it’s not available in scikit-learn. You could do the same thing for random forests. Because in the end, the way they predict is quite similar.

Wrapper methods

Can be applied for ANY model!
Shrink / grow feature set by greedy search
Called Forward or Backward selection
Run CV / train-val split per feature
Complexity: n_features * (n_features + 1) / 2
Implemented in mlxtend

Model Interpretation and Feature Selection 10/28/2022 Robert Utterback (based on slides by Andreas Muller)