使用 RandomForestClassifier 理解 TreeInterpreter 的输出

Question

我应用了随机森林分类器来获取数据集中特定行的特征。但是，我得到了 2 个特征值，而不是一个。我不太清楚为什么。这是我的代码。

import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from treeinterpreter import treeinterpreter as ti
from treeinterpreter import treeinterpreter as ti

X, y = make_classification(n_samples=1000,
                           n_features=6,
                           n_informative=3,
                           n_classes=2,
                           random_state=0,
                           shuffle=False)

# Creating a dataFrame
df = pd.DataFrame({'Feature 1':X[:,0],
                                  'Feature 2':X[:,1],
                                  'Feature 3':X[:,2],
                                  'Feature 4':X[:,3],
                                  'Feature 5':X[:,4],
                                  'Feature 6':X[:,5],
                                  'Class':y})


y_train = df['Class']
X_train = df.drop('Class',axis = 1)

rf = RandomForestClassifier(n_estimators=50,
                               random_state=0)

rf.fit(X_train, y_train)

print ("-"*20) 

importances = rf.feature_importances_

indices = X_train.columns

instances = X_train.loc[[60]]

print(rf.predict(instances))

print ("-"*20) 

prediction, biases, contributions = ti.predict(rf, instances)


for i in range(len(instances)):
    print ("Instance", i)
    print ("-"*20) 
    print ("Bias (trainset mean)", biases[i])
    print ("-"*20) 
    print ("Feature contributions:")
    print ("-"*20) 

    for c, feature in sorted(zip(contributions[i], 
                                 indices), 
                             key=lambda x: ~abs(x[0].any())):

        print (feature, np.round(c, 3))

    print ("-"*20)

这是我的代码的输出。有人可以解释为什么偏差和特征输出 2 个值而不是一个值吗？

--------------------
[0]
--------------------
Instance 0
--------------------
Bias (trainset mean) [ 0.49854  0.50146]
--------------------
Feature contributions:
--------------------
Feature 1 [ 0.16 -0.16]
Feature 2 [-0.024  0.024]
Feature 3 [-0.154  0.154]
Feature 4 [ 0.172 -0.172]
Feature 5 [ 0.029 -0.029]
Feature 6 [ 0.019 -0.019]

Answer 1

你得到长度为 2 的数组用于偏差和特征贡献，原因很简单，你有一个 2-class class化问题。

正如软件包创建者在 this blog post 中清楚解释的那样，在鸢尾花数据集的 3-class 情况下，您将获得长度为 3 的数组（即每个 class):

from treeinterpreter import treeinterpreter as ti
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris
iris = load_iris()

rf = RandomForestClassifier(max_depth = 4)
idx = range(len(iris.target))
np.random.shuffle(idx)

rf.fit(iris.data[idx][:100], iris.target[idx][:100])

prediction, bias, contributions = ti.predict(rf, instance)
print "Prediction", prediction
print "Bias (trainset prior)", bias
print "Feature contributions:"
for c, feature in zip(contributions[0], 
                             iris.feature_names):
    print feature, c

给出：

Prediction [[ 0. 0.9 0.1]]
Bias (trainset prior) [[ 0.36 0.262 0.378]]
Feature contributions:
sepal length (cm) [-0.1228614 0.07971035 0.04315104]
sepal width (cm) [ 0. -0.01352012 0.01352012]
petal length (cm) [-0.11716058 0.24709886 -0.12993828]
petal width (cm) [-0.11997802 0.32471091 -0.20473289]

公式

prediction = bias + feature_1_contribution + ... + feature_n_contribution

来自TreeInterpreter适用于每个class，在class化问题的情况下；因此，对于 k-class class 化问题，相应的数组长度为 k（在您的示例中 k=2，而对于 iris 数据集 k=3）。

使用 RandomForestClassifier 理解 TreeInterpreter 的输出

Understanding the output of the TreeInterpreter with RandomForestClassifier

python

machine-learning

random-forest

scikit-learn