如何在选定列上应用 sklearn Pipeline 上的每个步骤？

Question

当我在查找 sklearn.Pipeline 中的步骤如何准备好仅对某些列进行操作时，我在 Whosebug 上偶然发现了 sklearn.Pipeline.FeatureUnion from this answer。但是，我不太清楚如何不对我不想要的列应用任何内容并将完整数据传递给下一步。例如，在我的第一步中，我只想在某些列上应用 StandardScaler，这可以使用下面显示的代码来完成，但问题是下一步将只有标准缩放的列。我如何在下一步中使用从上一步按比例缩放的标准列获得完整数据？

下面是一些示例代码：

from sklearn.pipeline import Pipeline, FeatureUnion, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.neighbors import KNeighborsClassifier

class Columns(BaseEstimator, TransformerMixin):
    def __init__(self, names=None):
        self.names = names

    def fit(self, X, y=None, **fit_params):
        return self

    def transform(self, X):
        return X[self.names]


pipe = Pipeline([
    # steps below applies on only some columns
    ("features", FeatureUnion([
        ('numeric', make_pipeline(Columns(names=[list of numeric column names]), StandardScaler())),
    ])),
    ('feature_engineer_step1', FeatEng_1()),
    ('feature_engineer_step2', FeatEng_2()),
    ('feature_engineer_step3', FeatEng_3()),
    ('remove_skew', Skew_Remover()),

    # below step applies on all columns
    ('model', RandomForestRegressor())
])

编辑：

由于所选答案没有任何示例代码，我将我的代码粘贴在这里，以供可能遇到此问题并希望找到有效代码的任何人使用。 以下示例中使用的数据是 google colab.

附带的加州住房数据

from sklearn.preprocessing import StandardScaler
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import RandomForestRegressor

# writing a column transformer that operates on some columns
num_cols = ['housing_median_age', 'total_rooms','total_bedrooms', 'population', 'households', 'median_income']
p_stand_scaler_1 = ColumnTransformer(transformers=[('stand_scale', StandardScaler(), num_cols)],
                                     # set remainder to passthrough to pass along all the un-specified columns untouched to the next steps
                                     remainder='passthrough')

# make a pipeline now with all the steps
pipe_1 = Pipeline(steps=[('standard_scaler', p_stand_scaler_1),
                         ('rf_regressor', RandomForestRegressor(random_state=100))])

# pass the data now to fit
pipe_1.fit(house_train.drop('median_house_value', axis=1), house_train.loc[:,'median_house_value'])

# make predictions
pipe_predictions = pipe_1.predict(house_test.drop('median_house_value', axis=1))

Answer 1

我相信使用 ColumnTransformer（来自 sklearn.compose import ColumnTransformer）应该可以解决问题。

在实例化列转换器时，您可以设置 remainder='passthrough'，这只会让剩余的列保持不变。
然后你用列转换器实例化一个管道对象作为第一步。
这样，下一个管道步骤将根据需要接收所有列。

Answer 2

您可以使用 sklearn 中的 ColumnTransformer。这里有一个片段可以帮助你。

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
from sklearn.preprocessing import MinMaxScaler
from sklearn.ensemble import RandomForestClassifier

#transform columns
#num_cols = numerical columns, categorical_col = categorical columns
preprocessor = ColumnTransformer(transformers = [('minmax',MinMaxScaler(), num_cols),
                                                 ('onehot', OneHotEncoder(), categorical_col)])

#model
model = RandomForestClassifier(random_state=0)

#model pipeline
model_pipeline = Pipeline(steps=[('preprocessor', preprocessor), ('model', model)])

model_pipeline.fit(x_train, y_train)

如何在选定列上应用 sklearn Pipeline 上的每个步骤？

How to apply each step on sklearn Pipeline on selected columns?

python

pipeline

python-3.x

scikit-learn