SKlearn Tfidfvectorizer with Keras：预计 dense_input_1 具有形状

Question

我正在尝试将 SKLearn Tfidfvectorizer 与 Keras 结合使用，但我遇到了以下错误：异常：检查模型输入时出错：预期 dense_input_1 具有形状 (None, 126) 但得到形状为 (700, 116)

的数组

我知道它与矩阵的形状有关，但我不知道如何解决它。

vectorizer = TfidfVectorizer(analyzer=self.identity, use_idf=True, max_features=2000)

#a list of sentences
x_train_vec = vectorizer.fit_transform(x_train).toarray()
x_test_vec = vectorizer.fit_transform(self.x_test[i]).toarray()

#labels
y_train = np_utils.to_categorical(y_train, self.nb_classes)
y_test = np_utils.to_categorical(y_test, self.nb_classes)

#get model
model = self.build_model(x_train_vec.shape[1])
model.fit(x_train_vec, y_train, nb_epoch=self.n_epochs, batch_size=self.batch_size, shuffle='batch', verbose=1, validation_data=(x_test_vec, y_test), )

构建模型：

def build_model(self, nb_features):
    print("Building model...")

    model = Sequential()
    model.add(Dense(input_dim = nb_features, output_dim = self.hidden_units_1))
    model.add(Activation('relu'))

Answer 1

问题是 x_train 和 x_test 的维度不同。更改 tfidfvectorizer 中的最大特征解决了问题。

vectorizer = TfidfVectorizer(analyzer=self.identity, use_idf=True, max_features=100)

Answer 2

向量化测试集时，需要调用 transform 而不是 fit_transform:

x_train_vec = vectorizer.fit_transform(x_train).toarray()
x_test_vec = vectorizer.transform(self.x_test[i]).toarray()

SKlearn Tfidfvectorizer with Keras：预计 dense_input_1 具有形状

SKlearn Tfidfvectorizer with Keras: expected dense_input_1 to have shape

python

scikit-learn

keras