可视化交叉验证模型拟合的方法

Question

在使用交叉验证时，我该如何编写代码来可视化我的准确性和损失发展在训练中的进展？通常我会在训练模型时将变量名 'history' 分配给拟合函数，但在交叉验证的情况下，它不会显示验证曲线。我假设是这种情况，因为我没有在 fit 函数（下面）中调用 validation_data 。

kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=seed)
cvscores = []
for train, test in kfold.split(x_train, y_train):
   model = Sequential()
   model.add(layers.Conv2D(32,(4,4), activation = 'relu', input_shape = (224,224,3)))
   model.add(layers.MaxPooling2D((2, 2)))
   model.add(layers.Conv2D(64, (4,4), activation = 'relu'))
   model.add(layers.MaxPooling2D((2, 2)))
   model.add(layers.Flatten())
   model.add(layers.Dense(64, activation = 'relu'))
   model.add(layers.Dropout(0.5))
   model.add(layers.Dense(1, activation = 'sigmoid'))
  
   model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
 
   history = model.fit(x_train[train], y_train[train], epochs=15, batch_size=64)
 
   scores = model.evaluate(x_train[test], y_train[test], verbose=0)
   print("%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))
   cvscores.append(scores[1] * 100)
print("%.2f%% (+/- %.2f%%)" % (np.mean(cvscores), np.std(cvscores)))

通常我会使用如下代码，但由于我没有适合的验证数据，我不确定如何处理它。

plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='upper left')
plt.show()

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'val'], loc='upper left')
plt.show()

]1]1

Answer 1

您可以使用 TensorBoard. Generally, you make following splits: train, test, validation. You validate your model using that 3rd split. You can use your metrics from sklearn. Most of the time people don't cross-validate their DNN models, as it'd take too much time. However, once you have those models, it'd be nice to plot distribution of metrics using some boxplot 转储所有内容。

可视化交叉验证模型拟合的方法

Method of visualizing fit of cross validated model

python

cross-validation

keras

data-augmentation