Keras Val_acc 很好，但对相同数据的预测很差

Question

我正在使用 Keras 进行 CNN 两个 class class化。在训练时，我的 val_acc 超过 95%。但是当我预测相同验证数据的结果时，acc 低于 60%，这可能吗？这是我的代码：

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras import backend as K
from keras.callbacks import TensorBoard
from keras.preprocessing import image
import matplotlib.pyplot as plt
import numpy as np
np.random.seed(1337) # for reproducibility
%matplotlib inline

img_width, img_height = 230,170

train_data_dir = 'data/Train'
validation_data_dir = 'data/Validation'
nb_train_samples =  13044
nb_validation_samples = 200
epochs =14
batch_size = 32

if K.image_data_format() == 'channels_first':
    input_shape = (1, img_width, img_height)
else:
    input_shape = (img_width, img_height, 1)

model = Sequential()

model.add(Convolution2D(32, (3, 3),data_format='channels_first' , input_shape=(1,230,170))) 
convout1 = Activation('relu')
model.add(convout1)
convout2 = MaxPooling2D(pool_size=(2,2 ), strides= None , padding='valid', data_format='channels_first')
model.add(convout2)

model.add(Convolution2D(32, (3, 3),data_format='channels_first'))
convout3 = Activation('relu')
model.add(convout3)
model.add(MaxPooling2D(pool_size=(2, 2), data_format='channels_first'))

model.add(Convolution2D(64, (3, 3),data_format='channels_first'))
convout4 = Activation('relu')
model.add(convout4)
convout5 = MaxPooling2D(pool_size=(2, 2), data_format='channels_first')
model.add(convout5)

model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

train_datagen = ImageDataGenerator(rescale=1. / 255, 
                                   shear_range=0, 
                                   zoom_range=0.2, 
                                   horizontal_flip=False, 
                                   data_format='channels_first')

test_datagen = ImageDataGenerator(rescale=1. / 255, 
                                  data_format='channels_first')
train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary',
    color_mode= "grayscale",
    shuffle=True
)
validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary',
    color_mode= "grayscale",
    shuffle=True
)
model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    shuffle=True
    )

纪元 37/37

407/407[==============] - 1775 秒 4 秒/步 - 损失：0.12 - 加速：0.96 - val_loss：0.02 - val_acc: 0.99

#Prediction:
test_data_dir='data/test'
validgen = ImageDataGenerator(horizontal_flip=False, data_format='channels_first')
test_gen = validgen.flow_from_directory(
         test_data_dir,
         target_size=(img_width, img_height),
         batch_size=1,
         class_mode='binary',
         shuffle=False,
         color_mode= "grayscale")

preds = model.predict_generator(test_gen)

在下面的输出中，大约有 7 张图像属于 class 0。我对 class 0 验证数据的所有 100 张图像都进行了相同的尝试，只有 15 张图像被预测为 class 0 和剩余被预测为 class 1

Found 10 images belonging to 1 classes.
[[ 1.]
 [ 1.]
 [ 1.]
 [ 1.]
 [ 1.]
 [ 1.]
 [ 1.]
 [ 0.]
 [ 0.]
 [ 1.]]

Answer 1

您没有像在训练和验证图像中那样将测试图像缩放 1./255。理想情况下，您的测试数据的统计数据应该与训练数据相似。

Answer 2

所以，我决定 post 我在 Quora but with the essential part as advised. I too had a similar problem as this one and I hope my answer can help someone else as well. I decided to research on the Internet and came across this answer by cjbayron 中 post 的答案。

帮助我解决类似问题的是我在训练模型的代码中包含以下内容：

import keras
import os
from keras import backend as K
import tensorflow as tf
import random as rn
import numpy as np

os.environ['PYTHONHASHSEED'] = '0'
np.random.seed(70)
rn.seed(70)
tf.set_random_seed(70)

/******* code for my model ******/

#very important here to save session after completing model.fit 

model.fit_generator(train_batches, steps_per_epoch=4900, validation_data=valid_batches,validation_steps=1225, epochs=40, verbose=2, callbacks=callbacks_list)

saver = tf.train.Saver()
sess = keras.backend.get_session()
saver.save(sess, 'gdrive/My Drive/KerasCNN/model/keras_session/session.ckpt')

保存的会话还将生成以下文件：

/keras_session/checkpoint
/keras_session/session.ckpt.data-00000-of-00001
/keras_session/session.ckpt.index
/keras_session/session.ckpt.meta

我也从我的 Google 驱动器下载了所有这些文件，并将它们放在本地目录中。您可能会注意到似乎没有名为 session.ckpt 的文件，但正在 saver.restore() 中使用。这没关系。 Tensorflow 可以解决这个问题。它不会带来错误。

期间 model.load_model()

所以在我的 Pycharm 中，我按如下方式加载了模型：

model=load_model('C:\Users\Username\PycharmProjects\MyProject\mymodel\mymodel.h5')

saver = tf.train.Saver()
sess = keras.backend.get_session()
saver.restore(sess,'C:\Users\Username\PycharmProjects\MyProject\mymodel\keras_session\session.ckpt')

/***** then predict the images as you wish ******/
pred = model.predict_classes(load_image(os.path.join(test_path, file)))

如图所示放置恢复代码很重要，即在加载模型之后。完成此操作后，我尝试预测用于训练和验证的相同图像，而这一次，模型错误地预测了每个 class 大约 2 张图像。现在我确定我的模型没问题，我继续用我的测试图像进行预测，即它以前没有见过的图像并且表现非常好。

Keras Val_acc 很好，但对相同数据的预测很差

Keras Val_acc is good but prediction for same data is poor

python

predict

keras

tensorflow