在 Keras 中使用 CNN 时如何提取图像的特征向量

Question

我在做一个二分类问题，我的模型架构如下

def CNN_model(height, width, depth):
    input_shape = (height, width, depth)

    model = Sequential()
    # Block 1
    model.add(Conv2D(64, kernel_size=(3, 3), strides=1, activation='relu', input_shape=input_shape, padding='VALID'))
    model.add(Conv2D(64, kernel_size=(3, 3), strides=1, activation='relu', padding='VALID'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    # Block 2
    model.add(Conv2D(128, kernel_size=(3, 3), strides=1, activation='relu', padding='VALID'))
    model.add(Conv2D(128, kernel_size=(3, 3), strides=1, activation='relu', padding='VALID'))
    model.add(AveragePooling2D(pool_size=(19, 19)))

    # set of FC => RELU layers
    model.add(Flatten())
    model.add(Dense(128))
    model.add(Activation('relu'))
    model.add(BatchNormalization())
    model.add(Dense(num_classes, activation='softmax'))
    model.compile(loss=keras.losses.binary_crossentropy,
                  optimizer=keras.optimizers.Adadelta(),
                  metrics=['accuracy'])
    return model

我需要针对测试集上的每张图像，从 FC 层收集一个 128 维的特征向量，用于 SVM 分类。更多详细信息，来自 model.add(Dense(128))。你能告诉我如何解决这个问题吗？谢谢！

Answer 1

这里最简单的方法就是去掉Dense层。

I will answer with a counter example with similar layers but different input_shape:

from keras.layers import *
from keras.preprocessing import image
from keras.applications.vgg16 import VGG16
from keras.applications.vgg16 import preprocess_input
import numpy as np
from scipy.misc import imsave
import  numpy  as  np
from keras.layers import *
from keras.applications.vgg16 import VGG16
from keras.applications.vgg16 import preprocess_input
from keras.layers import Dropout, Flatten, Dense
from keras.applications import ResNet50
from keras.models import Model, Sequential
from keras.layers import Dense, GlobalAveragePooling2D
from keras import backend as K
import matplotlib.pyplot as plt
from keras.applications.resnet50 import preprocess_input

model = Sequential()
model.add(Conv2D(64, kernel_size=(3, 3), input_shape=(530, 700, 3), padding='VALID'))
model.add(Conv2D(64, kernel_size=(3, 3), padding='VALID'))
model.add(MaxPooling2D(pool_size=(2, 2)))

# Block 2
model.add(Conv2D(128, kernel_size=(3, 3), strides=1, activation='relu', padding='VALID'))
model.add(Conv2D(128, kernel_size=(3, 3), strides=1, activation='relu', padding='VALID'))
model.add(AveragePooling2D(pool_size=(19, 19)))

# set of FC => RELU layers
model.add(Flatten())

#getting the summary of the model (architecture)
model.summary()

img_path = '/home/sb0709/Desktop/dqn/DQN/data/data/2016_11_01-2017_11_01.png'
img = image.load_img(img_path, target_size=(530, 700))
img_data = image.img_to_array(img)
img_data = np.expand_dims(img_data, axis=0)
img_data = preprocess_input(img_data)

vgg_feature = model.predict(img_data)
#print the shape of the output (so from your architecture is clear will be (1, 128))
#print shape
print(vgg_feature.shape)

#print the numpy array output flatten layer
print(vgg_feature.shape)

Here is the output model architecture with all layers:

Also here is listed the feature vector:

Image used in the example:

第二种方法适用于使用 Functional Api 而不是 Sequencial() 来使用 How can I obtain the output of an intermediate layer?

from keras import backend as K
# with a Sequential model
get_6rd_layer_output = K.function([model.layers[0].input],
                                  [model.layers[6].output])
layer_output = get_6rd_layer_output([x])[0]

#print shape
print(layer_output.shape)

#print the numpy array output flatten layer
print(layer_output.shape)

One more useful step is the visualization of the features, I bet a lot of people want to see what see the computer and will illustrate only the "Flatten" layer output(better said the network):

def visualize_stock(img_data):
    plt.figure(1, figsize=(25, 25))
    stock = np.squeeze(img_data, axis=0)
    print(stock.shape)
    plt.imshow(stock)

和魔法：

visualize_stock(img_data)

注意：从 input_shape=(530, 700, 3) 更改为 input_shape=(84, 84, 3) 以便更好地显示 public.

P.S：决定 post 这样任何有此类问题的人都会受益（最近为相同类型的问题而苦苦挣扎）。

在 Keras 中使用 CNN 时如何提取图像的特征向量

How to extract feature vector for image when using CNN in Keras

feature-extraction

keras