Keras 中 CNN 特征可视化的优化和正则化

Question

我正在尝试在 VGG16 上实现此 distill article on feature visualization for VGGFace model. I was able to find a tutorial but it didn't go in detail about optimization and regularization, which the distill article emphasized are crucial in feature visualization. So my question is how to (1) optimize and (2) regularize (using a learned prior like distill article)? My code here used very simple techniques and achieved results that are far from those generated by OpenAI Microscope。有人可以帮我吗？

import numpy as np
from keras_vggface.vggface import VGGFace
model=VGGFace(model='vgg16')

def generate_pattern(layer_name,filter_index,size=150):
    layer_output=model.get_layer(layer_name).output
    loss=K.mean(layer_output[:,:,:,filter_index])
    grads=K.gradients(loss, model.input)[0]
    grads/=(K.sqrt(K.mean(K.square(grads))) + 1e-5)
    iterate=K.function([model.input],[loss,grads])
    rand_img=np.random.random((1,size,size,3))*20+128.
    step=1.;

    # optimization and regularization using bilateral filter
    for i in range(300):
        loss_value,grads_value=iterate([rand_img])
        rand_img+=grads_value*step
        rand_img=np.array([cv2.bilateralFilter(np.float32(rand_img[0]),3,25,25)])

    img=rand_img[0]
    img=(img-img.mean())/img.std()
    img=np.interp(img,(img.min(),img.max()),(0,1))
    return img

layer_names=[layer.name for layer in model.layers[1:19]]
for layer_name in layer_names:
    if '_1' in layer_name:
        size=64;margin=5;count=1
        results=np.zeros((int(np.sqrt(size))*size+7*margin,
                          int(np.sqrt(size))*size+7*margin,3))
        for i in range(int(np.sqrt(size))):
            for j in range(int(np.sqrt(size))):
                print('{}/{}'.format(count,size),end='\r');count+=1
                filter_img=generate_pattern(layer_name,int(i+(j*np.sqrt(size))),
                                        size=size)
                horizontal_start=i*size+i*margin
                horizontal_end=horizontal_start+size
                vertical_start=j*size+j*margin
                vertical_end=vertical_start+size
                results[horizontal_start:horizontal_end,
                    vertical_start:vertical_end,:]=filter_img
        plt.figure(figsize=(20,20))
        plt.imshow(results)
        plt.show()

提前致谢。

Answer 1

因此仔细查看脚注[9]中的提炼文章

Images were optimized for 2560 steps in a color-decorrelated fourier-transformed space, using Adam at a learning rate of 0.05. We used each of following transformations in the given order at each step of the optimization: • Padding the input by 16 pixels to avoid edge artifacts • Jittering by up to 16 pixels • Scaling by a factor randomly selected from this list: 1, 0.975, 1.025, 0.95, 1.05 • Rotating by an angle randomly selected from this list; in degrees: -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5 • Jittering a second time by up to 8 pixels • Cropping the padding

Keras 中 CNN 特征可视化的优化和正则化

Optimization and Regularization for CNN Feature Visualization in Keras

python

visualization

conv-neural-network

keras