在自定义损失中对张量应用高斯模糊

Question

我有一个自定义损失，我想将高斯滤波器应用于预测标签以对其进行一些操作。使用最大或平均池很简单，因为它是在 keras 中预定义的，但我必须自己制作 class 用于高斯池：

import numpy as np
from keras.layers import DepthwiseConv2D
from keras.layers import Input
from keras.models import Model
import tensorflow as tf

class Gaussian():
    
    def __init__(self,shape, f = 3):
        self.filt = f
        self.g = self.gaussFilter(shape)
        
    def doFilter(self, data):
        return self.g.predict(data, steps=1) #steps are for predicting on const tensor, I change it when predicting on predictions 
    
    def gauss2D(self,shape=(3,3),sigma=0.5):
    
        m,n = [(ss-1.)/2. for ss in shape]
        y,x = np.ogrid[-m:m+1,-n:n+1]
        h = np.exp( -(x*x + y*y) / (2.*sigma*sigma) )
        h[ h < np.finfo(h.dtype).eps*h.max() ] = 0
        sumh = h.sum()
        if sumh != 0:
            h /= sumh
        return h
    
    def gaussFilter(self, size=256):
        kernel_weights = self.gauss2D(shape=(self.filt,self.filt))
        
        
        in_channels = 1  # the number of input channels
        kernel_weights = np.expand_dims(kernel_weights, axis=-1)
        kernel_weights = np.repeat(kernel_weights, in_channels, axis=-1) # apply the same filter on all the input channels
        kernel_weights = np.expand_dims(kernel_weights, axis=-1)  # for shape compatibility reasons
        
        
        inp = Input(shape=(size,size,1))
        g_layer = DepthwiseConv2D(self.filt, use_bias=False, padding='same')(inp)
        model_network = Model(input=inp, output=g_layer)
        print(model_network.summary())
        model_network.layers[1].set_weights([kernel_weights])
        model_network.trainable= False
            
        return model_network

当向 doFilter 函数提供常量张量时，这会按预期工作，简单数据的示例：

a = np.array([[[1, 2, 3], [4, 5, 6], [4, 5, 6]]])
filt = Gaussian(3)
print(filt.doFilter(tf.constant(a.reshape(1,3,3,1))))

但是，如果我尝试在自定义损失中使用它：

def custom_loss_no_true(input_tensor, length):
    def loss(y_true, y_pred):
        gaus_pooler = Gaussian(256, length//8)
        a = gaus_pooler.doFilter(y_pred)
        ...more stuff comes after

我收到一个错误：

ValueError: When feeding symbolic tensors to a model, we expect the tensors to have a static batch size. Got tensor with shape: (None, 256, 256, 1)

正如我发现的那样，这是由以下事实引起的，即我正在输入一个张量，该张量是其他模型的输出，是符号数据，而不是实际值 ()。因此，我需要改变我的方法的逻辑，因为评估张量以提供我的 class 会破坏图表并导致损失内没有梯度传播（或者我不正确？）。如何在作为其他模型输出的张量上应用这种卷积运算？有可能吗？或者也许有一种方法可以在不向模型添加层的情况下使用它，例如 MaxPooling？

Answer 1

如果您只想将输入与高斯核进行卷积，那么您实际上并不需要复杂的 keras 模型或 keras 层。这是带有简单 tensorflow ops 的代码端口：

import tensorflow as tf
def get_gaussian_kernel(shape=(3,3), sigma=0.5):
    """build the gaussain filter"""
    m,n = [(ss-1.)/2. for ss in shape]
    x = tf.expand_dims(tf.range(-n,n+1,dtype=tf.float32),1)
    y = tf.expand_dims(tf.range(-m,m+1,dtype=tf.float32),0)
    h = tf.exp(tf.math.divide_no_nan(-((x*x) + (y*y)), 2*sigma*sigma))
    h = tf.math.divide_no_nan(h,tf.reduce_sum(h))
    return h

def gaussian_blur(inp, shape=(3,3), sigma=0.5):
    """Convolve using tf.nn.depthwise_conv2d"""
    in_channel = tf.shape(inp)[-1]
    k = get_gaussian_kernel(shape,sigma)
    k = tf.expand_dims(k,axis=-1)
    k = tf.repeat(k,in_channel,axis=-1)
    k = tf.reshape(k, (*shape, in_channel, 1))
    # using padding same to preserve size (H,W) of the input
    conv = tf.nn.depthwise_conv2d(inp, k, strides=[1,1,1,1],padding="SAME")
    return conv

您可以简单地在您的自定义损失中使用它（假设是 4D y_pred [batch, height width, channel]）：

a = gaussian_blur(y_pred)

在自定义损失中对张量应用高斯模糊

Applying Gaussian blur on tensor in custom loss

python

convolution

deep-learning

keras

tensorflow