将 Keras 生成器传递给我的模型的 call 方法

Question

我已经编写了自己的 keras 模型，我正尝试将 Keras 生成器作为输入传递给 model.fit。问题是当我在 MyModel 的 call 方法中时，我不知道如何处理生成器。我如何从生成器访问 x 和 y，以便将它们作为输入传递到我的编码器和解码器网络，并让生成器发挥其魔力，在每个时期加载批次？

好的，所以这个 MyModel class 继承了 tf.keras.Model

class MyModel(tf.keras.Model):

def __init__(self):
    super(MyModel, self).__init__()
    self.enc = Encoder()
    self.dec1 = Decoder1()
    self.dec2 = Decoder2()

def __call__(self, data_generator, **kwargs):

    ################################################ 
    ? how do I acces x and y in order to pass them to the encoder and decoder ? 
    and also keep the generator proprieties
    ###############################################
    x_train, y_train = data_generator # ?????????
    #####################################

    dec_inputs = tf.concat((tf.zeros_like(y_train[:, :1, :]), y_train[:, :-1, :]), 1)  
                                                                                      
    dec_inputs = dec_inputs[:, :, -hp.n_mels:] 

    print("########ENC INPUTS #####")
    #print(tf.shape(x_train))
    print("######################")

    print("#########DEC INPUTS #####")
    #print(tf.shape(dec_inputs))
    print("######################")

    memory = self.enc(x_train)
    y_hat = self.dec1(dec_inputs, memory)
    #z_hat = self.dec2(y_hat)
    return y_hat

这是我的生成器函数

class DataGenerator(keras.utils.Sequence):

def __init__(self, list_IDs, ID_dictionary, labels, batch_size=8, dim1=(32, 32, 32), dim2=(32, 32, 32),
             n_channels=None, n_classes=None, shuffle=True):
    'Initialization'

    self.dim1 = dim1  # dimensiune X
    self.dim2 = dim2  # dimensiune Y
    self.batch_size = batch_size

    self.ID_dictionary = ID_dictionary
    self.labels = labels
    self.list_IDs = list_IDs

    self.n_channels = n_channels
    self.n_classes = n_classes

    self.shuffle = shuffle
    self.on_epoch_end()

def __len__(self):
    'Denotes the number of batches per epoch'
    return int(np.floor(len(self.list_IDs) / self.batch_size))

# 3
def __getitem__(self, index):
    'Generate one batch of data'
    # Generate indexes of the batch
    indexes = self.indexes[index * self.batch_size:(index + 1) * self.batch_size] 

    # Find list of IDs
    list_IDs_temp = [self.list_IDs[k] for k in indexes]         

    # Generate data
    x, y = self.__data_generation(list_IDs_temp)  

    return x, y

# 1
def on_epoch_end(self):
    'Updates indexes after each epoch'
    self.indexes = np.arange(len(self.list_IDs))  
    if self.shuffle == True:
        np.random.shuffle(self.indexes)

# 2
def __data_generation(self, list_IDs_temp):
    # Initialization            
    x = np.empty((self.batch_size, self.dim1)) 
    y = np.empty((self.batch_size, *self.dim2), dtype=float)

    # Generate data
    for i, ID in enumerate(list_IDs_temp):  
        # Store sample
        x[i, ] = self.ID_dictionary[ID]       
        # Store class
        y[i] = self.labels[ID]                  
    return x, y

这就是我在 main

中调用 MyModel 的方式

 listID, dict1, dict2, text_shape, mel_shape = get_batch()
 # dict1 has the inputs ( text ) and dict2 has the labels ( the mels )
 
 training_generator = DataGenerator(listID, dict1, dict2, dim1=text_shape, dim2=mel_shape)

 model = MyModel()


 model.compile(
      optimizer=keras.optimizers.Adam(),
      metrics=["accuracy"],

      )

  #model.fit_generator(generator=training_generator, use_multiprocessing=True, workers=6)
  model.fit(training_generator, epochs=2)

Answer 1

call method 在 model.fit 中被调用，它需要一个输入，即 x_input，所以当使用 [=20= 时，你不能期望生成器作为调用方法的输入] 方法。请阅读 tensorflow.org/guide/keras/custom_layers_and_models 和 tensorflow.org/tutorials/text/nmt_with_attention 以更好地了解其工作原理。

编辑 1：如何在调用方法中传递两个变量

# pass list [x,y] to your call function instead of only x, we will club x and y into one variable
def __call__(self, inputs):
    x = inputs[0]
    y = inputs[1]
    # now you can use x and y coming from your generator without changing much


# update your generator to return [x,y] and y
def generator
    yield [x, y], y

# simply call model.fit like you were doing before
model.fit(generator)

将 Keras 生成器传递给我的模型的 call 方法

Passing a Keras Generator to call method of My Model

python

generator

keras

tf.keras

tensorflow2.0

将 Keras 生成器传递给我的模型的 __call__ 方法

Passing a Keras Generator to __call__ method of My Model

python

generator

keras

tf.keras

tensorflow2.0

将 Keras 生成器传递给我的模型的 call 方法

Passing a Keras Generator to call method of My Model