Layer.call不急于执行

Question

我写了一层，什么都不做

class Fractal2D(tf.keras.layers.Layer):
    def __init__(self, kernel_size_range):
        super(Fractal2D, self).__init__()
        self.kernel_size_range = kernel_size_range
    
    def build(self, inputs):
        print(f'build executes eagerly: {tf.executing_eagerly()}')
        return inputs
    
    def call(self, inputs):
        print(f'call executes eagerly: {tf.executing_eagerly()}')
        return inputs

并做了模型

model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(224, 224, 3), batch_size=32),
    Fractal2D(kernel_size_range=(3, 41)),
    hub.KerasLayer("https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4", output_shape=[1280],
                   trainable=False),
    tf.keras.layers.Dense(DIAGNOSIS_NUMBER, activation='softmax')
])

单元格的输出是

build executes eagerly: True 
call executes eagerly: False

当我训练模型时

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(training_set, validation_data=validation_set, epochs=20)

我明白了

 Epoch 1/20
 call executes eagerly: False
 call executes eagerly: False

问题：

为什么在实例化模型时执行构建和调用方法？
如果默认执行方法是急切执行，为什么不急切执行 call 方法？

Answer 1

自定义层的call方法自动用@tf.function修饰，本质上是在第一次调用时创建一个数据流图，然后在所有后续调用中执行此图。为什么这与您的问题相关？因为根据 tf.executing_eagerly() 上的 docs：

Eager execution is enabled by default and this API returns True in most of cases. However, this API might return False in the following use cases.

Executing inside tf.function, unless under tf.init_scope or tf.config.run_functions_eagerly(True) is previously called.

Executing inside a transformation function for tf.dataset.

tf.compat.v1.disable_eager_execution() is called.

所以让我们看看使用 tf.init_scope:

时会发生什么

import tensorflow_hub as hub
import tensorflow as tf

class Fractal2D(tf.keras.layers.Layer):
    def __init__(self, kernel_size_range):
        super(Fractal2D, self).__init__()
        self.kernel_size_range = kernel_size_range
    
    def build(self, inputs):
        print(f'build executes eagerly: {tf.executing_eagerly()}')
        return inputs
    
    def call(self, inputs):
        with tf.init_scope():
          print(f'call executes eagerly: {tf.executing_eagerly()}')
        return inputs

model = tf.keras.Sequential([
    tf.keras.layers.InputLayer(input_shape=(224, 224, 3), batch_size=1),
    Fractal2D(kernel_size_range=(3, 41)),
    hub.KerasLayer("https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4", output_shape=[1280],
                   trainable=False),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
training_set = tf.random.normal((1, 224, 224, 3))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit(training_set, tf.random.normal((1, 1)), epochs=2)

build executes eagerly: True
call executes eagerly: True
Epoch 1/2
call executes eagerly: True
call executes eagerly: True
1/1 [==============================] - 4s 4s/step - loss: 0.2856 - accuracy: 0.0000e+00
Epoch 2/2
1/1 [==============================] - 0s 36ms/step - loss: 0.1641 - accuracy: 0.0000e+00
<keras.callbacks.History at 0x7f8836515710>

似乎与文档一致。

Answer 2

我认为即使您使用自定义层也可以运行急切模式。由于 'model.fit()' 方法，您的模型运行s 在图形模式下，在 eager 模式下运行您必须从头开始编写自己的训练循环，您可以使用 GradientTape。 [1]: https://www.tensorflow.org/guide/keras/customizing_what_happens_in_fit

Layer.call不急于执行

Layer.call is not executed eagerly

python

machine-learning

deep-learning

keras

tensorflow