如何在 Keras 中实现自定义有状态指标

Question

我正在尝试在 Keras 中实现自定义有状态指标。 Keras API website 提供了有关如何执行此操作的简短演示。但是，该演示仅包括 class 定义、创建实例以及根据某些数据计算指标。他们没有演示如何在 model.fit 方法中使用它，而我（和大多数人）就是这样使用它的。当我使用它时，我得到的结果是 (1) 不改变时代到时代和 (2) 与内置指标冲突，所以我知道我得到的结果不正确。是我使用的代码不正确还是示例代码不起作用？

我在下面包含了该问题的最小演示：

# libraries
import numpy as np
import random
import tensorflow as tf
from sklearn import datasets
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.model_selection import train_test_split

# setting psuedorandom seeds for reproducibility
np.random.seed(0)
tf.random.set_seed(0)
random.seed(0)

# loads famous Iris dataset
iris = datasets.load_iris()
x = iris.data
y = iris.target

# converts to binary prediction problem
y = np.where(y == 2, 1, 0)

# train/validation split 
x_train, x_val, y_train, y_val = train_test_split(x, y, test_size = 0.2)

# reformatting 
x_train = tf.constant(x_train)
x_val = tf.constant(x_val)
y_train = tf.constant(y_train)
y_val = tf.constant(y_val)

# stateful metric code from keras website
class BinaryTruePositives(tf.keras.metrics.Metric):

    def __init__(self, name='binary_true_positives', **kwargs):
        super(BinaryTruePositives, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name='tp', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_true = tf.cast(y_true, tf.bool)
        y_pred = tf.cast(y_pred, tf.bool)

        values = tf.logical_and(tf.equal(y_true, True), tf.equal(y_pred, True))
        values = tf.cast(values, self.dtype)
        if sample_weight is not None:
            sample_weight = tf.cast(sample_weight, self.dtype)
            values = tf.multiply(values, sample_weight)
        self.true_positives.assign_add(tf.reduce_sum(values))

    def result(self):
        return self.true_positives

    def reset_states(self):
        self.true_positives.assign(0)

# demonstration from website (correct value is returned)
m = BinaryTruePositives()
m.update_state([0, 1, 1, 1], [0, 1, 0, 0])
print(m.result())

# instantiates metric
true_pos = BinaryTruePositives()

# defines a very simple model
model = Sequential()
model.add(Dense(1, activation = 'sigmoid'))

# compiles model
model.compile(optimizer = 'adam',
              loss = 'binary_crossentropy',
              metrics = ['Recall', true_pos])

# trains model and prints output
history = model.fit(x_train, y_train,
                    epochs = 30,
                    verbose = 1,
                    validation_data = (x_val, y_val),
                    batch_size = 10
                    )

请注意，召回率提高了，而真阳性率保持不变，这是不可能的。

Answer 1

我稍微修改了你的BinaryTruePositives class，因为在你的模型中，y_pred是(0, 1)中的连续变量，不是布尔值。例如，您需要使用 tf.where(y_pred > 0.5, True, False) 将其转换为布尔变量。

class BinaryTruePositives(tf.keras.metrics.Metric):

    def __init__(self, name='binary_true_positives', **kwargs):
        super(BinaryTruePositives, self).__init__(name=name, **kwargs)
        self.true_positives = self.add_weight(name='tp', initializer='zeros')

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_true = tf.cast(y_true, tf.bool)
        y_pred = tf.where(y_pred > 0.5, True, False)

        values = tf.logical_and(tf.equal(y_true, True), tf.equal(y_pred, True))
        values = tf.cast(values, self.dtype)
        if sample_weight is not None:
            sample_weight = tf.cast(sample_weight, self.dtype)
            values = tf.multiply(values, sample_weight)
        self.true_positives.assign_add(tf.reduce_sum(values))

    def result(self):
        return self.true_positives

    def reset_states(self):
        self.true_positives.assign(0)

如何在 Keras 中实现自定义有状态指标

How to implement custom stateful metric in Keras

python

machine-learning

keras

tensorflow