自定义损失函数 - Keras

Question

我正在开发一个回归模型来预测加密货币价格，并且我已经创建了一个简单的损失函数。这个想法很简单，Y 目标是某个查找 window 的价格变化，因此值要么是正值，要么是负值。这个想法是首先应用 mae 损失函数，然后在 y_pred 为正且 y_true 为负的情况下进行惩罚，反之亦然。并减少 y_pred 为正且 y_true 也为正的损失，反之亦然。然而，当我使用我的损失函数进行训练时，精度不会高于 0.50，而使用常规 mae 损失函数则达到 0.535 左右。知道是什么原因造成的吗？

损失函数：

def loss_fn(
    # the loss mode [mae, rmse, mape, huber].
    mode="mae",
    # the threshold.
    threshold=0.0,
    # penalize incorrect predictions (when predicted positive and is negative and reversed) (should be >= 1).
    penalizer=1.0, 
    # reduce correct predictions (when predicted positive and is positive and reversed) (should be <= 1).
    reducer=1.0,
):
    def loss_function(y_true, y_pred):
        if mode == "mae":
            loss = keras.losses.MAE(y_true, y_pred)
        elif mode == "rmse":
            loss = K.sqrt(K.mean(K.square(y_pred - y_true)))
        elif mode == "mape":
            loss = keras.losses.mean_absolute_percentage_error(y_true, y_pred)
        elif mode == "huber":
            loss = keras.losses.Huber()(y_true, y_pred)
        if penalizer != 1.0 or reducer != 1.0:
            # apply penalizer.
            mask = tf.where(
                tf.logical_or(
                    tf.logical_and(K.less_equal(y_pred, -1 * threshold), K.greater(y_true, 0.0)),
                    tf.logical_and(K.greater_equal(y_pred, threshold), K.less(y_true, 0.0)),
                ),
                penalizer,
                1.0,
            )[:, 0]
            loss = tf.multiply(loss, mask)
            # apply reducer.
            mask = tf.where(
                tf.logical_or(
                    tf.logical_and(K.less_equal(y_pred, -1 * threshold), K.less(y_true, 0.0)),
                    tf.logical_and(K.greater_equal(y_pred, threshold), K.greater(y_true, 0.0)),
                ),
                reducer,
                1.0,
            )[:, 0]
            loss = tf.multiply(loss, mask)
            loss = tf.math.reduce_mean(loss)
        return loss
    return loss_function
loss = loss_fn(mode="mae", threshold=0.0, penalizer=3.0, reducer=1.0/3.0)

有没有人看到任何可能导致此问题的错误或失误？

来自“mae”的训练日志作为损失：

Epoch 1/250
2829/2829 [==============================] - 44s 12ms/step - loss: 0.8713 - precision: 0.5311 - val_loss: 0.9731 - val_precision: 0.5343
Epoch 2/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8705 - precision: 0.5341 - val_loss: 0.9732 - val_precision: 0.5323
Epoch 3/250
2829/2829 [==============================] - 31s 11ms/step - loss: 0.8702 - precision: 0.5343 - val_loss: 0.9727 - val_precision: 0.5372
Epoch 4/250
2829/2829 [==============================] - 31s 11ms/step - loss: 0.8701 - precision: 0.5345 - val_loss: 0.9730 - val_precision: 0.5336
Epoch 5/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8700 - precision: 0.5344 - val_loss: 0.9732 - val_precision: 0.5316
Epoch 6/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8699 - precision: 0.5347 - val_loss: 0.9726 - val_precision: 0.5334
Epoch 7/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8697 - precision: 0.5346 - val_loss: 0.9731 - val_precision: 0.5331
Epoch 8/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8695 - precision: 0.5343 - val_loss: 0.9722 - val_precision: 0.5382
Epoch 9/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8693 - precision: 0.5346 - val_loss: 0.9724 - val_precision: 0.5330
Epoch 10/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8693 - precision: 0.5345 - val_loss: 0.9732 - val_precision: 0.5331
Epoch 11/250
2829/2829 [==============================] - 32s 11ms/step - loss: 0.8692 - precision: 0.5342 - val_loss: 0.9738 - val_precision: 0.5339
Epoch 12/250
2829/2829 [==============================] - 31s 11ms/step - loss: 0.8690 - precision: 0.5345 - val_loss: 0.9729 - val_precision: 0.5356
Epoch 13/250
2829/2829 [==============================] - 31s 11ms/step - loss: 0.8687 - precision: 0.5350 - val_loss: 0.9728 - val_precision: 0.5342

来自自定义损失函数的训练日志（启用 EarlyStopping）：

Epoch 1/250
2829/2829 [==============================] - 42s 12ms/step - loss: 1.4488 - precision: 0.5039 - val_loss: 1.5693 - val_precision: 0.5021
Epoch 2/250
2829/2829 [==============================] - 33s 12ms/step - loss: 1.4520 - precision: 0.5022 - val_loss: 1.6135 - val_precision: 0.5132
Epoch 3/250
2829/2829 [==============================] - 33s 12ms/step - loss: 1.4517 - precision: 0.5019 - val_loss: 1.6874 - val_precision: 0.4983
Epoch 4/250
2829/2829 [==============================] - 33s 12ms/step - loss: 1.4536 - precision: 0.5017 - val_loss: 1.6885 - val_precision: 0.4982
Epoch 5/250
2829/2829 [==============================] - 33s 12ms/step - loss: 1.4513 - precision: 0.5028 - val_loss: 1.6654 - val_precision: 0.5004
Epoch 6/250
2829/2829 [==============================] - 34s 12ms/step - loss: 1.4578 - precision: 0.4997 - val_loss: 1.5716 - val_precision: 0.5019

Answer 1

Any idea what could cause this?

假设您设置了可重复性的种子，否则它可能只是初始化，当您更改损失函数时，您会更改梯度将迭代以优化您的网络的域。

并且由于您不能保证您的模型将达到全局最小值，但很可能会停在局部最小值，这可能只是意味着，给定相同的初始化（设置种子）优化过程在不同的局部最小值处停止。

自定义损失函数 - Keras

Custom Loss Function - Keras

python

machine-learning

deep-learning

keras

tensorflow