为什么添加随机数不会破坏这个自定义损失函数？

Question

有人可以解释为什么将随机数添加到损失中不会影响此 Keras 模型的预测吗？每次我运行它都会得到两个模型非常相似的 AUC，但我希望第二个模型的 AUC 接近 0.5。我用 Colab。

对于为什么会发生这种情况有什么建议吗？


import numpy as np
import pandas as pd
import tensorflow as tf
import keras as keras
from keras import layers
import random
from keras import backend as K
from sklearn import metrics
from sklearn.metrics import roc_auc_score

opt = tf.keras.optimizers.Adam(learning_rate=1e-04)

#resetting seeds to ensure reproducibility

def reset_random_seeds():
   tf.random.set_seed(1)
   np.random.seed(1)
   random.seed(1)

def get_auc(y_test,y_pred):

  fpr, tpr, threshold = metrics.roc_curve(y_test, y_pred)
  auc = metrics.auc(fpr, tpr)
  return auc

#standard loss function with binary cross-entropy

def binary_crossentropy1(y_true, y_pred): 

  bin_cross = tf.keras.losses.BinaryCrossentropy(from_logits=False)
  bce1 = K.mean(bin_cross(y_true, y_pred))
  return bce1

#same loss function but with added random numbers

def binary_crossentropy2(y_true, y_pred): 

  bin_cross = tf.keras.losses.BinaryCrossentropy(from_logits=False)

  bce2 = K.mean(bin_cross(y_true, y_pred))
  penalty = tf.random.normal([], mean=50.0, stddev=100.0)
  bce2 = tf.math.add(bce2, penalty)
  return bce2

#model without randomness

reset_random_seeds()

input1 = keras.Input(shape=(9,))
x = layers.Dense(12, activation="relu", kernel_initializer=keras.initializers.glorot_uniform(seed=123))(input1)
x = layers.Dense(8, activation="relu", kernel_initializer=keras.initializers.glorot_uniform(seed=123))(x)
output = layers.Dense(1, activation="sigmoid", kernel_initializer=keras.initializers.glorot_uniform(seed=123))(x)
model1 = keras.Model(inputs=input1, outputs=output)


model1.compile(optimizer=opt, loss=binary_crossentropy1, metrics=['accuracy'])

model1.fit(x=X_train, y=y_train, epochs=10, batch_size = 32)

model1_pred = model1.predict(X_test)

#model with randomness

reset_random_seeds()

input1 = keras.Input(shape=(9,))
x = layers.Dense(12, activation="relu", kernel_initializer=keras.initializers.glorot_uniform(seed=123))(input1)
x = layers.Dense(8, activation="relu", kernel_initializer=keras.initializers.glorot_uniform(seed=123))(x)
output = layers.Dense(1, activation="sigmoid", kernel_initializer=keras.initializers.glorot_uniform(seed=123))(x)
model2 = keras.Model(inputs=input1, outputs=output)


model2.compile(optimizer=opt, loss=binary_crossentropy2, metrics=['accuracy'])

model2.fit(x=X_train, y=y_train, epochs=10, batch_size = 32)

model2_pred = model2.predict(X_test)


print(get_auc(y_test, model1_pred))
print(get_auc(y_test, model2_pred))

结果

0.7228943446346893
0.7231896873302319

惩罚是什么样的

penalty =  112.050842
penalty =  139.664017
penalty =  152.505341
penalty =  -37.1483
penalty =  -74.08284
penalty =  155.872528
penalty =  42.7903175

Answer 1

训练以损失相对于输入的梯度为指导。

您添加到第二个模型中的损失的随机值与输入无关，因此它不会影响训练期间的损失梯度。当您运行预测时，您正在采用模型输出（在损失函数之前），因此这也不会受到影响。

为什么添加随机数不会破坏这个自定义损失函数？

Why does adding random numbers not break this custom loss function?

python

machine-learning

deep-learning

keras

tensorflow