张量维度必须相等
Tensor Dimensions must be equal
我正在为 Keras 中的神经网络误报指标编写自己的代码。实现的神经网络给出了 (100,1) 维的输出，其中每个输出值都在 0 和 1 之间。批量大小为 1000。在 false_alarm1 函数中，我想 select 顶部k个概率，让它们为1，其余为0，然后计算混淆矩阵。我收到一条消息，指出尺寸不匹配。对于类似的问题，我尝试了互联网上所有可用的解决方案，但似乎没有任何效果。
这是我实现的功能
def false_alarm1(y_true, y_pred): #based on the Top K probabilities
values , indices =tf.math.top_k(y_pred, k =2)
topk_value = tf.math.argmin(input = values, axis =-1)
topk_value = tf.broadcast_to(topk_value , shape = tf.shape(y_pred))
topk_value = K.cast(topk_value, K.floatx())
y_pred = K.cast(y_pred, K.floatx())
y_pred = K.greater_equal(y_pred , topk_value)
y_pred = K.cast(y_pred, K.floatx())
y_true = K.cast(y_true, K.floatx())
a = K.reshape(y_pred, [-1])
b = K.reshape(y_true, [-1])
conf = tf.math.confusion_matrix(b, a)/tf.shape(a)[0]
score1 = conf[0][1]/(conf[0][0] + conf[0][1])
return score1
我也试过这个
topk_value = tf.broadcast_to(topk_value , shape = (1000,100))
NN的进度
Train on 8000 samples
1000/8000 [==>...........................] - ETA: 5s
我收到的错误
InvalidArgumentError: Dimensions must be equal, but are 1000 and 100 for 'metrics/false_alarm1/BroadcastTo' (op: 'BroadcastTo') with input shapes: [1000], [2] and with input tensors computed as partial shapes: input[1] = [1000,100].
ValueError: Dimensions must be equal, but are 1000 and 100 for 'metrics/false_alarm1/BroadcastTo' (op: 'BroadcastTo') with input shapes: [1000], [2] and with input tensors computed as partial shapes: input[1] = [1000,100].
我的模特：
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 140)] 0
__________________________________________________________________________________________________
dense (Dense) (None, 1000) 141000 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 1000) 4000 dense[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 1000) 1001000 batch_normalization[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 1000) 4000 dense_1[0][0]
__________________________________________________________________________________________________
re_lu (ReLU) (None, 1000) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1000) 0 re_lu[0][0]
__________________________________________________________________________________________________
add (Add) (None, 1000) 0 dropout[0][0]
batch_normalization[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 1000) 1001000 add[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 1000) 4000 dense_2[0][0]
__________________________________________________________________________________________________
re_lu_1 (ReLU) (None, 1000) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 1000) 0 re_lu_1[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 1000) 0 dropout_1[0][0]
add[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 1000) 1001000 add_1[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 1000) 4000 dense_3[0][0]
__________________________________________________________________________________________________
re_lu_2 (ReLU) (None, 1000) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 1000) 0 re_lu_2[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 1000) 0 dropout_2[0][0]
add_1[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 1000) 1001000 add_2[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 1000) 4000 dense_4[0][0]
__________________________________________________________________________________________________
re_lu_3 (ReLU) (None, 1000) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 1000) 0 re_lu_3[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 1000) 0 dropout_3[0][0]
add_2[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 1000) 1001000 add_3[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 1000) 4000 dense_5[0][0]
__________________________________________________________________________________________________
re_lu_4 (ReLU) (None, 1000) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, 1000) 0 re_lu_4[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 1000) 0 dropout_4[0][0]
add_3[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 1000) 1001000 add_4[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 1000) 4000 dense_6[0][0]
__________________________________________________________________________________________________
re_lu_5 (ReLU) (None, 1000) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, 1000) 0 re_lu_5[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 1000) 0 dropout_5[0][0]
add_4[0][0]
__________________________________________________________________________________________________
dense_7 (Dense) (None, 100) 100100 add_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sigmoid (TensorFlow (None, 100) 0 dense_7[0][0]
==================================================================================================
尝试使用 tf.tensor_scatter_nd_update 使 top-k 概率为 1，其余概率为 0：
import tensorflow as tf
y_true = [[0, 1, 0, 0], [0, 1, 1, 0]]
y_pred = [[0.7, 0.5, 0.94, 0.01], [0.3, 0.1, 0.33, 0.41]]
def false_alarm1(y_true, y_pred):
k = 2
new_y_pred = tf.zeros_like(y_pred)
_ , indices = tf.math.top_k(y_pred, k = k)
batch_indices = tf.repeat(tf.range(tf.shape(y_pred)[0]), repeats=k)
indices = tf.stack([batch_indices, tf.reshape(indices,[-1])], axis=1)
new_y_pred = tf.tensor_scatter_nd_update(new_y_pred, indices, tf.ones(tf.reduce_prod(tf.shape(indices)[0])))
print(new_y_pred)
return tf.math.confusion_matrix(tf.reshape(y_true,[-1]), tf.reshape(new_y_pred,[-1]))
print(false_alarm1(y_true, y_pred))
tf.Tensor(
[[1. 0. 1. 0.]
[0. 0. 1. 1.]], shape=(2, 4), dtype=float32)
tf.Tensor(
[[2 3]
[2 1]], shape=(2, 2), dtype=int32)
经过更改的原始实现如下所示：
def false_alarm1(y_true, y_pred):
k = 2
new_y_pred = tf.zeros_like(y_pred)
_ , indices = tf.math.top_k(y_pred, k = k)
batch_indices = tf.repeat(tf.range(tf.shape(y_pred)[0]), repeats=k)
indices = tf.stack([batch_indices, tf.reshape(indices,[-1])], axis=1)
new_y_pred = tf.tensor_scatter_nd_update(new_y_pred, indices, tf.ones(tf.reduce_prod(tf.shape(indices)[0])))
a = tf.reshape(new_y_pred,[-1])
b = tf.reshape(y_true,[-1])
conf = tf.math.confusion_matrix(b, a)/tf.shape(a)[0]
score1 = conf[0][1]/(conf[0][0] + conf[0][1])
return score1
我正在为 Keras 中的神经网络误报指标编写自己的代码。实现的神经网络给出了 (100,1) 维的输出，其中每个输出值都在 0 和 1 之间。批量大小为 1000。在 false_alarm1 函数中，我想 select 顶部k个概率，让它们为1，其余为0，然后计算混淆矩阵。我收到一条消息，指出尺寸不匹配。对于类似的问题，我尝试了互联网上所有可用的解决方案，但似乎没有任何效果。
这是我实现的功能
def false_alarm1(y_true, y_pred): #based on the Top K probabilities
values , indices =tf.math.top_k(y_pred, k =2)
topk_value = tf.math.argmin(input = values, axis =-1)
topk_value = tf.broadcast_to(topk_value , shape = tf.shape(y_pred))
topk_value = K.cast(topk_value, K.floatx())
y_pred = K.cast(y_pred, K.floatx())
y_pred = K.greater_equal(y_pred , topk_value)
y_pred = K.cast(y_pred, K.floatx())
y_true = K.cast(y_true, K.floatx())
a = K.reshape(y_pred, [-1])
b = K.reshape(y_true, [-1])
conf = tf.math.confusion_matrix(b, a)/tf.shape(a)[0]
score1 = conf[0][1]/(conf[0][0] + conf[0][1])
return score1
我也试过这个
topk_value = tf.broadcast_to(topk_value , shape = (1000,100))
NN的进度
Train on 8000 samples
1000/8000 [==>...........................] - ETA: 5s
我收到的错误
InvalidArgumentError: Dimensions must be equal, but are 1000 and 100 for 'metrics/false_alarm1/BroadcastTo' (op: 'BroadcastTo') with input shapes: [1000], [2] and with input tensors computed as partial shapes: input[1] = [1000,100].
ValueError: Dimensions must be equal, but are 1000 and 100 for 'metrics/false_alarm1/BroadcastTo' (op: 'BroadcastTo') with input shapes: [1000], [2] and with input tensors computed as partial shapes: input[1] = [1000,100].
我的模特：
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 140)] 0
__________________________________________________________________________________________________
dense (Dense) (None, 1000) 141000 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 1000) 4000 dense[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 1000) 1001000 batch_normalization[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 1000) 4000 dense_1[0][0]
__________________________________________________________________________________________________
re_lu (ReLU) (None, 1000) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1000) 0 re_lu[0][0]
__________________________________________________________________________________________________
add (Add) (None, 1000) 0 dropout[0][0]
batch_normalization[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 1000) 1001000 add[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 1000) 4000 dense_2[0][0]
__________________________________________________________________________________________________
re_lu_1 (ReLU) (None, 1000) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 1000) 0 re_lu_1[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, 1000) 0 dropout_1[0][0]
add[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 1000) 1001000 add_1[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 1000) 4000 dense_3[0][0]
__________________________________________________________________________________________________
re_lu_2 (ReLU) (None, 1000) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 1000) 0 re_lu_2[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, 1000) 0 dropout_2[0][0]
add_1[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 1000) 1001000 add_2[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 1000) 4000 dense_4[0][0]
__________________________________________________________________________________________________
re_lu_3 (ReLU) (None, 1000) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 1000) 0 re_lu_3[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, 1000) 0 dropout_3[0][0]
add_2[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 1000) 1001000 add_3[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 1000) 4000 dense_5[0][0]
__________________________________________________________________________________________________
re_lu_4 (ReLU) (None, 1000) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, 1000) 0 re_lu_4[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, 1000) 0 dropout_4[0][0]
add_3[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 1000) 1001000 add_4[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 1000) 4000 dense_6[0][0]
__________________________________________________________________________________________________
re_lu_5 (ReLU) (None, 1000) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, 1000) 0 re_lu_5[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, 1000) 0 dropout_5[0][0]
add_4[0][0]
__________________________________________________________________________________________________
dense_7 (Dense) (None, 100) 100100 add_5[0][0]
__________________________________________________________________________________________________
tf_op_layer_Sigmoid (TensorFlow (None, 100) 0 dense_7[0][0]
==================================================================================================
尝试使用 tf.tensor_scatter_nd_update 使 top-k 概率为 1，其余概率为 0：
import tensorflow as tf
y_true = [[0, 1, 0, 0], [0, 1, 1, 0]]
y_pred = [[0.7, 0.5, 0.94, 0.01], [0.3, 0.1, 0.33, 0.41]]
def false_alarm1(y_true, y_pred):
k = 2
new_y_pred = tf.zeros_like(y_pred)
_ , indices = tf.math.top_k(y_pred, k = k)
batch_indices = tf.repeat(tf.range(tf.shape(y_pred)[0]), repeats=k)
indices = tf.stack([batch_indices, tf.reshape(indices,[-1])], axis=1)
new_y_pred = tf.tensor_scatter_nd_update(new_y_pred, indices, tf.ones(tf.reduce_prod(tf.shape(indices)[0])))
print(new_y_pred)
return tf.math.confusion_matrix(tf.reshape(y_true,[-1]), tf.reshape(new_y_pred,[-1]))
print(false_alarm1(y_true, y_pred))
tf.Tensor(
[[1. 0. 1. 0.]
[0. 0. 1. 1.]], shape=(2, 4), dtype=float32)
tf.Tensor(
[[2 3]
[2 1]], shape=(2, 2), dtype=int32)
经过更改的原始实现如下所示：
def false_alarm1(y_true, y_pred):
k = 2
new_y_pred = tf.zeros_like(y_pred)
_ , indices = tf.math.top_k(y_pred, k = k)
batch_indices = tf.repeat(tf.range(tf.shape(y_pred)[0]), repeats=k)
indices = tf.stack([batch_indices, tf.reshape(indices,[-1])], axis=1)
new_y_pred = tf.tensor_scatter_nd_update(new_y_pred, indices, tf.ones(tf.reduce_prod(tf.shape(indices)[0])))
a = tf.reshape(new_y_pred,[-1])
b = tf.reshape(y_true,[-1])
conf = tf.math.confusion_matrix(b, a)/tf.shape(a)[0]
score1 = conf[0][1]/(conf[0][0] + conf[0][1])
return score1