Tensorflow：矩阵乘法和加法期间的nan

Question

我正在使用一个函数 (ndegree_poly)，它获取一个张量和一个权重数组，并根据它们计算多项式的结果。

代码看起来很直接，但是当度增加，或者函数重复多次时，生成的张量包含一堆nans和infs。 infs是合理的。但是如果数字变得非常小，它们不应该变成零，而不是 nan 吗？

import tensorflow as tf

function_degree = 10

def ndegree_poly(x, a, degree=6):
        op = tf.add_n([tf.multiply(tf.pow(x, i), a[i]) for i in range(1, degree)])
        return tf.add(op, a[0])

with tf.Session() as sess:
    poly_weight = tf.Variable(tf.random_normal([function_freedom, 1, 5]))

    mat = tf.Variable(tf.random_normal([2, 5]))

    result0 = ndegree_poly(mat, poly_weight, function_degree)
    result1 = ndegree_poly(result0, poly_weight, function_degree)
    result2 = ndegree_poly(result1, poly_weight, function_degree)
    result3 = ndegree_poly(result2, poly_weight, function_degree)
    result4 = ndegree_poly(result3, poly_weight, function_degree)


    sess.run(tf.global_variables_initializer())
    print(sess.run(result4))

它打印：

[[-0.28569764         nan         nan         nan        -inf]
 [        nan         nan  3.55561209         nan  0.53827095]]

Answer 1

nan 值不是来自非常小的系数，它只是尝试做 ∞ - ∞ 的 "natural" 结果，因为系数来自正态分布，所以都是正数和负面。

import math
import tensorflow as tf

tf_inf = tf.constant(inf)
res = tf_inf - tf_inf
with tf.Session() as sess:
    print(sess.run(res))

>>> nan

Tensorflow：矩阵乘法和加法期间的nan

Tensorflow: nan during matrix multiplication and addition

python

matrix

nan

tensorflow