Tensorflow：公认的 "clean" 组织大型网络的方式是什么？

Question

我有一个相当大的全连接网络，我开始对我将权重和偏差存储在字典中然后计算每一层这一事实感到困扰

layer_i+1 = relu(add(matmul(layer_i, weights['i']), biases['i']))

肯定有一些 "cleaner" 方法可以做到这一点？还是我想多了？

Answer 1

我按以下方式管理我的网络：

layers.py

vision = [
    ('conv', [5,5, 3,32], [32]),
    ('conv', [3,3,32,32], [32]),
    ('conv', [3,3,32,32], [32]),
    ('pool', 2),
    ('conv', [3,3,32,64], [64]),
    ('conv', [3,3,64,64], [64]),
    ('pool', 2),
    ('conv', [3,3,64,128], [128]),
    ('pool', 2),
    ('reshape', [-1,6*128]),
    ('dense', [6*128, 512], [512])
]


counter = [
    ('dense', [512, 256], [256]),
    ('dense', [256, max_digits], [max_digits])
]

tfmodel.py

def conv2d(x, W, b, strides=1, act='relu', name='convolution'):
    x = tf.nn.conv2d(x, W, strides=[1,strides,strides,1], padding="VALID", name=name)
    x = tf.nn.bias_add(x, b)

    if act=='relu':
        return tf.nn.relu(x)
    elif act=='tanh':
        return tf.nn.tanh(x)
    elif act=='softmax':
        return tf.nn.softmax(x)

def maxpool2d(x, k=2):
    return tf.nn.max_pool(x, ksize=[1,k,k,1], strides=[1,k,k,1], padding="VALID")

def process_network(X, layers, dropout, scope):
    with tf.variable_scope(scope):
        h = X
        i=0
        for layer in layers:
            if layer[0]=='conv':
                nameW = 'conv{}W'.format(i)
                nameb = 'conv{}b'.format(i)
                h = conv2d(h, tf.get_variable(nameW, layer[1], initializer=tf.random_normal_initializer()), tf.get_variable(nameb,layer[2], initializer=tf.random_normal_initializer()))
            elif layer[0]=='pool':
                h = maxpool2d(h, layer[1])
            elif layer[0]=='dense':
                nameW = 'dense{}W'.format(i)
                nameb = 'dense{}b'.format(i)
                h = tf.add(tf.matmul(h, tf.get_variable(nameW, layer[1], initializer=tf.random_normal_initializer())), tf.get_variable(nameb,layer[2], initializer=tf.random_normal_initializer()))
            elif layer[0]=='reshape':
                h = tf.reshape(h, layer[1])
            i = i+1
        h = tf.identity(h, 'out')
        return h

在创建图形时，只需像这样调用：

h = tfmodel.process_network(image, layers.vision, 0.1, 'vision')
c_ = tfmodel.process_network(h, layers.counter, 0.1, 'counter')

这也会在 TensorBoard 中创建一个干净的图形。它不完整，但我相信你明白了。

另一种干净的方法是使用 Keras 来定义层或模型。查看 Keras as a simplified interface to TensorFlow: tutorial

Tensorflow：公认的 "clean" 组织大型网络的方式是什么？

Tensorflow: What's the accepted "clean" way to organize a large network?

python

standards

tensorflow