共享层是否得到有效处理？

Question

共享层在 CNTK 中的处理是否高效？（即，激活计算不重复）

假设我有以下表达式：

def create_func(shared_layers, out1_layers, out2_layers):

    # ... there would be a with block specifying activations...omitted for brevity
    shared_hl_func = For(shared_layers, lambda n: Dense(n), name="Shared Hidden Layers")
    out1_hl_func = For(out1_layers, lambda n: Dense(n), name="Out1 Only Hidden Layers")
    out2_hl_func = For(out2_layers, lambda n: Dense(n), name="Sigma Only Hidden Layers")

    output1_func = Sequential([shared_hl_func, out1_hl_func,
                                  Dense(1, activation=None, init=init, name="Out1_Regression_Layer")], name="Out1")
    output2_func = Sequential([shared_hl_func, out2_hl_func,
                                Dense(1, activation=None, init=init, name="Out2_Regression_Layer")], name="Out2")
    return output1_func, output2_func

output1, output2 = create_func([50,25], [25, 10], [25, 10])
my_input = cntk.input_variable((70,))
dual_model = cntk.combine(output1(my_input), output2(my_input))

评估dual_model时计算是否有效？（即，前两个更宽的密集层是否只计算一次然后共享？如果不是这种情况，那么通过显式函数组合构造它是否有助于提高效率？

Answer 1

在您上面的代码中，shared_hl_func 在 output1_func 和 output2_func 中独立求值，但参数是共享的。查看计算图，请使用plot可视化。

要实现计算共享，需要将shared_hl_func输出变量传入output1_func和output2:

import cntk
from cntk.layers import *
def create_func(shared_layers, out1_layers, out2_layers):
    shared_hl_func = For(shared_layers, lambda n: Dense(n), name="Shared Hidden Layers")
    out1_hl_func = For(out1_layers, lambda n: Dense(n), name="Out1 Only Hidden Layers")
    out2_hl_func = For(out2_layers, lambda n: Dense(n), name="Sigma Only Hidden Layers")
    out1_regr_func = Dense(1, activation=None, name="Out1_Regression_Layer")
    out2_regr_func = Dense(1, activation=None, name="Out2_Regression_Layer")
    @cntk.Function
    def _func(x):
        # ... there would be a with block specifying activations...omitted for brevity
        shared_hl = shared_hl_func(x)
        output1 = Sequential([out1_hl_func, out1_regr_func], name="Out1")(shared_hl)
        output2 = Sequential([out2_hl_func, out2_regr_func], name="Out2")(shared_hl)
        return cntk.combine(output1, output2)
    return _func

output = create_func([50,25], [25, 10], [25, 10])
my_input = cntk.input_variable((70,))
dual_model = output(my_input)
# use plot to visualize the model
cntk.logging.graph.plot(dual_model, 'dual.pdf')

共享层是否得到有效处理？

Are shared layers handled efficiently?

deep-learning

cntk