在本地加载 GCMLE 并获取激活

Question

我想从部署到 GCMLE 预测服务的 saved_model 在本地（例如 jupyter notebook）查看激活，以便我可以试验可视化。我已成功将 saved_model 加载到图中：

with tf.Session(graph=tf.Graph()) as sess:
  tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], _VERSION)

我还有一个输入字典 request，我通常可以将其提供给已部署的预测服务（为简单起见，最后包含 predict_json()：

responses = predict_json(project, model, instances = [request], version)

有什么方法可以让我在本地使用我的 saved_model 并输入输入 request 然后查看特定层的激活（例如 logits 或卷积的输出)?我相信我正在尝试做的是这样的：

with tf.Session(graph=tf.Graph()) as sess:
    tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], _VERSION)
    graph = tf.get_default_graph()
    inputs = graph.get_tensor_by_name("input_layer:0")
    activations = graph.get_tensor_by_name("conv1d/bias:0")

    print(sess.run(inputs, activations))

但是，我无法找出与 predict_json 中的 body={'instances': instances} 等服务输入函数等效的张量名称。此外，我只是假设如果我通过名称得到卷积偏差，这将代表卷积的激活，但我对此也不是肯定的（因为我无法看到它们是什么） .

GCMLE predict_json() 供参考

def predict_json(project, model, instances, version=None):
"""Send json data to a deployed model for prediction.

Args:
    project (str): project where the Cloud ML Engine Model is deployed.
    model (str): model name.
    instances ([Mapping[str: Any]]): Keys should be the names of Tensors
        your deployed model expects as inputs. Values should be datatypes
        convertible to Tensors, or (potentially nested) lists of datatypes
        convertible to tensors.
    version: str, version of the model to target.
Returns:
    Mapping[str: any]: dictionary of prediction results defined by the
        model.
"""
# Create the ML Engine service object.
# To authenticate set the environment variable
# GOOGLE_APPLICATION_CREDENTIALS=<path_to_service_account_file>
service = googleapiclient.discovery.build('ml', 'v1')
name = 'projects/{}/models/{}'.format(project, model)

if version is not None:
    name += '/versions/{}'.format(version)

response = service.projects().predict(
    name=name,
    body={'instances': instances}
).execute(num_retries=2)

if 'error' in response:
    raise RuntimeError(response['error'])

return response['predictions']

Answer 1

你的总体做法是正确的。

您可以使用 saved_model_cli 检查输入和输出的张量名称，例如（来自上述文档）：

saved_model_cli show --dir \
/tmp/saved_model_dir --tag_set serve --signature_def serving_default

可能会输出如下内容：

The given SavedModel SignatureDef contains the following input(s):
inputs['x'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: x:0
The given SavedModel SignatureDef contains the following output(s):
outputs['y'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: y:0
Method name is: tensorflow/serving/predict

你可以看到输入 x 映射到张量名称 x:0。

当然，找到非输入张量的名称有点困难。如果您是从头开始构建图形，则可以通过向 ops 添加 name="XXX" 为张量起一个友好的名称。否则，您将不得不执行类似转储 SavedModel 的操作，例如

from tensorflow.core.protobuf import saved_model_pb2
s = saved_model_pb2.SavedModel()
with open("saved_model.pb") as f:
  s.ParseFromString(f.read())
print(s)

在本地加载 GCMLE 并获取激活

Load GCMLE locally and obtain activations

tensorflow

google-cloud-ml