使用 docker 服务多个张量流模型

Question

看过 this github issue and Whosebug post 我曾希望这会很简单。

好像传入环境变量MODEL_CONFIG_FILE没有影响。我是运行通过 docker-compose 但我使用 docker-run.

遇到同样的问题

错误：

I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config:  model_name: model model_base_path: /models/model
I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
I tensorflow_serving/model_servers/server_core.cc:558]  (Re-)adding model: model
E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path /models/model for servable model

Dockerfile

FROM tensorflow/serving:nightly

COPY ./models/first/ /models/first
COPY ./models/second/ /models/second

COPY ./config.conf /config/config.conf

ENV MODEL_CONFIG_FILE=/config/config.conf

撰写文件

version: '3'

services:
  serving:
    build: .
    image: testing-models
    container_name: tf

配置文件

model_config_list: {
  config: {
    name:  "first",
    base_path:  "/models/first",
    model_platform: "tensorflow",
    model_version_policy: {
        all: {}
    }
  },
  config: {
    name:  "second",
    base_path:  "/models/second",
    model_platform: "tensorflow",
    model_version_policy: {
        all: {}
    }
  }
}

Answer 1

没有 docker 名为“MODEL_CONFIG_FILE”的环境变量（这是一个 tensorflow/serving 变量，参见 docker 图片 link），因此 docker 图像将只使用默认的 docker 环境变量（"MODEL_NAME=model" 和 "MODEL_BASE_PATH=/models"），并且运行启动时模型“/models/model” docker 图片。 "config.conf" 应在 "tensorflow/serving" 启动时用作输入。尝试运行像这样的东西：

docker run -p 8500:8500 8501:8501 \
  --mount type=bind,source=/path/to/models/first/,target=/models/first \
  --mount type=bind,source=/path/to/models/second/,target=/models/second \
  --mount type=bind,source=/path/to/config/config.conf,target=/config/config.conf\
  -t tensorflow/serving --model_config_file=/config/config.conf

Answer 2

我运行在 windows.

上 git bash this 双斜杠问题

因此，我在 docker-compose.

中通过 command 传递了 @KrisR89 提到的论点

新的 docker-compose 看起来像这样并与提供的 dockerfile:

一起工作

version: '3'

services:
  serving:
    build: .
    image: testing-models
    container_name: tf
    command: --model_config_file=/config/config.conf

Answer 3

错误是因为服务找不到您的模型。

E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:369] FileSystemStoragePathSource encountered a file-system access error: Could not find base path /models/model for servable model

您的 docker 撰写文件没有将您的模型文件装载到容器中。所以 Serving 找不到您的模型。我建议设置三个配置文件。

1 docker-compose.yml

2.env

3 models.config

docker-compose.yml:

将您的模型文件从主机装载到容器。我认为你可以这样做 :

 version: "3"
  services:
        sv:
                image: tensorflow/serving:latest
                restart: unless-stopped
                ports:
                        - 8500:8500
                        - 8501:8501
                volumes:
                        - ${MODEL1_PATH}:/models/${MODEL1_NAME}
                        - ${MODEL2_PATH}:/models/${MODEL2_NAME}
                        - /home/deploy/dcp-file/tf_serving/models.config:/models/models.config
                command: --model_config_file=/models/models.config

.env: docker-compose.yml 从该文件加载信息。

MODEL1_PATH=/home/notebooks/water_model
MODEL1_NAME=water_model
MODEL2_PATH=/home/notebooks/ice_model
MODEL2_NAME=ice_model

models.config:

model_config_list: {
  config {
    name:  "water_model",
    base_path:  "/models/water_model",
    model_platform: "tensorflow",
    model_version_policy: {
        versions: 1588723537
        versions: 1588734567
    }
  },
  config {
    name:  "ice_model",
    base_path:  "/models/ice_model",
    model_platform: "tensorflow",
    model_version_policy: {
        versions: 1588799999
        versions: 1588788888
    }
  }
}

你可以看到这个 serving official document

使用 docker 服务多个张量流模型

Serving multiple tensorflow models using docker

docker

dockerfile

docker-compose

tensorflow-serving