android 如何将图像传递给 tflite 模型

Question

我已将 Yolo 模型转换为 .tflite 以用于 android。这就是它在 python -

中的使用方式

net = cv2.dnn.readNet("yolov2.weights", "yolov2.cfg")
classes = []
with open("yolov3.txt", "r") as f:
    classes = [line.strip() for line in f.readlines()]
layer_names = net.getLayerNames()
output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]
colors = np.random.uniform(0, 255, size=(len(classes), 3))

cap= cv2.VideoCapture(0)


while True:
    _,frame= cap.read()
    height,width,channel= frame.shape
    blob = cv2.dnn.blobFromImage(frame, 0.00392, (320, 320), (0, 0, 0), True, crop=False)
    net.setInput(blob)
    outs = net.forward(output_layers)
    for out in outs:
        for detection in out:
            scores = detection[5:]
            class_id = np.argmax(scores)
            confidence = scores[class_id]
            if confidence > 0.2:
            # Object detected
                center_x = int(detection[0] * width)
                center_y = int(detection[1] * height)
                w = int(detection[2] * width)
                h = int(detection[3] * height)
                # Rectangle coordinates
                x = int(center_x - w / 2)
                y = int(center_y - h / 2)

我使用 netron https://github.com/lutzroeder/netron 来可视化模型。输入描述为名称：输入，类型：float32[1,416,416,3]，量化：0≤q≤255，地点：399 输出为姓名：output_boxes，类型：float32[1,10647,8]，地点：400.

我的问题是关于在 android 中使用这个模型。我已经在“Interpreter tflite”中加载了模型，我正在以 byte[] 格式从相机获取输入帧。如何将其转换为 tflite.run(input, output) 所需的输入？

Answer 1

您需要调整输入图像的大小以匹配 TensorFlow-Lite 模型的输入大小，然后将其转换为 RGB 格式以提供给模型。

通过使用 TensorFlow-Lite 支持库中的 ImageProcessor，您可以轻松地进行图像大小调整和转换。

ImageProcessor imageProcessor =
        new ImageProcessor.Builder()
            .add(new ResizeWithCropOrPadOp(cropSize, cropSize))
            .add(new ResizeOp(imageSizeX, imageSizeY, ResizeMethod.NEAREST_NEIGHBOR))
            .add(new Rot90Op(numRoration))
            .add(getPreprocessNormalizeOp())
            .build();
return imageProcessor.process(inputImageBuffer);

在使用解释器进行运行推理之后，将预处理后的图像提供给 TensorFlow-Lite 解释器：

tflite.run(inputImageBuffer.getBuffer(), outputProbabilityBuffer.getBuffer().rewind());

同时参考 this official example for more details, additionally you can refer this 示例。

android 如何将图像传递给 tflite 模型

How to pass image to tflite model in android

android

computer-vision

tensorflow-lite