使用基于状态的标签注释视频帧

Question

我有一堆显示 Microsoft Kinect 人体姿势的视频和深度图。

我可以得到视频中人的骨架，但我想做的是从这个骨架数据中识别出特定的姿势。

为此，我需要用 0 或 1 注释视频中的每一帧，对应于 "bad pose" 和 "good pose"，即帧具有二进制状态变量。

我希望能够在 matlab 中播放 avi 文件，然后按 space 在这两种状态之间切换，同时将状态变量添加到数组中，给出视频中每一帧的状态。

matlab中有没有可以做到这一点的工具？否则matlab不限制，python，C++或者其他任何语言都可以。

我一直在谷歌搜索，我发现的大部分内容都是用多边形注释单个帧。我想以视频常规帧速率的一半执行此操作。

编辑：我使用了 miindlek 提供的解决方案，并决定在有人遇到这个问题时分享一些东西。我需要在视频中看到我为每一帧分配了什么注释，所以我在显示视频时在视频的左上角画了一个小圆圈。希望这对以后的其他人有用。我还捕获了用 waitKey 按下的键，然后根据输出执行某些操作。这允许在注释期间按下多个键。

import numpy as np
import cv2
import os
os.chdir('PathToVideo')

# Blue cicle means that the annotation haven't started
# Green circle is a good pose
# Red is a bad pose
# White circle means we are done, press d for that

# Instructions on how to use!
# Press space to swap between states, you have to press space when the person
# starts doing poses. 
# Press d when the person finishes.
# press q to quit early, then the annotations are not saved, you should only 
# use this if you made a mistake and need to start over.

cap = cv2.VideoCapture('Video.avi')

# You can INCREASE the value of speed to make the video SLOWER
speed = 33

# Start with the beginning state as 10 to indicate that the procedure has not started
current_state = 10
saveAnnotations = True
annotation_list = []
# We can check wether the video capture has been opened
cap.isOpened()
colCirc = (255,0,0)
# Iterate while the capture is open, i.e. while we still get new frames.
while(cap.isOpened()):
    # Read one frame.
    ret, frame = cap.read()
    # Break the loop if we don't get a new frame.
    if not ret:
        break
    # Add the colored circle on the image to know the state
    cv2.circle(frame,(50,50), 50, colCirc, -1)
    # Show one frame.
    cv2.imshow('frame', frame)
    # Wait for a keypress and act on it
    k = cv2.waitKey(speed)
    if k == ord(' '):
        if current_state==0:
            current_state = 1
            colCirc = (0,0,255)
        else:
            current_state = 0
            colCirc = (0,255,0)
        if current_state == 10:
            current_state = 0
            colCirc = (0,255,0)
    if k == ord('d'):
        current_state = 11
        colCirc = (255,255,255)

    # Press q to quit
    if k == ord('q'):
        print "You quit! Restart the annotations by running this script again!"
        saveAnnotations = False
        break

    annotation_list.append(current_state)

# Release the capture and close window
cap.release()
cv2.destroyAllWindows()

# Only save if you did not quit
if saveAnnotations:
    f = open('poseAnnot.txt', 'w')
    for item in annotation_list:
        print>>f, item
    f.close()

Answer 1

解决您的任务的一种方法是将 opencv 库与 python 一起使用，如 tutorial.

中所述

import numpy as np
import cv2

cap = cv2.VideoCapture('video.avi')

current_state = False
annotation_list = []

while(True):
    # Read one frame.
    ret, frame = cap.read()
    if not ret:
        break

    # Show one frame.
    cv2.imshow('frame', frame)

    # Check, if the space bar is pressed to switch the mode.
    if cv2.waitKey(1) & 0xFF == ord(' '):
        current_state = not current_state

    annotation_list.append(current_state)

# Convert the list of boolean values to a list of int values.    
annotation_list = map(int, annotation_list)
print annotation_list

cap.release()
cv2.destroyAllWindows()

变量annotation_list包含每一帧的所有注释。要在两种模式之间切换，您必须按 space 栏。

使用基于状态的标签注释视频帧

Annotating video frames with a label based on state

python

video

matlab

annotations

computer-vision