Microsoft Azure 认知服务 - 横向（人脸）模式下人脸检测失败

Question

我正在通过它使用 MS 认知服务 API 来分析图像并注意到当图像处于横向模式 ('faces are not vertically aligned') 时 returns 结果为空.我很困惑，想知道是我做错了什么，还是 MS 服务的工作原理。这是一个重现问题的简单示例。

您应该有 MS Face 订阅密钥才能使用该服务！

import requests
# If you are using a Jupyter notebook, uncomment the following line.
#%matplotlib inline
import matplotlib.pyplot as plt
from PIL import Image
from matplotlib import patches
from io import BytesIO

# send request to MS, use YOUR subscription key
subscription_key = "982374kwhXXXxxxx"
assert subscription_key

face_api_url ='https://westus.api.cognitive.microsoft.com/face/v1.0/detect'
header = {'Ocp-Apim-Subscription-Key': face_api_url }
headers  = {'Ocp-Apim-Subscription-Key': subscription_key, "Content-Type": "application/octet-stream" }

params = {
    'returnFaceId': 'true',
    'returnFaceLandmarks': 'false',
    'returnFaceAttributes': 'age,gender,headPose,smile,facialHair,glasses,' +
    'emotion,hair,makeup,occlusion,accessories,blur,exposure,noise'
}

# get a random image and convert to numpy array
image_url = 'https://how-old.net/Images/faces2/main007.jpg'
image = Image.open(BytesIO(requests.get(image_url).content))
image_np = np.array(image)

这是图片：

然后运行MS人脸识别分析：

    # save image into a buffer
    buf = io.BytesIO()
    plt.imsave(buf, image_np, format='jpg')


    response = requests.post(face_api_url, params=params, headers=headers, data=buf.getvalue())
    faces = response.json()
    print(faces)

正确输出：

[{'faceId': '56afb612-f737-44da-8b62-511070527e18',   'faceRectangle': {'top': 209, 'left': 229, 'width': 91, 'height': 91},   'faceAttributes': {'smile': 1.0,    'headPose': {'pitch': 0.0, 'roll':
3.7, 'yaw': 1.2},    'gender': 'female',    'age': 29.0,    'facialHair': {'moustache': 0.0, 'beard': 0.0, 'sideburns': 0.0},    'glasses': 'NoGlasses',    'emotion': {'anger': 0.0,
    'contempt': 0.0,
    'disgust': 0.0,
    'fear': 0.0,
    'happiness': 1.0,
    'neutral': 0.0,
    'sadness': 0.0,
    'surprise': 0.0},    'blur': {'blurLevel': 'low', 'value': 0.02},    'exposure': {'exposureLevel': 'goodExposure', 'value': 0.65},    'noise': {'noiseLevel': 'medium', 'value': 0.31},    'makeup': {'eyeMakeup': True, 'lipMakeup': True},    'accessories': [],    'occlusion': {'foreheadOccluded': False,
    'eyeOccluded': False,
    'mouthOccluded': False},    'hair': {'bald': 0.05,
    'invisible': False,
    'hairColor': [{'color': 'brown', 'confidence': 1.0},
     {'color': 'blond', 'confidence': 0.48},
     {'color': 'black', 'confidence': 0.35},
     {'color': 'red', 'confidence': 0.27},
     {'color': 'gray', 'confidence': 0.19},
     {'color': 'other', 'confidence': 0.03}]}}},  {'faceId': 'dfb667ba-3fe7-42fc-b9ba-06a86619e94d',   'faceRectangle': {'top': 110, 'left': 125, 'width': 76, 'height': 76},   'faceAttributes': {'smile': 1.0,    'headPose': {'pitch': 0.0, 'roll': 2.3, 'yaw': 2.2}, 'gender': 'male',    'age': 32.0,    'facialHair': {'moustache': 0.4, 'beard': 0.4, 'sideburns': 0.1},    'glasses': 'NoGlasses',    'emotion': {'anger': 0.0,
    'contempt': 0.0,
    'disgust': 0.0,
    'fear': 0.0,
    'happiness': 1.0,
    'neutral': 0.0,
    'sadness': 0.0,
    'surprise': 0.0},    'blur': {'blurLevel': 'low', 'value': 0.02},    'exposure': {'exposureLevel': 'goodExposure', 'value': 0.72},    'noise': {'noiseLevel': 'low', 'value': 0.01},    'makeup': {'eyeMakeup': False, 'lipMakeup': True},    'accessories': [],    'occlusion': {'foreheadOccluded': False,
    'eyeOccluded': False,
    'mouthOccluded': False},    'hair': {'bald': 0.02,
    'invisible': False,
    'hairColor': [{'color': 'brown', 'confidence': 1.0},
     {'color': 'blond', 'confidence': 0.92},
     {'color': 'red', 'confidence': 0.66},
     {'color': 'gray', 'confidence': 0.25},
     {'color': 'other', 'confidence': 0.02},
     {'color': 'black', 'confidence': 0.01}]}}}]

但是，如果我交换轴（转置图像），我得到的结果为零 ('empty')

# save image into a buffer
buf = io.BytesIO()
plt.imsave(buf, np.swapaxes(image_np, 0,1), format='jpg')

response = requests.post(face_api_url, params=params, headers=headers, data=buf.getvalue())
faces = response.json()
print(faces)

结果将是一个空列表：

[]

我想知道是否有人遇到过这个问题以及如何使 MS 服务正常工作，无论图像的旋转或位置如何。

Answer 1

它实际上指出 here 在某些情况下可能无法检测到人脸，包括错误的图像方向。

Face detector prefer frontal and near-frontal faces. There are cases that faces may not be detected, e.g. exceptionally large face angles (head-pose) or being occluded, or wrong image orientation.

你可以自己处理，通过检测人脸的方向，根据方向旋转图片，然后发送！

Microsoft Azure 认知服务 - 横向（人脸）模式下人脸检测失败

Microsoft Azure cognition service - Face detection fails in landscape (face) mode

python

image-processing

microsoft-cognitive