Microsoft Face API：不适用于批量图像

Question

我正在尝试从文件夹中的图像中获取面孔。如果图像数量相对较少，它可以工作，但如果图像数量很大（大约 1500 张），它会在处理大约 10 张图像后引发错误。
"MakeAnalysisRequest" 是向 Microsoft FaceAPI 发送请求的函数。此函数还将 "faceId" 添加到列表中，因为我仅在所有图像处理完成后才需要此列表。所以函数里面才会有这段代码

foreach (FileInfo file in files)
{
    tasks.Add(MakeAnalysisRequest(file.FullName, file.Name, delay, subscriptionKey, endPointURL));
}

//Waiting for "MakeAnalysisRequest" to finish.
Task.WaitAll(tasks.ToArray());

如果我处理大约 100 张图像，一切正常。但是，如果我尝试运行处理大约 1500 张图像，它会失败（引发 "There was an error while saving the file!"）。例外只说 "Aggregate Exception"。处理 "Aggregate" 异常也不会在内部异常中显示任何内容。这是完整的代码（不包括变量。只有相关代码可读性）：

public void ProcessImages(string imageFolderPath)
{
    try
    {
        DirectoryInfo dInfo = new DirectoryInfo(imageFolderPath);
        //Only jpeg|jpg files are needed for the process. Filtering the images in folder. 
        string[] extensions = new[] { ".jpg", ".jpeg" };
        FileInfo[] files = dInfo.GetFiles().Where(f => extensions.Contains(f.Extension.ToLower())).ToArray();
        var tasks = new List<Task>();

        foreach (FileInfo file in files)
        {
            tasks.Add(MakeAnalysisRequest(file.FullName, file.Name, delay, subscriptionKey, endPointURL));
        }

        //Waiting for "MakeAnalysisRequest" to finish.
        Task.WaitAll(tasks.ToArray());

        //faceIds are generated in MakeAnalysisRequest function.
        CreateFaceGroups(faceIds);
    }
    catch (Exception ex)
    {
        Console.WriteLine("There was an error while saving the file! ");
    }
}

static async Task MakeAnalysisRequest(string imageFilePath, string fileName, int delay, string subscriptionKey, string endPointURL)
{
    //To control the numberof requests sent per second. Client does not allow more than 10 per second.
    await Task.Delay(delay);

    HttpClient client = new HttpClient();

    HttpResponseMessage response;

    // Request body. Posts a locally stored JPEG image.
    byte[] byteData = GetImageAsByteArray(imageFilePath);

    using (ByteArrayContent content = new ByteArrayContent(byteData))
    {
        content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");

        // Execute the REST API call.
        response = await client.PostAsync(endPointURL, content);

        // Get the JSON response.
        string contentString = await response.Content.ReadAsStringAsync();
        if (!String.IsNullOrEmpty(contentString))
        {
            try
            {
                //Converting the response to DTO class object.
                List<RootObject> objRoot = JsonConvert.DeserializeObject<List<RootObject>>(contentString);
                foreach (var obj in objRoot)
                {
                    if (obj != null)
                    {
                        objFaceIdImage.Add(new FaceIdImageLink { faceId = obj.faceId, imageName = fileName });
                        faceIds.Add(obj.faceId);
                        //Based on the coordinates creating face image by cropping the source image.
                        CropImage(fileName, imageFilePath, obj.faceRectangle.left, obj.faceRectangle.top, obj.faceRectangle.width, obj.faceRectangle.height, obj.faceId);
                    }
                }
            }
            catch (Exception ex)
            {
                var a = ex;
            }
        }
        else
            ReStart("No faces found in the images!");
    }
}

找不到问题所在！

Answer 1

问题是您将处理限制为每秒 10 个请求的代码不起作用。事实上，它所做的只是确保每个请求都延迟 delay 然后它们都立即触发。一个更安全的解决方案是分批进行处理，并且只有在不到一秒的时间内完成一批处理时才会延迟。例如（未经测试的代码）并使用 this answer:

这个方便的批处理扩展方法

public void ProcessImages(string imageFolderPath)
{
    FileInfo[] files = ...

    foreach (var batch in files.Batch(10))
    {
        var stopWatch = new System.Diagnostics.Stopwatch();
        stopWatch.Start();

        //Process the files as before, but now only 10 at a time
        ProcessBatch(batch);

        stopWatch.Stop();

        //If less than 1 second has passed, let's hang around until it has
        if(stopWatch.ElapsedMilliseconds < 1000)
        {
            //Probably don't use Thread.Sleep here, but you get the idea
            Thread.Sleep(1000 - (int)stopWatch.ElapsedMilliseconds);
        }
    }
}

private void ProcessBatch(IEnumerable<FileInfo> batch)
{
    //Process the batch as before
    var tasks = new List<Task>();

    foreach (FileInfo file in batch)
    {
        //Remove the 'delay' parameter as it's no longer needed
        tasks.Add(MakeAnalysisRequest(file.FullName, file.Name, 
            subscriptionKey, endPointURL));
    }

    //Waiting for "MakeAnalysisRequest" to finish.
    Task.WaitAll(tasks.ToArray());
}

Microsoft Face API：不适用于批量图像

Microsoft Face API: Not working for bulk images

c#

asynchronous

microsoft-cognitive