无法从 Python 中的容器中删除多个 Blob

Question

我正在尝试删除容器中的 blob。每个容器至少有 1500-2000 个 blob。每个容器包含 jpeg 文件和一个 mp4 文件。如果 mp4 文件存在，那么我将删除该特定容器内的 blob。每次我尝试执行 content.delete_blobs(*blobsToDelete) 时，都会出现以下异常：

Exception in Non AI : Traceback (most recent call last):
  File "deleteblobs.py", line 278, in BlobsToDeleteeNonAI
    content.delete_blobs(*blobsToDelete)
  File "C:\Users\hp\AppData\Local\Programs\Python\Python38-32\lib\site-packages\azure\core\tracing\decorator.py", line 83, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "C:\Users\hp\AppData\Local\Programs\Python\Python38-32\lib\site-packages\azure\storage\blob\_container_client.py", line 1194, in delete_blobs
    return self._batch_send(*reqs, **options)
  File "C:\Users\hp\AppData\Local\Programs\Python\Python38-32\lib\site-packages\azure\storage\blob\_shared\base_client.py", line 304, in _batch_send
    raise error
azure.storage.blob._shared.response_handlers.PartialBatchErrorException: There is a partial failure in the batch operation.

我的代码如下所示：

def BlobsToDeleteeNonAI():
        blobsToDelete = []
        #Will delete all the photos except the video.
        try:
                for containerName in NonAICandidates:
                        try:
                                mp4Found = 0
                                content = blob_service_client.get_container_client(str(containerName))
                                for blobs in content.list_blobs():
                                        print("\n"+blobs.name)
                                        #file.write("\n" +blobs.name)
                                        if(blobs.name.endswith(".jpeg")):   #str(blobs.size)
                                                blobsToDelete.append(blobs.name)

                                        if(blobs.name.endswith(".mp4")):
                                                mp4Found = 1
                                                file.write("\nMP4 File Name : " +str(blobs.name))
                                #Will only Delete if and only if the Video File is Present
                                if(mp4Found == 1):
                                        #DeleteCodeHere
                                        
                                        file.write("\n Mp4 Found : " +str(mp4Found) + " for " +str(containerName))
                                        #file.write("\n Blobs to Delete : "+str(blobsToDelete))
                                        
                                        content.delete_blobs(*blobsToDelete)
                                        blobsToDelete.clear()
                                        file.write("\n Blobs Deleted for : " +str(containerName))
                                else:
                                        file.write("\nMp4 File Not found for Non AI Candidate : " +str(containerName) + ". Cannot Perform Deletion Operation.");
                                                
                                           
                        except:
                                file.write("\nException in Non AI : " +str(traceback.format_exc()))
                                blobsToDelete.clear()
        except:
                 file.write("\nException : " +str(traceback.format_exc()))




if __name__ == "__main__":

        NonAICandidates = ['container1', 'container2', 'container3', 'container4', 'container5', 'container6', ....]


        BlobsToDeleteeNonAI()

实施是否有任何问题或是否有任何其他问题阻止我删除 blob？

Answer 1

原因是单个批次只能支持256 sub-requests。而在你的容器中，至少有 1500-2000 个 blob 需要删除，当你试图用一种 delete_blobs 方法删除这些 blob 时，它超出了 256 个限制。

你应该修改你的代码，在一个delete_blobs方法中，只删除1到256个blob。这是一个示例：

#Will only Delete if and only if the Video File is Present
if(mp4Found == 1):
    blobs_lenth=len(blobsToDelete)

    start=0
    end=256

    while end<=blobs_lenth:
         #each time, delete 256 blobs at most
         container_client.delete_blobs(*blobsToDelete[start:end])
         start = start + 256
         end = end + 256

         if start < blobs_lenth and end > blobs_lenth:
            container_client.delete_blobs(*blobsToDelete[start:blobs_lenth])

Answer 2

if(mp4Found == 1):
    blobs_lenth=len(blobsToDelete)

    if blobs_lenth <= 256:
        container_client.delete_blobs(*blobsToDelete) 

    else:
        start=0
        end=256

        while end<=blobs_lenth:
             #each time, delete 256 blobs at most
             container_client.delete_blobs(*blobsToDelete[start:end])
             start = start + 256
             end = end + 256

             if start < blobs_lenth and end > blobs_lenth:
                container_client.delete_blobs(*blobsToDelete[start:blobs_lenth])

Answer 3

我发现这个示例可以简单地写成下面这样。工作正常。

if(mp4Found == 1):
    blobs_length=len(blobsToDelete)

    for i in range(0, blobs_length, 256):
        container_client.delete_blobs(*blobsToDelete[i: i+256])

无法从 Python 中的容器中删除多个 Blob

Unable to delete Multiple Blobs from Container in Python

python

containers

azure

azure-storage

azure-blob-storage