使用 Python 仅获取 Azure Data Lake 目录中的子文件夹名称列表

Question

我在名为 infaqa 的 Azure 数据湖容器中有数据，在 infaqa 中有目录，路径如下： `infqa/EIM//Sales/Raw/APXTConga4__Composer_Setting__mdt、infqa/EIM//Sales/Raw/Account等

我正在使用 Azure 笔记本和 blobserviceclient to list blobs 等库，但我看到所有子文件夹列表和子文件夹列表都在继续列出。当我从下面显示的输出 ['APXTConga4__Composer_Setting__mdt','Account'...] 中仅查找列表中的子文件夹名称时

    Input:
        blobPrefix = "/EIM/Sales/Raw/"     
        mylist=[]
        objects=[]
        blob_list = container_client.list_blobs(blobPrefix)
        for blob in blob_list:
            mylist.append(blob.name)
            print(blob.name)
    
    Ouptut:
    EIM/Sales/Raw/APXTConga4__Composer_Setting__mdt
    EIM/Sales/Raw/APXTConga4__Composer_Setting__mdt/2020
    EIM/Sales/Raw/APXTConga4__Composer_Setting__mdt/2020/12
    EIM/Sales/Raw/APXTConga4__Composer_Setting__mdt/2020/12/02
    EIM/Sales/Raw/Account
    EIM/Sales/Raw/Account/2020
    EIM/Sales/Raw/Account/2020/12
    EIM/Sales/Raw/Account/2020/12/02

Answer 1

试试这个解决方案，我在我的系统中试过

我的文件夹结构类似于 test 容器和 account 是文件夹

1)test/account/main/sub1/
2)test/account/test1/sub2
3)test/account/test2/sub3


from azure.storage.blob import BlobServiceClient
import os

source_key = 'Key'
source_account_name = 'Account Name'
block_blob_service = BlobServiceClient(
    account_url=f'https://{source_account_name}.blob.core.windows.net/', credential=source_key)
source_container_client = block_blob_service.get_container_client(
    'Container name')
result=[]
allfolders=[]
generator =source_container_client.list_blobs("account")

for file in source_container_client.walk_blobs('account/', delimiter='/'):
    print(file.name)
    text=file.name
    result.append(text)
for data in result:
    
    allfolders.append(data.replace("account/",""))
print(allfolders)
for res in allfolders:
    print(res)

输出

存储帐户中的文件夹结构

能够获取所有子文件夹名称

使用 Python 仅获取 Azure Data Lake 目录中的子文件夹名称列表

To get only list of subfolder names in Directory in Azure Data Lake using Python

python

azure

azure-data-lake-gen2