Azure Data Lake 存储通过 C# 脚本创建文件夹

Question

我正在尝试在 datalake store 中创建新文件夹，代码中没有错误，但 datalake store 中没有反映任何内容。

代码：

using System;
using System.Collections.Generic;

using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Threading;

using Microsoft.Azure.Management.DataLake.Store;
using Microsoft.Azure.Management.DataLake.Store.Models;
using Microsoft.Rest.Azure.Authentication;
namespace test_dlstore
{
    class Program
    {
        private static DataLakeStoreAccountManagementClient _adlsClient;
        private static DataLakeStoreFileSystemManagementClient _adlsFileSystemClient;

        private static string _adlsAccountName;
        private static string _subId;
        private static void Main(string[] args)
        {
            _adlsAccountName = "mycreds@mysite.com"; 

            _subId = "2342342-97ce-a54b2-ba6e-234234234234234";

            string localFolderPath = @"C:\myfolder\"; // TODO: Make sure this exists and can be overwritten.
            string localFilePath = Path.Combine(localFolderPath, "try.txt");
            string remoteFolderPath = "adl://mystore.azuredatalakestore.net/myfolder";
            string remoteFilePath = Path.Combine(remoteFolderPath, "try.txt");

            SynchronizationContext.SetSynchronizationContext(new SynchronizationContext());
            var tenant_id = "my-tenant-id"; 
            var nativeClientApp_clientId = "1950a258-227b-4e31-a9cf-717495945fc2";


            var activeDirectoryClientSettings = ActiveDirectoryClientSettings.UsePromptOnly(nativeClientApp_clientId, new Uri("urn:ietf:wg:oauth:2.0:oob"));
            var creds = UserTokenProvider.LoginWithPromptAsync(tenant_id, activeDirectoryClientSettings).Result;

            _adlsClient = new DataLakeStoreAccountManagementClient(creds) { SubscriptionId = _subId };
            _adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
            Console.WriteLine(ListAdlStoreAccounts());
            Console.WriteLine(AppendToFile("adl://mystore.azuredatalakestore.net/myfolder/stage/testfile.txt", "abcdefghijklmnopqrstuvwxyz"));
            CreateDirectory("adl://mystore.azuredatalakestore.net/myfolder/newdir");

            Console.ReadLine();

        }
        // Append to file
        public static string AppendToFile(string path, string content)
        {
            using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(content)))
            {
                Console.WriteLine(_adlsAccountName, path, content);
                Console.WriteLine(path);
                Console.WriteLine(content);
                _adlsFileSystemClient.FileSystem.AppendAsync(_adlsAccountName, path, stream);
                return "Tried";
            }
        }
        public static string CreateDirectory(string path)
        {
            _adlsFileSystemClient.FileSystem.MkdirsAsync(_adlsAccountName, path);
            return "Tried Creating directory.";
        }
}
}

执行以上代码后，程序无错退出。正在建立连接。

它还显示 datalake 存在但无法在整个数据湖存储中执行任何操作的存储。

我是 azure 的新手，请帮助我。

Answer 1

我在我这边做了一个演示测试，它在一边工作正常。以下是我的详细步骤：

准备：

注册 AD 应用程序并将角色分配给应用程序，请提供更多详细信息参考Azure official tutorials。之后我们可以从Azure Portal获取tenantId, appId, secretKey

步骤：

1.Create 一个 C# 控制台项目

2.Referece Microsoft.Azure.Management.DataLake.Store SDK，更多详情请参考packages.config文件部分。

3.Add 跟随代码

 var applicationId = "appid";
 var secretKey = "secretkey";
 var tenantId = "tenantid";
 var adlsAccountName = "adlsAccount Name";
 var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;
 var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds,clientTimeoutInMinutes:60); 
 adlsFileSystemClient.FileSystem.Mkdirs(adlsAccountName, "/tomtest/newfolder");

4.Run测试代码

5.Check 来自 Azure 门户。

Packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Azure.Management.DataLake.Store" version="2.2.0" targetFramework="net452" />
  <package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="3.13.8" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.8" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.7" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure.Authentication" version="2.2.0-preview" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="9.0.2-beta1" targetFramework="net452" />
</packages>

Azure Data Lake 存储通过 C# 脚本创建文件夹

Azure Data Lake store create folder via C# script

c#

azure

azure-data-lake