Azure 上 "Plan Service" 级别的 SocketException

SocketException on a "Plan Service" level on Azure

我开始在我的 Application Insights 日志中看到很多 SocketException,但是来自实际的 machine/azure 托管和 运行 我的服务的基础设施。我确定是服务以某种方式导致了这种情况,但我不知道如何进一步调试它。

考虑到它来自我无权控制的代码,堆栈跟踪并没有真正给我很多工作要做,因为它被扔到我的服务托管的实际机器上:

System.Net.Http.HttpRequestException:
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Net.Http.HttpClient+<FinishSendAsyncBuffered>d__58.MoveNext (System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Agent.FrontendClient.StampFrontendClient+<>c__DisplayClass9_0`1+<<HttpGetAsync>b__0>d.MoveNext (Microsoft.ServiceProfiler.Agent.FrontendClient, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Agent.FrontendClient.StampFrontendClient+<HttpRequestAsync>d__12`1.MoveNext (Microsoft.ServiceProfiler.Agent.FrontendClient, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Agent.FrontendClient.StampFrontendClient+<HttpGetAsync>d__9`1.MoveNext (Microsoft.ServiceProfiler.Agent.FrontendClient, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Agent.FrontendClient.ProfilerFrontendClient+<GetEtlUploadAccessAsync>d__3.MoveNext (Microsoft.ServiceProfiler.Agent.FrontendClient, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Uploaders.StampEtlUploader+<UploadEtlZipFileAsync>d__1.MoveNext (ApplicationInsightsProfiler, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Collectors.DetailedTraceCollector+<UploadTraceAsync>d__41.MoveNext (ApplicationInsightsProfiler, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Collectors.DetailedTraceCollector+<StopCollectingAndUploadTraceAsync>d__38.MoveNext (ApplicationInsightsProfiler, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Engine+<>c__DisplayClass1_1+<<RunAsync>b__2>d.MoveNext (ApplicationInsightsProfiler, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at Microsoft.ServiceProfiler.Agent.Orchestration.Orchestrator+<TraceAsyncOperation>d__18.MoveNext (Microsoft.ServiceProfiler.Agent.Orchestration, Version=2.6.1909.2701, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
Inner exception System.Net.WebException handled at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw:
   at System.Net.HttpWebRequest.EndGetResponse (System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Net.Http.HttpClientHandler.GetResponseCallback (System.Net.Http, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
Inner exception System.Net.Sockets.SocketException handled at System.Net.HttpWebRequest.EndGetResponse:
   at System.Net.Sockets.Socket.BeginConnectEx (System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Net.Sockets.Socket.UnsafeBeginConnect (System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
   at System.Net.ServicePoint.ConnectSocketInternal (System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)

如果我查看门户,我所看到的只是一条线索,表明它可能来自 ApplicationInsightsProfiler,而我在我的代码库中也没有直接接触过它。

恐怕我正在用尽 http 连接池,但我该如何进一步调查它?

要检查您的 http 连接和 SNAT 端口,请从您的应用打开“诊断和解决问题”,然后打开“可用性和性能”。从那里您可以访问 2 个选项卡:“SNAT 端口耗尽”和“TCP 连接”。 在那里您可以看到您的 http 连接及其目标。