NLog 初始发送和多个批量电子邮件

NLog initial send and multiple bulk emailing

背景

我有一个用 asp.net 核心 v2.2 编写的 Web 应用程序。我使用 NLog 作为我的第 3 方记录器,每次出现应用程序错误时它都会给我发电子邮件。

    <targets>
       <!--eg email on error -->
        <target xsi:type="Mail"
                  name="emailnotify"
                  header="An Error has been reported By foo.com (${machinename}) ${newline}  "
                  layout="${longdate} ${level:uppercase=true:padding=5} - ${logger:shortName=true} - ${message} ${exception:format=tostring} ${newline}"
                  html="true"
                  addNewLines="true"
                  replaceNewlineWithBrTagInHtml="true"
                  subject="Error on foo.com (${machinename})"
                  to="foo@foo.com"
                  from="foo@foo.com"
                  secureSocketOption="StartTls"
                  smtpAuthentication="basic"
                  smtpServer="${environment:EmailConfigNlogSMTPServer}"
                  smtpPort="25"
               smtpusername="${environment:EmailConfigNlogSMTPUsername}"
                smtppassword="${environment:EmailConfigNlogSMTPPassword}" />
        <!-- set up a blackhole log catcher -->
        <target xsi:type="Null" name="blackhole" />
    </targets>

     <rules>
        <!-- Skip Microsoft logs and so log only own logs-->
        <logger name="Microsoft.*" level="Info" writeTo="blackhole" final="true" />
        <!-- Send errors via emailnotify target -->
        <logger name="*" minlevel="Error" writeTo="emailnotify" final="true" />
      </rules>

但是,有时可能会在短时间内接连收到大量电子邮件,这让我觉得自己在 "spammed" 之前就已经有机会解决问题了。

  1. 一个例子可能在极少数情况下出现,例如我的应用程序的第 3 方搜索引擎已关闭。此时,每个在网站上进行搜索的用户都会产生一个错误,从而导致潜在的大量电子邮件。
  2. 另一个例子是,当公司收集网站技术统计数据或只是普通的狡猾请求时,会出现推测性 404。即 /bitcoin/xxx 或 /shop/xxx 或 git/.head/xxx.

从历史上看,我在收到的一些 404 请求中使用了 "filters" 功能,并设置了一些。请参见下面的示例。但是,在批量格式中,我希望收到有关所有 404 或 500 的通知。我认为尝试过滤掉看起来不相关的东西不是一个好主意,因为您可能会开始错过关键信息,例如黑客攻击、SQL 注入攻击或应用程序嗅探:

    <filters>
        <when condition="contains('${message}','.php')" action="Ignore" />
        <when condition="contains('${message}','wp-includes')" action="Ignore" />
        <when condition="contains('${message}','wordpress')" action="Ignore" />
        <when condition="contains('${message}','downloader')" action="Ignore" />
      </filters>
    </logger>

要求

我想在我的应用程序日志记录中引入以下逻辑:

  1. 当出现第一个 500 类型错误时(即在网站上执行搜索时出错,访问数据库时出错),我会立即知道应用程序出现初始错误时的电子邮件通知
  2. 在初始错误后批量通知连续 500 个错误,这样我就知道这个错误何时发生,但它不会继续 "spam" 我。
  3. 批量通知所有 404、405 错误

这些批量电子邮件会在出现 50 次错误后或每天一次通知我,以先到者为准。

理想情况下,我希望批量电子邮件的格式为:

群发电子邮件

主题:500 个错误 (13) | 404 错误 (2)

table 格式的正文:

500 Errors (13)
-----------------------------------------------------
| When                 | Message                    |
-----------------------------------------------------
| 2019-11-13 11:10:29  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:10:30  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:10:31  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:10:32  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:10:33  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:10:34  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:10:35  | Website search engine down |
-----------------------------------------------------
| 2019-11-13 11:11:01  | Login failed for user foo  |
-----------------------------------------------------
| 2019-11-13 11:11:01  | Login failed for user foo  |
-----------------------------------------------------
| 2019-11-13 11:11:01  | Login failed for user foo  |
-----------------------------------------------------
| 2019-11-13 11:11:02  | Login failed for user foo  |
-----------------------------------------------------
| 2019-11-13 11:11:02  | Login failed for user foo  |
-----------------------------------------------------
| 2019-11-13 11:11:02  | Login failed for user foo  |
-----------------------------------------------------    

404 Errors (2)
-----------------------------------------------------
| When                 | Message                    |
-----------------------------------------------------
| 2019-11-13 11:10:45  | /bitcoin not found         |
-----------------------------------------------------
| 2019-11-13 11:10:57  | /shop not found            |
-----------------------------------------------------

或者,我很乐意为 500s、404s 或 405s 提供单独的批量电子邮件。

在上面的批量电子邮件示例中,应用程序会立即通过 2 封单独的电子邮件通知我存在问题

  1. 随着网站搜索引擎的关闭
  2. 用户无法访问 SQL 数据库

然后在遇到 50 个错误或每天一次后,发送批量电子邮件。

问题

仅使用 NLog 配置是否可以实现我的要求?

我知道如何使用 NLog 的 buffer wrapper 配置执行批量通知,尽管我认为不可能以我理想的 table 格式输出。

      <target xsi:type="BufferingWrapper"
              name="bulkemailnotify"
              bufferSize="50"
              flushTimeout="86400000‬"
              slidingTimeout="False"
              overflowAction="Flush">
        <target xsi:type="emailnotify" />
      </target>

如果不是,它会查看以下内容之一吗?

  1. Extending NLog
  2. Removing NLog and adding a custom logger

理想情况下,我会继续使用 NLog,并且只会在万不得已时添加自定义记录器。

假设 ${aspnet-response-statuscode} 是返回 HTTP 状态代码的那个:

那么我们可以有以下两个目标:

  • 出现第一个错误的即时邮件目标(第一个事件立即触发,并在再次触发前等待 5 分钟)
  • 包含所有错误的群发邮件目标(所有事件在 5 分钟后触发)

那么你可能会这样做:

<targets async="true">
   <target xsi:type="SplitGroup" name="special-mail">
       <target xsi:type="FilteringWrapper" name="filter-mail">
          <filter type="whenRepeated" layout="${aspnet-response-statuscode}" timeoutSeconds="300" action="Ignore" />
          <target xsi:type="Mail" name="instant-mail">
                 <header>Instant Error Report</header>
          </target>
       </target>
       <target xsi:type="BufferingWrapper" name="buffer-mail" flushTimeout="300000">
           <target xsi:type="Mail" name="bulk-mail">
                <header>Bulk Error Report</header>
           </target>
       </target>
   </target>
</targets>
<rules>
    <logger name="Microsoft.*" maxlevel="Info" final="true" />
    <logger name="*" minlevel="Error" writeTo="special-mail" final="true" />
</rules>

不太擅长邮件目标布局(页眉+正文+页脚)和制作好看的html-电子邮件。但你或许可以从互联网上窃取一些东西。

我将@Rolf Kristensen 的回答标记为答案,因为他提供了我要求的大部分功能。

不过,我想我会分享我的最终配置,以防对其他人有所帮助。

理想情况下,我想使用 ${exception:format=HResult} 检查 "whenRepeated" 过滤器中的重复日志,但有一个错误已在 nlog 的未来版本中修复。

编辑: 根据 Rolf Kristensen 的建议更新了 <filter>WhenRepeated 属性以包含 ${exception:format=TargetSite} ${exception:format=Type}

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      autoReload="true"
      internalLogLevel="Warn"
      internalLogFile="c:\temp\internal.txt">

  <!-- We're using the Mail xsi:type so reference mailkit -->
  <extensions>
    <add assembly="NLog.MailKit"/>
  </extensions>

  <!-- Set up the variables that we want to use across the various email targets -->
  <variable name="exceptiontostring" value="${exception:format=tostring}" />
  <variable name="abbreviatedenvironment" value="${left:inner=${environment:ASPNETCORE_ENVIRONMENT}:length=1}" />
  <variable name="websiteshortname" value="${configsetting:cached=True:name=WebsiteConfig.ShortName}" />
  <!-- If you change the below value, ensure you update the nummillisecondstobufferkeyerrors variable too -->
  <variable name="numsecondstobufferkeyerrors" value="10" />
  <!-- This should always be set to the value of numsecondstobufferkeyerrors variable *1000 to ensure initial logs aren't missed being sent -->
  <variable name="nummillisecondstobufferkeyerrors" value="10000" />
  <!-- Set the timings around how often the email the non key error email gets sent. If you want an email once a day, set this to 86400000 -->
  <variable name="nummillisecondstobuffernonkeyerrors" value="5000" />
  <variable name="borderstyle" value="border: 1px solid #000000;" />

  <!-- Define various log targets -->
  <!-- Targets that write info and error logs to a file -->
  <targets async="true">
    <!-- Write logs to file -->
    <target xsi:type="File" name="error" fileName="D:\Dev\dotnet\Core\Logs\Logging2.2\error-log-${shortdate}.log"
                 layout="${longdate}|${logger}|${uppercase:${level}}|${message} ${exception}" />
    <target xsi:type="File" name="info" fileName="D:\Dev\dotnet\Core\Logs\Logging2.2\info-log-${shortdate}.log"
                 layout="${longdate}|${logger}|${uppercase:${level}}|${message}" />
  </targets>

  <!-- Targets that send emails -->
  <targets>
    <!-- Ensure we try for up to a minute to resend emails if there's a temporary issue with the email service so we still get to be notified of any logs -->
    <default-wrapper xsi:type="AsyncWrapper">
      <wrapper-target xsi:type="RetryingWrapper" retryDelayMilliseconds="6000" retryCount="10" />
    </default-wrapper>

    <!-- Set up the default parameters for each email target so we don't have to repeat them each time -->
    <default-target-parameters xsi:type="Mail">
      <to>${configsetting:cached=True:name=EmailConfig.SendErrorsTo}</to>
      <from>${configsetting:cached=True:name=EmailConfig.FromAddress}</from>
      <secureSocketOption>StartTls</secureSocketOption>
      <smtpAuthentication>basic</smtpAuthentication>
      <smtpServer>${environment:EmailConfigNlogSMTPServer}</smtpServer>
      <smtpPort>25</smtpPort>
      <smtpusername>${environment:EmailConfigNlogSMTPUsername}</smtpusername>
      <smtppassword>${environment:EmailConfigNlogSMTPPassword}</smtppassword>
      <html>true</html>
      <addNewLines>true</addNewLines>
      <replaceNewlineWithBrTagInHtml>true</replaceNewlineWithBrTagInHtml>
    </default-target-parameters>

    <!-- Set up a SplitGroup target that allows us to split out key errors between
    1) Sending an initial email notification for each key distinct error
    2) Buffer all key errors and send them later so we don't keep getting the same error multiple times straight away -->
    <target xsi:type="SplitGroup" name="key-error-mail">
      <target xsi:type="FilteringWrapper" name="filter-mail">
        <!-- Ignore any previous logs for the same controller, action and exception
        TO DO: Change ${exception:format=TargetSite} ${exception:format=Type} in the layout below to be ${exception:format=HResult} when bug is fixed -->
        <filter type="whenRepeated" layout="${exception:format=TargetSite} ${exception:format=Type}" timeoutSeconds="${numsecondstobufferkeyerrors}" action="Ignore" />
        <!-- If we get past the ignore filter above then we send the instant mail -->
        <target xsi:type="Mail" name="key-error-instant-mail">
          <subject>Error on ${websiteshortname} (${abbreviatedenvironment}) ${newline}</subject>
          <layout>${longdate} ${level:uppercase=true:padding=5} - ${logger:shortName=true} - ${message} ${exceptiontostring} ${newline}</layout>
        </target>
      </target>
      <!-- Ensure that we buffer all logs to the bulk email -->
      <target xsi:type="BufferingWrapper" name="buffer-key-error-mail" flushTimeout="${nummillisecondstobufferkeyerrors}">
        <!-- Send out bulk email of key logs in a table format -->
        <target xsi:type="Mail" name="key-error-bulk-mail">
          <subject>Bulk Key Errors on ${websiteshortname} (${abbreviatedenvironment})</subject>
          <layout>
            &lt;strong&gt;${aspnet-response-statuscode} Error ${when:when='${aspnet-request-url}'!='':inner=on ${aspnet-request-url} (${aspnet-mvc-controller} > ${aspnet-mvc-action})}&lt;/strong&gt;
            &lt;table width="100%" border="0" cellpadding="2" cellspacing="2" style="border-collapse:collapse; ${borderstyle}"&gt;
            &lt;thead&gt;
            &lt;tr style="background-color:#cccccc;"&gt;
            &lt;th style="${borderstyle}"&gt;Date&lt;/th&gt;
            &lt;th style="${borderstyle}"&gt;IP Address&lt;/th&gt;
            &lt;th style="${borderstyle}"&gt;User Agent&lt;/th&gt;
            &lt;th style="${borderstyle}"&gt;Method&lt;/th&gt;
            &lt;/tr&gt;
            &lt;/thead&gt;
            &lt;tbody&gt;
            &lt;tr&gt;
            &lt;td style="${borderstyle}"&gt;${longdate}&lt;/td&gt;
            &lt;td style="${borderstyle}"&gt;${aspnet-request-ip}&lt;/td&gt;
            &lt;td style="${borderstyle}"&gt;${aspnet-request-useragent}&lt;/td&gt;
            &lt;td style="${borderstyle}"&gt;${aspnet-request-method}&lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;&lt;td colspan="4" style="${borderstyle}"&gt;${message}&lt;/td&gt;&lt;/tr&gt;
            &lt;tr&gt;&lt;td colspan="4" style="${borderstyle}"&gt;${exceptiontostring}&lt;/td&gt;&lt;/tr&gt;
            &lt;/tbody&gt;&lt;/table&gt;
            ${newline}
          </layout>
        </target>
      </target>
    </target>

    <!-- Set up a non key error buffer wrapper that we're not interested in receiving an immediate notification for
    Send a periodic email with a list of the logs encountered since the last email was sent
    Send the email out in table format -->
    <target xsi:type="BufferingWrapper" name="buffer-mail-non-key-error" flushTimeout="${nummillisecondstobuffernonkeyerrors}">
      <target xsi:type="Mail" name="bulk-mail-non-key-error">
        <subject>Bulk Errors on ${websiteshortname} (${abbreviatedenvironment})</subject>
        <layout>
          &lt;strong&gt;${aspnet-response-statuscode} Error ${when:when='${aspnet-request-url}'!='':inner=on ${aspnet-request-url} (${aspnet-mvc-controller} > ${aspnet-mvc-action})}&lt;/strong&gt;
          &lt;table width="100%" border="0" cellpadding="2" cellspacing="2" style="border-collapse:collapse; ${borderstyle}"&gt;
          &lt;thead&gt;
          &lt;tr style="background-color:#cccccc;"&gt;
          &lt;th style="${borderstyle}"&gt;Date&lt;/th&gt;
          &lt;th style="${borderstyle}"&gt;IP Address&lt;/th&gt;
          &lt;th style="${borderstyle}"&gt;User Agent&lt;/th&gt;
          &lt;th style="${borderstyle}"&gt;Method&lt;/th&gt;
          &lt;/tr&gt;
          &lt;/thead&gt;
          &lt;tbody&gt;
          &lt;tr&gt;
          &lt;td style="${borderstyle}"&gt;${longdate}&lt;/td&gt;
          &lt;td style="${borderstyle}"&gt;${aspnet-request-ip}&lt;/td&gt;
          &lt;td style="${borderstyle}"&gt;${aspnet-request-useragent}&lt;/td&gt;
          &lt;td style="${borderstyle}"&gt;${aspnet-request-method}&lt;/td&gt;
          &lt;/tr&gt;
          &lt;tr&gt;&lt;td colspan="4" style="${borderstyle}"&gt;${message}&lt;/td&gt;&lt;/tr&gt;
          &lt;/tbody&gt;&lt;/table&gt;
          ${newline}
        </layout>
      </target>
    </target>

    <!-- Set up a blackhole log catcher -->
    <target xsi:type="Null" name="blackhole" />

  </targets>

  <rules>
    <!-- Skip Microsoft logs and so log only own logs-->
    <logger name="Microsoft.*" level="Info" writeTo="blackhole" final="true" />
    <!-- Now we've sent Microsoft logs to the blackhole, we only log non Microsoft info to the info log file-->
    <logger name="*" level="Info" writeTo="info" final="true" />
    <!-- Log errors to the error email buffer and error file -->
    <logger name="*" minlevel="Error" writeTo="error" />
    <!-- log any key errors to the SplitGroup target -->
    <logger name="*" minlevel="Error" writeTo="key-error-mail">
      <filters defaultAction="Log">
        <when condition="starts-with('${aspnet-response-statuscode}','4')" action="Ignore" />
      </filters>
    </logger>
    <!-- log any non key errors to the bufferwrapper target -->
    <logger name="*" minlevel="Error" writeTo="buffer-mail-non-key-error">
      <filters defaultAction="Log">
        <when condition="not starts-with('${aspnet-response-statuscode}','4')" action="Ignore" />
      </filters>
    </logger>
  </rules>
</nlog>