启动 NIFI 时 .. 出现异常：java.nio.file.FileSystemException: ./state/local/partition-0/107654.journal: 打开的文件太多

Question

启动 NiFi 时失败并出现以下错误

    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1086)
    at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:55)
    ... 37 common frames omitted

原因：org.springframework.beans.factory.BeanCreationException：创建名称为 'flowController' 的 bean 时出错：FactoryBean 在创建对象时抛出异常；嵌套异常是 java.lang.RuntimeException: java.nio.file.FileSystemException: ./state/local/partition-0/107654.journal: 打开的文件太多 在 org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)

我调试发现

的所有分区都创建了很多文件

/usr/lib/NiFi/NiFi/nifi-1.11.4/state/local

分区-0 分区-11 分区-14 分区-3 分区-6 分区-9 分区1 分区12 分区15 分区4 分区7 快照分区 10 分区 13 分区 2 分区 5 分区 8 wali.lock

如果有任何分区文件夹我 select ...假设我 select partition-0 所以里面有很多日记条目我可以看到

107018.journal 107632.journal 108246.journal 108860.journal 109474.journal 110088.journal 107019.journal 107633.journal 108247.journal 108861.journal 109475.journal 110089.journal 107020.journal 107634.journal 108248.journal 108862.journal 109476.journal 110090.journal 107021.journal 107635.journal 108249.journal 108863.journal 109477.journal 110091.journal 107022.journal 107636.journal 108250.journal 108864.journal 109478.journal 110092.journal 107023.journal 107637.journal 108251.journal 108865.journal 109479.journal 110093.journal 107024.journal 107638.journal 108252.journal 108866.journal 109480.journal 110094.journal 107025.journal 107639.journal 108253.journal 108867.journal 109481.journal 110095.journal 107026.journal 107640.journal 108254.journal 108868.journal 109482.journal 110096.journal 107027.journal 107641.journal 108255.journal 108869.journal 109483.journal 110097.journal 107028.journal 107642.journal 108256.journal 108870.journal 109484.journal 110098.journal 107029.journal 107643.journal 108257.journal 108871.journal 109485.journal 110099.journal 107030.journal 107644.journal 108258.journal 108872.journal 109486.journal 110100.journal 107031.journal 107645.journal 108259.journal 108873.journal 109487.journal 110101.journal 107032.journal 107646.journal 108260.journal 108874.journal 109488.journal 110102.journal 107033.journal 107647.journal 108261.journal 108875.journal 109489.journal 110103.journal 107034.journal 107648.journal 108262.journal 108876.journal 109490.journal 110104.journal 107035.journal 107649.journal 108263.journal 108877.journal 109491.journal 110105.journal

条目更多，我想这导致无法启动 Nifi。

希望我能在这里找到解决这个问题的方法

Answer 1

您是否遵循操作系统上的 instructions to raise the open file limit？当 reading/writing 到内部数据存储库时，NiFi 需要大量打开的文件，因此如果这受到限制，它通常会阻止成功操作。

来自文档：

Maximum File Handles

NiFi will at any one time potentially have a very large number of file handles open. Increase the limits by editing /etc/security/limits.conf to add something like

hard nofile 50000

soft nofile 50000

Maximum Forked Processes

NiFi may be configured to generate a significant number of threads. To increase the allowable number, edit /etc/security/limits.conf

hard nproc 10000

soft nproc 10000

And your distribution may require an edit to /etc/security/limits.d/90-nproc.conf by adding

soft nproc 10000

启动 NIFI 时 .. 出现异常：java.nio.file.FileSystemException: ./state/local/partition-0/107654.journal: 打开的文件太多

While starting NIFI.. getting Exception: java.nio.file.FileSystemException: ./state/local/partition-0/107654.journal: Too many open files

apache-nifi