Akka http 在高吞吐量时抛出异常 - java.io.IOException: 系统中打开的文件太多

Question

我正在尝试向 akka-http 发送 200k 条消息。

protected val someRouts: Route = pathPrefix("foo") {
        pathEndOrSingleSlash {
          put {
             entity(as[Foo]) { foo =>
              log.debug(s"/foo update $foo")
              complete(Future(Foo("a")).map(f => s"Got - $f "))
            }
          }
        }

Http().bindAndHandle(someRouts, "0.0.0.0", 9000)

我从另一个进程循环发送 200K 条消息。客户端代码（简化）：

lazy val apiFlow: Flow[HttpRequest, HttpResponse, Any] =
    Http().outgoingConnection("0.0.0.0", 9000)

  def request(request: HttpRequest): Future[HttpResponse] = Source.single(request).via(apiConnectionFlow).runWith(Sink.head)

for (i <- 1 to 200000){
    request(RequestBuilding.Put("/foo", Foo(i))
}

过了一会儿我得到这个异常：

[akka.actor.default-dispatcher-33] ERROR akka.io.TcpListener - Accept error: could not accept new connection
java.io.IOException: Too many open files in system
    at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
    at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
    at akka.io.TcpListener.acceptAllPending(TcpListener.scala:112)
    at akka.io.TcpListener$$anonfun$bound.applyOrElse(TcpListener.scala:85)
    at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
    at akka.io.TcpListener.aroundReceive(TcpListener.scala:34)
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
    at akka.actor.ActorCell.invoke(ActorCell.scala:495)
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

更新使用 lsof -i tcp:9000 | wc -l 检查打开的连接我可以看到该数字明显攀升至 6500

以上

Answer 1

您的问题是您正试图打开到 0.0.0.0:9000 的 200,000 个 TCP 连接，而您正在使用的机器未配置为允许同时激活那么多文件描述符。

来自 the docs（强调我的）：

Note that no connection is attempted until the returned flow is actually materialized! If the flow is materialized several times then several independent connections will be opened (one per materialization).

检查 ulimit -n - 它可能远低于 200K。

另请参阅：

Increasing the maximum number of tcp/ip connections in linux
How do I change the number of open files limit in Linux?

Akka http 在高吞吐量时抛出异常 - java.io.IOException: 系统中打开的文件太多

Akka http throwing exception on high throughput - java.io.IOException: Too many open files in system

scala

akka

akka-http