如何获取 Spark Java 应用程序自动定义的端口？

Question

在 Java Spark（不是 Apache spark）的 API 文档中，您可以指定端口 0 以使其自动 select 成为一个端口。太棒了！

但是，我不知道如何在服务器启动后获取该端口。我可以在日志中看到它：

15:41:12.459 [Thread-2] INFO  spark.webserver.JettySparkServer - >> Listening on 0.0.0.0:63134

但我需要能够以编程方式访问它，以便我的集成测试每次都能运行可靠地进行。

那么如何获取该端口？

Answer 1

我找不到在 API 中获取此信息的方法，因此我提交了一份 issue on their github。

我通过一堆丑陋的反思得到了它：

/**
 * Meant to be called from a different thread, once the spark app is running
 * This is probably only going to be used during the integration testing process, not ever in prod!
 *
 * @return the port it's running on
 */
public static int awaitRunningPort() throws Exception {
    awaitInitialization();
    //I have to get the port via reflection, which is fugly, but the API doesn't exist :(
    //Since we'll only use this in testing, it's not going to kill us
    Object instance = getInstance();
    Class theClass = instance.getClass();
    Field serverField = theClass.getDeclaredField("server");
    serverField.setAccessible(true);
    Object oneLevelDeepServer = serverField.get(instance);

    Class jettyServerClass = oneLevelDeepServer.getClass();
    Field jettyServerField = jettyServerClass.getDeclaredField("server");
    jettyServerField.setAccessible(true);
    //Have to pull in the jetty server stuff to do this mess
    Server jettyServer = (Server)jettyServerField.get(oneLevelDeepServer);

    int acquiredPort = ((ServerConnector)jettyServer.getConnectors()[0]).getLocalPort();

    log.debug("Acquired port: {}", acquiredPort);
    return acquiredPort;
}

这在我们的集成测试中对我来说效果很好，但我没有使用 https，它确实通过反射抓取受保护字段达到了 API 的大约两个级别。我找不到任何其他方法来做到这一点。很高兴被证明是错误的。

Answer 2

这将适用于 Spark 2.6.0：

public static int start (String keystoreFile, String keystorePw)
{
    secure(keystoreFile, keystorePw, null, null);
    port(0);

    staticFiles.location("/public");

    get(Path.CLOCK, ClockController.time);
    get(Path.CALENDAR, CalendarController.date);

    // This is the important line. It must be *after* creating the routes and *before* the call to port()
    awaitInitialization();

    return port();
}

如果不调用 awaitInitialization() port() 将 return 0.

如何获取 Spark Java 应用程序自动定义的端口？

How to get the automatically defined port for a Spark Java Application?

java

spark-java