在使用最新的 Jetty 和 Jersey 上传到 Heroku 的 WAR 文件中找不到依赖项 class

Question

我有一个用 Java 语言开发的简单网络应用程序，它利用 DL4J and ND4J 的库依赖来读取和处理某个文本文件（文本文件的大小很大，大约 100- 200mb) 位于 WAR 文件的 WEB-INF 文件夹内。我正在使用 Maven 构建工具，并且我在 POM.xml :

中包含了所有必要的依赖项

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

<modelVersion>4.0.0</modelVersion>

<groupId>com.example</groupId>
<artifactId>simple-heroku-webapp</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>simple-heroku-webapp</name>


<properties>
    <nd4j.version>0.6.0</nd4j.version>
    <dl4j.version>0.6.0</dl4j.version>
    <jersey.version>2.23.2</jersey.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <jetty.version>9.0.6.v20130930</jetty.version>
</properties>


<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.glassfish.jersey</groupId>
            <artifactId>jersey-bom</artifactId>
            <version>${jersey.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>



<dependencies>
    <dependency>
        <groupId>org.glassfish.jersey.containers</groupId>
        <artifactId>jersey-container-servlet</artifactId>
    </dependency>

    <dependency>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-servlet</artifactId>
        <version>${jetty.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-webapp</artifactId>
        <version>${jetty.version}</version>
        <scope>provided</scope>
    </dependency>

    <dependency>
        <groupId>org.glassfish.jersey.test-framework.providers</groupId>
        <artifactId>jersey-test-framework-provider-bundle</artifactId>
        <type>pom</type>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-ui</artifactId>
        <version>${dl4j.version}</version>
    </dependency>

    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-nlp</artifactId>
        <version>${dl4j.version}</version>
    </dependency>
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-native</artifactId>
        <version>${nd4j.version}</version>
    </dependency>
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-common</artifactId>
        <version>${nd4j.version}</version>
    </dependency>
</dependencies>

<build>
    <finalName>simple-heroku-webapp</finalName>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>2.5.1</version>
            <inherited>true</inherited>
            <configuration>
                <source>1.7</source>
                <target>1.7</target>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <executions>
                <execution>
                    <id>copy-dependencies</id>
                    <phase>package</phase>
                    <goals>
                        <goal>copy-dependencies</goal>
                    </goals>
                    <configuration>
                        <includeScope>compile</includeScope>
                    </configuration>
                </execution>
                <execution>
                    <phase>package</phase>
                    <goals><goal>copy</goal></goals>
                    <configuration>
                        <artifactItems>
                            <artifactItem>
                                <groupId>org.eclipse.jetty</groupId>
                                <artifactId>jetty-runner</artifactId>
                                <version>9.0.0.M5</version>
                                <destFileName>jetty-runner.jar</destFileName>
                            </artifactItem>
                        </artifactItems>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.eclipse.jetty</groupId>
            <artifactId>jetty-maven-plugin</artifactId>
            <version>${jetty.version}</version>
            <configuration>
                <contextPath>/</contextPath>
                <webApp>
                    <contextPath>/</contextPath>
                    <webInfIncludeJarPattern>.*/.*jersey-[^/]\.jar$</webInfIncludeJarPattern>
                </webApp>
                <war>${project.build.directory}/${project.build.finalName}.war</war>
            </configuration>
        </plugin>
        <plugin>
            <groupId>com.heroku.sdk</groupId>
            <artifactId>heroku-maven-plugin</artifactId>
            <version>1.1.1</version>
        </plugin>
    </plugins>
</build>

我使用 jetty-runner 以 WAR 格式在本地码头服务器上测试了应用程序，每个 GET 请求都按预期工作。因此，我使用来自 Intellij IDE 的 heroku:deploy-war 将它部署到 Heroku。部署成功，未显示 error/warning 日志。之后，我导航到 Heroku link 上的主页（在不使用 DL4J 和 ND4J 的页面上调用 GET 方法），页面正常运行。但是，当我尝试对使用 DL4J 和 ND4J 库的方法调用 GET 请求时，浏览器页面显示此错误，指出以下 NoClassFound error :

HTTP Status 500 - org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: Could not initialize class org.nd4j.linalg.factory.Nd4j

    type Exception report

    message org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: Could not initialize class org.nd4j.linalg.factory.Nd4j

    description The server encountered an internal error that prevented it from fulfilling this request.

    exception

    javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: Could not initialize class org.nd4j.linalg.factory.Nd4j
        org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:489)
        org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)

    root cause

    org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: Could not initialize class org.nd4j.linalg.factory.Nd4j
        org.glassfish.jersey.servlet.internal.ResponseWriter.rethrow(ResponseWriter.java:278)
        org.glassfish.jersey.servlet.internal.ResponseWriter.failure(ResponseWriter.java:260)
        org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:509)
        org.glassfish.jersey.server.ServerRuntime.run(ServerRuntime.java:334)
        org.glassfish.jersey.internal.Errors.call(Errors.java:271)
        org.glassfish.jersey.internal.Errors.call(Errors.java:267)
        org.glassfish.jersey.internal.Errors.process(Errors.java:315)
        org.glassfish.jersey.internal.Errors.process(Errors.java:297)
        org.glassfish.jersey.internal.Errors.process(Errors.java:267)
        org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
        org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
        org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
        org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
        org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)

    root cause

    java.lang.NoClassDefFoundError: Could not initialize class org.nd4j.linalg.factory.Nd4j
        org.deeplearning4j.models.embeddings.loader.WordVectorSerializer.loadTxt(WordVectorSerializer.java:1578)
        org.deeplearning4j.models.embeddings.loader.WordVectorSerializer.loadTxtVectors(WordVectorSerializer.java:1502)
        com.example.MyResource.train(MyResource.java:61)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.lang.reflect.Method.invoke(Method.java:606)
        org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.invoke(ResourceMethodInvocationHandlerFactory.java:81)
        org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.run(AbstractJavaResourceMethodDispatcher.java:144)
        org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
        org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
        org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
        org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
        org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
        org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
        org.glassfish.jersey.server.ServerRuntime.run(ServerRuntime.java:326)
        org.glassfish.jersey.internal.Errors.call(Errors.java:271)
        org.glassfish.jersey.internal.Errors.call(Errors.java:267)
        org.glassfish.jersey.internal.Errors.process(Errors.java:315)
        org.glassfish.jersey.internal.Errors.process(Errors.java:297)
        org.glassfish.jersey.internal.Errors.process(Errors.java:267)
        org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
        org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
        org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
        org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
        org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
        org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)

    note The full stack trace of the root cause is available in the Apache Tomcat/8.0.30 logs.

更具体地说，它只会在我从以下源代码调用 train 方法时发生：

package com.example;
import org.deeplearning4j.models.embeddings.loader.WordVectorSerializer;
import org.deeplearning4j.models.embeddings.wordvectors.WordVectors;

import javax.servlet.ServletContext;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;
import java.io.*;
import java.util.Arrays;
import java.util.Collection;

/**
 * Root resource (exposed at "myresource" path)
 */

@Path("test")
public class MyResource {

    WordVectors wordVectors=null;

    String loc="/WEB-INF/glove.6B.50d.txt";
    @javax.ws.rs.core.Context
    ServletContext context;


    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String getIt() {
        return  "Test";
     }

    @GET
    @Path("train")
    @Produces("text/plain")
    public String train(){


        String fullPath = context.getRealPath(loc);
        File file = new File(fullPath);
        try {
            wordVectors = WordVectorSerializer.loadTxtVectors(file);
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        Collection<String> lst = wordVectors.wordsNearest("day", 20);


        return Arrays.toString(lst.toArray());
    }

}

Class 启动 Jetty 嵌入式容器：

package com.example.heroku;

import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.webapp.WebAppContext;

public class Main {

    public static void main(String[] args) throws Exception{
        String webPort = System.getenv("PORT");
        if (webPort == null || webPort.isEmpty()) {
            webPort = "8080";
        }

        final Server server = new Server(Integer.valueOf(webPort));
        final WebAppContext root = new WebAppContext();

        root.setContextPath("/");
        root.setParentLoaderPriority(true);

        final String webappDirLocation = "src/main/webapp/";
        root.setDescriptor(webappDirLocation + "/WEB-INF/web.xml");
        root.setResourceBase(webappDirLocation);

        server.setHandler(root);
        server.start();
        server.join();
    }
}

错误似乎指出 ND4J 库中的 class 之一无法访问，尽管我已经检查了 WEB-INF/lib 文件夹，并且所有 ND4J 库都在那里。任何 guidance/help/tips 将不胜感激。

Answer 1

听起来标准的 Servlet WebApp 类加载器隔离正在发挥作用。

您必须注意您的 class 所在的位置，它是从服务器 class 路径加载的，还是从 WebApp 的 class路径。

当您使用...

java -cp target/classes:target/dependency/* com.example.heroku.Main

在target/classes:target/dependency/*中定义的classes和jar在服务器类路径上，因此，禁止WebApp看到来自服务器类路径的那些。

然而...

您还声明了 root.setParentLoaderPriority(true);，它打破了 Servlet WebApp 类加载器隔离，但是您现在在不同的地方有 2 个 classes 的副本。一次来自服务器类路径，一次来自 WebApp 的类路径。如果它们的一些 classes 是 1 classpath 的一部分而有些是另一个

path 的一部分，则有许多库将无法运行。

如果您打算使用这种 embedded-jetty 方法，请考虑让您的 WEB-INF/lib 完全为空，或者不在您的服务器类路径中声明 classes。

当你运行 ...

mvn jetty:run-war

然后 classes 是您的 WebApp classpath 的一部分，WebApp 可以看到它们，而服务器 classpath 没有它们的副本。

在使用最新的 Jetty 和 Jersey 上传到 Heroku 的 WAR 文件中找不到依赖项 class

Dependency class not found on uploaded WAR file to Heroku with latest Jetty and Jersey

java

jetty

heroku

jersey

maven