Hadoop HiPi - hibImport NoClassDefFoundError

Question

我正在使用 Hadoop 2.9.1 和来自 Github 的最新 HiPi。它 运行 在带有 OpenJDK 10 的 Ubuntu 18.04.3 上运行。

安装成功，我尝试运行来自HiPi的示例。下载一组图像后，我尝试将其转换为 HipiImageBundle。结果如下：

hadoop@master:~/workspace/hipi$ tools/hibImport.sh ~/workspace/SampleImages ~/workspace/SampleImages/sampleimages.hib
Input image directory: /home/hadoop/workspace/SampleImages
Input FS: local FS
Output HIB: /home/hadoop/workspace/SampleImages/sampleimages.hib
Overwrite HIB if it exists: false
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.hadoop.security.authentication.util.KerberosUtil (file:/opt/hadoop/share/hadoop/common/lib/hadoop-auth-2.9.1.jar) to method sun.security.krb5.Config.getInstance()
WARNING: Please consider reporting this to the maintainers of org.apache.hadoop.security.authentication.util.KerberosUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Exception in thread "main" java.lang.NoClassDefFoundError: org/hipi/imagebundle/HipiImageBundle
        at org.hipi.tools.HibImport.main(HibImport.java:124)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.lang.ClassNotFoundException: org.hipi.imagebundle.HipiImageBundle
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:466)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:566)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
        ... 7 more

Answer 1

问题已解决。 Hipi有bug

在tools/build.gradle中有错误的行被注释掉了：

hadoop@master:/opt/hipi/tools$ cat build.gradle 
subprojects {

  dependencies {
    compile project(':core')
  }

  jar {
    manifest {
      # Uncomment this line
      attributes("Class-Path" : configurations.runtime.collect { it.getAbsolutePath() }.join(' '));
      # Comment this line out
  //    attributes("Class-Path" : configurations.runtime.collect { it.toURI() }.join(' '));
    }
  }
}

第二个问题是图像损坏的问题。当你加载它们时，你会得到一个 NullPointerException。