波光粼粼的水滴项目运行 issue

Question

我从“https://github.com/h2oai/h2o-droplets/tree/master/sparkling-water-droplet”克隆了 sparkling droplet 项目。并分别使用 ./gradlew clean 和 ./gradlew build 清理和构建项目。之后尝试使用命令
运行项目 spark-submit --class water.droplets.SparklingWaterDroplet build/libs/sparkling-water-droplet-app.jar
然后收到以下错误消息：

Exception in thread "main" java.lang.NoClassDefFoundError: water/fvec/Frame
at water.droplets.SparklingWaterDroplet.main(SparklingWaterDroplet.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:674)
at org.apache.spark.deploy.SparkSubmit$.doRunMain(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Caused by: java.lang.ClassNotFoundException: water.fvec.Frame
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more

Answer 1

您必须运行 ./gradlew shadowJar 并使用它创建的 jar build/libs/sparkling-water-droplet-app.jar 因为它创建了一个 fat jar，其中包含提交作业时所需的所有类（build 任务不会那样做）。

波光粼粼的水滴项目运行 issue

Sparkling water droplet project running issue

scala

machine-learning

apache-spark

h2o