在 Windows 上安装 apache spark

Question

您好，我正在尝试安装 apache spark，但出现此错误。我已经安装了兼容apache spark、phyton和scala的java版本。非常感谢您的帮助。

    Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/05/18 22:50:07 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/05/18 22:50:09 ERROR SparkContext: Error initializing SparkContext.
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
...more
Caused by: java.net.URISyntaxException: Illegal character in path at index 32: spark://DESKTOP-H3SRII2:63213/C:\classes
        at java.net.URI$Parser.fail(Unknown Source)
...more
Caused by: java.net.URISyntaxException: Illegal character in path at index 32: spark://DESKTOP-H3SRII2:63213/C:\classes
        at java.net.URI$Parser.fail(Unknown Source)
        at java.net.URI.<init>(Unknown Source)
        at org.apache.spark.repl.ExecutorClassLoader.<init>(ExecutorClassLoader.scala:57)
22/05/18 22:50:09 ERROR Utils: Uncaught exception in thread shutdown-hook-0
java.lang.ExceptionInInitializerError
        at org.apache.spark.executor.Executor.stop(Executor.scala:333)
...more
Caused by: java.lang.NullPointerException
        at org.apache.spark.shuffle.ShuffleBlockPusher$.<init>(ShuffleBlockPusher.scala:465)
        at org.apache.spark.shuffle.ShuffleBlockPusher$.<clinit>(ShuffleBlockPusher.scala)
22/05/18 22:50:09 WARN ShutdownHookManager: ShutdownHook '$anon' failed, java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
        at java.util.concurrent.FutureTask.report(Unknown Source)
        at java.util.concurrent.FutureTask.get(Unknown Source)
        at org.apache.hadoop.util.ShutdownHookManager.executeShutdown(ShutdownHookManager.java:124)
        at org.apache.hadoop.util.ShutdownHookManager.run(ShutdownHookManager.java:95)

...更多原因：java.lang.ExceptionInInitializerError 在 org.apache.spark.executor.Executor.stop(Executor.scala:333) ...更多的原因：java.lang.NullPointerException 在 org.apache.spark.shuffle.ShuffleBlockPusher$.(ShuffleBlockPusher.scala:465) 在 org.apache.spark.shuffle.ShuffleBlockPusher$.(ShuffleBlockPusher.scala)

Answer 1

我下载了较低版本的spark 3.2.1，一切正常。我不知道这有什么解释，但显然这是解决方案。

在 Windows 上安装 apache spark

install apache spark on Windows

apache-spark