显示 python 未找到的 spark-submit 命令

Question

当我运行命令 spark-submit spark.py 时出现以下错误，我无法理解我已经安装了 python。我已经检查了它们正确的路径并且 python 版本也是 3.9。我找不到错误。请帮忙！

java.io.FileNotFoundException: Hadoop bin directory does not exist: D:\spark-3.1.2-bin-hadoop3.2\bin\bin -see https://wiki.apache.org/hadoop/WindowsProblems
        at org.apache.hadoop.util.Shell.getQualifiedBinInner(Shell.java:608)
        at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:592)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:689)
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:78)
        at org.apache.hadoop.conf.Configuration.getTimeDurationHelper(Configuration.java:1814)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1791)
        at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
        at org.apache.hadoop.util.ShutdownHookManager$HookEntry.<init>(ShutdownHookManager.java:207)
        at org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:302)
        at org.apache.spark.util.SparkShutdownHookManager.install(ShutdownHookManager.scala:181)
        at org.apache.spark.util.ShutdownHookManager$.shutdownHooks$lzycompute(ShutdownHookManager.scala:50)
        at org.apache.spark.util.ShutdownHookManager$.shutdownHooks(ShutdownHookManager.scala:48)
        at org.apache.spark.util.ShutdownHookManager$.addShutdownHook(ShutdownHookManager.scala:153)
        at org.apache.spark.util.ShutdownHookManager$.<init>(ShutdownHookManager.scala:58)
        at org.apache.spark.util.ShutdownHookManager$.<clinit>(ShutdownHookManager.scala)
        at org.apache.spark.util.Utils$.createTempDir(Utils.scala:326)
        at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:343)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
        at org.apache.spark.deploy.SparkSubmit.doRunMain(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
        at org.apache.spark.deploy.SparkSubmit$$anon.doSubmit(SparkSubmit.scala:1039)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
21/06/25 14:13:46 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
log4j:WARN No appenders could be found for logger (org.apache.spark.util.ShutdownHookManager).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Answer 1

您的第一个错误没有说明 Python。

说找不到HADOOP_HOME\bin\bin

您是否不小心将 bin 文件夹包含在 HADOOP_HOME 变量中？也许如果你解决这个问题，那么它也会修复 Python 错误

显示 python 未找到的 spark-submit 命令

spark-submit command showing python not found

hadoop

apache-spark