最终应用状态：FAILED，exitCode：16 RECEIVED SIGNAL 15：SIGTERM

Question

当运行 spark-1.6.2 集群上的 spark yarn 应用程序时，我在驱动程序中遇到以下错误：

2017-03-27 18:41:50 SIGTERM handler [INFO ] ApplicationMaster - Final app status: FAILED, exitCode: 16
2017-03-27 18:41:50 SIGTERM handler [ERROR] ApplicationMaster - RECEIVED SIGNAL 15: SIGTERM
2017-03-27 18:41:50 Driver [ERROR] SparkDriver - Failed to start the driver for TestApp
java.lang.InterruptedException
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
    at org.apache.spark.streaming.ContextWaiter.waitForStopOrError(ContextWaiter.scala:63)
    at org.apache.spark.streaming.StreamingContext.awaitTermination(StreamingContext.scala:642)
    at org.apache.spark.streaming.api.java.JavaStreamingContext.awaitTermination(JavaStreamingContext.scala:632)
    at com.test.example.SparkDriver.submit(SparkDriver.java:119)
    at com.test.example.SparkDriver.main(SparkDriver.java:94)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)

Answer 1

这个问题出现在 HDP(hortonworks) Cluster 上，现在是 HDP upgrade 问题。 HDP 环境的升级介于两者之间，并且在此 HDP 集群上暂停了升级。升级完成后，问题得到解决，我能够运行集群上的 spark yarn applications。

最终应用状态：FAILED，exitCode：16 RECEIVED SIGNAL 15：SIGTERM

Final app status: FAILED, exitCode: 16 RECEIVED SIGNAL 15: SIGTERM

hadoop-yarn

apache-spark