Databricks 作业获取 javax.net.ssl.SSLHandshakeException：收到致命警报：在 Google 云中调用 api 运行 时收到致命警报：handshake_failure
Databricks job getting javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure when calling api running in Google Cloud
作为 Databricks 作业的 spark 作业 运行 尝试通过 http 访问外部 rest api 并发生以下错误：错误 ScalaDriverLocal：用户代码堆栈跟踪：
javax.net.ssl.SSLHandshakeException：收到致命警报：handshake_failure
这是进行 http 调用的代码
Request request = new Request.Builder()
.url("https://some_url")
.get()
.addHeader("cache-control", "no-cache")
.build();
Response response = client.newCall(request).execute();
我试过在代码中设置https.protocols系统变量如下
System.setProperty("https.protocols","TLSv1,TLSv1.1,TLSv1.2");
没有结果。
这里是错误的完整堆栈跟踪：
ERROR ScalaDriverLocal: User Code Stack Trace:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2020)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1127)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:351)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:310)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:178)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:236)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:109)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:77)
at okhttp3.internal.connection.Transmitter.newExchange$okhttp(Transmitter.kt:162)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:35)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:84)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:71)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
at okhttp3.RealCall.execute(RealCall.kt:66)
at com.mycompany.metadata.MetadataRepository.loadAggregations(MetadataRepository.java:50)
at com.mycompany.jobs.DefaultJob.run(DefaultJob.java:50)
at com.mycompany.run.Main.main(Main.java:26)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command--1:1)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw$$iw$$iw.<init>(command--1:44)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw$$iw.<init>(command--1:46)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw.<init>(command--1:48)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw.<init>(command--1:50)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw.<init>(command--1:52)
at line7ccb0b1a0bd6475aac11185531c9050025.$read.<init>(command--1:54)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$.<init>(command--1:58)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$.<clinit>(command--1)
at line7ccb0b1a0bd6475aac11185531c9050025.$eval$.$print$lzycompute(<notebook>:7)
at line7ccb0b1a0bd6475aac11185531c9050025.$eval$.$print(<notebook>:6)
at line7ccb0b1a0bd6475aac11185531c9050025.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq.apply(IMain.scala:645)
at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq.apply(IMain.scala:644)
at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:215)
at com.databricks.backend.daemon.driver.ScalaDriverLocal$$anonfun$repl.apply$mcV$sp(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.ScalaDriverLocal$$anonfun$repl.apply(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.ScalaDriverLocal$$anonfun$repl.apply(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:679)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:632)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.DriverLocal$$anonfun$execute.apply(DriverLocal.scala:368)
at com.databricks.backend.daemon.driver.DriverLocal$$anonfun$execute.apply(DriverLocal.scala:345)
at com.databricks.logging.UsageLogging$$anonfun$withAttributionContext.apply(UsageLogging.scala:238)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at com.databricks.logging.UsageLogging$class.withAttributionContext(UsageLogging.scala:233)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:48)
at com.databricks.logging.UsageLogging$class.withAttributionTags(UsageLogging.scala:271)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:48)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:345)
at com.databricks.backend.daemon.driver.DriverWrapper$$anonfun$tryExecutingCommand.apply(DriverWrapper.scala:644)
at com.databricks.backend.daemon.driver.DriverWrapper$$anonfun$tryExecutingCommand.apply(DriverWrapper.scala:644)
at scala.util.Try$.apply(Try.scala:192)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:639)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:485)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:597)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:390)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:337)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:219)
at java.lang.Thread.run(Thread.java:748)
我无法 ping 指出问题的原因，但我找到了一个不使用 OkHttp 的解决方法，用它替换发出请求的代码。
HttpResponse<String> response = Unirest.get("<some_url>")
.header("cache-control", "no-cache")
.asString();
这是因为安全密码套件不兼容：
Databricks 正在选择高效的密码套件
OkHttp - 现代安全密码套件
要使其正常工作，您可以使用 https://www.ssllabs.com/ssltest/
分析 API
API 支持密码套件的部分，您必须像这样明确设置它们（对于我的情况 CBC_SHA256 是交易）：
ConnectionSpec spec = new ConnectionSpec.Builder(ConnectionSpec.MODERN_TLS)
.tlsVersions(TlsVersion.TLS_1_2)
.cipherSuites(
CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
)
.build();
OkHttpClient client = new OkHttpClient.Builder()
.connectionSpecs(Collections.singletonList(spec))
.build();
https://square.github.io/okhttp/https/
我在这里找到了很多信息和描述：
https://github.com/square/okhttp/issues/6138
作为 Databricks 作业的 spark 作业 运行 尝试通过 http 访问外部 rest api 并发生以下错误：错误 ScalaDriverLocal：用户代码堆栈跟踪： javax.net.ssl.SSLHandshakeException：收到致命警报：handshake_failure
这是进行 http 调用的代码
Request request = new Request.Builder()
.url("https://some_url")
.get()
.addHeader("cache-control", "no-cache")
.build();
Response response = client.newCall(request).execute();
我试过在代码中设置https.protocols系统变量如下
System.setProperty("https.protocols","TLSv1,TLSv1.1,TLSv1.2");
没有结果。
这里是错误的完整堆栈跟踪：
ERROR ScalaDriverLocal: User Code Stack Trace:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2020)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1127)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:351)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:310)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:178)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:236)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:109)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:77)
at okhttp3.internal.connection.Transmitter.newExchange$okhttp(Transmitter.kt:162)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:35)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:82)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:84)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:71)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:112)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:87)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.kt:184)
at okhttp3.RealCall.execute(RealCall.kt:66)
at com.mycompany.metadata.MetadataRepository.loadAggregations(MetadataRepository.java:50)
at com.mycompany.jobs.DefaultJob.run(DefaultJob.java:50)
at com.mycompany.run.Main.main(Main.java:26)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw$$iw$$iw$$iw.<init>(command--1:1)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw$$iw$$iw.<init>(command--1:44)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw$$iw.<init>(command--1:46)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw$$iw.<init>(command--1:48)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw$$iw.<init>(command--1:50)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$$iw.<init>(command--1:52)
at line7ccb0b1a0bd6475aac11185531c9050025.$read.<init>(command--1:54)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$.<init>(command--1:58)
at line7ccb0b1a0bd6475aac11185531c9050025.$read$.<clinit>(command--1)
at line7ccb0b1a0bd6475aac11185531c9050025.$eval$.$print$lzycompute(<notebook>:7)
at line7ccb0b1a0bd6475aac11185531c9050025.$eval$.$print(<notebook>:6)
at line7ccb0b1a0bd6475aac11185531c9050025.$eval.$print(<notebook>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:793)
at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1054)
at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq.apply(IMain.scala:645)
at scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq.apply(IMain.scala:644)
at scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:644)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:576)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:572)
at com.databricks.backend.daemon.driver.DriverILoop.execute(DriverILoop.scala:215)
at com.databricks.backend.daemon.driver.ScalaDriverLocal$$anonfun$repl.apply$mcV$sp(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.ScalaDriverLocal$$anonfun$repl.apply(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.ScalaDriverLocal$$anonfun$repl.apply(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExitInternal$.trapExit(DriverLocal.scala:679)
at com.databricks.backend.daemon.driver.DriverLocal$TrapExit$.apply(DriverLocal.scala:632)
at com.databricks.backend.daemon.driver.ScalaDriverLocal.repl(ScalaDriverLocal.scala:197)
at com.databricks.backend.daemon.driver.DriverLocal$$anonfun$execute.apply(DriverLocal.scala:368)
at com.databricks.backend.daemon.driver.DriverLocal$$anonfun$execute.apply(DriverLocal.scala:345)
at com.databricks.logging.UsageLogging$$anonfun$withAttributionContext.apply(UsageLogging.scala:238)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at com.databricks.logging.UsageLogging$class.withAttributionContext(UsageLogging.scala:233)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionContext(DriverLocal.scala:48)
at com.databricks.logging.UsageLogging$class.withAttributionTags(UsageLogging.scala:271)
at com.databricks.backend.daemon.driver.DriverLocal.withAttributionTags(DriverLocal.scala:48)
at com.databricks.backend.daemon.driver.DriverLocal.execute(DriverLocal.scala:345)
at com.databricks.backend.daemon.driver.DriverWrapper$$anonfun$tryExecutingCommand.apply(DriverWrapper.scala:644)
at com.databricks.backend.daemon.driver.DriverWrapper$$anonfun$tryExecutingCommand.apply(DriverWrapper.scala:644)
at scala.util.Try$.apply(Try.scala:192)
at com.databricks.backend.daemon.driver.DriverWrapper.tryExecutingCommand(DriverWrapper.scala:639)
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:485)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:597)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:390)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:337)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:219)
at java.lang.Thread.run(Thread.java:748)
我无法 ping 指出问题的原因，但我找到了一个不使用 OkHttp 的解决方法，用它替换发出请求的代码。
HttpResponse<String> response = Unirest.get("<some_url>")
.header("cache-control", "no-cache")
.asString();
这是因为安全密码套件不兼容：
Databricks 正在选择高效的密码套件
OkHttp - 现代安全密码套件
要使其正常工作，您可以使用 https://www.ssllabs.com/ssltest/
分析 APIAPI 支持密码套件的部分，您必须像这样明确设置它们（对于我的情况 CBC_SHA256 是交易）：
ConnectionSpec spec = new ConnectionSpec.Builder(ConnectionSpec.MODERN_TLS)
.tlsVersions(TlsVersion.TLS_1_2)
.cipherSuites(
CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
)
.build();
OkHttpClient client = new OkHttpClient.Builder()
.connectionSpecs(Collections.singletonList(spec))
.build();
https://square.github.io/okhttp/https/
我在这里找到了很多信息和描述： https://github.com/square/okhttp/issues/6138