Scala lambda 仅在 AWS 中失败

Question

我正在编写我的第一个 scala lambda，并且在本地一切都连接并且工作正常。但是，当我尝试在 AWS 中测试我的 lambda 时，出现以下错误。

{
  "errorMessage": "Error loading class FooBar.Main: scala/collection/Seq",
  "errorType": "java.lang.NoClassDefFoundError"
}

根据我的谷歌搜索，这似乎是因为我需要将 scala 库添加到我的依赖项中，我确实这样做了。

name := "FooBar"
version := "0.1"
scalaVersion := "2.12.12"
javacOptions ++= Seq("-source", "1.8", "-target", "1.8", "-Xlint")
lazy val root = (project in file(".")).
settings(
  name := "FooBar",
  version := "1.0",
  scalaVersion := "2.12.12",
  retrieveManaged := true
)
libraryDependencies += "software.amazon.awssdk" % "ec2" % "2.5.60"
libraryDependencies += "com.amazonaws" % "aws-lambda-java-core" % "1.2.0"
libraryDependencies += "com.amazonaws" % "aws-lambda-java-events" % "2.1.0"
libraryDependencies += "com.amazonaws" % "aws-java-sdk-dynamodb" % "1.11.313"
libraryDependencies += "org.scalikejdbc" %% "scalikejdbc" % "3.4.0"
libraryDependencies += "org.apache.phoenix" % "phoenix-core" % "4.14.3-HBase-1.4"
libraryDependencies += "org.apache.hbase" % "hbase-common" % "1.4.10"
libraryDependencies += "org.apache.hbase" % "hbase-server" % "1.4.10"
libraryDependencies += "io.spray" %%  "spray-json" % "1.3.2"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.5" % "test"
libraryDependencies += "org.scala-lang" % "scala-library" % "2.12.12"

assemblyShadeRules in assembly := Seq(
  ShadeRule.keep("x.**").inAll,
  ShadeRule.keep("FooBar.**").inProject
)

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs@_*) => MergeStrategy.discard
  case x => MergeStrategy.first
}

同样，本地一切正常，只是永远无法在 AWS 上执行。有人有想法吗？

Answer 1

sbt-assembly 插件着色规则 ShadeRule.keep 文档说明

The ShadeRule.keep rule marks all matched classes as "roots". If any keep rules are defined all classes which are not reachable from the roots via dependency analysis are discarded when writing the output jar.

https://github.com/sbt/sbt-assembly#shading

所以在这种情况下，class 路径 x.* 和 FooBar.* 的所有 classes 在创建 fat jar 时都会保留。其余所有其他 classes，包括 scala-library 中的 classes 都被丢弃。

要解决此问题，请删除所有 ShadeRule.keep 规则，然后尝试 ShadeRule.zap 有选择地丢弃 class 不需要的规则。

例如，以下阴影规则从远处的 jar 中删除所有 HDFS classes：

  assemblyShadeRules in assembly := Seq(
    ShadeRule.zap("org.apache.hadoop.hdfs.**").inAll
  )

PS：在解压 fat jar 之后，AWS Lambda 的代码大小硬性限制为 256MB 。

Scala lambda 仅在 AWS 中失败

Scala lambda only failing in AWS

scala

amazon-web-services

sbt