断言失败：运行时反射宇宙中的不安全符号 TimeStamp（<none> 的子项）

Question

我们使用scala编程语言，使用spark作为数据处理。我们有 3 个不同的项目

项目 A // 我们在其中定义了此注释

package Common {
      case class TimeStamp(format: DateFormat) extends StaticAnnotation
    }

项目 B // 我们有这种情况 class 使用上面的注释

case class CommonSchema(
    @Common.TimeStamp(DateFormat.DateTime)
    __datetime: Timestamp
)

C 项目 // 我们使用上述案例 class 作为类型

val events = events.map(c => CommonSchema(
  c.__datetime,
))
events.as[CommonSchema]

我们在项目 C 中遇到以下异常

User class threw exception: java.lang.RuntimeException: error reading Scala signature of commonschema: assertion failed: unsafe symbol TimeStamp (child of <none>) in runtime reflection universe
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:46)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete.apply$mcV$sp(SymbolLoaders.scala:28)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete.apply(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete.apply(SymbolLoaders.scala:25)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.load(SymbolLoaders.scala:33)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams.apply(SynchronizedSymbols.scala:140)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams.apply(SynchronizedSymbols.scala:133)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:168)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.typeParams(SynchronizedSymbols.scala:132)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon.typeParams(SynchronizedSymbols.scala:168)
at scala.reflect.internal.Types$NoArgsTypeRef.typeParams(Types.scala:1931)
at scala.reflect.internal.Types$NoArgsTypeRef.isHigherKinded(Types.scala:1930)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType2(TypeComparers.scala:377)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType1(TypeComparers.scala:320)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType(TypeComparers.scala:278)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:16)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:784)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:260)
at org.apache.spark.sql.catalyst.ScalaReflection$.isSubtype(ScalaReflection.scala:83)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType.apply$mcZ$sp(ScalaReflection.scala:677)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType.apply(ScalaReflection.scala:676)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType.apply(ScalaReflection.scala:676)
at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:56)
at org.apache.spark.sql.catalyst.ScalaReflection$class.cleanUpReflectionObjects(ScalaReflection.scala:926)
at org.apache.spark.sql.catalyst.ScalaReflection$.cleanUpReflectionObjects(ScalaReflection.scala:49)
at org.apache.spark.sql.catalyst.ScalaReflection$.optionOfProductType(ScalaReflection.scala:675)
at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:51)
at org.apache.spark.sql.Encoders$.product(Encoders.scala:275)

类似的问题在 and 中得到了回答，但是我们已经为项目C构建了uber jar，所以所有的classes都应该包含在包中。在本地，命中此代码的单元测试工作正常，但为什么我们会收到运行时错误？

Answer 1

发生这种情况是因为我们使用了 maven-shade-plugin 插件并最小化了 jar。我们更改了配置，这样类就不会从 jar 中删除。它已经解决了问题

断言失败：运行时反射宇宙中的不安全符号 TimeStamp（<none> 的子项）

assertion failed: unsafe symbol TimeStamp (child of <none>) in runtime reflection universe

java

annotations

scala

jar

apache-spark