Dstream 火花流
Dstream Spark Streaming
你好我写了下面的代码
val receivedStream: DStream[PubsubSparkMessage] =
PubsubSource.createStream(ssc, pubsubConfig, pubsubSubscriber)
@volatile var receivedMessages: List[PubsubSparkMessage] = List() //scalastyle:ignore
receivedStream.foreachRDD { rdd =>
if (!rdd.isEmpty()) {
receivedMessages = List(rdd.collect)
receivedMessages
}
}
我收到一个错误,因为类型不匹配;
我的问题是如何将 List[Array[PubsubSparkMessage]] 修改为 List[PubsubSparkMessage]
类似于:
val list : List[Array[PubsubSparkMessage]] = List()
val result = list.flatMap(x => x)
你好我写了下面的代码
val receivedStream: DStream[PubsubSparkMessage] =
PubsubSource.createStream(ssc, pubsubConfig, pubsubSubscriber)
@volatile var receivedMessages: List[PubsubSparkMessage] = List() //scalastyle:ignore
receivedStream.foreachRDD { rdd =>
if (!rdd.isEmpty()) {
receivedMessages = List(rdd.collect)
receivedMessages
}
}
我收到一个错误,因为类型不匹配; 我的问题是如何将 List[Array[PubsubSparkMessage]] 修改为 List[PubsubSparkMessage]
类似于:
val list : List[Array[PubsubSparkMessage]] = List()
val result = list.flatMap(x => x)