Window Spark 结构化流中的长度
Window length in Spark Structured Streaming
使用 Spark DStreams,我可以像这样指定滑动 Window 长度:
val windowedStream = stream.window(Seconds(20))
如何使用结构化流来做到这一点?
查看事件时间文档的 window 操作:https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#window-operations-on-event-time
示例:
val windowedCounts = words.groupBy(window($"timestamp", "10 minutes", "5 minutes"),$"word").count()
使用 Spark DStreams,我可以像这样指定滑动 Window 长度:
val windowedStream = stream.window(Seconds(20))
如何使用结构化流来做到这一点?
查看事件时间文档的 window 操作:https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#window-operations-on-event-time
示例:
val windowedCounts = words.groupBy(window($"timestamp", "10 minutes", "5 minutes"),$"word").count()