Beam/Dataflow:未定义 Window 的拓扑的 maxTimestamp
Beam/Dataflow: maxTimestamp for topology with no Window defined
对于全局 window,maxTimestamp 的预期行为是什么?
我有一个源无限的拓扑,它没有指定 windowing 策略。当我访问 BoundedWindow 的 maxTimestamp 字段时,我得到一个未来的时间戳。这是预期的行为吗?
是的,这是有意为之的行为。全局 window 的末尾必须略小于 Beam 中可能的最大时间戳值,在实践中通常被称为 +infinity。
来自GlobalWindow.java的源代码:
// Triggers use maxTimestamp to set timers' timestamp. Timers fires when
// the watermark passes their timestamps. So, the maxTimestamp needs to be
// smaller than the TIMESTAMP_MAX_VALUE.
// One standard day is subtracted from TIMESTAMP_MAX_VALUE to make sure
// the maxTimestamp is smaller than TIMESTAMP_MAX_VALUE even after rounding up
// to seconds or minutes.
private static final Instant END_OF_GLOBAL_WINDOW = extractMaxTimestampFromProto();
对于全局 window,maxTimestamp 的预期行为是什么? 我有一个源无限的拓扑,它没有指定 windowing 策略。当我访问 BoundedWindow 的 maxTimestamp 字段时,我得到一个未来的时间戳。这是预期的行为吗?
是的,这是有意为之的行为。全局 window 的末尾必须略小于 Beam 中可能的最大时间戳值,在实践中通常被称为 +infinity。
来自GlobalWindow.java的源代码:
// Triggers use maxTimestamp to set timers' timestamp. Timers fires when
// the watermark passes their timestamps. So, the maxTimestamp needs to be
// smaller than the TIMESTAMP_MAX_VALUE.
// One standard day is subtracted from TIMESTAMP_MAX_VALUE to make sure
// the maxTimestamp is smaller than TIMESTAMP_MAX_VALUE even after rounding up
// to seconds or minutes.
private static final Instant END_OF_GLOBAL_WINDOW = extractMaxTimestampFromProto();