Continuous queries: event-time windows (watermarking) instead of processing-time boundaries

[xe-nvdk]

Summary

CQ window boundaries are based on processing/wall-clock time (endTime = time.Now()), not the event time in the data. Correctness of a given window therefore assumes the data for that window has been ingested by the moment the interval fires. For feeds that are bursty, delayed, or out-of-order (common in financial data), this couples aggregate correctness to ingestion timing rather than to the data's own timestamps.

Current behavior (code)

• internal/api/continuous_query.go:464-470 — endTime = time.Now().UTC(); the window is [last_processed_time, now) on wall-clock. The user's query filters on the data's time column via {start_time}/{end_time} placeholders, but the boundary advance is wall-clock, so events whose event-time falls in a window that has already been closed are missed.

Proposed direction (larger, architectural)

• Track a watermark on the data's event-time column (e.g. max observed event time minus an allowed-lateness bound) and advance CQ windows by event-time watermark rather than wall-clock.
• Lets windows close based on observed data progress, with bounded allowed-lateness, which is the standard streaming-aggregation model.

Relationship to other issues

• Heavier lift than Continuous queries: handle late / out-of-order data (lookback window + idempotent upsert) #522 (lookback) and Continuous queries can produce duplicate aggregate rows (non-idempotent output) #521 (idempotency). Those two together cover the near-term financial need (recent late data folded in + no duplicates). Event-time watermarking is the more complete long-term model.
• Likely Enterprise-tier / roadmap, not near-term.

Evidence

• internal/api/continuous_query.go:464-470