perf: linearize DISTINCT aggregate preflight deduplication

[XuPeng-SH]

What this changes

Make saved-argument DISTINCT preflight adaptive instead of forcing either quadratic scans or hashing every row:

• split DISTINCT and non-DISTINCT admission into separate functions, so ordinary aggregates do not clear or retain the DISTINCT hash table stack frame;
• keep up to 8 exact (target group, DISTINCT arguments) representatives and use exact equality only;
• initialize the fixed 512-slot open-addressing table only when the ninth unique tuple arrives, bounding high-cardinality admission to expected O(B) for the 256-row work unit;
• delay single-column payload access until a tuple is actually admitted, so duplicate-heavy fixed and varlen inputs are faster than the old scan;
• retain collision-safe canonical equality for NULL filtering, const vectors, offsets, multi-column boundaries, and floating-point signed zero;
• add a direct non-DISTINCT single-argument loop so splitting the paths does not regress ordinary saved arguments.

The implementation keeps regular and DISTINCT admission separate and pays the bounded hash-table cost only after the exact low-cardinality path overflows. A full aggregate-framework rewrite would be disproportionate for this bounded 256-row preflight unit.

arenaskl correctness fix

A reused arenaskl.Inserter could miss equality when the requested key was exactly its cached base-level successor. The PR now checks that successor before leaving the cached-splice path and adds Inserter.AddWithPlan so sorted planned publications can safely retain the splice cache.

Fixes #27687.
Fixes #27691.
Related to #27672; the mixed-aggregation query optimization remains in #27693.

Performance

Pinned to one CPU, -cpu=1, 256 rows, median of 3 runs (focused non-DISTINCT fixed result uses 5 runs at 2s):

input	cardinality	base	head	result
INT64 DISTINCT	1	7.813 us	6.987 us	10.6% faster
INT64 DISTINCT	2	10.451 us	9.959 us	4.7% faster
INT64 DISTINCT	8	25.741 us	25.561 us	neutral / 0.7% faster
INT64 DISTINCT	9	27.920 us	10.896 us	2.56x faster
INT64 DISTINCT	256	648.647 us	14.664 us	44.2x faster
1 KiB VARCHAR DISTINCT	1	12.282 us	10.770 us	12.3% faster
1 KiB VARCHAR DISTINCT	2	16.562 us	15.863 us	4.2% faster
1 KiB VARCHAR DISTINCT	8	44.797 us	43.617 us	2.6% faster
1 KiB VARCHAR DISTINCT	9	48.153 us	28.928 us	1.66x faster
1 KiB VARCHAR DISTINCT	256	999.855 us	46.526 us	21.5x faster
INT64 non-DISTINCT	n/a	7.632 us	7.542 us	1.2% faster
1 KiB VARCHAR non-DISTINCT	n/a	24.638 us	24.605 us	neutral

All focused cases remain at 0 B/op and 0 allocs/op. The existing 65,536-row BenchmarkCountDistinctSavedArguments remains about 48-49 ms/op.

Stack frames from the compiled test binary:

• base combined path: 12,832 bytes;
• head non-DISTINCT single path: 12,448 bytes;
• head low-cardinality DISTINCT path: 8,400 bytes;
• high-cardinality hash helper: 5,296 bytes in its separate frame.

Regression coverage

• true 256-unique-candidate hash-table test plus a complete second duplicate pass;
• threshold transition at 8/9 unique tuples;
• end-to-end adaptive multi-column DISTINCT test with offsets, NULL, unmatched groups, two groups, repeat preflight, publication, and result cardinality;
• non-DISTINCT multi-argument preflight remains publication-free;
• cached Inserter sorted-overlap regression checks duplicate rejection and final cardinality;
• benchmark matrix covers cardinality 1, 2, 8, 9, and 256 for fixed/varlen plus non-DISTINCT controls.

Exact package coverage comparison (pkg/common/arenaskl + pkg/sql/colexec/aggexec):

• base: 10,520 / 13,316 statements = 79.002704%;
• head: 10,669 / 13,483 statements = 79.129274%.

Validation

• full normal tests for both packages;
• full race tests for both packages;
• go vet for both packages;
• deterministic CGo compile/link/runtime environment;
• pinned-CPU benchmark matrix;
• git diff --check;
• rebased onto latest origin/main (e8332cb8cc).

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

[XuPeng-SH]

Deep review（base a560c6e3a8，head d67e2dc5e7）结论：建议先 request changes，暂不 approve。

1. P1：低基数 DISTINCT 有确定的性能回退

seenOrInsert 在确认重复前无条件计算 xxhash，重复值随后还会再次做完整 equality。独立 A/B（Apple M4，同一 benchmark）：

• 65,536 个 unique INT64：base 173.39 ms，head 48.81 ms，约 3.55x 加速；
• 单 group、全部重复 INT64：base 3.76 ms，head 4.35 ms，退化 15.6%；
• 单 group、全部重复、1 KiB VARCHAR：base 16.04 ms，head 21.06 ms，退化 31.4%。

后两种是正常可达的 COUNT(DISTINCT col) 场景，不能用高基数收益覆盖。相关代码见 capacity_preflight.go。

2. P2：非 DISTINCT 路径也支付 hash table 成本

distinctArgumentBatch 在 distinct=false 时仍被无条件放入 preflightBatchFillArgs 栈帧并清零。head 的编译栈帧约 17.8 KiB，base 约 12.7 KiB；非 DISTINCT saved-argument 基准也有约 0.5% 回退。见 capacity_preflight.go。

建议：

• 将 DISTINCT 与非 DISTINCT preflight 拆成独立路径；
• 低基数先用少量 unique representative 做精确去重，不要每行 hash；
• 超过小阈值后再初始化/进入 open-addressing hash 路径；
• benchmark 覆盖 cardinality 1、2、阈值附近、256，以及定长/varlen；
• 单测真正覆盖 256 个 unique candidate；当前 “full work unit” 测试最多只有 65 个 unique key，见 capacity_preflight_edge_test.go。

正确性方面目前未发现问题：hash collision 有完整比较兜底，NULL、const、offset、多列、signed-zero 语义保持一致；表容量有界且无共享状态。精确 head 的 aggexec 包测试和 git diff --check 已通过。

[XuPeng-SH]

Addressed the deep-review findings in 353ed46e82:

• DISTINCT/non-DISTINCT preflight paths are physically split; compiled non-DISTINCT single-path frame is 12,448 bytes versus base 12,832 bytes.
• Up to 8 exact representatives stay hash-free; the ninth unique tuple switches to the bounded hash table.
• Duplicate single-column rows no longer fetch/account payload before admission.
• Pinned-CPU A/B now shows cardinality 1/2/8 neutral or faster for both INT64 and 1 KiB VARCHAR; cardinality 9/256 retains the large improvement. Non-DISTINCT controls are neutral or faster, all at 0 allocs/op.
• The full-work-unit test now has 256 actual unique candidates; threshold, multi-column/NULL/offset/publication, and non-DISTINCT multi-argument regressions were added.
• arenaskl: cached Inserter can admit a duplicate successor key #27691 cached-Inserter successor correctness fix and regression are included here.
• Exact combined package coverage increases from 79.002704% to 79.129274%.

Normal tests, full race tests, vet, and diff-check pass locally after rebasing to current main. Full commands and benchmark numbers are in the updated PR body.

[LeftHandCold]

Reviewed the exact head 353ed46. Checked the adaptive DISTINCT preflight paths, NULL/const/offset/multi-column/signed-zero semantics, bounded hash-table transition, allocation accounting, and cached Inserter equality fix. Also ran the focused arenaskl tests and diff-check. I found no reachable correctness or regression blocker. The remaining Ubuntu/SCA/BVT checks were still running at review time.