Repository navigation
bench(array): measure RowFn Boolean retry in CI - #9988
Conversation
Merging this PR will regress 1 benchmark
|
1d65020 to
f812b78
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Keep only two-column inputs and a constant left operand against a column. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015jJxqdFc4B1PsoctjdFZhd Signed-off-by: Claude <noreply@anthropic.com>
Keep only the multiversioned collector over partially valid input, for an accepted dense attempt and a null-only retry, each with two columns and with a constant left operand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015jJxqdFc4B1PsoctjdFZhd Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015jJxqdFc4B1PsoctjdFZhd Signed-off-by: Claude <noreply@anthropic.com>
830f4b3 to
81ace30
Compare
Stacked on #9988, which adds the CI benchmark without changing production code. ## Summary Routes deferred Boolean dense attempts through the packed collector, preserving the requested multiversioning flag. Keeps the source and failure accumulator in one borrowed state, and inlines the collector's scalar tail so LLVM can retain the alias information needed to vectorize the row loop. Decode errors remain terminal, and null-only failures still retry valid rows. The earlier x86 regressions came from scalarized collector loops. Optimized Linux IR and the native CI executables confirm that these loops now vectorize. ## Changes Native CI medians in microseconds per eight 16,384-row batches, with partial validity and the multiversioned collector: | Target | Columns before → after | Constant LHS before → after | | --- | ---: | ---: | | AVX2 | 111.0 → 44.53 | 217.1 → 34.58 | | AVX-512 | 42.81 → 45.07 | 206.9 → 37.25 | | NEON | 68.32 → 71.01 | 173.3 → 57.62 | Constant RHS also improves on every target. Small column-only movements remain uncertain after one paired run. All final cases are below 1 ms, with a maximum printed sample of 555.8 µs. Passed 175 RowFn tests, 14 Boolean packing tests, and scoped Clippy. The CodSpeed check flags no Boolean retry regressions, but remains red on three other, unattributed benchmark alerts. This remains a draft. [Four-stage comparison, compiler evidence, and raw results](https://github.com/vortex-data/vortex/blob/ct/row-fn-performance-evidence/research/row-fn-engine/performance/ci-bool-capture.md). Evidence stays on the separate branch. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
…a#9986) Stacked on vortex-data#9988, which adds the CI benchmark without changing production code. ## Summary Routes deferred Boolean dense attempts through the packed collector, preserving the requested multiversioning flag. Keeps the source and failure accumulator in one borrowed state, and inlines the collector's scalar tail so LLVM can retain the alias information needed to vectorize the row loop. Decode errors remain terminal, and null-only failures still retry valid rows. The earlier x86 regressions came from scalarized collector loops. Optimized Linux IR and the native CI executables confirm that these loops now vectorize. ## Changes Native CI medians in microseconds per eight 16,384-row batches, with partial validity and the multiversioned collector: | Target | Columns before → after | Constant LHS before → after | | --- | ---: | ---: | | AVX2 | 111.0 → 44.53 | 217.1 → 34.58 | | AVX-512 | 42.81 → 45.07 | 206.9 → 37.25 | | NEON | 68.32 → 71.01 | 173.3 → 57.62 | Constant RHS also improves on every target. Small column-only movements remain uncertain after one paired run. All final cases are below 1 ms, with a maximum printed sample of 555.8 µs. Passed 175 RowFn tests, 14 Boolean packing tests, and scoped Clippy. The CodSpeed check flags no Boolean retry regressions, but remains red on three other, unattributed benchmark alerts. This remains a draft. [Four-stage comparison, compiler evidence, and raw results](https://github.com/vortex-data/vortex/blob/ct/row-fn-performance-evidence/research/row-fn-engine/performance/ci-bool-capture.md). Evidence stays on the separate branch. Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Summary
Adds a deferred Boolean RowFn benchmark to establish the CI baseline for #9986. It uses the multiversioned collector over partially valid input and covers four cases: an accepted dense attempt and a null-only retry, each with two columns and with a constant left operand against a column.
Changes
Uses the existing native AVX2, AVX-512, and NEON jobs, so there are 12 benchmarks in total. Each timed iteration executes eight 16,384-row batches to clear the wall-time floor. Fixture and execution-context setup and result destruction stay outside timing. Backtraces follow the CI setting. This layer changes no production code.
An earlier revision ran 72 cases (both collector flags, constants in either position, all-valid input, and observable failures). All of them completed in native CI, with medians from 32.31 to 490.6 µs and a maximum printed sample of 712.1 µs.
Comparison with #9986 and raw results come from that earlier revision.