Repository navigation
feat(array): push arithmetic and selection through Narrow - #10332
connortsui20 wants to merge 4 commits into
Conversation
Merging this PR will improve performance by 26.55%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | fsst_decompress_string |
526 µs | 187.9 µs | ×2.8 |
| ⚡ | Simulation | chunked_canonicalize_into[(20, 100, 16, 4)] |
739.9 µs | 340.9 µs | ×2.2 |
| ⚡ | Simulation | chunked_dict_fsst_into_canonical[(250, 10, 2)] |
681.9 µs | 344.7 µs | +97.85% |
| ⚡ | Simulation | chunked_canonicalize_into[(20, 50, 8, 4)] |
274.4 µs | 213.4 µs | +28.58% |
| ⚡ | Simulation | chunked_canonicalize_into[(20, 50, 8, 16)] |
279.5 µs | 218.6 µs | +27.85% |
| ⚡ | Simulation | chunked_canonicalize_into[(20, 50, 8, 64)] |
306.3 µs | 243.2 µs | +25.98% |
| ⚡ | Simulation | chunked_into_canonical[(20, 50, 8, 4)] |
314.3 µs | 253 µs | +24.22% |
| ⚡ | Simulation | chunked_into_canonical[(20, 50, 8, 16)] |
323.3 µs | 262 µs | +23.4% |
| ⚡ | Simulation | chunked_into_canonical[(20, 50, 8, 64)] |
348.2 µs | 286.4 µs | +21.54% |
| ⚡ | Simulation | decompress_fsst[(200, 4, 4)] |
60.8 µs | 51 µs | +19.04% |
| ⚡ | Simulation | decompress_fsst[(200, 4, 8)] |
61.6 µs | 51.9 µs | +18.6% |
| ⚡ | Simulation | chunked_canonicalize_into[(10, 150, 4, 4)] |
235.3 µs | 199.5 µs | +17.94% |
| ⚡ | Simulation | decompress_fsst[(200, 16, 4)] |
66.4 µs | 56.7 µs | +17.13% |
| ⚡ | Simulation | decompress_fsst[(200, 16, 8)] |
69.4 µs | 59.8 µs | +15.98% |
| ⚡ | Simulation | chunked_canonicalize_into[(10, 150, 16, 4)] |
266.6 µs | 231 µs | +15.42% |
| ⚡ | Simulation | chunked_canonicalize_into[(20, 100, 16, 16)] |
461.3 µs | 400 µs | +15.33% |
| ⚡ | Simulation | chunked_into_canonical[(10, 150, 4, 4)] |
264.9 µs | 231 µs | +14.7% |
| ⚡ | Simulation | chunked_into_canonical[(20, 100, 16, 4)] |
452.7 µs | 398.1 µs | +13.72% |
| ⚡ | Simulation | chunked_into_canonical[(10, 150, 16, 4)] |
298.7 µs | 263.3 µs | +13.44% |
| ⚡ | Simulation | chunked_into_canonical[(20, 100, 16, 16)] |
508.7 µs | 448.6 µs | +13.41% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/narrow-pushdown (557a613) with develop (3c87da0)2
Footnotes
-
518 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
No successful run was found on
ct/narrow-encoding(44780e8) during the generation of this report, sodevelop(3c87da0) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
6996aa4 to
2be0bae
Compare
2be0bae to
7bb3a4a
Compare
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
7bb3a4a to
557a613
Compare
Summary
Progress towards #10020. Stacked on #10223.
A result can overflow the stored integer width while still fitting its logical type. Non-null bounds now select a sufficient width before checked arithmetic, so
i8(127) + 1can remain a narrowi64result withi16storage. If separate column extrema exceed the logical range, the existing checked kernel tests the actual row pairs at logical width.Changes
Casts, fill, zip, between, concatenation, and interleave can use the stored children. Gather indices can also use their stored width directly. The rules retain logical dtypes, null behavior, and checked overflow. Buffers remain immutable, and the change does not add primitive integer types.