Skip to content

feat(array): push arithmetic and selection through Narrow - #10332

Draft
connortsui20 wants to merge 4 commits into
ct/narrow-encodingfrom
ct/narrow-pushdown
Draft

connortsui20 wants to merge 4 commits into
ct/narrow-encodingfrom
ct/narrow-pushdown

Conversation

@connortsui20

Copy link
Copy Markdown
Member

Summary

Progress towards #10020. Stacked on #10223.

A result can overflow the stored integer width while still fitting its logical type. Non-null bounds now select a sufficient width before checked arithmetic, so i8(127) + 1 can remain a narrow i64 result with i16 storage. If separate column extrema exceed the logical range, the existing checked kernel tests the actual row pairs at logical width.

Changes

Casts, fill, zip, between, concatenation, and interleave can use the stored children. Gather indices can also use their stored width directly. The rules retain logical dtypes, null behavior, and checked overflow. Buffers remain immutable, and the change does not add primitive integer types.

@codspeed

codspeed Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Merging this PR will improve performance by 26.55%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 26 improved benchmarks
✅ 2075 untouched benchmarks
🆕 3 new benchmarks
⏩ 518 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
⚡ Simulation fsst_decompress_string 526 µs 187.9 µs ×2.8
⚡ Simulation chunked_canonicalize_into[(20, 100, 16, 4)] 739.9 µs 340.9 µs ×2.2
⚡ Simulation chunked_dict_fsst_into_canonical[(250, 10, 2)] 681.9 µs 344.7 µs +97.85%
⚡ Simulation chunked_canonicalize_into[(20, 50, 8, 4)] 274.4 µs 213.4 µs +28.58%
⚡ Simulation chunked_canonicalize_into[(20, 50, 8, 16)] 279.5 µs 218.6 µs +27.85%
⚡ Simulation chunked_canonicalize_into[(20, 50, 8, 64)] 306.3 µs 243.2 µs +25.98%
⚡ Simulation chunked_into_canonical[(20, 50, 8, 4)] 314.3 µs 253 µs +24.22%
⚡ Simulation chunked_into_canonical[(20, 50, 8, 16)] 323.3 µs 262 µs +23.4%
⚡ Simulation chunked_into_canonical[(20, 50, 8, 64)] 348.2 µs 286.4 µs +21.54%
⚡ Simulation decompress_fsst[(200, 4, 4)] 60.8 µs 51 µs +19.04%
⚡ Simulation decompress_fsst[(200, 4, 8)] 61.6 µs 51.9 µs +18.6%
⚡ Simulation chunked_canonicalize_into[(10, 150, 4, 4)] 235.3 µs 199.5 µs +17.94%
⚡ Simulation decompress_fsst[(200, 16, 4)] 66.4 µs 56.7 µs +17.13%
⚡ Simulation decompress_fsst[(200, 16, 8)] 69.4 µs 59.8 µs +15.98%
⚡ Simulation chunked_canonicalize_into[(10, 150, 16, 4)] 266.6 µs 231 µs +15.42%
⚡ Simulation chunked_canonicalize_into[(20, 100, 16, 16)] 461.3 µs 400 µs +15.33%
⚡ Simulation chunked_into_canonical[(10, 150, 4, 4)] 264.9 µs 231 µs +14.7%
⚡ Simulation chunked_into_canonical[(20, 100, 16, 4)] 452.7 µs 398.1 µs +13.72%
⚡ Simulation chunked_into_canonical[(10, 150, 16, 4)] 298.7 µs 263.3 µs +13.44%
⚡ Simulation chunked_into_canonical[(20, 100, 16, 16)] 508.7 µs 448.6 µs +13.41%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/narrow-pushdown (557a613) with develop (3c87da0)2

Open in CodSpeed

Footnotes

  1. 518 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on ct/narrow-encoding (44780e8) during the generation of this report, so develop (3c87da0) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@connortsui20
connortsui20 removed this pull request from stack #10334 October 6, 2026 13:44
@connortsui20
connortsui20 added this pull request to stack #10335 October 6, 2026 13:44
@connortsui20
connortsui20 removed this pull request from stack #10335 October 6, 2026 14:12
@connortsui20
connortsui20 added this pull request to stack #10336 October 6, 2026 14:17
@connortsui20
connortsui20 removed this pull request from stack #10336 October 6, 2026 14:18
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant