[mhk197]

Summary

Keep an integer array's logical dtype while storing a narrower integer child. For example, a logical i64 array can hold an i8 buffer and compare at that width without widening first.

Stacked on #9948, the 8-bit primitive comparison optimization. This draft targets mk/primitive-compare-bitmap; it adds no further changes to the primitive comparison kernels or their existing benchmarks.

Changes

• Add NarrowArray::try_new for an existing narrow child and NarrowArray::encode for selecting storage from non-null bounds. Preserve signedness and nullability; flatten nested wrappers.
• Compare fitting constants and matching-width NarrowArrays through their children. Handle out-of-range constants without scanning values.
• Push slice, filter, take, and mask into the child. Canonical execution uses a widening cast; scalar access retains the logical dtype.
• Serialize the child PType and reconstruct the stored width on deserialization. Cover primitive, dictionary, and bitpacked children, nulls, boundaries, malformed metadata, and arithmetic fallback.
• Add NarrowArray-specific benchmarks and a reproducible results report. The performance fixtures use ordinary uncompressed primitive buffers.

Design · Full benchmark report · All results, quartiles, and per-pass medians

Benchmarks

Apple M5 Max, Rust 1.98.0, 1,048,576 non-null rows. Three interleaved passes of 140 samples per case; pooled medians. All Narrow representations retain logical i64. Compute starts with prebuilt inputs and excludes encoding.

Representation	Value buffers	Savings	< constant	< column
Primitive i64	8 MiB	baseline	171.50 µs	296.02 µs
Narrow over i32	4 MiB	50%	196.48 µs	243.25 µs
Narrow over i16	2 MiB	75%	208.60 µs	256.94 µs
Narrow over i8	1 MiB	87.5%	38.98 µs	49.17 µs

The i8 comparison gains are 4.4× against a constant and 6.0× against a column. Across individual passes these ratios range from 4.33–4.41× and 6.03–7.02×. Absolute times vary across passes; raw variation is included in the report. Buffer sizes exclude array objects, allocator metadata, and RSS.

Operation	Primitive i64	Narrow/i32	Narrow/i16	Narrow/i8
Filter → i64	304.85 µs	227.27 µs	156.02 µs	342.69 µs
Take → i64	215.77 µs	331.96 µs	308.33 µs	321.40 µs
Compare → filter → i64	458.10 µs	402.62 µs	342.27 µs	382.17 µs
Filter → compare	379.71 µs	211.58 µs	156.58 µs	264.73 µs
Take → compare	167.62 µs	165.17 µs	153.58 µs	120.75 µs

Filter selects approximately 50%; take gathers N/8 pseudorandom indices. Every operation executes to its final primitive buffer or boolean bitmap.

Encoding i64 into i8, including a fresh bounds scan, costs 334 µs. The constant-comparison savings amortize this in roughly 3–4 comparisons in this fixture. Widening a full i8 child back to i64 costs 189 µs. One comparison alone does not justify converting an existing wide buffer.

Draft limitations

• i32/i16 constant comparisons are slower than i64; smaller storage does not universally improve compute.
• Take → i64 regresses: the canonical widening cast is pushed into the dictionary values before gathering. Avoiding that early widening remains follow-up work.
• No automatic compressor selection, released file-edition change, decimal/DecimalByteParts integration, or specialized arithmetic/aggregate kernels. Mixed child widths use canonical fallback.
• Local array microbenchmarks only; x86 and end-to-end query benefits have not been measured. The report includes 8K/16M cases, equality, and native-width controls.

Validation

• cargo nextest run -p vortex-array -p vortex-fastlanes: 4,121 passed, one existing skip.
• cargo test --doc -p vortex-array: 76 passed, 21 ignored.
• cargo clippy -p vortex-array -p vortex-fastlanes --all-targets --all-features -- -D warnings: passed.
• cargo clippy --all-targets --all-features: passed across the workspace.
• cargo +nightly-2026-09-10 fmt --all --check and git diff --check: passed.
• Three complete passes of both benchmark matrices: result-equivalence and size assertions passed.

The workspace build warned that clang-format was unavailable for a generated DuckDB header. No C++ or CUDA sources were changed.

[codspeed]

Merging this PR will regress 8 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 52 improved benchmarks
❌ 8 regressed benchmarks
✅ 159 untouched benchmarks
⏩ 2282 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	dbp_split_kernel_all_valid_neon[(I256, 8192)]	8.2 µs	36.2 µs	-77.39%
❌	dbp_split_kernel_all_valid_avx512[(I256, 8192)]	13.6 µs	50.4 µs	-73.09%
❌	dbp_split_kernel_all_valid_avx2[(I256, 8192)]	15.5 µs	53.5 µs	-71.06%
❌	words_gather_dispatch_avx512[65536]	1 µs	1.4 µs	-25.67%
❌	filtered_sink_i64_avx2[OneNullInEight]	21.8 µs	26.2 µs	-16.83%
❌	filtered_sink_i64_avx512[OneNullInEight]	22.5 µs	26.6 µs	-15.42%
❌	filtered_owned_i64_avx2[OneNullInEight]	21.8 µs	25.5 µs	-14.51%
❌	dict_canonicalize_gt_u8_neon[1000000]	488.3 µs	560.2 µs	-12.85%
⚡	mul_u64_nonnull_neon	40.4 µs	15.3 µs	×2.6
⚡	mul_i16_nonnull_avx2	29 µs	11.6 µs	×2.5
⚡	mul_i32_nonnull_avx2	33 µs	13.4 µs	×2.5
⚡	mul_i8_nonnull_avx2	30.3 µs	12.3 µs	×2.5
⚡	mul_u8_nonnull_avx512	22.4 µs	9.5 µs	×2.4
⚡	mul_i32_nullable_avx2	34.6 µs	15 µs	×2.3
⚡	mul_i8_nonnull_neon	25.7 µs	11.2 µs	×2.3
⚡	mul_u8_nonnull_avx2	20.9 µs	9.2 µs	×2.3
⚡	mul_i32_nonnull_neon	23.4 µs	10.4 µs	×2.3
⚡	mul_i16_nonnull_neon	24 µs	10.7 µs	×2.3
⚡	mul_u64_nonnull_avx2	39.3 µs	17.5 µs	×2.3
⚡	mul_i64_nonnull_neon	38.6 µs	17.2 µs	×2.2
...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/narrow-array-pr (d7d9a98) with mk/primitive-compare-bitmap (c215b0e)}

Footnotes

1. 2282 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[github-actions]

This PR has been marked as stale because it has been open for 14 days with no activity. Please comment or remove the stale label if you wish to keep it active, otherwise it will be closed in 7 days

[github-actions]

This PR was closed because it has been inactive for 7 days since being marked as stale.