perf(array): extend small-table take SIMD

[joseph-isaacs]

Summary

Small dictionary take falls back to general lookup once byte tables exceed 16 entries, and wider values miss the register-table optimization. Extend the existing AVX2 and NEON kernels using u8 codes, selecting only the value widths that showed wins on each measured architecture.

Value width	AVX2 maximum entries	AVX-512 VBMI maximum entries	NEON maximum entries
1 byte	64	256	256
2 bytes	32	128	128
4 bytes	Existing fallback	Existing fallback	64
8 bytes	Existing fallback	Existing fallback	32

Arbitrary cardinalities within each limit are supported. NEON requires at least 64 output rows, or 1,024 rows for eight-byte values. Wider codes and unsupported widths/cardinalities retain existing fallbacks. Physical widths cover signed/unsigned integers, floats, and corresponding decimal storage; variable-length strings and bit-packed booleans are outside this path.

Implementation

• AVX2 uses up to four 16-byte tables with VPSHUFB and blends. Runtime-gated AVX-512F/BW/VBMI uses VPERMB/VPERMT2B. Two-byte lookup supports the limits above.
• NEON byte lookup uses one- through four-register TBL, combining 64-byte banks for larger tables. Independent maximum-code accumulators shorten the bounds-check dependency chain.
• NEON wider lookup transposes dictionary values into separate byte tables, looks up 16 codes, and interleaves output with ST2/ST4.
• Retain perf(array): extend small-table take SIMD #10317's shared try_take dispatch and allocator behavior. Enable four-/eight-byte small-table dispatch only on little-endian AArch64; the existing x86 implementations and tests are unchanged by consolidation.
• Preserve checked indices, scalar tails, and the small-cardinality AVX2 table-construction fix.
• Keep the direct-kernel/fallback, allocation, stable fearless_simd, and opt-in nightly std::simd benchmarks. Add the full dictionary benchmark across all primitive types and the standalone NEON/generic comparison from [codex] perf: extend small dictionary take across NEON value widths #10315.
• No public API changes. The standalone comparison's dependency is isolated from the production workspace.

x86 performance

Intel Xeon 6975P-C, 1,048,576 indices:

Path	SIMD	Scalar fallback	Improvement
u8, 128–256 values	30–32 µs	~271 µs	8.5–9x
u16, 2–128 values	92–95 µs	273–276 µs	~2.9x

The final u8 kernel measured ~29.95 µs at 32 values versus ~30.77 µs for the stable generic fearless_simd comparison in the same run. The raw allocation controls measured ~19.81 µs for reusable memcpy and ~21.48 µs for a fresh Vortex buffer plus copy.

Profiling showed the apparent ~30% Vortex gap was primarily checked-kernel instruction cost plus an allocation mismatch in the benchmark, not framework dispatch. Replacing per-vector subtract/OR bounds checks with max reduction removed those hotspots; framework/execution accounted for roughly 6% of the profiled sample.

On x86, u32/u64 register-table variants were benchmarked but intentionally not selected in production because they did not beat the existing memory-bound fallback.

These x86 measurements were reported on the pre-consolidation branch ending at b3d8fab. The consolidation does not alter those x86 kernels.

NEON — Apple M3 Max

End-to-end dictionary execution, including allocation, one million uniform non-null u8 codes. Times are microseconds. Rust 1.98.0, native aarch64 macOS, no CPU pinning.

Baseline is 3c0a33e with the expanded benchmark source. It already has the earlier PR's 32-entry byte fast path; larger byte tables and wider values use the scalar default. Candidate is cac5eca from #10315, before consolidation. These are historical measurements of the imported kernels; the combined PR head has not been benchmarked.

Value type	Dictionary values	Baseline	Candidate	Speedup
u8	2	36.27	20.06	1.81×
u8	4	36.30	20.02	1.81×
u8	8	36.23	19.90	1.82×
u8	16	36.11	19.99	1.81×
u8	32	36.32	21.70	1.67×
u8	48	279.40	20.74	13.47×
u8	64	281.00	22.60	12.43×
u8	96	282.30	45.96	6.14×
u8	128	281.90	46.15	6.11×
u8	192	280.10	68.35	4.10×
u8	256	281.60	95.80	2.94×
u16	64	280.10	47.18	5.94×
u16	128	281.90	90.60	3.11×
u32	32	280.40	68.76	4.08×
u32	64	280.70	106.80	2.63×
u64	16	557.30	151.50	3.68×
u64	32	560.40	166.50	3.37×
f16	128	281.60	89.10	3.16×
f32	64	281.30	105.80	2.66×
f64	32	559.20	156.40	3.58×

Entries are medians of three run medians, with 31 samples of four iterations. Run order was baseline/candidate, candidate/baseline, baseline/candidate. All 1,408 cases ran in each executable: 11 primitive types × 16 cardinalities × four lengths × two distributions.

The host showed scheduling variation, including in unchanged fallback cases. For example, u64/32 candidate run medians were 166.5, 156.4, and 325.3 µs. Small differences at 64 rows should not be treated as stable wins. These measurements establish gains on this M3 Max; they are not measurements of other NEON implementations.

Native NEON versus generic fearless_simd

Kernel-only comparison on the same M3 Max with Rust 1.98.0, one million uniform codes. Times are microseconds. Output is preallocated; table preparation, dispatch, bounds checks, lookup, and tails are timed. The generator differs from the end-to-end benchmark, so compare columns within this table.

Value width	Dictionary values	Checked scalar	Native NEON	Generic fearless_simd
1	16	301.65	18.71	34.88
1	32	298.15	18.71	18.69
1	48	298.65	18.75	20.23
1	64	300.02	20.44	20.12
1	128	295.08	43.54	43.29
1	256	296.29	89.29	88.73
2	64	277.21	43.54	43.38
2	128	277.58	83.54	85.12
4	32	278.08	63.37	62.63
4	64	277.29	99.96	99.71
8	16	278.46	144.83	164.42
8	32	278.00	149.92	148.33
16	2	309.54	571.25	327.50
32	2	388.50	1037.88	910.04

The 16-/32-byte native entries are experimental kernels, not enabled production paths. Most enabled cases are effectively tied between native and generic SIMD. The byte/16 native loop uses four independent accumulators while the generic version uses a single 16-lane vector; this comparison applies to the supplied kernels, not to the libraries in general.

Each entry is the median of three run medians. Each run rotates the three variants over 31 samples, with clamp(2_000_000 / output_bytes, 1, 256) iterations per sample. All 504 datasets and setup correctness checks ran successfully.

Reproduction

For the end-to-end NEON comparison, keep take_small_dict.rs identical in the baseline and candidate and save each emitted executable:

cargo bench -p vortex-array --bench take_small_dict --no-run
<saved-benchmark-binary> --bench --sample-count 31 --sample-size 4

For the standalone generic comparison on AArch64:

cargo run --release --manifest-path benchmarks/take-small-dict/Cargo.toml > results.jsonl

Use cargo bench -p vortex-array --bench dict_take_small for the existing direct-kernel, fallback, allocation, and generic SIMD comparisons.

NEON comparison methodology

Validation

The combined revision 04f4c4e was inspected for dispatch compatibility and preservation of the x86 source and existing tests. Builds, tests, linting, formatting, and benchmarks were not rerun during consolidation; the results below belong to the earlier revisions.

Earlier x86 revision b3d8fab

• cargo nextest run -p vortex-array — 4,004 passed, 1 skipped
• Focused fixed-width take tests — 141 passed
• cargo clippy -p vortex-array --all-targets --all-features -- -D warnings
• cargo check -p vortex-array --lib --target aarch64-unknown-linux-gnu
• RUSTFLAGS='--cfg vortex_nightly_portable_simd -C target-cpu=native' cargo +nightly check -p vortex-array --bench dict_take_small
• cargo +nightly fmt --all -- --check
• git diff --check

Imported NEON revision cac5eca

• Native baseline/candidate benchmark builds succeeded; all 1,408 scalar-reference checks passed in all six end-to-end runs.
• The standalone benchmark built with Rust 1.98.0. All three runs passed comparisons across record widths, non-power-of-two counts, unaligned inputs, vector tails, and invalid-code rejection.
• Unit tests, formatting, and Clippy were not run for that revision.

[codspeed]

Merging this PR will improve performance by 18.08%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 3 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
✅ 2099 untouched benchmarks
🆕 85 new benchmarks
⏩ 518 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	decode_primitives[u8, (2000, 32)]	43.3 µs	34.3 µs	+26.22%
⚡	Simulation	decode_primitives[u8, (1000, 32)]	35.9 µs	32.5 µs	+10.46%
🆕	Simulation	allocate_and_copy	N/A	2 ms	N/A
🆕	Simulation	u8[16]	N/A	2 ms	N/A
🆕	Simulation	u8[2]	N/A	2 ms	N/A
🆕	Simulation	u8[32]	N/A	2 ms	N/A
🆕	Simulation	u8[4]	N/A	2 ms	N/A
🆕	Simulation	u8[8]	N/A	2 ms	N/A
🆕	Simulation	gather_u16[128]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u16[16]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u16[2]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u16[32]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u16[4]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u16[64]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u16[8]	N/A	8.8 ms	N/A
🆕	Simulation	gather_u32[16]	N/A	7 ms	N/A
🆕	Simulation	gather_u32[2]	N/A	7 ms	N/A
🆕	Simulation	gather_u32[32]	N/A	7 ms	N/A
🆕	Simulation	gather_u32[4]	N/A	7 ms	N/A
🆕	Simulation	gather_u32[64]	N/A	7 ms	N/A
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing perf/small-table-take-vbmi (b3d8fab) with develop (c6e51ba)}

Footnotes

1. 518 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[codspeed]

Too many benchmarks in a single upload

The performance report could not be generated because there were too many benchmarks in a single upload to CodSpeed. We recommend sharding your benchmarks into smaller uploads, with max 1000 benchmarks per upload. See the documentation for more information.