Refine BitPackingScheme to per-block bit widths when fastlanes.bitpacked.v2 is allowed

[mhk197]

Tracking Issue: #10167

Stacked on #10244; this PR targets mk/bitpacked-v2-wire.

Summary

BitPackingScheme packs each 1024-element block at its own bit width when the compressor's allowed serialized IDs include fastlanes.bitpacked.v2, following FoRScheme (#10136). Default writes are unchanged: the core edition allows only fastlanes.bitpacked, so refine keeps v1.

BitPackingScheme

• BitPackingScheme gains v1 and v2 modes with v1() and v2() constructors, and Default is v1. The session registers BITPACKING_V1.
• refine returns v2 when fastlanes.bitpacked.v2 is allowed, and v1 otherwise.
• produced_encodings: v1 declares fastlanes.bitpacked; v2 declares fastlanes.bitpacked.v2.
• Compress: v1 is unchanged. v2 always produces a blocked array, like FoRScheme v2 always encodes per-chunk references. It chooses every block's width with bitpack_to_best_bit_widths (Blocked BitPacked: encode blocked bitwidths #10203), even when every block chooses the same width, and compresses the block offsets as child 0 with compress_child. Patches are handled as in v1: compressed in place, or moved into Patched under experimental patches.
• Estimate: unchanged. Both modes sample, and the sampler stitches 64-value runs from across the array, so sampled blocks don't resemble real ones and the sample tends toward one width. That is conservative for v2.

FoRScheme still bit-packs its encoded child with BITPACKING_V1, so FoR arrays keep a global width for now.

CUDA

The CUDA preset excludes fastlanes.bitpacked.v2, as it does fastlanes.for.v2. CUDA decoding returns an error for per-block bit widths.

Testing

• refine picks v2 exactly when the v2 ID is allowed, from either mode.
• tests/bitpacking_config.rs: with only BitPacking and every ID allowed, both blocks needing 1 to 8 bits and uniform 7-bit blocks serialize as v2, and nullable values round trip. The core edition never writes v2, and the CUDA preset writes v1.

Follow-ups

• Bit-pack FoR's encoded child per block. FoRScheme would bit-pack with v2 when fastlanes.bitpacked.v2 is allowed, so per-chunk references and per-block widths combine.
• Estimate per-block widths, for the same reason as FoR's estimate follow-up in Refine FoRScheme to per-chunk references when fastlanes.for.v2 is allowed #10136.

[codspeed]

Merging this PR will degrade performance by 24.89%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 2187 untouched benchmarks
⏩ 409 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Simulation	decode_primitives[i64, (1000, 32)]	24.1 µs	38 µs	-36.41%
❌	WallTime	bitpack_blocked_compress_avx2	6.8 µs	7.6 µs	-11.28%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/bitpacked-v2-scheme (f638117) with mk/bitpacked-v2-wire (fc9ce15)}

Footnotes

1. 409 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩