Read RowFn UTF-8 inputs from native offsets

[connortsui20]

Summary

String RowFns currently construct 16-byte string views from VarBin inputs before callbacks can read them. Utf8OffsetColumn makes direct access to validated native u32 offsets available through the existing input-element API, while retaining payload ownership and strict null propagation.

Changes

Selection is explicit because RowFn::dispatch receives logical dtypes. Other nonempty layouts are rejected, and Utf8Column remains the general decoder. Batch constants use an opaque validated value so safe callers cannot pass arbitrary bytes to unchecked string access.

This remains a draft pending a consumer integration and measurements of the exact API port. Decode reports fallibility because it can reject a physical layout, so partially valid batches use the existing valid-only policy. The research prototype declared infallible decode, and its percentages do not qualify this port. Existing native LIKE specializations also remain stronger baselines in several cases.

Research evidence, before the API port

At develop 9a08b82dc683cd3fff8dcde834c382141f425f48, a generic prefix RowFn on 65,536 dense 64-byte VarBin strings measured 919,427 ns/batch through Utf8Column and 418,615 ns through the private offset prototype. The median paired ratio was 0.450, with five process-pair ratios from 0.444 to 0.465. This removes 1,048,576 bytes of view metadata for one input, while both paths share the payload and validate readable UTF-8.

Measurements used c7i.4xlarge, Rust 1.98.0, mimalloc, AVX2, the benchmark profile with 16 codegen units and LTO disabled, CPU affinity with the SMT sibling offline, and five alternating process pairs with 30 samples per process. This comparison measures the research binding, not the renamed library port or a query. Native LIKE must remain available, and there is no production CPU-share evidence supporting a query-gain claim.

[codspeed]

Merging this PR will improve performance by 11.98%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 2100 untouched benchmarks
⏩ 518 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	WallTime	scalar_subtract_neon	13.5 µs	12 µs	+11.98%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/row-fn-utf8-offsets (41ae7ce) with develop (c6e51ba)}

Footnotes

1. 518 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[connortsui20]

Closing in favor of a simpler change: #10340.

Instead of adding an opt-in Utf8OffsetColumn, Utf8Column itself now reads a VarBin input through its offsets when every offset fits in u32 and every row (null rows included) is valid UTF-8. Every other input still executes to string views. So every existing RowFn gets the fast path without choosing a layout before dtype-only dispatch, and no new public element type is needed. It also covers i32 offsets (Arrow Utf8 imports) without a copy.

Measured with the new row_fn_utf8_input bench (a starts_with RowFn, 16,384 rows, median of 7 alternating runs):

Input	develop	New
VarBin, 8-byte values	227.5 µs	34.1 µs
VarBin, 64-byte values	252.9 µs	47.9 µs
VarBinView, 8-byte values	79.7 µs	65.3 µs
VarBinView, 64-byte values	120.4 µs	113.1 µs

The VarBinView gain comes from marking the row accessors #[inline]. They are not generic, so a row loop instantiated in another crate called them once per row.

---

Generated by Claude Code