Skip to content

Validate VarBin arrays with an explicit execution context - #10593

Merged
robert3005 merged 1 commit into
developfrom
rk/legacy-session-varbin
Oct 20, 2026
Merged

robert3005 merged 1 commit into
developfrom
rk/legacy-session-varbin

Conversation

@robert3005

@robert3005 robert3005 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor
  • The VarBin constructors take an ExecutionCtx to validate offsets and UTF-8 data. This covers VarBinArray::{new, try_new} and VarBinData::{build, build_from_handle, try_build, try_build_from_handle, validate}.
  • UTF-8 validation stays complete, because other code reads these bytes with from_utf8_unchecked.
  • Deserialization validates with a context from its session.
  • offset_at, bytes_at and sliced_bytes take a ctx.
  • The VarBin and FSST nullability casts don't change offsets or bytes, so they rebuild with new_unchecked_from_handle instead of revalidating.
  • The builder's debug sortedness check reads its own offsets slice directly.

part of #9446

@myrrc myrrc added the changelog/break A breaking API change label Oct 19, 2026
@robert3005
robert3005 marked this pull request as ready for review October 19, 2026 13:25
@robert3005
robert3005 force-pushed the rk/legacy-session-patches branch from 471c088 to b473671 Compare October 19, 2026 15:22
Base automatically changed from rk/legacy-session-patches to develop October 20, 2026 11:08
@robert3005
robert3005 force-pushed the rk/legacy-session-varbin branch from 7281c06 to 8105637 Compare October 20, 2026 11:25
Comment thread vortex-array/src/arrays/varbin/builder.rs Outdated
Comment thread vortex-array/src/arrays/varbin/builder.rs Outdated
@connortsui20

connortsui20 commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

codex found some stray cfg debug assertions (and also there is a merge conflict now)

@robert3005
robert3005 enabled auto-merge (squash) October 20, 2026 11:41
The VarBin constructors validated offsets and UTF-8 data, and `offset_at`,
`bytes_at` and `sliced_bytes` read offsets, with the hidden global
`legacy_session()`. Take an `ExecutionCtx` instead, and validate during
deserialization with a context from the session.

The VarBin and FSST nullability casts rebuild unchanged offsets and bytes
with `new_unchecked_from_handle` instead of revalidating them.

Signed-off-by: Claude <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y1zHpE1K1Y9HvyU4kt24Tq
@robert3005
robert3005 force-pushed the rk/legacy-session-varbin branch from 8105637 to cbb2829 Compare October 20, 2026 11:42
@robert3005
robert3005 merged commit 7216cb5 into develop Oct 20, 2026
100 of 101 checks passed
@robert3005
robert3005 deleted the rk/legacy-session-varbin branch October 20, 2026 11:59
@codspeed

codspeed Bot commented Oct 20, 2026

Copy link
Copy Markdown

Merging this PR will regress 5 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 1 benchmark measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. This result is not comparable, so it counts as unchanged.

Preventing compiler optimizations

⚠️ 43 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

ℹ️ 40 benchmarks have no flame graph

The flamegraph could not be generated because we failed to resolve the symbols in the profiling trace. This happens when we fail to harvest information about language frames.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 6 improved benchmarks
❌ 5 regressed benchmarks
✅ 2397 untouched benchmarks
⏩ 12 skipped benchmarks1
🗄️ 43 archived benchmarks run2

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation decode_primitives[i64, (1000, 8)] 24.7 µs 37.4 µs -34%
❌ WallTime threshold_neon[8] 1.8 µs 2.6 µs -30.67%
❌ ⚠️ Simulation test_scan_scalar_at[int] 785.2 µs 929.1 µs -15.49%
❌ ℹ️ Simulation test_scan_filter[int] 3 ms 3.4 ms -10.84%
❌ WallTime words_gather_dispatch_neon[65536] 1.8 µs 2 µs -10.17%
⚡ ⚠️ Simulation test_repeated_scan_scalar_at[int_str] 832.5 µs 424.5 µs +96.15%
⚡ WallTime filter_neon[8] 1,755 ns 911 ns +92.65%
⚡ WallTime filter_neon[64] 2.5 µs 1.8 µs +39.89%
⚡ WallTime filter_neon[80] 3.1 µs 2.6 µs +20.46%
⚡ ⚠️ Simulation bench_compare_sliced_dict_primitive[(2000, 10000)] 78.8 µs 68.6 µs +14.89%
⚡ WallTime threshold_avx512[80] 2.8 µs 2.5 µs +10.88%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing rk/legacy-session-varbin (cbb2829) with develop (3bf14ba)

Open in CodSpeed

Footnotes

  1. 12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. 43 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/break A breaking API change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants