Repository navigation
fix(bench): write random-access Parquet with zstd level 3 - #10157
Merged
Merged
Conversation
The three synthetic random-access generators (feature_vectors, nested_lists, nested_structs) wrote Parquet with default writer properties: uncompressed and, since every dataset is under the 1Mi-row default, one row group per file. Readers select row groups before rows, so a point lookup had to decode the whole file. Write them with 32Ki-row row groups, 1024-row data pages and zstd level 3, matching the other benchmark data generators. The row group size is a whole multiple of the 1024-row batches the Parquet-to-Vortex conversion reads, so the derived Vortex files are byte-identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L627uApatcD62FGrQPutCL Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Merging this PR will improve performance by 12.21%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | WallTime | dict_canonicalize_gt_u8_neon[1000000] |
548.2 µs | 488.5 µs | +12.21% |
Tip
Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ji/random-access-parquet-row-groups (2036089) with develop (91285b8)
Footnotes
-
503 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
joseph-isaacs
changed the title
…arquet Only the codec changes: zstd level 3 like the other generators. Row group and data page sizes stay at the parquet-rs defaults. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L627uApatcD62FGrQPutCL Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
joseph-isaacs
added
the
action/bench-random-access
github-actions
Bot
removed
the
action/bench-random-access
Contributor
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Contributor
Benchmarks: Random Access 📖Commits: PR How to read Verdict and Engines
vortex / arrow-ipc / ns (0.981x ➖, 0↑ 0↓)
random-access / vortex-file-compressed / ns (0.990x ➖, 1↑ 0↓)
random-access / parquet / ns (1.495x ❌, 1↑ 4↓)
random-access / lance / ns (1.002x ➖, 0↑ 0↓)
|
robert3005
approved these changes
Sep 30, 2026
robert3005
pushed a commit
that referenced
this pull request
Oct 5, 2026
## Summary The random-access benchmark only ever read its data from local NVMe, which hides the cost that matters most for point lookups: how many bytes and round trips a format needs against an object store. This adds an S3 variant of the same benchmark so Vortex, Parquet, and Lance random access can be tracked against remote storage, both on `develop` and on demand for PRs. The synthetic Parquet files keep the parquet-rs default row group and page sizes; only the codec (zstd level 3, from #10157) is set. ## Changes - `random-access-bench` gains `--remote-data-dir s3://bucket/prefix/` and `--prepare-data`. Data is materialized locally as before, uploaded verbatim, and then opened through `object_store` (Vortex and Parquet) or Lance's own `s3://` provider. Remote measurements are named `...-tokio-s3` and tagged with `s3` storage so they form a separate series from the local-disk numbers. Arrow IPC has no object-store reader and is skipped for remote runs. - v3 ingest records have no storage field and their measurement ID hashes only commit, dataset, format and open mode, so S3 runs use an `-s3` dataset suffix (e.g. `taxi-s3/uniform`). This keeps them from overwriting the local-disk rows without a schema change; local-disk IDs are unchanged. - `vortex-bench` adds `RemoteDataDir`, which maps a local data path to its object key, plus `open_object_store` constructors for the Vortex and Parquet accessors. The Parquet accessor keeps its cached footer and re-opens through a boxed `AsyncFileReader` on each take, so the local and object-store paths share one read path. - CI: new `pr-bench-random-access-s3.yml` reusable workflow, wired into `pr-bench-dispatch.yml` behind the `action/bench-random-access-s3` label and `action/bench-all`. `pr-bench-runner.yml` learns `variant_id` and `remote_data_dir` inputs, uploads the data before the run and deletes it afterwards. `develop-bench.yml` gains a `Random Access (S3)` matrix entry that refreshes `s3://vortex-ci-benchmark-datasets/develop/random-access/` on every run. - `lance-bench` enables Lance's `aws` feature so `Dataset::open` accepts `s3://` URIs. - Docs: README section for running against S3 and the new label in the benchmarking guide. Unit tests cover the key and URI mapping of `RemoteDataDir` and the separate S3 ingest dataset. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01L627uApatcD62FGrQPutCL --------- Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Split out of #9412 so the change to the benchmark data files lands on its own and the random-access baseline reset is attributed to it, rather than to the S3 variant.
The three synthetic random-access generators (
feature_vectors,nested_lists,nested_structs) passed no writer properties to parquet-rs, so their files were uncompressed. #10103 moved the other generators from Snappy to zstd level 3 but did not cover these, since they never set a codec at all.Changes
random_access_writer_propertiesinvortex-bench, which sets zstd level 3 and leaves row group and page sizes at the parquet-rs defaults.The Parquet-to-Vortex conversion is unaffected, so the derived Vortex files are byte-identical.