Repository navigation
Resolve page-cached segments in place with preadv2(RWF_NOWAIT), skipping IO coalescing - #10355
joseph-isaacs wants to merge 1 commit into
Conversation
…AIT) Add PageCacheSegmentCache, a SegmentCache that tries a non-blocking positional read of a segment's byte range. If the bytes are resident in the page cache the segment is returned straight from the calling task, skipping the coalescing read driver and the blocking thread pool; otherwise it is a miss and the regular read path fetches it. Stores are no-ops because the OS owns eviction. open_path now uses it by default when no segment cache is configured. The Python bindings keep the Moka cache only for object stores. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PHeout6F88LHmMuW8B7kq5
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (0.987x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.024x ➖, 0↑ 1↓)
datafusion / vortex-compact / ns (0.994x ➖, 2↑ 1↓)
datafusion / parquet / ns (1.002x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.071x ➖, 2↑ 4↓)
duckdb / vortex-compact / ns (1.023x ➖, 1↑ 2↓)
duckdb / parquet / ns (0.984x ➖, 1↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (0.995x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.996x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.006x ➖, 1↑ 1↓)
duckdb / vortex-compact / ns (1.011x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.999x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.023x ➖, 0↑ 1↓)
datafusion / vortex-compact / ns (0.981x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.976x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.032x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.973x ➖, 2↑ 1↓)
File Size Changes (200 files changed, +0.0% overall, 92↑ 108↓)
Totals:
|
Benchmarks: String Encoding 📖Commits: PR vortex / vortex-file-compressed / ms (1.004x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
|
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.909x ➖, 1↑ 0↓)
datafusion / vortex-compact / ns (0.885x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.827x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed / ns (1.057x ➖, 0↑ 1↓)
duckdb / vortex-compact / ns (0.975x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.010x ➖, 0↑ 1↓)
|
Benchmarks: Appian on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-compact / ns (0.995x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.004x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (1.000x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.001x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.999x ➖, 1↑ 1↓)
duckdb / vortex-compact / ns (0.999x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.996x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.996x ➖, 1↑ 0↓)
datafusion / vortex-compact / ns (1.000x ➖, 1↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (1.002x ➖, 2↑ 2↓)
duckdb / vortex-compact / ns (1.002x ➖, 3↑ 2↓)
duckdb / parquet / ns (0.995x ➖, 4↑ 5↓)
No file size changes detected. |
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.006x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (0.999x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.997x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (0.993x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.005x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.014x ➖, 0↑ 0↓)
datafusion / vortex-compact / ns (1.010x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.019x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.996x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.018x ➖, 0↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.944x ➖, 2↑ 1↓)
datafusion / vortex-compact / ns (0.898x ➖, 4↑ 2↓)
datafusion / parquet / ns (0.975x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed / ns (0.982x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (1.000x ➖, 1↑ 1↓)
duckdb / parquet / ns (1.015x ➖, 0↑ 1↓)
|
Benchmarks: Compression 📖Commits: PR vortex / vortex-file-compressed / ns (0.978x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / ratio (0.989x ➖, 0↑ 0↓)
vortex / parquet / ns (0.994x ➖, 0↑ 0↓)
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
vortex / arrow-ipc / ns (0.997x ➖, 0↑ 1↓)
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
|
Benchmarks: TPC-H SF=10 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-compact / ns (0.884x ➖, 2↑ 0↓)
datafusion / parquet / ns (0.932x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (0.937x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.920x ➖, 1↑ 0↓)
|
Benchmarks: Random Access (S3) 📖Commits: PR How to read Verdict and Engines
random-access / vortex-file-compressed / ns (1.091x ➖, 0↑ 3↓)
random-access / parquet / ns (1.015x ➖, 0↑ 0↓)
random-access / lance / ns (0.998x ➖, 2↑ 2↓)
|
Benchmarks: Random Access 📖Commits: PR How to read Verdict and Engines
vortex / arrow-ipc / ns (1.083x ➖, 0↑ 3↓)
random-access / vortex-file-compressed / ns (1.030x ➖, 0↑ 3↓)
random-access / parquet / ns (0.999x ➖, 0↑ 0↓)
random-access / lance / ns (1.010x ➖, 0↑ 0↓)
|
Summary
Local files currently rely on an in-memory Moka segment cache (in the Python bindings) to avoid re-reading segments. This PR instead leans on the OS page cache, resolving cached segments synchronously at request time.
PageCacheSegmentSourceis the outermost segment source for files opened withopen_path. Eachrequest(id)first issues a non-blockingpreadv2(RWF_NOWAIT)of the segment's byte range on the calling thread. If every byte is resident it returns a ready future, never touching:SegmentCacheSourceAdapter/InstrumentedSegmentCacheasync lookups,SharedSegmentSourcein-flightDashMap,FileSegmentSourcecoalescing read driver (event channel,Mutex-guarded driver state),On a miss the request is forwarded unchanged to the existing stack, whose read also populates the page cache.
Changes
vortex-io: addstd_file::try_read_exact_at_cached, which loopspreadv2(..., RWF_NOWAIT)until the buffer is full.EAGAIN,EOPNOTSUPP/EINVALand EOF are treated as a miss;EINTRretries. Always a miss on non-Linux platforms. Adds a Linux-onlyrustixdependency. ExposeFileReadAt::file().vortex-file: addsegments::PageCacheSegmentSource(file + segment specs + allocator + fallback source).VortexOpenOptions::open_pathwraps the assembled segment source inPageCacheSegmentSourceunless a cache was configured viawith_segment_cache.open_readis unchanged and delegates to a privateopen_read_wrapping_source.vortex-python: the 256 MiB Moka cache is now only used for object stores; local paths use the page cache.Trade-offs:
EAGAINsyscall per segment before it joins the coalescing path.InstrumentedSegmentCache, sovortex.file.segments.cache.*metrics no longer count them, and footer-read segments are re-copied from the page cache rather than served from the initial-read map.Tested with a new Linux-only unit test,
serves_resident_segments_and_forwards_misses, covering a resident read, alignment, and forwarding of a range past EOF to the fallback source.🤖 Generated with Claude Code
https://claude.ai/code/session_01PHeout6F88LHmMuW8B7kq5