Skip to content

Use mimalloc with purging disabled in each e2e benchmark binary - #10490

Merged
robert3005 merged 5 commits into
developfrom
ji/e2e-bench-mimalloc
Oct 18, 2026
Merged

robert3005 merged 5 commits into
developfrom
ji/e2e-bench-mimalloc

Conversation

@joseph-isaacs

@joseph-isaacs joseph-isaacs commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The end-to-end benchmark binaries (SQL via DataFusion and DuckDB, random access, compression, string, Lance) only ran with mimalloc implicitly, through a #[global_allocator] declared in the vortex-bench library. This makes the choice explicit in each benchmark binary, and stops mimalloc from purging freed memory back to the OS during a run: mimalloc v3 purges 1s after a free, and that work lands in the middle of measured queries.

Changes

  • Declare #[global_allocator] static GLOBAL: MiMalloc = MiMalloc; in the main.rs of compress-bench, datafusion-bench, duckdb-bench, lance-bench, random-access-bench, string-bench, and in vortex-bench's data-gen binary.
  • Remove the global allocator from vortex-bench/src/lib.rs, so the library no longer picks the allocator for whatever links it.
  • Add vortex_bench::configure_allocator(), which sets mimalloc's purge_delay option to -1 (never purge), and call it first thing in each of those binaries. This uses libmimalloc-sys with its extended feature (adds cty to the lockfile).
  • Add mimalloc to each benchmark crate's dependencies and update Cargo.lock.

Checks: cargo clippy --locked --bins -D warnings for vortex-bench, compress-bench, random-access-bench, string-bench and datafusion-bench; formatted with the pinned nightly rustfmt. duckdb-bench and lance-bench could not be built locally (DuckDB source download and the lance-encoding build script fail in this environment), so CI covers them.

🤖 Generated with Claude Code

https://claude.ai/code/session_0122rGn9fTSV47KLv191MTHB

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants