From 91b0405cad17314e3406b8d029e175190b5a1a3c Mon Sep 17 00:00:00 2001 From: jyn Date: Mon, 28 Sep 2026 22:15:10 +0200 Subject: [PATCH 01/16] add target dir goal --- src/2026/target-dir-size.md | 250 ++++++++++++++++++++++++++++++++++++ 1 file changed, 250 insertions(+) create mode 100644 src/2026/target-dir-size.md diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md new file mode 100644 index 00000000..9d2c1b43 --- /dev/null +++ b/src/2026/target-dir-size.md @@ -0,0 +1,250 @@ +# Target directory size + +| Metadata | | +| :--------------- | -------------------------------------------------------------------------------- | +| Contact | @jyn514 | +| Status | Proposed | +| What and why | Reduce the size of temporary compilation artifacts so developers need less disk space | +| Zulip channel | N/A (an existing stream can be re-used or new streams can be created on request) | + +## Summary + +Investigate, triage, and decrease the size of intermediate compilation directories (e.g. `target/`). +With a funded team of 3-5 engineers, we aim over the next year to decrease the size of the directory by 60% or more for fresh builds and by 80% or more for repeated builds across version branches. +We will benchmark based on real crates in the Rust ecosystem, focusing on disproportionately large and widely used crates. +After benchmarking, we will work on multi-pronged approaches that allow parallel work on high-impact interventions. +Our work will be incremental, in the sense that partial work outputs will still be useful; we do not need a full year to start seeing improvements. + +## Motivation + +### The status quo + +Compiling Rust programs taks three primary resources: +1. CPU time and wall time +2. Memory use +3. Disk space + +CPU time, and to some extent wall time and memory use, has been relentlessly optimized and micro-optimized for many years in Rustc. +Disk space is currently under investigation in Cargo, via the [cross workspace cache] goal, but has not yet been significantly investigated in Rustc. +Reclaiming space between builds has several tracking issues ([cargo#5026], [cargo#13136], [cargo#13060]) but no project goal. +Reusing caching between `check` and `build` is also being investigated in the [incremental system redesign] goal, but cannot help with initial full builds. +Work such as [`-Zembed-metadata=no`][embed-metadata] avoids creating unnecessary metadata sections, but does not shrink the sections themselves when they're created. +Build script disk usage to date has not been investigated in detail. + +[embed-metadata]: https://github.com/rust-lang/rust/issues/139165 +[cross workspace cache]: https://goals.rust-lang.org/2026/cargo-cross-workspace-cache.html +[cargo#5026]: https://github.com/rust-lang/cargo/issues/5026 +[cargo#13060]: https://github.com/rust-lang/cargo/issues/13060 +[cargo#13136]: https://github.com/rust-lang/cargo/issues/13136 +[incremental system redesign]: https://goals.rust-lang.org/2026/incremental-system-rethought.html + +Disk space is documented as a repeated concern; +the [2025 "State of Rust Survey"][2025-survey] shows it as the second-most common complaint about the Rust toolchain, +second only to slow compilation. + +[2025-survey]: https://blog.rust-lang.org/2026/03/02/2025-State-Of-Rust-Survey-results/#challenges-and-wishes-about-rust + +We expect reducing disk space to benefit all environments that build Rust programs, but especially: +- developers who heavily use multiple git worktrees/jj workspaces, such as AI-first workflows or contributors to the Rust compiler itself; +- CI jobs that cache intermediate artifacts, both in storage space and in upload/download speeds; +- persistent remote caches, such as used by Bazel and Buck2, primarily in storage space + +We also theorize that by reducing disk IO, we can speed up wall time for compilation itself. +If our efforts are sufficiently fruitful, we could further decrease compilation time by allowing target directories to fit on a RAM disk (sometimes known as `tmpfs`). + +Note that disk space at *compile time* (of intermediate artifacts) is different than disk space at *runtime* (of final artifacts). +The [binary size reduction] roadmap tracks final artifacts; this goal is focused on intermediate artifacts. +We expect that we may need to coordinate between teams, but that our efforts will not substantially overlap. + +[Binary size reduction]: https://goals.rust-lang.org/2026/roadmap-binary-size-reduction.html + +### What we propose to do about it + +We believe the primary obstacles to shrinking intermediate artifacts are that it's currently difficult to measure sizes, and a lack of funding for the work. +The closest tools are `llvm-size` to show overall section sizes; `rustc -Z meta-stats` to show `.rmeta` contents; and `size:query_cache` on [perf.rust-lang.org][prlo], but none give a breakdown of `.rmeta` or incremental files by query, and are difficult to correlate across invocations. + +[prlo]: https://perf.rust-lang.org + +At a high level, our plan is: +1. Benchmark what takes up the space, and include those benchmarks in `rustc-perf` so they are triaged alongside other performance improvements and regressions. +2. Near-term (1-6 months), use those measurements to guide simple improvements, with a focus on "easy wins" that greatly reduce size without much implementation work. +3. Mid-term (3-12 months), use measurements to guide structural improvements that need design work or extensive review. +4. Long-term (6-18 months), make large refactors, and change best-practices in the ecosystem to avoid disk usage. + +We will document our approach and tooling so that sizes can be further reduced even after the goal period ends. + +We do not believe our long-term goals will be necessary for our proposed 60-80% size improvement in the first year. +We expect mid-term and long-term redesigns to "compose" with the short-term improvements; in other words, major improvements before a redesign will also be major improvements after the redesign. +If we find this not to be the case after experience during implementation, we will revisit our approach. + +All structural redesigns will be conditional on benchmarks that show that they're actually a problem. + +#### Axioms + +- **Avoid an all-or-nothing approach.** Steps can depend on previous steps, but between each step we should have a coherent work product that will be an improvement to the ecosystem even if later steps are never completed. + Size improvements should stand on their own without requiring a rewrite of subsystems of Rustc or Cargo. + Only work on sweeping refactors once we have fixed all the "low-hanging fruit". + +- **Focus on architectural improvements.** Micro-optimizations look good in the short-term and are hard to maintain in the long-term. + They can also easily be specific to the precise hardware that was used to run the benchmark. + By prioritizing algorithmic and architectural improvements, we hope to end up with code that is *more* maintainable than it was to start, while still generating significantly smaller artifacts. + +- **Work together.** There are many people who want to work on the compiler and few who have the expertise to do so. + We have an explicit goal of teaching people about the compiler, especially about the query system and `rustc_metadata`, so that they can continue to maintain these subsystems even after the goal is closed. + As a corollary, we will choose optimizations that can be worked on in parallel, so that multiple people can be working at once, collaborating and bouncing ideas off of each other. + *Note*: A common problem when optimizing CPU time is that improvements in one area regress other areas, especially when touching the query system, forcing work to essentially be done in serial. + We do *not* expect that to be the case here, because so little optimization has been done to date. + We will benchmark each change to confirm, and will revist our scheduling assumptions every 3 months based on observed conflicts. + +- **Use good defaults but allow configuration.** Many software projects are *either* highly configurable (e.g. Kakoune) or work well out of the box (Helix), but few are both. We aim to make improvements that greatly improve disk space without needlessly regressing performance, while still allowing developers to choose higher reductions in exchange for slower compilation time, or vice-versa. For example, environments that already use an in-kernel compressed file system for `target/` may wish to disable debuginfo compression, or remote caching servers that rarely rebuild but have to maintain artifacts indefinitely may wish to prioritize size over speed. + +### Work items over the next year + +Subgoals (heading level 5) are independent and can be worked on in parallel unless otherwise labeled. +Within a subgoal, tasks are serially dependent unless otherwise labeled. + +#### Measurement + +##### Benchmark intermediate artifact sizes + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Create a prototype PoC that breaks down `.rmeta` files and `incremental/` directories by size and query across multiple build invocations | @jyn514 | | +| Extend the PoC to a maintainable tool that can be used outside the project goal team | ? | Likely implemented as a `rustc_driver` or `-Z` flag | +| Add support for running that tool to `rustc-perf` benchmarks | ? | Subgoal id A2 | +| Add a graph of artifact sizes over time to the `rustc-perf` site | ? | | +| (Optional) Add support for auto-triaging whether a size change is statistically relevant | ? | | + +##### Collect representative benchmarks + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Collect a sample of crates that have unusually large artifact sizes | ? | Subgoal id B1 | +| Add those crates to the `rustc-perf` benchmark suite | ? | Depends on subgoal A2. May require adding support for chains of dependencies to rustc-perf, since artifact size is impacted by monomorphization | +| Identify the largest sub-sections of `.rmeta`, `.rlib`, and `incremental/` files for those benchmarks | ? | Subgoal id B3 | + +#### Near-term improvements +These will focus on improving intermediate artifact size for gathered benchmarks. +Each of these can be worked on in parallel. + +| Task | Owner(s) | Notes | +| ---- | -------- | ----- | +| Stabilize and enable `-Z embed-metadata=no` by default | @Kobzol | stablization PR already open but not yet merged | +| Decrease debuginfo size for .rlib files | ? | likely through enabling compression; early benchmarks show near-original performance when compressed and unpacked split-dwarf is enabled | +| Strip debuginfo from build scripts | ? | needs care to make sure that panics still show a symbolicated backtrace | +| Avoid serializing unnecessary incremental state | ? | needs further investigation | +| Avoid serializing queries on disk where possible. | ? | changes must backed by benchmarks showing that this has little effect on compilation speed | +| Dynamically link the standard library in build scripts | ? | | + +#### Structural improvements + +##### Reduce duplicate storage and unnecessary serialized state +Subtasks in this category can be worked on in parallel. + +| Task | Owner(s) | Notes | +| ---- | -------- | ----- | +| Remove duplicate sections between incremental cache and `.rlib` files | ? | depends on subgoal B3. needs design work. | +| Use DWARF type signature computation to avoid duplicating debuginfo in the final binary | ? | overlaps with binary-size roadmap, needs coordination | +| Use `dwz` to avoid duplicating debuginfo in the final binary | ? | unclear whether this should be Cargo or Rustc's responsibility, needs design work. overlaps with binary-size roadmap | +| Extend Rustc with equivalents of `-gmodules` and `-fno-standalone-debug` to avoid duplicating debuginfo in intermediate artifacts | ? | large task, needs compiler design work. needs care to avoid making intermediate files non-portable. | + +##### Decrease build script artifact sizes + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Determine which build scripts generate the largest outputs | ? | Depends on subgoal B1 | +| Make targeted improvements to crates in the ecosystem | ? | for example, get build scripts to delete unused temporary artifacts after a successful build | + +##### Redesign `incremental/` directories to be GC-able + +| Task | Owner(s) | Notes | +| ---- | -------- | ----- | +| Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. | +| Emit structured info in Rustc that gives Cargo enough info to GC | ? | | +| Extend Cargo to automatically GC incremental directories | ? | | +##### Redesign Cargo's incremental caching for build scripts + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. | +| Delete build script executables between reruns | ? | needs care to ensure that outputs can be reused even when the original build script is gone | + +##### Distinguish temporary build script outputs from final outputs + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. | +| Add Cargo APIs that allow build scripts to communicate this | ? | | +| Extend Cargo to GC temporary build script outputs | ? | | +| Make targeted PRs within the ecosystem to switch crates to use the new APIs | ? | | + ##### Redesign the `.rmeta` and `.rlib` formats for better disk usage. +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Design discussions with Compiler team | @jyn514 | will benefit from a compiler team sponsor, but not strictly necessary | +| ? | ? | needs design discussion before further subtasks can be created | + +## Team asks + + +| Team | Support level | Notes | +| ---------- | ------------- | --------------------------------------- | +| [cargo] | Large | We will likely want to substantially redesign how `incremental/` directories and build script outputs are cached. We are confident we can find an approach that works for both t-cargo and t-compiler, and we can be responsible for implementation, but we will need design discussions and review from the Cargo team. | +| [compiler] | Medium | Advice on where to look and which approaches are best; reviews for small-to-medium compiler PRs that improve binary size. We expect efforts to be focused in and around the query system and `rustc_metadata`. We understand that both these areas are quite complicated and fragile; we will clearly distinguish refactors, rearchitectures, and micro-optimizations. Note: Whether this is "medium" or "large" depends on our findings from the PoC. | +| [compiler/performance] | Medium | Assistance selecting benchmarks; reviews for new intermediate artifact measurement support. | +| [infra/bootstrap] | Small | We may make small tweaks to bootstrap code or to how `rustc-dev` .rlibs are uploaded. We will endevour to minimize changes to these areas. | + + +Suggested reviewers: +- cargo: ? +- compiler: Petrochenkov, Nick Nethercote +- compiler/performance: Jakub Beranek +- infra/bootstrap: Jakub + +## Help wanted + +|Task |Experience level |Time investment | +|-------------------------------------------------|---------------------------------------------------------|----------------------------------------------------------------| +|Determine representative benchmarks |No Rust experience needed. |1-2 weeks part-time | +|Create a maintainable benchmarking tool |Medium Rust experience, compiler experience preferable |1-3 weeks full-time | +|Extend rustc-perf suite to measure artifact sizes|Medium Rust experience, compiler experience not necessary|2-4 weeks full-time | +|Improve artifact sizes |Medium Rust experience, compiler experience preferable |1-12 months full-time. Multiple positions available. | + + +## Funding + +There are over a dozen workstreams within this proposed goal. +By myself, I cannot implement them all within a year. +Additional funding will allow me to hire and mentor other compiler and build system engineers to split up the work, +not only accelerating timelines but also completing more overall workstreams. +Furthermore, it will allow me to focus on design work rather than implementation, unlocking the larger structural improvements that require coordination between teams. + +This goal needs a minimum of (Ask) to run, but for that amount we could only complete 3-5 small workstreams. +Further funding will lead to much more effective results. + +I have extensive experience working on the compiler, including the query system and integration work between Rustc and Cargo. +I also have project management experience from my time at Ferrocene, +as well as consensus building experience on policy, design, and implementation work within the Rust project. +As part of this work, I would lead implementation, mentor contributors, and coordinate between the Compiler and Cargo teams. + +Kobzol has been responsible for implementing and stabilizing `-Z embed-metadata=no`, and is a current member of the Council. + +Zeklandia has a non-traditional programming background, but has the logistical flexibility to do short-term contract work, +and I hope to mentor her to be a long-term maintainer on these compiler subsystems. + + +|Purpose |Cost |Funded |Sponsor(s) | +|--------------------------------------------------------------------------------|-------------------------------|--------|---------------------------------------------| +|@jyn514 as owner (6-12 months, full-time) |Ask |No | | +|@Kobzol as advisor (6-12 months, part-time) |Ask |No | | +|@Zeklandia as benchmark selector and analyst |Ask |No | | +|Contributor (1-12 months, part- or full-time, repeatable, multiple openings) |TBD |No | | +|Cargo maintainer funding (1-12 months, part-time, possibly multiple maintainers)|TBD |No | | + + +## Frequently asked questions + +TBD + + + + From 74d115319097f4fac9da0a0639d31f3110314c11 Mon Sep 17 00:00:00 2001 From: jyn Date: Thu, 1 Oct 2026 19:12:45 +0200 Subject: [PATCH 02/16] extend goal with ongoing work and suggested cargo designs --- src/2026/target-dir-size.md | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 9d2c1b43..a5db7f78 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -25,17 +25,31 @@ Compiling Rust programs taks three primary resources: 3. Disk space CPU time, and to some extent wall time and memory use, has been relentlessly optimized and micro-optimized for many years in Rustc. -Disk space is currently under investigation in Cargo, via the [cross workspace cache] goal, but has not yet been significantly investigated in Rustc. -Reclaiming space between builds has several tracking issues ([cargo#5026], [cargo#13136], [cargo#13060]) but no project goal. -Reusing caching between `check` and `build` is also being investigated in the [incremental system redesign] goal, but cannot help with initial full builds. -Work such as [`-Zembed-metadata=no`][embed-metadata] avoids creating unnecessary metadata sections, but does not shrink the sections themselves when they're created. -Build script disk usage to date has not been investigated in detail. - +Target directory sizes have had various ideas suggested on the Cargo, but not as much ongoing work due to funding limitaions, and hardly anything has been done on the Rustc side. + +I am aware of the following ongoing work: +- The [cross workspace cache] project goal, which reduces the number of total artifacts, but does not reduce the size of each artifact. +- [Deduplicate build artifacts across workspaces][cargo#17453] again reduces the number of total artifacts. +- Reclaiming space between builds has several tracking issues ([cargo#5026], [cargo#13136], [cargo#13060]), but no project goal. +- Reusing caching between `check` and `build` is being investigated in the [incremental system redesign] goal, but cannot help with initial full builds. +- [`-Zembed-metadata=no`][embed-metadata] avoids creating unnecessary metadata sections, but does not shrink the sections themselves when they're created. +- [Reduce debuginfo to `line-tables-only` in the `dev` profile][cargo#17518] reduces the size of debuginfo in the most common scenarios, but does not improve builds that have full debuginfo. + +Furthermore, the Cargo team has suggested several possible improvements to disk size which could be incorporated into the project goal: +- [Reduce how frequently build scripts need to be written][cargo#14948] +- [Give build scripts a dedicated scratchpad for temporary artifacts](https://rust-lang.zulipchat.com/#narrow/channel/628857-t-cargo.2Fbuild-script/topic/Providing.20a.20dedicated.20scratchpad.2C.20instead.20of.20using.20.60OUT_DIR.60) +- [Convert build scripts to artifact dependencies][cargo#14903] +- [Pipeline build scripts, not just library crates](https://rust-lang.zulipchat.com/#narrow/channel/628857-t-cargo.2Fbuild-script/topic/Pipelined.20builds/with/620427777) + +[cargo#14948]: https://github.com/rust-lang/cargo/issues/14948 [embed-metadata]: https://github.com/rust-lang/rust/issues/139165 [cross workspace cache]: https://goals.rust-lang.org/2026/cargo-cross-workspace-cache.html [cargo#5026]: https://github.com/rust-lang/cargo/issues/5026 [cargo#13060]: https://github.com/rust-lang/cargo/issues/13060 [cargo#13136]: https://github.com/rust-lang/cargo/issues/13136 +[cargo#14903]: https://github.com/rust-lang/cargo/issues/14903 +[cargo#17453]: https://github.com/rust-lang/cargo/issues/17453 +[cargo#17518]: https://github.com/rust-lang/cargo/pull/17518 [incremental system redesign]: https://goals.rust-lang.org/2026/incremental-system-rethought.html Disk space is documented as a repeated concern; @@ -130,7 +144,6 @@ Each of these can be worked on in parallel. | Task | Owner(s) | Notes | | ---- | -------- | ----- | | Stabilize and enable `-Z embed-metadata=no` by default | @Kobzol | stablization PR already open but not yet merged | -| Decrease debuginfo size for .rlib files | ? | likely through enabling compression; early benchmarks show near-original performance when compressed and unpacked split-dwarf is enabled | | Strip debuginfo from build scripts | ? | needs care to make sure that panics still show a symbolicated backtrace | | Avoid serializing unnecessary incremental state | ? | needs further investigation | | Avoid serializing queries on disk where possible. | ? | changes must backed by benchmarks showing that this has little effect on compilation speed | @@ -144,6 +157,7 @@ Subtasks in this category can be worked on in parallel. | Task | Owner(s) | Notes | | ---- | -------- | ----- | | Remove duplicate sections between incremental cache and `.rlib` files | ? | depends on subgoal B3. needs design work. | +| Decrease debuginfo size for .rlib files | ? | likely through enabling compression; early benchmarks show near-original performance when compressed and unpacked split-dwarf is enabled. needs careful design if binaries are to remain static and portable. may be less urgent once `dev` profiles use `line-tables-only`. | | Use DWARF type signature computation to avoid duplicating debuginfo in the final binary | ? | overlaps with binary-size roadmap, needs coordination | | Use `dwz` to avoid duplicating debuginfo in the final binary | ? | unclear whether this should be Cargo or Rustc's responsibility, needs design work. overlaps with binary-size roadmap | | Extend Rustc with equivalents of `-gmodules` and `-fno-standalone-debug` to avoid duplicating debuginfo in intermediate artifacts | ? | large task, needs compiler design work. needs care to avoid making intermediate files non-portable. | @@ -167,7 +181,7 @@ Subtasks in this category can be worked on in parallel. | Task | Owner(s) | Notes | | ----------- | -------- | ----- | | Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. | -| Delete build script executables between reruns | ? | needs care to ensure that outputs can be reused even when the original build script is gone | +| Delete build script executables between reruns | ? | needs care to ensure that outputs can be reused even when the original build script is gone. needs care to ensure that rebuilding a build script doesn't unnecessarily slow down builds. | ##### Distinguish temporary build script outputs from final outputs @@ -195,7 +209,7 @@ Subtasks in this category can be worked on in parallel. Suggested reviewers: -- cargo: ? +- cargo: Ed Page, Ross Sullivan - compiler: Petrochenkov, Nick Nethercote - compiler/performance: Jakub Beranek - infra/bootstrap: Jakub From f8fd379d00eb0c9e01a41b40663a6f4b63f9d1f3 Mon Sep 17 00:00:00 2001 From: jyn Date: Thu, 1 Oct 2026 19:39:01 +0200 Subject: [PATCH 03/16] edit links into subtasks --- src/2026/target-dir-size.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index a5db7f78..4bd118d1 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -37,7 +37,7 @@ I am aware of the following ongoing work: Furthermore, the Cargo team has suggested several possible improvements to disk size which could be incorporated into the project goal: - [Reduce how frequently build scripts need to be written][cargo#14948] -- [Give build scripts a dedicated scratchpad for temporary artifacts](https://rust-lang.zulipchat.com/#narrow/channel/628857-t-cargo.2Fbuild-script/topic/Providing.20a.20dedicated.20scratchpad.2C.20instead.20of.20using.20.60OUT_DIR.60) +- [Give build scripts a dedicated scratchpad for temporary artifacts][scratchpad] - [Convert build scripts to artifact dependencies][cargo#14903] - [Pipeline build scripts, not just library crates](https://rust-lang.zulipchat.com/#narrow/channel/628857-t-cargo.2Fbuild-script/topic/Pipelined.20builds/with/620427777) @@ -51,6 +51,7 @@ Furthermore, the Cargo team has suggested several possible improvements to disk [cargo#17453]: https://github.com/rust-lang/cargo/issues/17453 [cargo#17518]: https://github.com/rust-lang/cargo/pull/17518 [incremental system redesign]: https://goals.rust-lang.org/2026/incremental-system-rethought.html +[scratchpad]: https://rust-lang.zulipchat.com/#narrow/channel/628857-t-cargo.2Fbuild-script/topic/Providing.20a.20dedicated.20scratchpad.2C.20instead.20of.20using.20.60OUT_DIR.60 Disk space is documented as a repeated concern; the [2025 "State of Rust Survey"][2025-survey] shows it as the second-most common complaint about the Rust toolchain, @@ -144,7 +145,7 @@ Each of these can be worked on in parallel. | Task | Owner(s) | Notes | | ---- | -------- | ----- | | Stabilize and enable `-Z embed-metadata=no` by default | @Kobzol | stablization PR already open but not yet merged | -| Strip debuginfo from build scripts | ? | needs care to make sure that panics still show a symbolicated backtrace | +| Reduce debuginfo in build scripts | ? | if [cargo#17518] doesn't already apply to build scripts, it could be extended | | Avoid serializing unnecessary incremental state | ? | needs further investigation | | Avoid serializing queries on disk where possible. | ? | changes must backed by benchmarks showing that this has little effect on compilation speed | | Dynamically link the standard library in build scripts | ? | | @@ -187,7 +188,7 @@ Subtasks in this category can be worked on in parallel. | Task | Owner(s) | Notes | | ----------- | -------- | ----- | -| Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. | +| Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. See [existing cargo discussion][scratchpad]. | | Add Cargo APIs that allow build scripts to communicate this | ? | | | Extend Cargo to GC temporary build script outputs | ? | | | Make targeted PRs within the ecosystem to switch crates to use the new APIs | ? | | From 52a8377f79842092a4c26e56980eaeee9a3965ff Mon Sep 17 00:00:00 2001 From: jyn Date: Thu, 1 Oct 2026 13:46:27 -0400 Subject: [PATCH 04/16] Update src/2026/target-dir-size.md Co-authored-by: Nurzhan Saken --- src/2026/target-dir-size.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 4bd118d1..c1ad2bfc 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -205,8 +205,8 @@ Subtasks in this category can be worked on in parallel. | ---------- | ------------- | --------------------------------------- | | [cargo] | Large | We will likely want to substantially redesign how `incremental/` directories and build script outputs are cached. We are confident we can find an approach that works for both t-cargo and t-compiler, and we can be responsible for implementation, but we will need design discussions and review from the Cargo team. | | [compiler] | Medium | Advice on where to look and which approaches are best; reviews for small-to-medium compiler PRs that improve binary size. We expect efforts to be focused in and around the query system and `rustc_metadata`. We understand that both these areas are quite complicated and fragile; we will clearly distinguish refactors, rearchitectures, and micro-optimizations. Note: Whether this is "medium" or "large" depends on our findings from the PoC. | -| [compiler/performance] | Medium | Assistance selecting benchmarks; reviews for new intermediate artifact measurement support. | -| [infra/bootstrap] | Small | We may make small tweaks to bootstrap code or to how `rustc-dev` .rlibs are uploaded. We will endevour to minimize changes to these areas. | +| [wg-compiler-performance] | Medium | Assistance selecting benchmarks; reviews for new intermediate artifact measurement support. | +| [bootstrap] | Small | We may make small tweaks to bootstrap code or to how `rustc-dev` .rlibs are uploaded. We will endeavour to minimize changes to these areas. | Suggested reviewers: From 254a2f271d58c565dc18f190bf6488e2209178a1 Mon Sep 17 00:00:00 2001 From: jyn Date: Thu, 1 Oct 2026 22:27:47 +0200 Subject: [PATCH 05/16] separate out 'Why it matters' section --- src/2026/target-dir-size.md | 42 +++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 20 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index c1ad2bfc..d60eed62 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -17,6 +17,28 @@ Our work will be incremental, in the sense that partial work outputs will still ## Motivation +### Why it matters + +Disk space is documented as a repeated concern; +the [2025 "State of Rust Survey"][2025-survey] shows it as the second-most common complaint about the Rust toolchain, +second only to slow compilation. + +[2025-survey]: https://blog.rust-lang.org/2026/03/02/2025-State-Of-Rust-Survey-results/#challenges-and-wishes-about-rust + +We expect reducing disk space to benefit all environments that build Rust programs, but especially: +- developers who heavily use multiple git worktrees/jj workspaces, such as AI-first workflows or contributors to the Rust compiler itself; +- CI jobs that cache intermediate artifacts, both in storage space and in upload/download speeds; +- persistent remote caches, such as used by Bazel and Buck2, primarily in storage space + +We also theorize that by reducing disk IO, we can speed up wall time for compilation itself. +If our efforts are sufficiently fruitful, we could further decrease compilation time by allowing target directories to fit on a RAM disk (sometimes known as `tmpfs`). + +Note that disk space at *compile time* (of intermediate artifacts) is different than disk space at *runtime* (of final artifacts). +The [binary size reduction] roadmap tracks final artifacts; this goal is focused on intermediate artifacts. +We expect that we may need to coordinate between teams, but that our efforts will not substantially overlap. + +[Binary size reduction]: https://goals.rust-lang.org/2026/roadmap-binary-size-reduction.html + ### The status quo Compiling Rust programs taks three primary resources: @@ -53,26 +75,6 @@ Furthermore, the Cargo team has suggested several possible improvements to disk [incremental system redesign]: https://goals.rust-lang.org/2026/incremental-system-rethought.html [scratchpad]: https://rust-lang.zulipchat.com/#narrow/channel/628857-t-cargo.2Fbuild-script/topic/Providing.20a.20dedicated.20scratchpad.2C.20instead.20of.20using.20.60OUT_DIR.60 -Disk space is documented as a repeated concern; -the [2025 "State of Rust Survey"][2025-survey] shows it as the second-most common complaint about the Rust toolchain, -second only to slow compilation. - -[2025-survey]: https://blog.rust-lang.org/2026/03/02/2025-State-Of-Rust-Survey-results/#challenges-and-wishes-about-rust - -We expect reducing disk space to benefit all environments that build Rust programs, but especially: -- developers who heavily use multiple git worktrees/jj workspaces, such as AI-first workflows or contributors to the Rust compiler itself; -- CI jobs that cache intermediate artifacts, both in storage space and in upload/download speeds; -- persistent remote caches, such as used by Bazel and Buck2, primarily in storage space - -We also theorize that by reducing disk IO, we can speed up wall time for compilation itself. -If our efforts are sufficiently fruitful, we could further decrease compilation time by allowing target directories to fit on a RAM disk (sometimes known as `tmpfs`). - -Note that disk space at *compile time* (of intermediate artifacts) is different than disk space at *runtime* (of final artifacts). -The [binary size reduction] roadmap tracks final artifacts; this goal is focused on intermediate artifacts. -We expect that we may need to coordinate between teams, but that our efforts will not substantially overlap. - -[Binary size reduction]: https://goals.rust-lang.org/2026/roadmap-binary-size-reduction.html - ### What we propose to do about it We believe the primary obstacles to shrinking intermediate artifacts are that it's currently difficult to measure sizes, and a lack of funding for the work. From 2a8de5308cf8e43f2021f29106e327b7e0e5634d Mon Sep 17 00:00:00 2001 From: jyn Date: Thu, 1 Oct 2026 22:28:07 +0200 Subject: [PATCH 06/16] tweaks --- src/2026/target-dir-size.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index d60eed62..faa564fd 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -28,7 +28,7 @@ second only to slow compilation. We expect reducing disk space to benefit all environments that build Rust programs, but especially: - developers who heavily use multiple git worktrees/jj workspaces, such as AI-first workflows or contributors to the Rust compiler itself; - CI jobs that cache intermediate artifacts, both in storage space and in upload/download speeds; -- persistent remote caches, such as used by Bazel and Buck2, primarily in storage space +- persistent remote caches, such as used by Bazel and Buck2, primarily in upload/download speeds and disk IO usage. We also theorize that by reducing disk IO, we can speed up wall time for compilation itself. If our efforts are sufficiently fruitful, we could further decrease compilation time by allowing target directories to fit on a RAM disk (sometimes known as `tmpfs`). @@ -52,7 +52,7 @@ Target directory sizes have had various ideas suggested on the Cargo, but not as I am aware of the following ongoing work: - The [cross workspace cache] project goal, which reduces the number of total artifacts, but does not reduce the size of each artifact. - [Deduplicate build artifacts across workspaces][cargo#17453] again reduces the number of total artifacts. -- Reclaiming space between builds has several tracking issues ([cargo#5026], [cargo#13136], [cargo#13060]), but no project goal. +- Garbage-collection work to reclaim space between builds has several tracking issues ([cargo#5026], [cargo#13136], [cargo#13060]), but no project goal. - Reusing caching between `check` and `build` is being investigated in the [incremental system redesign] goal, but cannot help with initial full builds. - [`-Zembed-metadata=no`][embed-metadata] avoids creating unnecessary metadata sections, but does not shrink the sections themselves when they're created. - [Reduce debuginfo to `line-tables-only` in the `dev` profile][cargo#17518] reduces the size of debuginfo in the most common scenarios, but does not improve builds that have full debuginfo. @@ -235,8 +235,8 @@ Additional funding will allow me to hire and mentor other compiler and build sys not only accelerating timelines but also completing more overall workstreams. Furthermore, it will allow me to focus on design work rather than implementation, unlocking the larger structural improvements that require coordination between teams. -This goal needs a minimum of (Ask) to run, but for that amount we could only complete 3-5 small workstreams. -Further funding will lead to much more effective results. +This goal needs a minimum of (Ask) to run, but for that amount we could only complete 3-5 small workstreams, most likely the items in the "Near-term improvements" section. +Funding up to the full requested amount will allow us to complete all or almost all the proposed workstreams. I have extensive experience working on the compiler, including the query system and integration work between Rustc and Cargo. I also have project management experience from my time at Ferrocene, From 3fa863535996b7953387cf5c2210a902c3c19c18 Mon Sep 17 00:00:00 2001 From: jyn Date: Thu, 1 Oct 2026 22:28:07 +0200 Subject: [PATCH 07/16] more numbers and mention parallel GC work --- src/2026/target-dir-size.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index faa564fd..41e4cd17 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -12,8 +12,14 @@ Investigate, triage, and decrease the size of intermediate compilation directories (e.g. `target/`). With a funded team of 3-5 engineers, we aim over the next year to decrease the size of the directory by 60% or more for fresh builds and by 80% or more for repeated builds across version branches. We will benchmark based on real crates in the Rust ecosystem, focusing on disproportionately large and widely used crates. + After benchmarking, we will work on multi-pronged approaches that allow parallel work on high-impact interventions. Our work will be incremental, in the sense that partial work outputs will still be useful; we do not need a full year to start seeing improvements. +The more funding we receive, the more work we will be able to deliver. + +This goal does not currently include garbage-collecting entire target directories within Cargo. +There is [existing work][cargo#13136] by Ross Sullivan on those improvements, but Ross is not a member of this goal. +This goal *does* include garbage-collection within a target directory, such as artifacts from an earlier edit. ## Motivation @@ -244,6 +250,9 @@ as well as consensus building experience on policy, design, and implementation w As part of this work, I would lead implementation, mentor contributors, and coordinate between the Compiler and Cargo teams. Kobzol has been responsible for implementing and stabilizing `-Z embed-metadata=no`, and is a current member of the Council. +`embed-metadata` has reduced the size of release builds by [almost 30%][brlo] in the past, and we are confident that further similar improvements are possible. + +[brlo]: https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/ Zeklandia has a non-traditional programming background, but has the logistical flexibility to do short-term contract work, and I hope to mentor her to be a long-term maintainer on these compiler subsystems. From 7c8b90f8d358991ed635ddca1f6ee5c2d23057ac Mon Sep 17 00:00:00 2001 From: jyn Date: Fri, 2 Oct 2026 08:35:39 +0200 Subject: [PATCH 08/16] add Ross to the goal --- src/2026/target-dir-size.md | 43 ++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 41e4cd17..0ae68ed5 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -11,15 +11,15 @@ Investigate, triage, and decrease the size of intermediate compilation directories (e.g. `target/`). With a funded team of 3-5 engineers, we aim over the next year to decrease the size of the directory by 60% or more for fresh builds and by 80% or more for repeated builds across version branches. +This includes both decreases in artifact size and garbage-collection within a target directory, such as artifacts from an earlier edit. We will benchmark based on real crates in the Rust ecosystem, focusing on disproportionately large and widely used crates. - After benchmarking, we will work on multi-pronged approaches that allow parallel work on high-impact interventions. Our work will be incremental, in the sense that partial work outputs will still be useful; we do not need a full year to start seeing improvements. -The more funding we receive, the more work we will be able to deliver. -This goal does not currently include garbage-collecting entire target directories within Cargo. -There is [existing work][cargo#13136] by Ross Sullivan on those improvements, but Ross is not a member of this goal. -This goal *does* include garbage-collection within a target directory, such as artifacts from an earlier edit. +Note that this goal does *not* include garbage-collection across directories, such as deleting `cargo -Z script` artifacts. +There is [parallel work][cargo#17524] on that, but it is not currently part of this goal. + +[cargo#17524]: https://github.com/rust-lang/cargo/pull/17524 ## Motivation @@ -196,7 +196,7 @@ Subtasks in this category can be worked on in parallel. | Task | Owner(s) | Notes | | ----------- | -------- | ----- | -| Design discussions with Cargo team | @jyn514 | Most uncertainty. Needs Cargo team capacity. See [existing cargo discussion][scratchpad]. | +| Design discussions with Cargo team | ? | Most uncertainty. Needs Cargo team capacity. See [existing cargo discussion][scratchpad]. | | Add Cargo APIs that allow build scripts to communicate this | ? | | | Extend Cargo to GC temporary build script outputs | ? | | | Make targeted PRs within the ecosystem to switch crates to use the new APIs | ? | | @@ -206,6 +206,13 @@ Subtasks in this category can be worked on in parallel. | Design discussions with Compiler team | @jyn514 | will benefit from a compiler team sponsor, but not strictly necessary | | ? | ? | needs design discussion before further subtasks can be created | +##### Garbage collect target directories within a project + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Design discussions with Cargo team | @ranger-ross | discussions already ongoing in [cargo#17512], less risk than other design tasks | +| ? | ? | needs design discussion before further subtasks can be created | + ## Team asks @@ -249,22 +256,28 @@ I also have project management experience from my time at Ferrocene, as well as consensus building experience on policy, design, and implementation work within the Rust project. As part of this work, I would lead implementation, mentor contributors, and coordinate between the Compiler and Cargo teams. -Kobzol has been responsible for implementing and stabilizing `-Z embed-metadata=no`, and is a current member of the Council. -`embed-metadata` has reduced the size of release builds by [almost 30%][brlo] in the past, and we are confident that further similar improvements are possible. +Ross Sullivan is a current member of the Cargo team and has ongoing work to reduce target directory sizes. +[Early research][cargo#17512] shows the possibility of a 56% or more size reduction in cross-project caches from his ongoing work alone. +Further reductions are possible with more workstreams. +For Ross to join, we would need a substantial amount of the funding to be paid "up-front" at the start of the goal. +[cargo#17512]: https://github.com/rust-lang/cargo/pull/17512 [brlo]: https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/ +Kobzol has been responsible for implementing and stabilizing `-Z embed-metadata=no`, and is a current member of the Council. +`embed-metadata` has reduced the size of release builds by [almost 30%][brlo] in the past, and we are confident that further similar improvements are possible. + Zeklandia has a non-traditional programming background, but has the logistical flexibility to do short-term contract work, and I hope to mentor her to be a long-term maintainer on these compiler subsystems. -|Purpose |Cost |Funded |Sponsor(s) | -|--------------------------------------------------------------------------------|-------------------------------|--------|---------------------------------------------| -|@jyn514 as owner (6-12 months, full-time) |Ask |No | | -|@Kobzol as advisor (6-12 months, part-time) |Ask |No | | -|@Zeklandia as benchmark selector and analyst |Ask |No | | -|Contributor (1-12 months, part- or full-time, repeatable, multiple openings) |TBD |No | | -|Cargo maintainer funding (1-12 months, part-time, possibly multiple maintainers)|TBD |No | | +|Purpose |Cost |Funded |Sponsor(s) | +|-------------------------------------------------------------------------------------|-------------------------------|--------|---------------------------------------------| +|@jyn514 as owner (6-12 months, full-time) |Ask |No | | +|@ranger-ross as Cargo design contact and implementor (6-12 months, variable)|Ask |No | | +|@Kobzol as advisor (6-12 months, part-time) |Ask |No | | +|@Zeklandia as benchmark selector and analyst (variable, part-time) |Ask |No | | +|Contributor (1-12 months, part- or full-time, multiple openings) |TBD |No | | ## Frequently asked questions From 6affdd1bcea61b9fdef44548594a47054cc805f9 Mon Sep 17 00:00:00 2001 From: jyn Date: Fri, 2 Oct 2026 09:03:52 +0200 Subject: [PATCH 09/16] remove filled positions --- src/2026/target-dir-size.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 0ae68ed5..199c4dba 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -232,12 +232,11 @@ Suggested reviewers: ## Help wanted -|Task |Experience level |Time investment | -|-------------------------------------------------|---------------------------------------------------------|----------------------------------------------------------------| -|Determine representative benchmarks |No Rust experience needed. |1-2 weeks part-time | -|Create a maintainable benchmarking tool |Medium Rust experience, compiler experience preferable |1-3 weeks full-time | -|Extend rustc-perf suite to measure artifact sizes|Medium Rust experience, compiler experience not necessary|2-4 weeks full-time | -|Improve artifact sizes |Medium Rust experience, compiler experience preferable |1-12 months full-time. Multiple positions available. | +|Task |Experience level |Time investment | +|-------------------------------------------------|----------------------------------------------------------------------|-----------------------------------------------------------------| +|Create a maintainable benchmarking tool |Medium Rust experience, compiler experience preferable |1-3 weeks full-time | +|Extend rustc-perf suite to measure artifact sizes|Medium Rust experience, compiler experience not necessary |2-4 weeks full-time | +|Improve artifact sizes |Medium Rust experience, compiler or build system experience preferable|1-12 months part-time or full-time. Multiple positions available.| ## Funding From 668638b2ef169238e80c539f77a72ecafb0ee688 Mon Sep 17 00:00:00 2001 From: jyn Date: Fri, 2 Oct 2026 20:57:53 +0200 Subject: [PATCH 10/16] note that metadata size is stable --- src/2026/target-dir-size.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 199c4dba..37df78f0 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -136,7 +136,7 @@ Within a subgoal, tasks are serially dependent unless otherwise labeled. | Extend the PoC to a maintainable tool that can be used outside the project goal team | ? | Likely implemented as a `rustc_driver` or `-Z` flag | | Add support for running that tool to `rustc-perf` benchmarks | ? | Subgoal id A2 | | Add a graph of artifact sizes over time to the `rustc-perf` site | ? | | -| (Optional) Add support for auto-triaging whether a size change is statistically relevant | ? | | +| (Optional) Add support for auto-triaging whether a size change is statistically relevant | ? | likely not necessary to start since metadata and query cache sizes are stable | ##### Collect representative benchmarks From e573c8d8872ae8c979fb2f8a847c1c6b9f17cffb Mon Sep 17 00:00:00 2001 From: jyn Date: Sat, 3 Oct 2026 20:49:17 +0200 Subject: [PATCH 11/16] address kobzol's comment --- src/2026/target-dir-size.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 37df78f0..7169a0f2 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -134,9 +134,8 @@ Within a subgoal, tasks are serially dependent unless otherwise labeled. | ----------- | -------- | ----- | | Create a prototype PoC that breaks down `.rmeta` files and `incremental/` directories by size and query across multiple build invocations | @jyn514 | | | Extend the PoC to a maintainable tool that can be used outside the project goal team | ? | Likely implemented as a `rustc_driver` or `-Z` flag | -| Add support for running that tool to `rustc-perf` benchmarks | ? | Subgoal id A2 | -| Add a graph of artifact sizes over time to the `rustc-perf` site | ? | | -| (Optional) Add support for auto-triaging whether a size change is statistically relevant | ? | likely not necessary to start since metadata and query cache sizes are stable | +| Add support for running that tool to `rustc-perf` benchmarks | ? | Subgoal id A2. Will automatically give us a graph over time once metrics are hooked up. | + | (Optional) Add support for auto-triaging whether a size change is statistically relevant | ? | likely not necessary to start since metadata and query cache sizes are stable | ##### Collect representative benchmarks From 03f1636f8e545440edb5ba6e019fcffbd24d7b79 Mon Sep 17 00:00:00 2001 From: jyn Date: Sat, 3 Oct 2026 20:56:51 +0200 Subject: [PATCH 12/16] assign some tasks --- src/2026/target-dir-size.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 7169a0f2..89a741c8 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -141,9 +141,9 @@ Within a subgoal, tasks are serially dependent unless otherwise labeled. | Task | Owner(s) | Notes | | ----------- | -------- | ----- | -| Collect a sample of crates that have unusually large artifact sizes | ? | Subgoal id B1 | -| Add those crates to the `rustc-perf` benchmark suite | ? | Depends on subgoal A2. May require adding support for chains of dependencies to rustc-perf, since artifact size is impacted by monomorphization | -| Identify the largest sub-sections of `.rmeta`, `.rlib`, and `incremental/` files for those benchmarks | ? | Subgoal id B3 | +| Collect a sample of crates that have unusually large artifact sizes | @Zeklandia | Subgoal id B1 | +| Add those crates to the `rustc-perf` benchmark suite | @Zeklandia | Depends on subgoal A2. May require adding support for chains of dependencies to rustc-perf, since artifact size is impacted by monomorphization | +| Identify the largest sub-sections of `.rmeta`, `.rlib`, and `incremental/` files for those benchmarks | @Zeklandia | Subgoal id B3 | #### Near-term improvements These will focus on improving intermediate artifact size for gathered benchmarks. @@ -152,10 +152,10 @@ Each of these can be worked on in parallel. | Task | Owner(s) | Notes | | ---- | -------- | ----- | | Stabilize and enable `-Z embed-metadata=no` by default | @Kobzol | stablization PR already open but not yet merged | -| Reduce debuginfo in build scripts | ? | if [cargo#17518] doesn't already apply to build scripts, it could be extended | +| Reduce debuginfo in build scripts | @jyn514 | if [cargo#17518] doesn't already apply to build scripts, it could be extended | | Avoid serializing unnecessary incremental state | ? | needs further investigation | | Avoid serializing queries on disk where possible. | ? | changes must backed by benchmarks showing that this has little effect on compilation speed | -| Dynamically link the standard library in build scripts | ? | | +| Dynamically link the standard library in build scripts | @jyn514 | | #### Structural improvements @@ -165,16 +165,16 @@ Subtasks in this category can be worked on in parallel. | Task | Owner(s) | Notes | | ---- | -------- | ----- | | Remove duplicate sections between incremental cache and `.rlib` files | ? | depends on subgoal B3. needs design work. | -| Decrease debuginfo size for .rlib files | ? | likely through enabling compression; early benchmarks show near-original performance when compressed and unpacked split-dwarf is enabled. needs careful design if binaries are to remain static and portable. may be less urgent once `dev` profiles use `line-tables-only`. | -| Use DWARF type signature computation to avoid duplicating debuginfo in the final binary | ? | overlaps with binary-size roadmap, needs coordination | -| Use `dwz` to avoid duplicating debuginfo in the final binary | ? | unclear whether this should be Cargo or Rustc's responsibility, needs design work. overlaps with binary-size roadmap | -| Extend Rustc with equivalents of `-gmodules` and `-fno-standalone-debug` to avoid duplicating debuginfo in intermediate artifacts | ? | large task, needs compiler design work. needs care to avoid making intermediate files non-portable. | +| Decrease debuginfo size for .rlib files | @jyn514 | likely through enabling compression; early benchmarks show near-original performance when compressed and unpacked split-dwarf is enabled. needs careful design if binaries are to remain static and portable. may be less urgent once `dev` profiles use `line-tables-only`. | +| Use DWARF type signature computation to avoid duplicating debuginfo in the final binary | @jyn514 | overlaps with binary-size roadmap, needs coordination | +| Use `dwz` to avoid duplicating debuginfo in the final binary | @jyn514 | unclear whether this should be Cargo or Rustc's responsibility, needs design work. overlaps with binary-size roadmap | +| Extend Rustc with equivalents of `-gmodules` and `-fno-standalone-debug` to avoid duplicating debuginfo in intermediate artifacts | @jyn514 | large task, needs compiler design work. needs care to avoid making intermediate files non-portable. | ##### Decrease build script artifact sizes | Task | Owner(s) | Notes | | ----------- | -------- | ----- | -| Determine which build scripts generate the largest outputs | ? | Depends on subgoal B1 | +| Determine which build scripts generate the largest outputs | @Zeklandia | Depends on subgoal B1 | | Make targeted improvements to crates in the ecosystem | ? | for example, get build scripts to delete unused temporary artifacts after a successful build | ##### Redesign `incremental/` directories to be GC-able From 1091f041df5d4bc6decea58ccc5b4a42b22695ae Mon Sep 17 00:00:00 2001 From: jyn Date: Sat, 3 Oct 2026 20:57:34 +0200 Subject: [PATCH 13/16] Add panstromek to the goal --- src/2026/target-dir-size.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 89a741c8..1ff3e0ab 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -164,7 +164,7 @@ Subtasks in this category can be worked on in parallel. | Task | Owner(s) | Notes | | ---- | -------- | ----- | -| Remove duplicate sections between incremental cache and `.rlib` files | ? | depends on subgoal B3. needs design work. | +| Remove duplicate sections between incremental cache and `.rlib` files | @panstromek | depends on subgoal B3. needs design work. | | Decrease debuginfo size for .rlib files | @jyn514 | likely through enabling compression; early benchmarks show near-original performance when compressed and unpacked split-dwarf is enabled. needs careful design if binaries are to remain static and portable. may be less urgent once `dev` profiles use `line-tables-only`. | | Use DWARF type signature computation to avoid duplicating debuginfo in the final binary | @jyn514 | overlaps with binary-size roadmap, needs coordination | | Use `dwz` to avoid duplicating debuginfo in the final binary | @jyn514 | unclear whether this should be Cargo or Rustc's responsibility, needs design work. overlaps with binary-size roadmap | @@ -199,10 +199,12 @@ Subtasks in this category can be worked on in parallel. | Add Cargo APIs that allow build scripts to communicate this | ? | | | Extend Cargo to GC temporary build script outputs | ? | | | Make targeted PRs within the ecosystem to switch crates to use the new APIs | ? | | - ##### Redesign the `.rmeta` and `.rlib` formats for better disk usage. + ##### Redesign the `.rmeta`, `.rlib`, and incremental formats for better disk usage. +Improvements to the `Encodable` serialization will result in decreased disk usage across all three file types. + | Task | Owner(s) | Notes | | ----------- | -------- | ----- | -| Design discussions with Compiler team | @jyn514 | will benefit from a compiler team sponsor, but not strictly necessary | +| Design discussions with Compiler team | @panstromek | will benefit from a compiler team sponsor, but not strictly necessary | | ? | ? | needs design discussion before further subtasks can be created | ##### Garbage collect target directories within a project @@ -259,6 +261,9 @@ Ross Sullivan is a current member of the Cargo team and has ongoing work to redu Further reductions are possible with more workstreams. For Ross to join, we would need a substantial amount of the funding to be paid "up-front" at the start of the goal. +Matyáš Racek is a current member of the Compiler Performance working group and has ongoing work to reduce encoding sizes in rmeta files and incremental caches. +He has experience measuring, triaging, and improving the compiler's performance, and his assistance will be invaluable in optimizations within the compiler itself. + [cargo#17512]: https://github.com/rust-lang/cargo/pull/17512 [brlo]: https://blog.rust-lang.org/inside-rust/2026/08/18/reducing-target-dir-size-on-nightly/ @@ -272,10 +277,11 @@ and I hope to mentor her to be a long-term maintainer on these compiler subsyste |Purpose |Cost |Funded |Sponsor(s) | |-------------------------------------------------------------------------------------|-------------------------------|--------|---------------------------------------------| |@jyn514 as owner (6-12 months, full-time) |Ask |No | | -|@ranger-ross as Cargo design contact and implementor (6-12 months, variable)|Ask |No | | +|@ranger-ross as Cargo design contact and implementor (6-12 months, variable) |Ask |No | | +|@panstromek as wg-perf design contact and implementor (6-12 months, part-time) |Ask |No | | |@Kobzol as advisor (6-12 months, part-time) |Ask |No | | |@Zeklandia as benchmark selector and analyst (variable, part-time) |Ask |No | | -|Contributor (1-12 months, part- or full-time, multiple openings) |TBD |No | | +|Intern (1-12 months, part- or full-time, multiple openings) |TBD |No | | ## Frequently asked questions From 5c45d0a6cf214d344030d8293b1c2779e371d78b Mon Sep 17 00:00:00 2001 From: jyn Date: Sat, 3 Oct 2026 21:15:33 +0200 Subject: [PATCH 14/16] update Help Wanted section --- src/2026/target-dir-size.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 1ff3e0ab..303cf834 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -232,12 +232,13 @@ Suggested reviewers: - infra/bootstrap: Jakub ## Help wanted +Note: at this stage we are primarily looking for contributors who are *new* to this area of the compiler, or new to the Rust project overall, and wish to be mentored as part of this goal. +We have sufficient contributors with existing experience. |Task |Experience level |Time investment | |-------------------------------------------------|----------------------------------------------------------------------|-----------------------------------------------------------------| |Create a maintainable benchmarking tool |Medium Rust experience, compiler experience preferable |1-3 weeks full-time | -|Extend rustc-perf suite to measure artifact sizes|Medium Rust experience, compiler experience not necessary |2-4 weeks full-time | -|Improve artifact sizes |Medium Rust experience, compiler or build system experience preferable|1-12 months part-time or full-time. Multiple positions available.| +|Interns to improve artifact sizes |Medium Rust experience, compiler or build system experience preferable|1-12 months part-time or full-time. Multiple positions available.| ## Funding From fa2cfae935431b3e25439bc705d467677df93c6e Mon Sep 17 00:00:00 2001 From: jyn Date: Sun, 4 Oct 2026 06:01:37 +0200 Subject: [PATCH 15/16] Update src/2026/target-dir-size.md Co-authored-by: Makai --- src/2026/target-dir-size.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index 303cf834..c4ca0d4c 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -47,7 +47,7 @@ We expect that we may need to coordinate between teams, but that our efforts wil ### The status quo -Compiling Rust programs taks three primary resources: +Compiling Rust programs takes three primary resources: 1. CPU time and wall time 2. Memory use 3. Disk space From 570a24d68d35564eab5dde9f39209e809d9af54d Mon Sep 17 00:00:00 2001 From: jyn Date: Tue, 6 Oct 2026 13:25:14 +0200 Subject: [PATCH 16/16] mention duplication in target binaries --- src/2026/target-dir-size.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/2026/target-dir-size.md b/src/2026/target-dir-size.md index c4ca0d4c..40dd6403 100644 --- a/src/2026/target-dir-size.md +++ b/src/2026/target-dir-size.md @@ -170,6 +170,16 @@ Subtasks in this category can be worked on in parallel. | Use `dwz` to avoid duplicating debuginfo in the final binary | @jyn514 | unclear whether this should be Cargo or Rustc's responsibility, needs design work. overlaps with binary-size roadmap | | Extend Rustc with equivalents of `-gmodules` and `-fno-standalone-debug` to avoid duplicating debuginfo in intermediate artifacts | @jyn514 | large task, needs compiler design work. needs care to avoid making intermediate files non-portable. | +##### Reduce overlap between test binaries +Currently, `cargo test` produces one `libtest` harness binary for each crate in a workspace. +In crates with many workspaces, this leads to a large amount of time spent linking and space taken up by the resulting many binaries. +Can we consolidate these into a single binary, or use dynamic linking to reduce the amount of duplicated space? + +| Task | Owner(s) | Notes | +| ----------- | -------- | ----- | +| Test on crates with large workspaces to see where the space is taken up | ? | | +| Investigate possible approaches to see which are feasible | ? | | +| ? | ? | needs design discussion before further subtasks can be created | ##### Decrease build script artifact sizes | Task | Owner(s) | Notes |