fix: use merge_all() to avoid O(n)-depth runfiles NestedSet nesting#86
Merged
Merged
Conversation
Replace sequential runfiles.merge() calls in loops with collecting into a list and a single merge_all() call. This changes the NestedSet depth from O(n) to O(1), preventing StackOverflowError in large multirun targets. When a multirun target has many commands (e.g. 1000+), each merge() wraps the previous result as a transitive child, creating a linear chain. When Bazel fingerprints this via the recursive NestedSetFingerprintCache.addToFingerprint, it overflows the JVM stack. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
keith
approved these changes
Jun 11, 2026
keith
left a comment
Owner
There was a problem hiding this comment.
that's a lot of runfiles! thanks!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runfiles.merge()calls in loops with collecting into a list + singlemerge_all()call in bothmultirun.bzlandcommand.bzlStackOverflowErrorin large multirun targetsProblem
When a
multiruntarget has many commands (e.g. 1000+), the current code callsrunfiles.merge()in a loop:Each
merge()wraps the previous result as a transitive child, creating a linear chain of NestedSets with depth proportional to the number of commands. When Bazel later fingerprints this structure via the recursiveNestedSetFingerprintCache.addToFingerprint, it overflows the JVM stack.We hit this as a
java.lang.StackOverflowError(1022 recursive frames deep) when upgrading to Bazel 9.1.1 in a monorepo with ~1000 deploy targets aggregated via a singlemultiruntarget. The crash is in Bazel's internal NestedSet fingerprinting:Fix
Collect all runfiles objects into a flat list, then call
merge_all()once. This produces a shallow NestedSet tree regardless of the number of commands.merge_all()has been available since Bazel 5.x.Test plan
StackOverflowError🤖 Generated with Claude Code