refactor(shm): decompose and optimize label propagation - #156
Merged
Conversation
DanielSeemaier
marked this pull request as ready for review
July 28, 2026 11:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
label_propagation/from 24 to 16 headers without creating an omnibus interface:rating_map.h;node_processing.h;two_hop_clustering.h;.ccfiles.local: aggregate the complete neighborhood in worker-local storage;deferred-parallel: defer neighborhoods that exceed local capacity and aggregate them in parallel.Why
The old CRTP design coupled iteration, state management, rating aggregation, selection, and moves in one large template hierarchy. The new components can be tested independently and reused without growing the template parameter surface of every LP consumer.
The performance changes retain only local, behavior-preserving work: fewer rating-map probes, smaller cache footprints, cached aggregation state, capacity-check elision when the bound proves safety, and cheaper iteration-order bookkeeping. Schedule-changing round-policy optimizations are deliberately excluded.
Performance
Local alternating smoke benchmark on an Apple M5 Pro, 12 paired repetitions per graph/preset:
-10.82%default /-6.85%fast; total LP-18.62%/-13.95%;-15.30%/-9.28%; total LP-20.94%/-19.21%;UFM cluster validation (
k=16, seed 1, epsilon 0.03, 64 threads, 73 graphs):-3.44%default (56/0/17W/T/L) and-7.34%fast (52/0/21);+2.00%default and-4.73%fast; after repeating the two dominant outliers per preset overseeds 2--6 and replacing only those first-sample observations with their
fresh-seed means, the sensitivity result is
-3.31%default and-4.48%fast;
-14.80%default (73/0/0) and-10.57%fast (
63/1/9); summed total LP time:-11.71%and-6.81%;-16.12%default /-10.87%fast;refinement LP:
-12.00%/-10.89%;-0.47%default /-0.53%fast. The raw summed cut was-0.59%/+1.13%; five fresh seeds on the two dominant fast-presetsocial outliers improved summed cut by
-1.28%(twitter-2010) and-1.14%(com-Friendster), changing the same outlier-replacementsensitivity to
-0.75%default /-0.88%fast.The restored strategies do not alter the default/fast preset selections, and their worker-local matching hot loop remains unchanged. A paired check against the previous PR head on
com-lj(4.0M nodes / 69.4M adjacency entries, 12 performance cores) found no detectable regression:0.00%, mean+0.35%(approximate 95% CI-0.25%..+0.96%);0.00%, mean+0.10%(approximate 95% CI-0.64%..+0.84%);The 24-to-16 header consolidation was checked against the preceding PR head on
com-lj(4.0M nodes / 69.4M adjacency entries, 12 performance cores) with no stable regression signal:-0.74%for five iterations and-1.30%for one iteration;-P fastconfirmation: paired medians-0.59%partitioning and-1.14%summed LP;-0.0053%median-cut change.Validation
348/348tests passed.44/44tests passed in both 32-bit and 64-bit node-ID builds.800/800).44/44focused LP tests and10/10SHM end-to-end tests passed with no diagnostics.6f1f99e4:20/20substantive checks passed, including GCC/Clang, macOS, package, wheel, bindings, and example builds; seven release-only publication/signing jobs were correctly skipped.292/292exact CSV/log pairs validated, all feasible, with exact graph pairing and CSV/log timer, cut, and imbalance reconciliation.40/40exact logs (20baseline/candidate pairs), all feasible; exact commits,k=16, seeds 2--6, epsilon 0.03, and 64 threads validated.iversoncompute node with GCC 14.2 for Neoverse N1; no compilation ran on a login node.