Consume SC kernels via scmp_kernels submodule - #3
Open
AE-W wants to merge 10 commits into
Open
Conversation
Mirror the scmp_llm layout so the stochastic-computing kernels are shared,
not vendored by hand. The kernel had been pulled in via a hardcoded
sys.path hack (/home/dingqy/Bench/scmp_llm/SC) against the pre-extraction
flat API (sc_triton / config_helpers), which no longer exists.
- Add scmp_kernels as a git submodule at kernels/ (tracks branch=main),
installed editable via `pip install -e ./kernels`. Kernel updates now
flow in with `git submodule update --remote kernels`.
- Migrate models/sc_integration to the public scmp_kernels.sc_matmul
dispatcher:
* sc_qk_matmul / sc_av_matmul -> granularity="per_head" (drops the
local _sc_matmul_enable_batched_bipolar_nm reimplementation that
reached into private Triton kernels).
* sc_linear_forward -> granularity="per_tensor" (preserves prior
per-tensor behavior).
Public surface (sc_qk_matmul, sc_av_matmul, sc_linear_forward, SCMlp,
controller fns) is unchanged, so downstream evaluate/ scripts and
irasim.py need no edits.
- Remove the sys.path injection + hardcoded path from sc_integration.
- Update install.sh, README (submodule + SC backend docs), .gitignore.
AE-W
force-pushed
the
reorg/sc-kernel-submodule
branch
from
July 10, 2026 17:35
49c6115 to
bd9b480
Compare
…ng, full-test schedulers - sc_linear/sc_attention: env-gated per_row granularity (SC_LINEAR_GRANULARITY), uSystolic stream halving (SC_HALVE), SmoothQuant scales (module attr), aligned with scmp_kernels README and scmp_diffusion/scmp_llm integrations. DPM10 n=8 full-SC: per_tensor 14.16 -> per_row 20.21 -> +halve 23.93 -> +SmoothQuant 24.43 dB (FP 24.51). - evaluate/: LOI (leave-one-in) per-(op,block) sensitivity shards + GPU auto-grab schedulers with per-sample resume; diverse FPS sample selection; SmoothQuant calibration; final-recipe decider; full-test two-line scheduler (FP vs SC recipe, gated); resume support + SC_SMOOTH_SCALES loader in eval_local_n_samples; plotting + HF progress backup. - configs/sc_spec.yaml: group SC spec (symmetric+bipolar, bitrev 64 masks, int8 uniform, MP levels [128,96,64,32], fractions=search protocol). - kernels submodule: bump 06ec99b -> fd0982e (halve guard, per-(op,block) MP dispatch, scramble cache-key fix, sc_conv2d). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, MP calibration, video backup - sc_linear: mixed-precision per-row dispatch (SC_MP_CONFIG), uniform fixed stream length (SC_UNIFORM_STOC_LEN), SC_PREC / SC_MP_FIXED_PREC knobs. HPCA config table adopted (scmp_llm feat/mp-config-wiring): sc_prec≡8, level values are halved cycle counts, int=uniform / avg=mixed-precision. - evaluate/calibrate_mp_fractions.py: Lagrangian budget allocation over measured per-row error curves -> MPConfig fractions hitting the target avg cycles. - eval_local_n_samples: --keys_file for diverse-sample selection; resume. - scripts/run_sc_ladder.sh, run_mp_ladder.sh: 5-config ladder, n=300, GPU auto-grab (excl. contended GPUs), per-sample resume. - evaluate/compute_fid_fvd_stream.py: streaming FID/FVD (rescale fix pending official cross-check), chunked to avoid OOM. - evaluate/hf_backup_videos.py: near-realtime video backup + verified prune. - configs/sc_spec.yaml: HPCA config table, int/avg classification locked. Partial bridge results (n=300 diverse): sc_int8 23.27 / sc_int7 22.73 / sc_int6 21.25 dB (uniform ladder); avg192/avg96 running. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…trix, TODO) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Baselines: the naive-int8 path was pinned to 8-bit symmetric (/127, clamp +/-127), so the W7A7..W4A4 and asymmetric cells could not be run at all. Generalise the fake-quant to any width in [2,8] and to asymmetric (zero-point) quantisation, exposed as --naive_bits / --naive_asymm. The default (8, symmetric) is bit-identical to the previous implementation, so existing W8A8_symm numbers are unaffected. Mixed precision: calibrate_mp_fractions.py solved a per-row Lagrangian and then collapsed it with bincount into a single global fraction triple. That discards which module each row came from, so every operator and block ended up spending an identical average budget and the only mixing left was within a layer. Keep the per-module split the solver already produced and emit it as per_module_fractions; sc_linear.py consumes it via SC_MP_PER_MODULE and falls back to the global triple when unset. Threading (op, block_idx) to sc_linear_forward is what makes the lookup possible. Also adds the 6.32-bit tier sc_avg80 (levels [48,40,32], budget 40) — its target sits inside the level set, unlike sc_int6 whose budget equals its minimum level and therefore degenerates to uniform. Portability: hf_backup.py, hf_backup_videos.py and compute_fid_fvd_stream.py hard-coded /home/qiuyid and /edrive2 paths. Make them overridable via SCMP_HF_TOKEN_FILE / SCMP_ROOT / SCMP_RESULTS / SCMP_EVAL_MODEL_DIR and default the detector directory relative to BRIDGE_ROOT. hf_backup.py also ignored *.pt wholesale, which is how smoothquant_scales.pt went missing from the backup; small calibration artifacts are now uploaded explicitly. RUNNING_ON_SLURM.md documents how these experiments are actually driven on a contended cluster: shard sizing from measured per-sample cost, the sample-set and small-sample-FID hazards, and the environment traps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The HPCA table fixes each MP tier to {ref, 0.75ref, 0.5ref} — three levels
spanning 2x. That bounds what mixed precision can do before any calibration
runs: at a 96-cycle budget, at most 50% of rows can reach 128. The scmp_llm
ladder instead uses 5-8 irregular levels per (model, target), values like
[111, 85, 49, 48, 33] that no rule generates, so this searches for them.
search_mp_levels.py has two phases. `measure` records the real per-row SC
error (via sc_matmul, not a surrogate) at every candidate stream length in one
GPU pass. `search` then scores candidate subsets on CPU with the same
Lagrangian assignment the calibrator uses — a few ms each, so every feasible
subset can be enumerated: ~18k for k=7 over a 17-point grid. `refine` does
coordinate descent from that optimum over a dense integer grid, which is what
lets a level land on 111 rather than the nearest grid point. Subsets that do
not bracket the budget are skipped; they collapse onto one level and degenerate
to uniform, which is how sc_int6 ([64,48,32] at budget 32) ended up with
fractions [0, 0, 1].
Rounding each fraction to 4 places independently lets the sum drift ~1e-4 off
1.0, and MPConfig rejects that. The drift grows with the level count, so the
k>=4 sets raised ValueError at import time and the job exited 0 before loading
the model — Slurm reported COMPLETED and seven candidates silently produced no
samples. Both the search output and calibrate_mp_fractions.py now absorb the
residual into the largest fraction.
Adds the 6.32-bit tier sc_avg80 and six level-set ablations that vary level
count and span independently, so the two can be told apart in the results.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ules, log MP_INVESTIGATION_LOG.md records the full row-level-MP investigation: twelve experiment families, six bugs with root causes, and the two measurements that explain every null result — the runtime ranking metric carries no information about true PSNR sensitivity (Spearman 0.02 against measured gradient weights) and per-row marginal returns are nearly homogeneous (CV 5%). Under those two facts mixing cannot pay in the smooth regime and strictly loses at the cliff, which is exactly what every intervention measured. Code that the log references: - measure_row_sensitivity.py: per-row downstream sensitivity via Hutchinson probes; eps-probes are provably ranked identically to final-latent L2 (x0_hat is affine in eps under PNDM), pix-probes push through the VAE decoder so the weights represent PSNR. - select_mp_or_uniform.py: the validation gate with uniform fallback. Uniform is a feasible point of the MP search space, so the deployed schedule is >= uniform by construction — the guarantee holds at the selection layer, where the objective is the true metric, not at the proxy-calibration layer. - sc_linear.py / sc_controller.py / pipeline: SC_MP_PER_MODULE fractions with per-module direction, SC_MP_INVERT, and SC_STEP_SCHEDULE — a per-diffusion- step stream-length table. The step index is exact at runtime, so this axis needs none of the row-ordering information whose absence killed row-level MP, and it is the one axis that showed real directionality (0.56 dB between late-heavy and early-heavy at matched budget). - search_mp_levels.py: scmp_llm objectives (sigma/sigma2/delta_sigma2) with the delta baseline taken per candidate subset — taken against the ascending grid it is identically zero and the search degenerates to the cheapest level, which is how the first cliff search collapsed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…sholds Qwen3-4B cross-check settled the earlier puzzle: naive per-row MP loses on LLMs too (uniform@32 ppl 296 vs MP 922/1099), so the reference table's wins come from the FULL scmp_llm pipeline, not from a substrate difference. Porting the missing pieces: - measure_block_gamma.py: the 6Bit-Diffusion signal Gamma = ||Y-X||_1/||X||_1 per (block, timestep). Block-level and the step index is exact, so it needs none of the row-ordering information that row-level MP lacked (Spearman 0.02). Measured range 0.12–1.07 across blocks; deep blocks are far more sensitive. - sc_linear.py: salient-channel split (protected columns run at a fixed high stream length, row dispatch sees only the residual columns, partials sum) and calibrated-threshold classification (min-max normalise the metric per call, bucket against absolute descending thresholds) — the two mechanisms the naive quantile port was missing. Plus SC_STEP_SCHEDULE per-diffusion-step stream lengths. - calibrate_full_mp.py: gradient-weighted salient-channel selector (E[x^2]·Σ W^2·E[g^2]) + per-module thresholds under a residual budget that charges the protected columns' extra cycles to the same average. - measure_row_sensitivity.py: three candidate dispatch metrics (amax/l2/crest) scored by rank correlation with the true gradient weight, mirroring scmp_llm's auto metric selection — the earlier rho=0.02 verdict only tested amax. W_t timestep probes (drop step t to L=16, measure end-to-end PSNR loss) show the back half of denoising is 2.2x more sensitive than the front — opposite to the text-to-image prior, because IRASim's first frame is a clean conditioning frame so PSNR bottlenecks on late-step detail. This is the axis with real directionality (0.52 dB between late-heavy and early-heavy at matched budget). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e runner
The MP configs and calibration artifacts a second machine needs to run the
four-column table live under results/, which is gitignored, so a fresh clone
could not run RUN_MP_QUEUE.sh. Force-add the small ones (~5 MB total): the
SmoothQuant-consistent error grid, the per-block Gamma table, the four
Gamma-weighted per-block MP configs (g758/g658/g632/g600, low-bit ones with an
L>=32 cliff floor), the SmoothQuant scales, diverse-{10,100,300} key lists, the
W_t timestep schedule, and the deployed skip recipe. The dataset (133 GB) and
HF token still come separately.
RUN_MP_QUEUE.sh gains `upload` (push scr10_* result json to the HF progress
dataset via hf_backup.py) and `all` (run every unfinished column serially then
upload). Each column is n=10, one job, ~1h at SC's ~350 s/sample.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Reorganize
scmp_worldmodelto mirror thescmp_llmlayout: consume the stochastic-computing kernels from the sharedscmp_kernelsrepo as a git submodule instead of a hardcodedsys.pathhack. Kernel updates now propagate to every SC application via the submodule.Why
models/sc_integration/previously injected/home/dingqy/Bench/scmp_llm/SCintosys.pathand imported the pre-extraction flat API (sc_triton,config_helpers). That API no longer exists —scmp_kernelshas since been refactored into a package with a unifiedsc_matmuldispatcher, so the old imports are broken against the current kernel.Changes
scmp_kernelsatkernels/(tracksbranch=main), installed editable viapip install -e ./kernels. Pull updates later withgit submodule update --remote kernels.models/sc_integration/to the publicscmp_kernels.sc_matmuldispatcher:sc_qk_matmul/sc_av_matmul->granularity="per_head". Deletes the local_sc_matmul_enable_batched_bipolar_nmreimplementation that reached into private Triton kernels.sc_linear_forward->granularity="per_tensor"(preserves prior per-tensor behavior).sc_qk_matmul,sc_av_matmul,sc_linear_forward,SCMlp, controller fns) is unchanged, so theevaluate/scripts andirasim.pyneed no edits.sys.pathinjection + hardcoded path.scripts/install.sh,README.md(submodule + SC backend docs),.gitignore.Verification and caveats
from scmp_kernels import sc_matmul,make_sobol_simple_config, and all passed kwargs (granularity/mode/sc_prec/stoc_len/config) resolve against the pinned submodule.configs/evaluation/bridge/frame_ada_sc.yaml.