fix(mp): FLOP-weighted calibration budget, per-row qk MP, drop α/β - #7
Closed
heroarmor wants to merge 6 commits into
Closed
fix(mp): FLOP-weighted calibration budget, per-row qk MP, drop α/β#7heroarmor wants to merge 6 commits into
heroarmor wants to merge 6 commits into
Conversation
…beta - calibrate_mp_thresholds.py: budget is now sum(rows * macs * stoc_len) instead of sum(rows * stoc_len), so an "avgX" target means X in real compute, not row count. macs_per_row (out*in for linears, N*head_dim for qk/av) is threaded through add() -> the global lambda solve + iso-budget check. qk recording is switched to per-row. Validated: realized FLOP-weighted avg = 47.97 for target 48. - sc_attention.py: _sc_qk MP is now per-query-row (||Q_row||_inf), matching the per_row quant scale; removed the per-head classifier and static per-head path. - quant_sc_main.py: removed all alpha/beta remnants -- --mp_alpha/--mp_beta and the 14 per-op variants, the crashing AdaptiveMPConfig(alpha=,beta=, operator_params=) construction, and the run-name tag. --adaptive_mp now requires --adaptive_mp_table. - sc_attention.py + sc_mlp.py: repaired the adaptive_classify_rows signature drift vs kernel 9358239 (6 call sites passed the old arg order -> TypeError, so the adaptive path could not run at all). Invalidates existing calibration threshold tables (new cost model -> new allocations); re-calibration required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Run artifacts (slurm-*.out, figures/) are regenerated, not source. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tools/: compare_old_vs_new{,_isolated,_thorough}.py, compare_vs_cpu_reference.py, sc_call_counter{,_full}.py, smoke_test_e2e.py
scripts/: global-PR calib + MP sbatch (calib_globalpr{,_flop}{,_halve}, mp_globalpr_{auto,fill,flop_halve_auto}, uniform{,_halve}_auto, opfreeze_probe) + eval/ CPU eval
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eanup Submodule now at local branch local/trace+scramble-cache (6c4539a): PR#25 trace subsystem + cherry-picked PR#28 enable-table scramble cache-key fix + mp/config doc cleanup. NOTE: 6c4539a is LOCAL-ONLY (not yet pushed to the heroarmor fork); a fresh 'git submodule update' cannot resolve it until pushed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion; mse_rel metric The halve MP line calibrated and ran levels above the halve-mode maximum (2^(sc_prec-1)=128): the 256/192 levels are unrealizable on halve hardware and silently inflated attention accuracy (up to 99% of qk rows ran 256/192- cycle streams). Fixes, one per layer: - scmp_kernels bump -> c967d3a: sc_matmul now raises on explicit stoc_len/rng_levels above 2^(sc_prec-1) under halve+bipolar (upstream kernels PR #30), so stale 256-max tables crash instead of overspending. - calibrate_mp_thresholds.py: hard-validate mp_levels and budget_ref_stoc_len <= 2^(sc_prec-1) under --sc_halve; add --metric mse_rel (squared relative L2 = cosine's quadratic curvature + gain-error sensitivity; a pure 10% gain error scores 0.0 under cosine, 0.01 under mse_rel). - sbatch_calib_globalpr_flop_halve.sb / sbatch_calib_globalpr_halve.sb: 128-capped level grid, budget_ref 128, BUDGET_RATIO computed from AVG in-script (avg/128 — passing the old avg/256 ratios silently halved every budget), optional METRIC arg with suffixed OUTDIR, and calibration now runs at the deployment CFG (teacher_cfg_scale 1.5, batch 16 so the CFG-doubled batch keeps the old 32-row memory footprint) instead of cfg=0. - sbatch_mp_globalpr_flop_halve_auto.sb: [METRIC] and [NUM_FID] args (metric-suffixed calib/output dirs; NUM_FID for fast probes), points at the 128-capped tables; old 256-grid tables/arms preserved in the un-suffixed dirs for comparison. Validated: avg48 recalibration hits expected_avg_stoc_len=48.00; runtime realized per-op distribution matches the table within 1-3pp per level; on the noise-aligned index prefix the fixed adaptive arms (cosine and mse_rel) beat uniform48 on fidelity-to-FP (2.5% vs 5% trajectory divergence). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Small single-purpose probes developed while root-causing the halve-line
regressions; kept because each verifies an invariant worth re-checking:
- eval/sbatch_eval_uniform_halve.sb: KID + FID/IS/Prec/Recall for one
uniform-halve arm (companion to sbatch_eval_mp_flop_halve.sb).
- eval/sbatch_eval_ab_kid100.sb: matched-index KID across probe arms + an FP
anchor on the same indices (the anchor exposes the class-composition floor
that dominates absolute KID at small n — FP scores the same ~30x1e-3 as SC
arms on 10 classes).
- eval/sbatch_probe_realized_dist{,_old}.sb: validate-path run that dumps
debug_mp_distribution.csv — compares the RUNTIME-realized per-op level
fractions against the calibration table's intent (A4 check).
- eval/sbatch_kernel_level_err.sb: kernel-level rel-err vs stoc_len under
halve on qk/av/mlp shapes (catches level-grid pathologies without running
the diffusion model).
- sbatch_uniform48_rerun100.sb: regenerate uniform48 idx 0-99 into a fresh
dir — reproducibility check. NOTE: latent noise pairs by position in each
worker's index list, not by global index, so PSNR comparisons against runs
with a different GPU split are only valid on the aligned worker-0 prefix.
- sbatch_only1op_sc32.sb: single-op SC isolation (everything FP except one op
at stoc_len 32 + halve) via per-op config JSONs; the legacy layerwise flags
are ignored whenever --sc_config supplies a precision map.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner
Author
|
Duplicate of the canonical upstream PR: CrucibleComputingGroup#6. Closing this fork-internal PR; all review/merge happens upstream. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three fixes to the SC mixed-precision pipeline plus one necessary drift repair. Diffusion-scoped (each sibling repo vendors its own
scmp_kernels, so blast radius is contained).1. FLOP-weighted calibration budget (
calibrate_mp_thresholds.py)The MP budget was row-weighted (
cost = rows × stoc_len). That over-weighted attention —av/qkare counted per-head-per-row (~80% of rows) but are only ~2% of FLOPs — and under-weighted the FLOP-heavy MLP. Result: anavg48table was ~avg70in real compute.Now
cost = rows × macs_per_row × stoc_len, so a targetavgXmeans X in real compute.macs_per_row(out×infor linears,N×head_dimfor qk/av) is threaded throughadd()→ the global-lambda solve (_global_lambdagetsR·macscounts), the per-grouprep, and the iso-budget check. Validated by standalone replication of_global_lambda: realized FLOP-weighted avg = 47.97 for target 48 (the same solution is 237 row-weighted). qk recording is switched to per-row to match the runtime.2. qk MP per-head → per-row (
sc_attention.py::_sc_qk)qk quantized per-row but assigned
stoc_lenper-head — inconsistent._sc_qknow classifies per query-row (||Q_row||_inf), mirroring_sc_av. The per-head classifier and static per-headgroup_stoc_lenspath are removed. MP granularity now matches the per_row quant scale.3. Removed all α/β (
quant_sc_main.py)The
α·progress+βmethod was already gone from kernel9358239; the leftovers were dead/broken:--mp_alpha/--mp_beta+ 14 per-op variants, and anAdaptiveMPConfig(alpha=,beta=,operator_params=)construction that wouldTypeErroron--adaptive_mp. All removed.--adaptive_mpnow requires--adaptive_mp_table.4. Repaired adaptive-MP signature drift (
sc_attention.py,sc_mlp.py)All 6
adaptive_classify_rowscall sites used the pre-bump arg order; kernel9358239reorderedconfigto the 2nd positional, so every callTypeErrord before running — the entire adaptive path was broken on the current tree. Fixed to the new signature.Verification
amax(dim=(0,2,3))) anywhere.Follow-ups
calib_*/*.jsontables are stale under the new cost model. Oldavg48(row) ≈avg70(FLOP).scmp_kernels/mp/config.pyhas an α/β docstring scrub in the working tree — intentionally not in this PR; it needs a separatescmp_kernelsPR + submodule bump.a576b83→9358239) before trusting the full pipeline.🤖 Generated with Claude Code