Repository navigation
feat(qmoe): support the router_weights input - #1241
Merged
Merged
Conversation
QMoE input 14 (router_weights) separates expert selection from aggregation: router_probs picks the top-k experts and the mixing weight of each selected expert is gathered from router_weights, renormalised only when normalize_routing_weights=1 (matching the ORT CPU kernel). wrap_qmoe rejected the input and returned -1, which left every MoE layer of such models (e.g. Laguna-XS) uncomputed. hip_qmoe_topk_routing takes an optional router_weights pointer and writes the gathered weights; every decode and prefill path consumes those weights unchanged. Selected weights are now staged in fp32 before the optional renormalisation. Made-with: Cursor Co-Authored-By: Cursor <cursoragent@cursor.com>
|
Thanks for opening a PR! This project follows LLVM's incremental-development and AI-tool-use Before requesting review, please check that:
Reviewers are assigned through |
L2 Accuracy Results (EP vs CPU)
Threshold: 0.01 | Run: 7663 - Commit: |
Binary sizesBaseline: run 39194929987 -
MorphiZen EP Performance Results
EPContext Export Performance
EPContext Import Performance
OGA Benchmark Results
OGA Wheel Smoke (Python benchmark_e2e.py)
VLM Benchmark Results
Run: 7663 - Commit: |
4 tasks done
Contributor
Author
|
ROCm GenAI CI finished with result: UNSTABLE. PR: #1243 |
BoarQing
marked this pull request as ready for review
October 19, 2026 07:35
BoarQing
requested review from
CaymanYang,
amd-bqiao,
amd-mingw,
asparkhi,
edelaye,
fhanuman,
qianglin-amd,
sambanerxilinxamd and
wcy123
as code owners
October 19, 2026 07:35
amd-mingw
approved these changes
Oct 20, 2026
amd-mingw
pushed a commit
that referenced
this pull request
Oct 21, 2026
## Summary Test PR, not for merge. It adds the onnxruntime-extensions tokenizer fix ([microsoft/onnxruntime-extensions#1136](microsoft/onnxruntime-extensions#1136)) to the Windows OGA build, so the CI package can run Laguna-XS 2.1 in Jenkins. The whole change is one entry in `OGA_PR_PATCHES`. ## Related issue or design - [microsoft/onnxruntime-extensions#1136](microsoft/onnxruntime-extensions#1136): the tokenizer fix. - [microsoft/onnxruntime-genai#2763](microsoft/onnxruntime-genai#2763): draft, not for merge. It moves OGA's onnxruntime-extensions pin to `99fec2db` plus the two #1136 commits, hosted on a fork until #1136 merges. - #1241: QMoE `router_weights` support, already on main. ## Why OGA 0.17 (with PRs 2685, 2691, 2705, 2712) pins onnxruntime-extensions `99fec2db`, which throws `invalid string_view position` when encoding a Laguna prompt that has a non-ASCII character before a newline. Both Jenkins prompts (`prompt_128.txt`, `prompt_2k.txt`) hit it. ## What `windows-deps.yml`: `OGA_PR_PATCHES` gains `2763`. Only the Windows OGA build changes. ## Test plan - I replayed the "Apply OGA PR patches" step locally on v0.17.0 with `2685 2691 2705 2712 2763`. All five applied, and the resulting ortx pin is the fork commit `c114ea9`. Its `bpe_utils.hpp` is byte-identical to the #1136 head. - A local OGA build with this onnxruntime-extensions fix encodes both Jenkins prompts. The token IDs match Hugging Face `tokenizers` exactly, apart from the BOS token that OGA omits by default. - With that OGA, #1241's `hipgpu.dll` and `enable_graph_capture: "0"`, Laguna completes both Jenkins prompts on gfx1151. Its 24-token greedy output on `prompt_128.txt` matches the ORT CPU EP. ## Notes for reviewers - Laguna still needs `"enable_graph_capture": "0"` in its AMDGPU provider options under OGA 0.17. That is a model-config change and is not part of this PR. - Once #1136 merges and OGA pins a commit containing it, replace 2763 with that OGA change and close 2763. ## Checklist - [x] The change is focused, or links a design/series explaining its scope. - [x] Relevant tests were added or updated and the results are documented. - [x] User-facing or design documentation was updated when needed. Not applicable for a CI test change. - [x] Substantial AI assistance is disclosed, and I reviewed and understand the result. Cursor prepared the OGA PR, this change and this description; validated as described above. Co-authored-by: Cursor <cursoragent@cursor.com>
ZhanxingPu
added a commit
that referenced
this pull request
Oct 21, 2026
Brings in #1237, #1241, #1243, #1245, #1259 and #1265. The only conflicts are the gfx1151 LUTs, which both sides changed: - gqa/lut/gfx1151.{json,fb}: this branch's re-measured table, with the 24:4 d256 decode rows taken from main. #1259 measured them on the new HpG 6 flash-decode kernel; this branch's rows timed an empty dispatch. 2076 -> 2071 points. - matmul_nbits/lut/gfx1151.{json,fb}: this branch's re-measured table, with the DecodeDp4a rows taken from main. #1245 re-tuned them with the cold, burst-timed tuner and checked them in-model; this branch measured them with the warm timer that #1245 replaced. Phase fallbacks unchanged. Every other row is this branch's, and main's configs were already in the config pools. The .fb files are flatc output of the merged JSON (the same flatc reproduces both sides' committed .fb byte for byte). The sweep's dp4a M-argument fix exists on both sides (7177b21 here, change 1 of #1245) and merged cleanly. This branch does not touch the flash-decode kernel or its launcher, so the HpG 6 rows main measured stay valid. Checked on the merged tree: check_kernel_resources.py, 0 errors on gfx11-generic, gfx1170 and gfx1151; the new HpG 6 decode instances do not spill. The qmoe_decode_fc1 / fc2 occupancy warning (12 -> 10 waves on gfx11-generic and gfx1170) comes from main's qmoe_kernel.hip (#1241), so the baselines are left as main has them. The 13 kernel unit-test leaves, lookup mode with the gfx1151 tables, pass on gfx1151. Scoped pre-commit passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ZhanxingPu
added a commit
that referenced
this pull request
Oct 26, 2026
…ills, restore QMoE prefill speed (#1307) ## Summary Follow-ups to #1240 (the `matmul_nbits` / `gqa` regressions with the current SDK), found while verifying it: - **GQA prefill could write inf / NaN on gfx11** (gpt-oss d64 shapes with the current SDK, intermittent from 1 in 30 runs to every run; v8x d256 at 4K queries x 64K keys with the previous SDK as well). Fixed. - **The GQA prefill tiles no longer spill.** The v6x (d64-d128) and v8x (d256) tiles sat at the 256-VGPR limit and 35 of them spilled on gfx11-generic. They now fit with headroom (at most 237 VGPRs on gfx11, 246 on gfx1170) and GQA prefill is 4-7 % faster than main. - **QMoE grouped prefill is back to its previous speed** (the current compiler serialized the small-M grouped WMMA kernel's prologue). - **gfx1151-target builds** are at least as fast as before #1240 on two GEMV paths it changed; the product binaries (gfx11-generic, gfx1170) are unchanged by these two commits. - The resource check gains a gfx1151 baseline; the kernel unit tests and the LUT sweeps default to the release target (gfx11-generic, `-O2`) and pick the LUT by the real chip; a unit-test leaf started outside its directory no longer crashes. - **The gfx1151, gfx1152 and gfx1150 matmul_nbits and GQA LUTs are re-measured** with the release binary, each table on its own chip, after four fixes to the sweep tools (one of them made the dp4a sweep fault the GPU). gfx1170 keeps its tables. - main is merged in at the end (#1245, #1259 and others); the only conflicts were the gfx1151 tables, resolved by keeping the rows main had just re-measured (see "Notes for reviewers"). ## Related issue or design Follow-up to #1240. Replaces #1258. ## Why - **GQA inf / NaN.** `v6_swap_halves()` exchanges a value with lane ^ 16 through `v_permlanex16`, whose "old" operand is tied to the result, so the compiler first copies the value. On gfx11 / gfx11.5 that copy can be dual-issued (VOPD) and hit the interlock hazard of LLVM #220348; the swap then reads a stale value. With the previous SDK the VOPD pair is visible in the ISA; with the current one the copies are plain `v_mov_b32` and the exact instruction that loses the interlock was not pinned down, but an inline-asm copy (never paired) removes every case found. - **GQA prefill spills.** The tiles prefetched the whole next K / V tile into registers, read every LDS fragment of a GEMM step up front and kept an address register per fragment. GQA asks for at most 2 waves per SIMD, so the budget is the 256-VGPR architectural limit on every target; the tiles needed 251-256, and the current compiler tipped 29 v6x and 6 v8x kernels into scratch on gfx11-generic. - **QMoE.** The prologue of `MatMulNBitsWMMA_SmallM_Grouped` (the QMoE prefill experts) tested the zero-point mode per k chunk; the current compiler keeps a branch per chunk and waits for each scale load inside it, so its 18 loads run one after another (55 -> 79 waits for the same source). - **gfx1151-target GEMVs.** #1240's per-group A row pointer cost the TILE_N=16 col-major and the TILE_M=8 M-blocked int4 GEMV registers on the 1536-VGPR target (an extra spill / a different occupancy step); they now keep their old code there, selected by `HIPDNN_VGPRS_PER_SIMD`. - **Tool defaults.** The unit tests and the LUT sweeps built a gfx1151 `-O3` binary that no release ships; gfx1151 budgets registers for a 1536-VGPR file and gfx11-generic for 1024, so tests and tables could pass or rank code that users do not run. - **LUTs.** The tables were measured on binaries that are no longer shipped (the kernels above changed, and the target moved to gfx11-generic). Re-measuring them showed four sweep problems: the u4 / i8 WMMA tuners timed one sample without settling the clock; the GQA sweep did not re-settle between shapes; the matmul sweep called the dp4a entry point without its M argument (since #1043), so on gfx1150 / gfx1151 / gfx1170 the dp4a point faulted the GPU and every later launch failed; and `hipEventElapsedTime` returned rare phantom-fast samples (0.7 % of readings on gfx1151) that picked false winners. ## What - `e0b416ed` fix(matmul_nbits): keep the gfx1151 TILE_N=16 int4 GEMV on its old col-major code - `6c7c205f` feat(kernels): check the gfx1151 target too, with baselines from this tree - `a139c8fa` fix(matmul_nbits): keep the gfx1151 TILE_M=8 int4 GEMV on its old A index - `b4f572eb` fix(gqa): copy the half-wave swap operand with inline asm on gfx11 - `e70fc759` fix(matmul_nbits): load the small-M grouped WMMA scales and zero points up front - `f609f295` perf(gqa): give the d256 v8x prefill tiles register headroom - `9404573c` perf(gqa): give the d64-d128 v6x prefill tiles register headroom - `64340c0c` perf(gqa): one staging rule per WMMA encoding for v6x / v8x prefill - `4bff1263` fix(kernel-ut): keep the stderr capture file beside the leaf executable - `23aa78e9` build(kernel-ut): test the release target and pick the LUT by the real chip - `b0845685` build(autotune): sweep the release target, keep the table per chip - `74c3d05a` chore(kernels): add the SwiGLU kernel to the resource baselines - `d7e9e3d1` perf(matmul_nbits): settle the clock and time the u4 / i8 WMMA tuners at their peak - `c97b28fa` fix(gqa): re-settle the clock before each shape in the autotune sweep - `e5ed8c61` fix(matmul_nbits): call the dp4a entry point with its M argument in the autotune sweep - `c3cd842e` fix(matmul_nbits): leave glitch samples out of the peak timer - `031db247` / `1fb7d9ce` perf(gqa / matmul_nbits): re-measure the gfx1151 LUT with the release binary - `a0c8d7b7` / `ce814ac8` perf(gqa / matmul_nbits): re-measure the gfx1152 LUT with the release binary - `df3ffec4` / `e1512239` perf(gqa / matmul_nbits): re-measure the gfx1150 LUT with the release binary - `f062a629` Merge origin/main into fix/pr1240-followups Each table commit touches only `lut/<chip>.{json,fb}`. `e5ed8c61` is the same fix as change 1 of #1245; the merge took it cleanly. The three GQA headroom commits are kept as history: the first two fit the tiles with per-tile choices, the third replaces those with one rule per WMMA encoding — on the gfx11 encoding K and V are staged separately and both GEMMs stream LDS fragments 2 reads ahead; on the gfx1170 encoding K and V stay in flight together, 3 reads ahead. A gfx11 WMMA fragment is twice the gfx1170 one: on gfx11 the tiles only fit without scratch with K / V staged separately, on gfx1170 splitting only gives up prefetch. M_TILES=2 with BKV=64 cannot fit the gfx11 encoding at all, so it is no longer built there and LUT rows that name it run the same block at BKV=32. Each commit message carries the root cause and the before / after numbers for its change. ## Test plan - The kernel results below were measured on this branch's `matmul_nbits_kernel.hip` and `gqa_kernel.hip` before main was merged in; the merge adds only main's changes to them (the dp4a tuner of #1245 and the HpG 6 decode case of #1259). The branch had first been rebased onto main, with conflicts only in the unit-test leaves (main's cold-cache timing, #1238). - `check_kernel_resources.py`, current SDK, all sources: 0 errors / 0 warnings on gfx11-generic, gfx1151 and gfx1170 (every v6x / v8x prefill kernel spill-free with both the current and the previous SDK). - Unit tests, `MODE=autotune`, coverage 1: all 13 leaves (matmul_nbits u4 / u2 / u3 / i8 fp16 / fp32 X, gqa prefill / decode fp16 / i8, qmoe_ragged_mm) built with the new defaults (gfx11-generic `-O2`) PASS on gfx1151; matmul_nbits and gqa leaves PASS for {current, previous SDK} x {gfx11-generic, gfx1151}. A leaf started from another directory PASSES (was 0xC0000409). - GQA correctness: every v6x / v8x config of 11 shapes (windows, sinks, chunked prefill, partial tiles), 184 configs, forced on gfx1151 (gfx11-generic exe) and gfx1170 (gfx1170 exe): outputs bit-identical to main. Determinism, 30 runs per shape, gfx11-generic, H64 G8 d64 HB8_QW1_MT1_BKV32 at sq 128 / 150 / 520 / 1000 and a sink shape: 1 / 3 / 30 / 30 / 30 bad runs on main -> 0 on all, and equal to a second config of the same shape. - QMoE prefill layer outputs bit-identical to before (144 / 144 rows). - On the branch head before the merge: resource check 0 errors / 0 warnings; the 13 leaves PASS in `MODE=autotune` on gfx1151 (and the four u4 / i8 matmul leaves at coverage 3, 4 x 72 cases), and in `MODE=lookup` with each chip's new table on that chip (gfx1151, gfx1152, gfx1150: 13 / 13 each); every matmul point of the old coverage is an exact point of the new tables and every GQA shape resolves; every GQA candidate in the sweeps matched the reference (rel L2 <= 4.4e-4). - After merging main: resource check 0 errors on gfx11-generic, gfx1170 and gfx1151 (the warnings are main's, see notes); the 13 leaves PASS in `MODE=lookup` with the merged gfx1151 tables on gfx1151; scoped pre-commit passes. ## Performance | Metric | Before | After | |---|---:|---:| | GQA prefill, gfx1151, 326 LUT points (time / main) | 1.000 | **0.928** | | GQA prefill, gfx1170, 130 LUT points | 1.000 | **0.965** | | GQA prefill, gfx1152, 38 model shapes | 1.000 | **0.941** | | GQA prefill, gfx1150, 67 points (incl. 26 remapped BKV=64 rows: 1.008) | 1.000 | **0.956** | | QMoE prefill layer, gpt-oss-120b T=128 / 512 / 2048 (ms; main with the previous SDK: 8.93 / 10.94 / 29.13) | 9.54 / 11.47 / 29.83 | **8.71 / 10.68 / 28.21** | | QMoE prefill layer, gpt-oss-20b T=512 / 2048 (ms; previous SDK: 6.94 / 23.52) | 7.17 / 24.30 | **6.93 / 23.64** | | gfx1151-target build: TILE_N=16 int4 GEMV decode points (time / pre-#1240 gfx1151 build) | 1.08-1.13 | **0.97-1.02** | | gfx1151-target build: M-blocked GEMV at M=6 | 0.78-1.31 | **0.98-1.05** | Before = main with the current SDK (for GQA: main with this PR's inf fix, which leaves its speed unchanged — 0.998x on 150 points). Still slower than main on a few latency-bound GQA shapes with few blocks: d128 BKV=64 8 waves at 256-1024 queries x 64K keys on gfx1151 (1.12-1.34x), d64 M_TILES=1 at 128-256 queries x 64K keys (1.07-1.23x), gpt-oss window 128 at 512 tokens on gfx1152 at the old table's pick (1.18-1.21x; the best other config is 1.11x), and sq = skv = 128 kernels of 1-2 us (1.05-1.24x). LUT re-measure, sampled points timed in fresh processes on the same binary (pick time / best config in the same process; the old tables read on the new binary in brackets): | Chip | matmul, time-weighted | matmul, within 5 % of best | GQA prefill, within 5 % | GQA decode, within 5 % | |---|---:|---:|---:|---:| | gfx1151 | **1.036** (1.041) | **52 %** (47 %) | **95 %** (77 %) | 94 % (95 %) | | gfx1152 | **1.021** (1.124) | **74 %** (57 %) | 90 % (88 %) | 90 % (91 %) | | gfx1150 | **1.057** (1.106) | **58 %** (49 %) | 91 % (86 %) | 79 % (74 %) | The per-point share stays low because near-tied tiles reorder between processes: handing a second process the first one's winners scores 54 / 82 / 54 %. phi4-14B prefill on gfx1151 is unchanged (MatMulNBits 1.001x / 0.997x / 1.011x at 512 / 1024 / 2048 tokens, GQA within 0.3 %): its old picks were already good. gfx1150 is 8.7 % faster at 2048 tokens. Hardware: Radeon 8060S (gfx1151), Radeon 860M (gfx1152), Radeon 890M (gfx1150), gfx1170 iGPU; Windows; the SDK pinned in `cmake/deps.txt`. Build: product flags (`-O2`, gfx11-generic for gfx115x, gfx1170 for gfx1170). Method: idle GPU; GQA builds compared in one process, alternated under one clock (A/A spread < 1 %), median of rounds; QMoE and GEMV rows A/B/A interleaved builds. ## Notes for reviewers - **Size.** Most of the changed lines are the generated LUT JSONs (six tables) and `kernel_resources_<arch>.json` baselines (including the new gfx1151 one). Kernel source changes: about 110 lines in `matmul_nbits_kernel.hip` (the QMoE prologue and the two gfx1151-target GEMV paths) and about 680 in `gqa_kernel.hip` (the inf fix plus the prefill headroom, which adds the shared `KvStage` and `v6_frag_stream` helpers). - **QMoE.** The fix is in `matmul_nbits_kernel.hip`, but it is the kernel behind the QMoE prefill experts (`real/qmoe.cpp`), so the QMoE owners (Cayman Yang, Yueqing Zhang) may want to look. - **Known, not fixed here (compiler side):** 256x128 WT4x2 (cfg 22-25) 3-11 % slower on gfx1151 with the current compiler (it packs the four A fragments into one register group and waits for every LDS read; a minimal repro exists), and the Llama-3.1 row-major M=1 GEMV with uint8 zero points ~3 % slower. If a later compiler fixes them, only the LUT points that pick these configs need re-measuring. - **Merge with main.** Both sides changed the gfx1151 tables. The merge keeps this branch's re-measured tables except for the rows main re-measured after the fork: the GQA 24:4 d256 decode rows (#1259, on the new HpG 6 kernel; this branch's rows had timed an empty dispatch) and the matmul DecodeDp4a rows (#1245, cold burst timing, checked in-model). This branch does not touch the flash-decode kernel or its launcher, so those rows stay valid. - **LUT gaps left for later.** gfx1170 is not re-measured (its box was unstable). On gfx1150 and gfx1152 the 24:4 d256 decode rows are still the empty-dispatch picks (as on main) and the DecodeDp4a rows come from the warm timer #1245 replaced; both can be re-measured with `update_lut.py --geometry 24:4:256` and #1245's tuner. gfx1152 GQA prefill is not better than the old table (geomean 1.024 vs 1.017 of best, worst 1.51 vs 1.19; time-weighted 1.004 vs 1.015). - **Resource check after the merge.** It warns that `qmoe_decode_fc1` / `fc2` drop from 12 to 10 waves per SIMD on gfx11-generic and gfx1170 (74 -> 94 VGPRs). That is main's `qmoe_kernel.hip` (#1241); this PR leaves those baselines as main has them. - **AI assistance.** Cursor agents (Claude) did the investigation (resource and ISA comparison, determinism and correctness checks), drafted the changes and ran the benchmarks and tests above. Commits carry `Co-Authored-By` trailers. I reviewed the changes and the measurements. ## Checklist - [x] The change is focused, or links a design/series explaining its scope. - [x] Relevant tests were added or updated and the results are documented. - [x] User-facing or design documentation was updated when needed. - [x] Substantial AI assistance is disclosed, and I reviewed and understand the result. --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
wrap_qmoenow supports QMoE's optionalrouter_weightsinput (input 14). Before this change it rejected the input and returned -1. Every MoE layer of a model that uses it was left uncomputed, but the run still exited 0 and produced plausible text. On Laguna's Jenkinsprompt_128.txt,mainmatches the ORT CPU EP's greedy output for 5 tokens and then diverges (" AMD has gone over generative AI" instead of " AMD has gone all-in on generative AI"). With this change, all 24 tokens compared match. Laguna-XS 2.1 (39 QMoE nodes, k=8,normalize_routing_weights=0) now runs end to end on gfx1151.Related issue or design
None. Found while running the Jenkins Laguna job. The tokenizer crash that job hits first is fixed separately in microsoft/onnxruntime-extensions#1136.
Why
router_weightsseparates expert selection from expert mixing, as in the ORT CPU kernel.router_probsstill picks the top-k experts. Each selected expert's mixing weight is then read fromrouter_weightsat that expert's index, and the selected weights are divided by their sum only whennormalize_routing_weights=1. If that sum is zero, the weights become zero.What
hip_qmoe_topk_routingtakes an optionalrouter_weightspointer. When it is set, the routing kernel writes the gathered weights instead of the softmax of the logits.Test plan
test/numeric/tests/test_qmoe.py,HIPDNN_EP_STRICT=1, gfx1151): 10 passed, 2 xfailed, 1 failed.test_qmoe_gpt_oss_shape_mix_weights:normalize0 and 1, atseq_len1, 37 and 128. The mixing weights are random and uncorrelated with the logits, so using the softmax of the logits instead would not match.seq_len=1cases are markedxfail(strict=False). The fused decode path exceedsatol=0.1there, as the existingtest_qmoe_gpt_oss_shape[1]does withoutrouter_weights.test_qmoe_gpt_oss_shape[1]. It fails with identical numbers onmain(max abs diff 0.615, mean 0.135).Dialect/Hipsr/Pipelines/embedding.mlir,Dialect/hip-externalize-constants.mlir) come from an unquoted Python path in my local setup and fail the same way without this change.model_benchmark.exefrom themainCI package at05ad6706, with this branch'shipgpu.dllandcustom_kernels_gfx1151.dll. OGA was rebuilt with the ort-extensions fix above, andenable_graph_capturewas set to"0"(see Notes).HIPDNN_EP_STRICT=1andHIPDNN_EP_DEBUG=1(verification only), there are 312 routing calls (39 layers × 8 forward passes), all withrouter_weights=yes, and no errors.prompt_128.txtis identical to the ORT CPU EP on the same model.prompt_128.txt,prompt_2k.txt) each completed 128 generated tokens × 3 repetitions after 1 warmup.Notes for reviewers
"enable_graph_capture": "0"in its AMDGPU provider options with OGA 0.17, because graph capture requirespast_present_share_buffer=true. That is a model-config question and is not part of this PR.planQMoEin HipToTosa still declinesrouter_weights, so those nodes keep thewrap_qmoepath this PR fixes. Its comment ("router_weights overrides router_probs and skips the softmax") doesn't match the semantics above. I left it unchanged to keep this PR focused.wrap_qmoealready accepted and loggedrouter_weights, so it is unchanged.Checklist
router_weightslimitation.