Skip to content

Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk - #692

Open
MengYu10151 wants to merge 1 commit into
deepseek-ai:mainfrom
MengYu10151:fix-st-bulk-size-operand
Open

Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk#692
MengYu10151 wants to merge 1 commit into
deepseek-ai:mainfrom
MengYu10151:fix-st-bulk-size-operand

Conversation

@MengYu10151

@MengYu10151 MengYu10151 commented Jul 20, 2026

Copy link
Copy Markdown

Summary

st_bulk() in deep_ep/include/deep_ep/common/ptx.cuh passes the st.bulk size operand in a 32-bit register ("r" constraint). The PTX ISA defines this operand as .u64 — the 32-bit form only became legal in PTX ISA 9.0 and is first implemented in ptxas 13.1 (its own diagnostic states: Feature 'st.bulk with 32-bit size argument' requires PTX ISA .version 9.0 or later).

As a result, on CUDA 12.8–13.0 toolchains, runtime JIT compilation of dispatch_copy_epilogue fails on SM100-family GPUs whenever dispatch(..., do_expand=True, do_zero_padding=True) is used. This is not a rare parameter combination: the standard test suite tests/elastic/test_ep.py exercises it in its default flow, so on a clean SM100-family environment (no JIT cache) with CUDA <= 13.0 the first test run fails:

ptxas /tmp/xxx.ptx, line 401; error   : Arguments mismatch for instruction 'mov'
ptxas fatal   : Ptx assembly aborted due to errors

This is likely invisible in environments already on CUDA 13.1+.

Changes

One line: pass the size as a 64-bit operand ("l" constraint), which is accepted by every toolkit version.

Verification

Verified on 8x B300 (sm_103) with tests/elastic/test_ep.py --test-first-only (EP_DISABLE_GIN=1), and by directly compiling the JIT-generated dispatch_copy_epilogue_impl<true, true, true, ...> kernel across toolkits:

ptxas version 32-bit "r" (before) 64-bit "l" (after)
CUDA 12.9 Arguments mismatch for instruction 'st.bulk' ✅ compiles
CUDA 13.0 Arguments mismatch for instruction 'mov' — runtime JIT failure on all ranks, test aborts ✅ compiles, full test passes
CUDA 13.1 ✅ compiles ✅ compiles

No behavior or performance change where both forms compile: with CUDA 13.1 the generated SASS for the affected kernel is byte-identical before/after (cuobjdump -sass diff is empty, identical register/shared-memory usage), and an 8-GPU A/B run shows identical dispatch bandwidth and passing correctness checks.

To pin down the exact trigger condition, all 8 <kDoExpand, kCachedMode, kDoZeroPadding> template combinations of dispatch_copy_epilogue_impl were compiled with CUDA 13.0 against the unfixed header: only the two instantiations with kDoExpand && kDoZeroPadding (<true, false, true> and <true, true, true>) fail; every other combination compiles fine, since the st_bulk() call sits behind if constexpr (kDoZeroPadding and kDoExpand).

…for st.bulk

ptxas on CUDA 12.8-13.0 rejects st.bulk with a 32-bit size operand
(the PTX ISA defines it as .u64; the 32-bit form only became legal in
PTX ISA 9.0 and is first implemented in ptxas 13.1). This makes runtime
JIT compilation of dispatch_copy_epilogue fail on SM100-family GPUs
whenever do_expand=True and do_zero_padding=True are used:

    ptxas error   : Arguments mismatch for instruction 'mov'

Pass the size as a 64-bit operand instead, which every toolkit version
accepts. With CUDA 13.1 (where both forms compile) the generated SASS
is byte-identical before/after this change.
@MengYu10151
MengYu10151 marked this pull request as draft July 20, 2026 08:29
@MengYu10151
MengYu10151 marked this pull request as ready for review July 20, 2026 08:51
intermezzi added a commit to intermezzi/DeepEP that referenced this pull request Jul 21, 2026
Cherry-pick of deepseek-ai/DeepEP#692. ptxas before 13.1 rejects the
32-bit size register form with 'Arguments mismatch for instruction mov'.
leewxgit pushed a commit to intermezzi/DeepEP that referenced this pull request Jul 23, 2026
Cherry-pick of deepseek-ai/DeepEP#692. ptxas before 13.1 rejects the
32-bit size register form with 'Arguments mismatch for instruction mov'.
@alpha-baby

Copy link
Copy Markdown
Contributor

LGTM

@ds-review-bot

Copy link
Copy Markdown
Collaborator

⚠️ 抱歉,只有对该仓库有 Write 权限的用户才能触发代码审查。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants