Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk - #692
Open
MengYu10151 wants to merge 1 commit into
Open
Fix JIT failure on SM100+ with CUDA <= 13.0: use 64-bit size operand for st.bulk#692MengYu10151 wants to merge 1 commit into
MengYu10151 wants to merge 1 commit into
Conversation
…for st.bulk
ptxas on CUDA 12.8-13.0 rejects st.bulk with a 32-bit size operand
(the PTX ISA defines it as .u64; the 32-bit form only became legal in
PTX ISA 9.0 and is first implemented in ptxas 13.1). This makes runtime
JIT compilation of dispatch_copy_epilogue fail on SM100-family GPUs
whenever do_expand=True and do_zero_padding=True are used:
ptxas error : Arguments mismatch for instruction 'mov'
Pass the size as a 64-bit operand instead, which every toolkit version
accepts. With CUDA 13.1 (where both forms compile) the generated SASS
is byte-identical before/after this change.
MengYu10151
marked this pull request as draft
July 20, 2026 08:29
MengYu10151
marked this pull request as ready for review
July 20, 2026 08:51
intermezzi
added a commit
to intermezzi/DeepEP
that referenced
this pull request
Jul 21, 2026
Cherry-pick of deepseek-ai/DeepEP#692. ptxas before 13.1 rejects the 32-bit size register form with 'Arguments mismatch for instruction mov'.
leewxgit
pushed a commit
to intermezzi/DeepEP
that referenced
this pull request
Jul 23, 2026
Cherry-pick of deepseek-ai/DeepEP#692. ptxas before 13.1 rejects the 32-bit size register form with 'Arguments mismatch for instruction mov'.
Contributor
|
LGTM |
Collaborator
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
st_bulk()indeep_ep/include/deep_ep/common/ptx.cuhpasses thest.bulksize operand in a 32-bit register ("r"constraint). The PTX ISA defines this operand as.u64— the 32-bit form only became legal in PTX ISA 9.0 and is first implemented in ptxas 13.1 (its own diagnostic states:Feature 'st.bulk with 32-bit size argument' requires PTX ISA .version 9.0 or later).As a result, on CUDA 12.8–13.0 toolchains, runtime JIT compilation of
dispatch_copy_epiloguefails on SM100-family GPUs wheneverdispatch(..., do_expand=True, do_zero_padding=True)is used. This is not a rare parameter combination: the standard test suitetests/elastic/test_ep.pyexercises it in its default flow, so on a clean SM100-family environment (no JIT cache) with CUDA <= 13.0 the first test run fails:This is likely invisible in environments already on CUDA 13.1+.
Changes
One line: pass the size as a 64-bit operand (
"l"constraint), which is accepted by every toolkit version.Verification
Verified on 8x B300 (sm_103) with
tests/elastic/test_ep.py --test-first-only(EP_DISABLE_GIN=1), and by directly compiling the JIT-generateddispatch_copy_epilogue_impl<true, true, true, ...>kernel across toolkits:"r"(before)"l"(after)Arguments mismatch for instruction 'st.bulk'Arguments mismatch for instruction 'mov'— runtime JIT failure on all ranks, test abortsNo behavior or performance change where both forms compile: with CUDA 13.1 the generated SASS for the affected kernel is byte-identical before/after (
cuobjdump -sassdiff is empty, identical register/shared-memory usage), and an 8-GPU A/B run shows identical dispatch bandwidth and passing correctness checks.To pin down the exact trigger condition, all 8
<kDoExpand, kCachedMode, kDoZeroPadding>template combinations ofdispatch_copy_epilogue_implwere compiled with CUDA 13.0 against the unfixed header: only the two instantiations withkDoExpand && kDoZeroPadding(<true, false, true>and<true, true, true>) fail; every other combination compiles fine, since thest_bulk()call sits behindif constexpr (kDoZeroPadding and kDoExpand).