[Draft]SM100 persistent sparse attention for V4 CSA/HCA - #207
Open
uchihatmtkinu wants to merge 4 commits into
Open
[Draft]SM100 persistent sparse attention for V4 CSA/HCA#207uchihatmtkinu wants to merge 4 commits into
uchihatmtkinu wants to merge 4 commits into
Conversation
`nvvm.fmax` used to take the result type as its first positional argument; newer MLIR python bindings infer it from the operands and reject the extra one, so the suite's two CuTe DSL kernels raise TypeError on a current nvidia-cutlass-dsl. Both forms build the same op and emit the same PTX, so detect which one this install exposes on first use rather than pinning the package from inside a kernel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both grids are instantiated -- `logits_aot.cu` dispatches launch<148> and launch<152> -- but the host guard required `num_sms` to equal the device's SM count, so a 152-SM GB200 could only run the 152 grid and the 148 instantiation was unreachable there. That is over-strict for a persistent kernel. Sizing one deliberately below the device is how a schedule leaves SMs for a concurrent node, and the CSA schedule does exactly that: its checked-in GB200 profile runs this stage at 148 on a 152-SM part, next to work that wants the other four. Forcing it to 152 undoes a tuned value. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title
Add SM100 persistent sparse-attention kernels and V4 CSA/HCA interfaces
Summary
This PR adds an optional SM100 persistent sparse-attention component for V4
CSA/HCA pipelines. It combines persistent scheduling with kernel fusion and
exposes the required prefill, decode, frontend, indexing, readiness, and output
interfaces through one FlashMLA extension.
The default FlashMLA call path remains unchanged. Specialized behavior is
selected only when the caller supplies persistent-attention options or invokes
the component API directly.
What changed
launches, optional Q preprocessing, ready signaling, an additional KV pool,
and fused output handling.
CSA/HCA decode integration.
fusion, logits/WqB-ready adapters, index/cache operations, learned TopK,
state/cache updates, and output quantization.
flash_mla.persistent_sparse_attention_cudaand expose Python wrappers fromflash_mla.persistent_sparse_attention.installed wheel does not use runtime JIT, NVCC, or NVRTC.
dependency model. Unmodified headers come directly from the submodules and
project-specific adaptations remain in a small overlay.
under the persistent sparse-attention component directories.
Build
The full V4 CSA/HCA suite is opt-in. Normal FlashMLA builds keep the existing
sparse-attention surface and do not include the additional pipeline kernels.
Validation
packed WqB-ready 0.994 (25% acceptance band): PASS
csa_cuda, or runtimecompiler dependency
Quick reproduction:
Detailed architecture and dependency boundaries are documented in
docs/persistent_sparse_attention/DESIGN.md. Test coverage and summarizedresults are documented in
docs/persistent_sparse_attention/VALIDATION.md.