torch_musa: boolean masked assignment fails past 2^31 elements

[cheersluvs]

Component: torch_musa / MUDNN

Environment

component	version
card	MTT S5000, 79.9 GiB
MUSA toolkit / mcc	4.3.5 (release_musa_4.3.0)
torch	2.9.0
torch_musa	2.9.0

Summary

RuntimeError: FillOp MUDNN failed in: RunMask

from tensor[bool_mask] = 0 on a tensor of 3.22e9 elements (98304 × 64 × 512),
just past 2^31 = 2.147e9.

Reproduction

import torch
n = 98304 * 64 * 512          # 3.22e9, past 2^31
a = torch.randn(n, dtype=torch.float32, device="musa")
m = a > 0
a[m] = 0.0                    # RuntimeError: FillOp MUDNN failed in: RunMask
torch.musa.synchronize()

Impact

Beyond the direct failure, this breaks torch.testing.assert_close: its
mismatch-reporting path performs exactly abs_diff[matches_flat] = 0 in order to
compute max_abs_diff. On any large tensor a comparison that should print a
diff instead raises an opaque RuntimeError: Comparing ... containing no
numbers. That misdirects debugging — we spent a round chasing a phantom numerical
regression before noticing the crash was in the reporting code and not in the
comparison itself.

Any test suite validating large tensors on MUSA is affected.

[cheersluvs]

Additional detail: the muDNN error, which names the failing site.

Re-running the reproduction with muDNN's own log visible gives more than the Python-level RuntimeError:

muDNN(v3105) ERROR# EXECUTION_FAILED in Fill::Run, Reason:
     musa runtime failed fill.mu:810: err 9 = invalid configuration argument

RuntimeError: FillOp MUDNN failed in: RunMask

err 9 = invalid configuration argument is a launch configuration failure, not an allocation one — consistent with a grid dimension computed from the element count exceeding a limit once the tensor passes 2^31. The card had ample free memory at the time.

Confirmed on torch 2.9.0 / torch_musa 2.9.0+0680da1, muDNN v3105, with the reproduction exactly as written above.