Skip to content

Build C++/CUDA extensions with -std=c++20 (torch 2.14 ATen requires C++20) - #1000

Open
atalman wants to merge 1 commit into
state-spaces:mainfrom
atalman:atalman/cpp20-for-torch-2.14
Open

atalman wants to merge 1 commit into
state-spaces:mainfrom
atalman:atalman/cpp20-for-torch-2.14

Conversation

@atalman

@atalman atalman commented Jul 24, 2026

Copy link
Copy Markdown

Purpose

torch>=2.14 enforces a C++20 minimum in its ATen/torch headers (pytorch/pytorch#178150). Any translation unit that includes an ATen header while compiling under an older standard now trips:

#error C++20 or later compatible compiler is required to use ATen.

setup.py hardcodes -std=c++17 in both the cxx and nvcc flag lists. torch's cpp_extension only appends its own default -std=c++20 when the extension passes no -std= flag:

if not any(flag.startswith('-std=') for flag in cuda_flags):
    cuda_flags.append('-std=c++20')

so the pinned c++17 wins and the host compile of selective_scan.cpp (which pulls in ATen headers) fails against torch 2.14 nightly.

Change

Bump the pinned standard from c++17 to c++20 in both the CUDA and HIP branches of setup.py (4 occurrences). c++20 is already required by the toolchains this codebase targets (CUDA 12.x / Blackwell sm_121), so this is a no-op for stable torch builds and unblocks torch 2.14+.

Context

Currently worked around downstream in vLLM CI (vllm-project/vllm#49600) by patching this exact -std= flag at build time; this PR fixes it at the source so that workaround can be dropped.

Test plan

  • pip install --no-build-isolation . against a torch 2.14 nightly build (previously failed at the ATen C++20 header guard, now compiles).
  • Stable-torch builds are unaffected (compiling the c++17-era sources under c++20 is backward compatible).

This PR was drafted with the assistance of an AI agent and reviewed by the author.

torch>=2.14 enforces a C++20 minimum in its ATen/torch headers
(pytorch/pytorch#178150): including any ATen header under an older
standard trips '#error C++20 or later compatible compiler is required
to use ATen'.

mamba's setup.py hardcodes -std=c++17 for both the cxx and nvcc flag
lists. torch's cpp_extension only appends its default -std=c++20 when
the extension passes no -std= flag, so the pinned c++17 wins and the
host compile of selective_scan.cpp fails against torch 2.14 nightly.

Bump the pinned standard to c++20 (CUDA and HIP branches). c++20 is
already required by the toolchains this codebase targets (CUDA 12.x /
Blackwell sm_121), so this is safe on stable torch as well.

This PR was drafted with the assistance of an AI agent and reviewed by the author.
@Chessing234

Copy link
Copy Markdown
Contributor

c++20 for torch 2.14 makes sense. worth calling out in the readme that older nvcc toolchains might need a bump.

@janbernloehr

Copy link
Copy Markdown

Would it be preferable to remove the explicit -std= flags instead of changing them to C++20?
torch.utils.cpp_extension.BuildExtension already selects the minimum compatible standard when none is provided: older PyTorch versions add C++17, while current PyTorch adds C++20. Keeping an explicit flag overrides that version-aware behavior.
This also retains compatibility with older PyTorch/toolchain combinations.

@e271p314

e271p314 commented Sep 5, 2026

Copy link
Copy Markdown

checkout setup.py in causal-conv1d repo, the code simply does not state -std=c++XX, I assume just remove the hard coded -std=c++17

stmcgovern added a commit to stmcgovern/mamba that referenced this pull request Sep 10, 2026
Rewrite selective_scan.cpp and the CUDA kernel headers to use only
torch/csrc/stable/, torch/headeronly/ and cuda_runtime.h. No c10/, ATen/
or torch/csrc/api/ includes remain, so the extension no longer depends on
libtorch's C++ symbols and stays loadable across torch versions.

C++:
- Replace PYBIND11_MODULE with STABLE_TORCH_LIBRARY + TORCH_BOX
- Replace at::Tensor with torch::stable::Tensor, passed by const-ref
- Replace c10::cuda::CUDAGuard/CUDAStream with the stable accelerator API
- Replace AT_ERROR/TORCH_CHECK with STD_TORCH_CHECK
- Replace C10_CUDA_CHECK with STD_CUDA_CHECK
- Replace ATen/cuda/Atomic.cuh with an inline gpuAtomicAdd
- Replace at::Half/at::BFloat16 with their torch::headeronly equivalents
- Use a std::conj replacement for complex conj in device code

Python:
- Dispatch through torch.ops.selective_scan.fwd/bwd, cached at import,
  since the kernel is now registered as a dispatcher op rather than
  exposed as pybind11 module functions

Build:
- Add -DTORCH_STABLE_ONLY to the host compile and -DUSE_CUDA to nvcc
- Link with --as-needed

The -std= flags are deliberately left at their existing values. This
translation unit no longer includes ATen, so ATen's "C++20 or later
compatible compiler is required" guard does not apply to it, and the
standard question is being settled separately in
state-spaces#1000.

Closes state-spaces#813

Verified against torch 2.15.0a0+git560a746 / CUDA 12.9 on an H100:
builds clean, and the suite reports 672 passed with a failure and error
set identical to main's baseline on the same machine. The built extension
has no undefined at::, c10:: or torch:: C++ symbols; all 73 torch symbols
it imports are C shim entry points.

torch.compile behaviour is unchanged from main: the default path
(fullgraph=False) works on both, and fullgraph=True fails on both -- on
main because the pybind11 call cannot be traced, here because the op has
no registered fake implementation. Making fullgraph=True work is a new
capability the dispatcher registration makes possible, and is left to a
follow-up rather than folded in here.
stmcgovern added a commit to stmcgovern/mamba that referenced this pull request Sep 10, 2026
Rewrite selective_scan.cpp and the CUDA kernel headers to use only
torch/csrc/stable/, torch/headeronly/ and cuda_runtime.h. No c10/, ATen/
or torch/csrc/api/ includes remain, so the extension no longer depends on
libtorch's C++ symbols and stays loadable across torch versions.

C++:
- Replace PYBIND11_MODULE with STABLE_TORCH_LIBRARY + TORCH_BOX
- Replace at::Tensor with torch::stable::Tensor, passed by const-ref
- Replace c10::cuda::CUDAGuard/CUDAStream with the stable accelerator API
- Replace AT_ERROR/TORCH_CHECK with STD_TORCH_CHECK
- Replace C10_CUDA_CHECK with STD_CUDA_CHECK
- Replace ATen/cuda/Atomic.cuh with an inline gpuAtomicAdd
- Replace at::Half/at::BFloat16 with their torch::headeronly equivalents
- Use a std::conj replacement for complex conj in device code

Python:
- Dispatch through torch.ops.selective_scan.fwd/bwd, cached at import,
  since the kernel is now registered as a dispatcher op rather than
  exposed as pybind11 module functions

Build:
- Add -DTORCH_STABLE_ONLY to the host compile and -DUSE_CUDA to nvcc
- Link with --as-needed

The -std= flags are deliberately left at their existing values. This
translation unit no longer includes ATen, so ATen's "C++20 or later
compatible compiler is required" guard does not apply to it, and the
standard question is being settled separately in
state-spaces#1000.

Closes state-spaces#813

Verified against torch 2.15.0a0+git560a746 / CUDA 12.9 on an H100:
builds clean, and the suite reports 672 passed with a failure and error
set identical to main's baseline on the same machine. The built extension
has no undefined at::, c10:: or torch:: C++ symbols; all 73 torch symbols
it imports are C shim entry points.

torch.compile behaviour is unchanged from main: the default path
(fullgraph=False) works on both, and fullgraph=True fails on both -- on
main because the pybind11 call cannot be traced, here because the op has
no registered fake implementation. Making fullgraph=True work is a new
capability the dispatcher registration makes possible, and is left to a
follow-up rather than folded in here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants