[BUG] Missing vLLM dependency

[Uxito-Ada]

Describe the bug

I found vllm is not declared in pyproject.toml default dependencies or optional extras, while telefuser.ops.fp8_gemm.enable_fp8_gemm() etc. logics depend on vllm, where the error is like the below:

To Reproduce

calling optimizations on models involving vllm like telefuser.ops.fp8_gemm.enable_fp8_gemm

Expected behavior

append vllm to dependencies, or remove relevant codes if vllm is too heavy as a dependency

Environment

• OS: [e.g. Ubuntu 22.04]
• Python version: [e.g. 3.10.12]
• PyTorch version: [e.g. 2.9.1]
• CUDA version: [e.g. 12.4]
• GPU: NVIDIA A100
• TeleFuser version: [e.g. 0.1.0]

Error logs

raceback (most recent call last):
  File "/root/heyang/profile_wan21_bf16_fp8_fp4.py", line 103, in apply_mode
    enable_fp8_gemm(
  File "/root/heyang/TeleFuser/telefuser/ops/fp8_gemm.py", line 347, in enable_fp8_gemm
    _recurse("", model)
  File "/root/heyang/TeleFuser/telefuser/ops/fp8_gemm.py", line 338, in _recurse
    fp8_mod = FP8Linear.from_linear(child, options=options)
  File "/root/heyang/TeleFuser/telefuser/ops/fp8_gemm.py", line 131, in from_linear
    return cls(linear, options=options)
  File "/root/heyang/TeleFuser/telefuser/ops/fp8_gemm.py", line 97, in __init__
    from vllm.model_executor.layers.quantization.utils.quant_utils import (
ModuleNotFoundError: No module named 'vllm'

Additional context

No response

[lzx1413]

Thanks for raising this.

We don't plan to add vLLM as a hard dependency of TeleFuser. The FP8 GEMM path currently depends on vLLM custom ops, but vLLM is tightly coupled with specific PyTorch/CUDA versions. Making it a default dependency would make the base installation fragile and could force unwanted
PyTorch upgrades or downgrades for users who do not use this path.

For now, users who need FP8 GEMM should install a vLLM build that matches their own PyTorch/CUDA environment.

Going forward, we plan to consolidate these FP8 GEMM kernels into tf-kernel, so TeleFuser can depend on a lighter kernel package instead of the full vLLM package. Once that is ready, TeleFuser will prefer the tf-kernel backend and keep the dependency surface smaller and easier
to manage.