Refactor/vllm parser detection actor#112
Conversation
… actor Now that llama_server does its own parsing internally and llama_cpp/transformers are gone, tool/reasoning parser-name detection is vLLM-only and no longer belongs in loader-agnostic preflight. Moves detection into a new modelship/infer/vllm/parsing/ package consumed directly by init_serving_chat, sourcing the chat template from the loaded engine's own tokenizer instead of a driver-side disk read. Also drops the reasoning-suppression probe entirely: vLLM's per-request is_reasoning_end already handles the suppression boundary correctly, so the deploy-time downgrade was redundant and, in one direction, less correct.
…er module Completes the parser-detection migration: drops driver-side resolve_all_tool_parsers/resolve_all_reasoning_parsers and their PrivateAttr handoff, and repoints init_serving_chat at the new modelship.infer.vllm.parsing.detect resolvers sourced from the engine's own tokenizer.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the chat-template parser-name detection for the vLLM loader by moving the resolution logic from the driver-side pre-flight check to run directly inside the vLLM actor. This simplifies the configuration models and ensures that the native vLLM parsers are resolved dynamically during initialization. A critical issue was identified in modelship/infer/vllm/vllm_infer.py where self.engine.get_tokenizer() is called synchronously, but it is an asynchronous method and must be awaited to avoid an AttributeError.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
1 participant
No description provided.