backend/utils/llm_utils.py:15-55 is the kind of code I usually expect to be a mess, and isn't. The function correctly handles three independently tricky cases:
- A single streaming token containing both
<think> and </think> (the partition order on lines 26 and 40 is deliberate and necessary).
- A provider that emits a closing
</think> without an opening one (line 28-31 — clear token_join and emit "" to the callback so the UI can drop already-streamed reasoning).
- Content split across multiple tokens where the tag boundary falls mid-stream.
Two things I appreciated:
- The function's return value
is_thinking plus the explicit token_join: List[str] argument keeps state out of the function — the caller in call_llm_for_system_prompt (line 116) owns both. That makes it trivially unit-testable without mocking anything.
- The "treat everything accumulated so far as reasoning and clear it" branch comes with a callback nudge to the UI (
callback("")). That's exactly the kind of correctness-for-the-user behaviour that tends to get dropped in the rush to ship streaming.
The accompanying check on line 124 (if not result and content_tokens_seen > 0: logger.warning(...)) is a nice operational touch — it gives you a single log line that explains "the user got an empty response because everything matched <think>…</think>", which is exactly the kind of diagnostic you want when investigating "the model returned nothing" tickets.
Filing because clean streaming-tag handling is rare and this one is worth pointing at as a reference for similar code elsewhere in the codebase.
backend/utils/llm_utils.py:15-55is the kind of code I usually expect to be a mess, and isn't. The function correctly handles three independently tricky cases:<think>and</think>(the partition order on lines 26 and 40 is deliberate and necessary).</think>without an opening one (line 28-31 — cleartoken_joinand emit""to the callback so the UI can drop already-streamed reasoning).Two things I appreciated:
is_thinkingplus the explicittoken_join: List[str]argument keeps state out of the function — the caller incall_llm_for_system_prompt(line 116) owns both. That makes it trivially unit-testable without mocking anything.callback("")). That's exactly the kind of correctness-for-the-user behaviour that tends to get dropped in the rush to ship streaming.The accompanying check on line 124 (
if not result and content_tokens_seen > 0: logger.warning(...)) is a nice operational touch — it gives you a single log line that explains "the user got an empty response because everything matched<think>…</think>", which is exactly the kind of diagnostic you want when investigating "the model returned nothing" tickets.Filing because clean streaming-tag handling is rare and this one is worth pointing at as a reference for similar code elsewhere in the codebase.