Skip to content

fix(proxy): keep tool_use stop_reason when the stream ends with [DONE] - #238

Closed
pcbeingused333 wants to merge 1 commit into
BlockRunAI:mainfrom
pcbeingused333:c0555-done-stop-reason
Closed

pcbeingused333 wants to merge 1 commit into
BlockRunAI:mainfrom
pcbeingused333:c0555-done-stop-reason

Conversation

@pcbeingused333

Copy link
Copy Markdown

Fixes the streaming proxy ending every OpenAI-format response with stop_reason: end_turn, even when the model asked for a tool call.

Cause

SSETranslator.processChunk handles finish_reason by emitting a message_delta with the mapped stop reason (tool_calls -> tool_use). When the upstream then sends [DONE], it emitted a second message_delta with a hard-coded end_turn. Anthropic-format clients take the last message_delta, so tool_use was overwritten and the tool call was treated as a finished turn.

Receipt (event sequence produced on main for an upstream stream ending in finish_reason: "tool_calls" followed by [DONE], as asserted by the new test):

event: message_delta  {"delta":{"stop_reason":"tool_use",...}}
event: message_delta  {"delta":{"stop_reason":"end_turn",...}}
event: message_stop

Fix

Track whether a stop reason was already emitted and, on [DONE], only send the fallback end_turn message_delta when none was. message_stop is still always sent. The case where [DONE] arrives with no finish_reason keeps the old fallback.

Tests

New test/sse-translator.local.mjs (added to npm test): tool_calls finish keeps a single tool_use; normal stop gives a single end_turn; [DONE] alone falls back to end_turn. Two of the three fail on main; all pass with the change.


This PR was prepared with the help of an AI assistant, and the author is responsible for the change.

The OpenAI->Anthropic SSE translator emitted a message_delta with the real
stop_reason on finish_reason, then a second one with a hard-coded end_turn on
[DONE], overwriting tool_use. Skip the fallback message_delta when a stop
reason was already sent, and add regression tests.
VickyXAI pushed a commit that referenced this pull request Nov 3, 2026
src/proxy/sse-translator.ts (OpenAI-to-Anthropic SSE translation) has
never been imported by any file in the repo's history. The proxy pipes
the gateway's stream and reads usage from message_delta itself, so the
class only drew fixes that had no runtime effect (PR #238).
@VickyXAI

VickyXAI commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Thanks for the careful write-up and the tests. The bug you found in SSETranslator is real, but it turns out that class was never wired in: no file in the repo imports src/proxy/sse-translator.ts, in any commit of its history. The proxy pipes the gateway's stream through as-is and reads usage from its message_delta events itself, so the double message_delta could not reach a client.

Rather than fix code that never runs, we removed the file in 1c108dd, so I'm closing this PR.

One note in case you take on similar work: keeping the message_delta sent at finish_reason would also have dropped the trailing usage chunk that OpenAI-style streams send after it. Emitting the stop reason once, at [DONE], keeps both the stop reason and the final usage.

@VickyXAI VickyXAI closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants