Request for improving /v1/responses soft-failure diagnostics and pre-output failover for official Codex OAuth accounts

[youngchan919]

Summary

/v1/responses sometimes returns:

{"error":{"code":"responses_unsupported","message":"no healthy responses account available","type":"invalid_request_error"}}

This happens even when the active official OpenAI/Codex OAuth account still has remaining quota and is marked active/
supporting responses.

Environment

• OS: Windows + WSL
• Client: Codex CLI
• ai-gate mode: local router / thin gateway
• Codex config:
  • base_url = "http://127.0.0.1:6789/ai-router/api"
  • wire_api = "responses"
• Upstream proxy: Clash Verge / mihomo
• Account setup:
  • one official OpenAI/Codex OAuth imported account
  • optional OpenAI-compatible fallback accounts
• Model: gpt-5.5
• Request type: streaming /v1/responses

Observed Behavior

The router intermittently returns responses_unsupported/no healthy responses account available.

From the local usage_events table, the official OAuth account shows many soft_failed events with zero tokens, often
after long latency.

Example aggregate from a pre-restore DB snapshot:

account_id=1 openai-official responses gpt-5.5 completed 595 avg_latency≈17s max≈260s
account_id=1 openai-official responses gpt-5.5 soft_failed 203 avg_latency≈71s max≈907s

Recent examples:

status=soft_failed input_tokens=0 output_tokens=0 latency≈34s
status=soft_failed input_tokens=0 output_tokens=0 latency≈24s
status=soft_failed input_tokens=0 output_tokens=0 latency≈46s

The account was not disabled, not invalid, not locked, and not obviously exhausted. Usage snapshot still showed
remaining quota.

Why This Is Hard To Diagnose

usage_events.status = soft_failed is too coarse. It is not clear whether the failure occurred at:

• session refresh / ensureOfficialAccountSession
• upstream request creation
• upstream HTTP status
• streaming read
• EOF before response.completed
• timeout
• proxy/network transport error
• official Codex backend error

The final client-facing error responses_unsupported also seems misleading when the account does support /responses;
the actual problem appears to be that no healthy candidate survived routing/failover.

Expected Behavior

1. If an account supports /responses but the upstream stream/request fails, the returned error should distinguish this
   from unsupported responses.
2. usage_events or another local diagnostic table/log should include failure stage and compact error reason.
3. If failure happens before any response bytes are streamed to the client, ai-gate should retry/fail over more
   robustly.
4. If fallback accounts are available, the router should log from-account/to-account failover decisions clearly.

Suggested Improvements

Diagnostics

Add structured fields to usage/logs, for example:

failure_stage = ensure_session | upstream_request | upstream_status | read_stream
error_class = soft | rate_limit | capacity | hard
error_reason = eof | timeout | stream_closed_before_response_completed | http_429 | http_403 | upstream_5xx | unknown
from_account_id
failover_succeeded
request_input_token_estimate or request_body_size

Behavior

For official Codex OAuth /responses streaming:

• retry EOF/timeout/transport errors with short backoff before returning to Codex
• if no bytes have been written to the client, attempt failover to the next healthy /responses account
• consider a temporary circuit breaker for accounts with repeated soft_failed events
• return a more accurate error code than responses_unsupported when the account supports /responses but all candidates
  failed

Notes

I understand ai-gate is intentionally a thin gateway and cannot safely replay a stream after partial output has
already been sent to the client. The main request here is better pre-output retry/failover and better observability
for soft_failed, so users can distinguish quota, upstream instability, stream EOF, timeout, and routing problems.