Skip to content

ChatRequest reasoning cannot express max_tokens — API accepts it, SDK silently strips it #693

Description

@daveycodez

Summary

The chat-completions HTTP API accepts a per-request reasoning token budget:

{ "reasoning": { "max_tokens": 2000 } }

but the SDK cannot send it. ChatRequestReasoning (models/chatrequest.ts) only models effort and summary:

export type ChatRequestReasoning = {
    effort?: ChatRequestEffort | null | undefined;
    summary?: ChatReasoningSummaryVerbosityEnum | null | undefined;
};

and because ChatRequestReasoning$outboundSchema is a plain z.object(...), an unknown key passed by a caller (e.g. via a type cast from JS/TS) is silently stripped before the request is serialized — the request goes out with no reasoning config at all, with no error or warning. That silent-drop behavior is the sharp edge: a caller who casts around the type believes they are testing a capped configuration while actually measuring the default.

Notably the SDK already models the right shape elsewhere — ReasoningConfig (models/reasoningconfig.ts) has maxTokens with a max_tokens wire mapping — it just isn't referenced by ChatRequest.

Ask

Add maxTokens to ChatRequestReasoning (serializing to max_tokens), or reference the existing ReasoningConfig from ChatRequest, so the SDK matches the documented API surface.

Use case

Effort tiers are too coarse for cost control on budget-based thinking models (Gemini flash family, Anthropic). On a large-batch classification workload the thinking tail scales with batch size — a per-request budget like max_tokens: batchItems * K caps the tail proportionally, which no fixed effort tier can express. Today the only SDK-expressible step below our current tier is a whole effort level down, which measurably degrades task quality.

Tested on @openrouter/sdk 0.13.20.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions