[Needs review] Update openai provider types - #451
Draft
github-actions[bot] wants to merge 1 commit into
Draft
github-actions[bot] wants to merge 1 commit into
github-actions[bot] wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated update of Lingua provider types.
Provider:
openaiPublication mode:
needs-review draftContinuation required
This draft preserves the automation's partial work. One or more validation or advisory checks did not complete successfully; continue from this branch instead of restarting the provider update.
Feedback: comment
/bt goodor/bt badto log review feedback to the Braintrust trace.Human decisions required
The agent completed every unblocked item. These decisions require human input before the update can be finished:
model enum membergpt-5.5(specs/openai/openapi.yml:62019 and Beta mirror :96172)(openai-gpt-5-5-reasoning-effort-family)gpt-5.5accept — i.e. whichEffortFamilyarm shouldreasoning_effort_family_for_modelreturn for point release 5? Concretely: doesgpt-5.5supportxhigh(matching gpt-5.4) and does it supportmax(introduced with gpt-5.6)? Secondary: should the same change also close the adjacent hole for non-codexgpt-5.3, which falls through identically today?gpt-5.5to two model enums (specs/openai/openapi.yml:62019 and :96172) and nothing else — it contains no per-model reasoning-effort enum, so the supported set is not derivable from this update. In crates/lingua/src/providers/openai/capabilities.rs:188-217,reasoning_effort_family_for_modeldispatches on the point-release digit fromgpt_5_point_release_suffix(:182-186) and has arms for 6, 4, 2, 3-codex, 1-codex, and 1, but none for 5. The fallback at :208-212 cannot catch it becausemodel == "gpt-5"is false andmodel.starts_with("gpt-5-")fails on the.separator. The predicate therefore returnsNone, andclamp_reasoning_effort_for_modelat :223-226 early-returns the effort unclamped, so a synthesizedReasoningEffort::Maxreaches adapter.rs:573 and responses_adapter.rs:1045 verbatim. The omission is invisible because the clamp test table at :519-570 enumerates 5.1, 5.2, 5.2-codex, 5.3-codex, 5.4, 5.6-terra, 5.6-luna, 5, 5-mini, and 5-nano but not 5.5. Sibling arms bracket the answer without settling it: 5.1 -> NoneLowMediumHigh, 5.2 and 5.4 -> NoneLowMediumHighXhigh, 5.6 -> NoneLowMediumHighXhighMax. Corroborating signal thatmaxarrived at 5.6 and not earlier:supports_prompt_cache_breakpoint(:78-95) gates onmajor.minor >= (5,6). Notegpt-5.5is already referenced elsewhere in the tree for unrelated predicates — capabilities.rs:465 and crates/braintrust-llm-router/src/catalog/spec.rs:91,96 — so only the effort family is missing.EffortFamily::NoneLowMediumHighXhighfor point release 5, matching gpt-5.4. Rationale: the enum ordersgpt-5.5immediately below the 5.6 family,xhighhas been present continuously since 5.2, andmaxappears to be a 5.6-and-later capability on the same boundary thatsupports_prompt_cache_breakpointalready encodes as>= (5,6). Implement it by extending the existingstarts_with('4') || starts_with('2')arm at :194-196 to include'5'so the dotted anddatabricks-gpt-5-5dashed forms are both covered by the shared suffix helper. Please confirm against the OpenAI model documentation before merging — this recommendation is an inference from sibling versions, not a fact stated by the specification.EffortFamily::NoneLowMediumHighXhighMax(treat 5.5 like 5.6). Tradeoff: safe ifgpt-5.5does supportmax, because nothing is silently downgraded; but if it does not, Lingua emitsreasoning.effort: "max"and the request fails at the API with a 400 that surfaces to the caller as a provider error rather than as a Lingua mapping decision. Fails loudly rather than silently, which is the better failure mode of the two wrong answers.EffortFamily::NoneLowMediumHighXhigh(treat 5.5 like 5.4) — the recommendation. Tradeoff: ifgpt-5.5actually supportsmax, a caller requestingmaxsilently receivesxhigh. This is a silent semantic downgrade, which AGENTS.md:19 disfavours, and it is invisible in the response.gpt-5.5unmatched, i.e. accept today's behavior and document it. Tradeoff: zero code change and exact user-supplied efforts pass through untouched, but it is inconsistent with every other supported point release, and a synthesized cross-provider effort (for example Anthropic thinking mapped tomax) would reach the API unvalidated. This also leaves the identical non-codexgpt-5.3hole in place.gpt-5.5. Recommended as a follow-up, not as part of this change.crates/lingua/src/providers/openai/capabilities.rscrates/lingua/src/providers/openai/adapter.rscrates/lingua/src/providers/openai/responses_adapter.rscrates/braintrust-llm-router/src/catalog/spec.rscargo test -p lingua --lib providers::openai::capabilities::tests::test_clamp_reasoning_effort_for_modelcargo test -p lingua --lib providers::openai::capabilities::testscargo test -p lingua --lib providers::openaicargo test -p braintrust-llm-router --lib catalog::spec::testsmake test-payloadsmake typed-boundary-checkProvider-only changes
These native wire features are accepted and passed through unchanged, but cross-provider transformation is intentionally unsupported:
FunctionToolCallOutput.namespace / FunctionCallOutputItemParam.namespace (and Beta mirrors)(openai-function-call-output-namespace)namespaceunchanged (generated.rs:6094), and native request/response validation and format detection recognize a function_call_output item carrying it. An unmodified same-format OpenAI request or response containingnamespacetakes the byte-preserving passthrough path atcrates/lingua/src/processing/transform.rs:462-464and is returned byte-identical. A cross-provider transform of a function_call_output item carrying a non-emptynamespacereturns an explicitConvertError::UnsupportedMappingnaming the field and item kind, raised before the field can be dropped or coerced. No universal field, no provider-options marker, no opaque replay carrier, and no cross-provider mapping is added.namespaceis a provider-defined toolset/registry scope, not portable message content. Three independent signals: the specification groups it withcaller/ToolCallCallerParam, explicitly described as the execution context that produced the tool call; the param-side constraint^[a-zA-Z0-9_-]+$with maxLength 64 describes an opaque provider-side registry key rather than user-meaningful content; and no other provider has an equivalent scope with compatible semantics. This is provider-defined harness state inside the provider-only boundary. Note the contrast withnamein the sibling change: generic caller-defined function-tool identity remains portable and is mapped — only the provider-scoped namespace is provider-only, and it is provider-only because of what it means, not because OpenAI uses a particular wire envelope. Provider-only is a completed scope decision, so this item is unblocked and is not listed as a human blocker.Validation
./pipelines/generate-provider-types.sh openai: successmake generate-types PROVIDER=openai: successmake lingua-wasm: successresponsesFunctionCallOutputToolNameParammake test-payloads: failuremake typed-boundary-check: successcargo test -p coverage-report --test cross_provider_test cross_provider_transformations_have_no_unexpected_failures: failureOnly safely scoped patches with a locally archived binary diff are published. Any incomplete AI phase or failed deterministic check produces a needs-review draft so work can continue from the PR branch.