Repository navigation
feat(model): capability-driven model management and unified provider invocation layer - #3278
Closed
Greenplumwine wants to merge 89 commits into
Closed
Greenplumwine wants to merge 89 commits into
Greenplumwine wants to merge 89 commits into
Conversation
Co-Authored-By: Claude <noreply@anthropic.com>
…ng levels, unified invocation core Backend implementation of the model-management refactor + unified invocation layer (task 09-08-model-mgmt-unified-invocation; ADRs 0001-0003, CONTEXT.md): - Capabilities: ProviderInfo.EffectiveCapabilities() synthesizes provider-level capability declarations (thinking levels, protocol family, per-type shards); ModelProviderDTO serves capabilities + extra_fields (design §5.1) - Parameter sharding: ModelParameters.Chat shard (context window, thinking level/selection, input modalities) with dual-read accessors over the deprecated top-level fields — zero DDL, legacy rows keep working (design §5.2) - Thinking levels: ChatOptions/AgentConfig/CustomAgentConfig/QA-request ThinkingLevel fields + ResolveThinkingLevel chain (session > agent > model > provider default, SelectedLevels-governed fallback, design §5.4) - Wire mapping: OpenAI/Azure reasoning_effort passthrough (gated to o-series/GPT-5); Anthropic thinking.budget_tokens with max_tokens ceiling enforcement (design §4.3, continuous-vendor收编) - Unified error model: ProviderError taxonomy with mandatory classification at every response path + provider-level retry with backoff (retryable: 429/5xx/ network/timeout; fast-fail: auth/context/policy, design §4.6) - Model catalog: WeKnora-own models.json schema (15 providers / 857 models converted from the models.dev reference), LOCAL-embedded loader with GITHUB/CNB enum hooks, GET /models/catalog + POST /models/remote-catalog backend probe (SSRF-gated, rate-limited, key never logged, design §5.10) Deferred: RPM/TPM rate limiting (no token-bucket infra exists), fallback chain (D8), Qwen/Gemini numeric level mapping (pending vendor docs). Env-dependent pre-existing failure: TestSkillPythonVerifier venv check.
…log prefill Frontend half of the model-management refactor (task 09-08-model-mgmt- unified-invocation): - Vendor-knowledge mirror deleted: utils/thinkingControl.ts + test removed (zero residual references); the wire-format dropdown and its supporting state/styles removed from ModelEditorDialog (D3) - Thinking controls rendered from the provider capability declaration: on/off switch (hidden when can_disable=false), level multi-select + default select fed by capabilities.chat.thinking.supported_levels; unsupported models show the degradation hint instead (design §5.4) - Catalog prefill per the value chain (ADR 0001/D1): new or changed model id probes POST /models/remote-catalog then reads GET /models/catalog (exact → normalized id); saved values are never overwritten, manual edits win, and source badges mark catalog/remote origin - Model id input becomes a searchable select backed by the remote-catalog probe (debounced, falls back to free input) - 13-provider hardcoded fallback collapsed to the generic custom-vendor entry - Saving writes the parameters.chat shard (thinking_level, selected_levels, input_modalities); legacy flat fields no longer written for chat — the backend dual-read accessors keep old rows working (design §8) - i18n: new keys across zh-CN/en-US/ko-KR/ru-RU/ja-JP (audit requires all locales share keys); legacy thinkingControl keys removed - ModelSettings tab counts and dropdown menu untouched (5.6.6) Verified: vue-tsc type-check, check-i18n audit, and the full frontend test suite pass.
…ultimodal retry, lint Quality-check fixes over the model-management branch: - POST /models/remote-catalog was registered inside the /models apiKeyGroup and inherited its manage_models/full-access API-key policy, violating the design constraint that the key-carrying probe serves JWT only. Moved out of the group (APIKeyGate fails closed → API keys 403), dropped the ApiKeyAuth swagger annotation, and added a policy regression assertion. - The multimodal-degradation retry in chatOnce/chatStreamOnce re-ran shapedRequest with the raw opts, bypassing resolveThinkingLevelOpts — a second attempt could emit an unvalidated reasoning_effort. Resolution now happens once per attempt. - 29 diff-scoped lint findings fixed (errcheck, lll, gofumpt, revive: catalog type-name stutters, const-block comments, builtin shadowing). golangci-lint --new-from-rev=main: 0 issues. All targeted tests green.
…den wire baseline (P1a/P1b) - invoke entry (Chat/ChatStream/Embed/Rerank/Transcribe/List) with langfuse middleware, entry-level llm_debug records (v1-format), image preprocessing, multimodal degradation retry, thinking-level entry fold - single HTTP executor: model-level concurrency, SSRF gate without vendor exceptions, retry/backoff, SSE+NDJSON demux, RPM/TPM placeholder slot - adapters dispatch by provider name (openai family incl. weknoracloud HMAC, anthropic native Messages, ollama native /api/chat + /api/tags list facet); model-conditional shaping via shapeFor/thinkingFor - typed Role enum with //exhaustive:enforce switches + golangci exhaustive (explicit-exhaustive-switch); JSON on-disk representation unchanged - 32 golden wire scenarios reconciled byte-for-byte against v1 - provider capability shards + Credentials spec DTO for /models/providers
…vlm path (P1c/P1d) - ~40 callers migrated to invoke.Chat/ChatStream: agent engine (think/finalize/observe/compaction/token estimator), chat_pipeline, session QA, memory/*, message_suggestion, extract, graph, knowledge auto-tag/process, wiki ingest, evaluation, handlers, modelcontext, container assembly - interfaces.ModelService drops GetChatModel/GetVLMModel; session title, agent image describer, temporary-document VLM go through the shared buildModelConfigByID + invoke.Chat seam - single shared constructor BuildModelConfig: WeKnoraCloud tenant credential fallback, legacy local-record ollama mapping (with /v1 strip), extra_config.remote_model_name wire-name override - budget unified on ChatOptions.MaxCompletionTokens; legacy summary.max_tokens migrated at config load (backfillSummaryBudget, outside the prompt-templates gate); tenant_skill installer deduped onto the shared helper - deleted internal/models/chat and internal/models/vlm (strangler complete for chat+vlm; embedding/rerank/asr follow in P2/P3)
…pec forms (P-FE) - extract ThinkingControls.vue shared component (dual mode: single level selector / multi-level allow-set, badge slot for catalog prefill) - agent editor: static thinking toggle replaced by ThinkingControls under the model selector; level cleared with toast when switching to a model that does not support it; unsupported models greyed out - chat page: ThinkingControls next to the model picker (session-scoped, in-memory only with reset); thinking_level now sent on the QA stream (closes the v1-deferred send path) - model editor: credential inputs rendered dynamically from the provider Credentials spec (edit shows Configured state + CredentialResource card, empty submit = no change) - i18n: locale keys audited across zh-CN/en-US/ja-JP/ko-KR/ru-RU
P2-a/P2-b strangler (task 09-08): adapters produce invoke.Request for the embedding facet; the executor stays the only network egress. - invoke.Embed: record-derived knobs on EmbeddingOptions (truncate budget, dimension-override gate); entry-level per-input fan-out via the optional SingleInputEmbedder (volcengine Ark multimodal yields one vector per request); llm_debug + langfuse middleware ported from the v1 decorator stack (exact v1 record format, v1 input shapes). - adapters/embedding.go: shared openai-shape build/parse (openai/zhipu/ generic family) with per-vendor deltas — azure deployment URL + Api-Key, nvidia input_type=passage (v1 ctx variant was dead code), jina truncate boolean (newly registered, embedding-only), aliyun text/vision dual-path, volcengine multimodal, gemini native batchEmbedContents (ported ahead of P5; only generateContent chat remains doc-gated). - ollama /api/embed facet (SDK-equivalent wire; auto-pull retired per design §11); weknoracloud HMAC-signed /api/v1/embeddings with index integrity validation. - Every embedding Build sets the v1 60s timeout; registration lock #2 now covers 12 dual/single-facet providers (jina joins the registry). - Entry applyCustomHeaders gains the always-protected auth-header set (v1 reservedHeaderKeys minus content-type): review finding 3. - 12 golden wire baselines recorded from the v1 clients and replayed byte-for-byte through invoke.Embed; unit coverage for parse semantics, ollama SDK-equivalent wire, and auth-header protection.
…odels/embedding P2-c/P2-e strangler (task 09-08): the v1 client package (23 files) is deleted; every caller now rides the unified entry. - Embedder/EmbedderPooler caller seam moves to types/interfaces (method set unchanged; test fakes are mechanical renames). - service/embedding_client.go: invokeEmbedder over invoke.Embed — v1 empty-result 3x retry on Embed, count contract on BatchEmbed (review finding 4), BATCH_EMBED_SIZE sub-batch pooler through the process ants pool; concurrency gating rides the executor limiter (background-only, same semantics as the v1 innermost wrapper). - GetEmbeddingModel/GetEmbeddingModelForTenant rebuild through the shared constructor (BuildModelConfig): credential slots, WeKnoraCloud tenant fallback, legacy local-record mapping — v1 ConfigFromModel promise kept. - TestEmbeddingModel goes through BuildModelConfig + invoke.Embed with the form's EmbeddingParameters carried over (v1 production/test parity); dimension now reported from the returned vector (review findings 1-2). - aliyun multimodal base URL trims the trailing slash like the v1 constructor (review finding 6).
P3-a/P3-b strangler (task 09-08): adapters produce invoke.Request for the rerank and ASR facets; the executor stays the only network egress. - adapters/rerank.go: openai-shape fallback (truncate_prompt_tokens only via the Endpoint fold of extra_config — never sent by default, issue Tencent#2143) + per-vendor deltas: jina (return_documents), zhipu (full-endpoint base posted directly), nvidia (query/passages objects, rankings envelope, raw logit sigmoid-normalized like v1 normalizeNvidiaLogit), aliyun (DashScope envelope output.results, top_n=len(documents)), weknoracloud (HMAC-signed /api/v1/rerank). Composites per shard set: quad (chat+embedding+rerank+ASR), rerank/ASR triples, jina embed+rerank. - P3 ruling: lkeap (TC3) and volcengine (IAM) rerank stay on the v1 SDK clients until P5 — signature protocols are native-adapter work. Their adapters trim the rerank shard (P1a interim convention; the frontend DTO still reads the provider package) and the service layer branches them. - adapters/asr.go: one openai-shape multipart transcriptions facet covers every vendor (field set mirrors the v1 go-openai SDK writer; default audio.mp3; multipart Content-Type protected); verbose_json parse with v1 TrimSpace semantics; body rides replayable Request.Body. - invoke.Rerank/Transcribe gain the v1 decorator equivalents (langfuse generation shapes + llm_debug record formats); invokeFacet gains the design §6.2 lock-#3 facet guard (ErrUnsupportedType, never a panic). - 7 golden rerank baselines recorded from the v1 clients and replayed byte-for-byte (request side) with response-side assertions on the two envelope-sensitive vendors; ASR wire asserted semantically (random boundary).
P3-c/P3-d strangler (task 09-08): models/asr is deleted; models/rerank is trimmed to the two P5-deferred SDK vendors. - types/interfaces/asr.go hosts the caller seam (ASR/TranscriptionResult/ Segment) — the deleted package cannot; GetASRModel returns it. - service/rerank_client.go: invokeReranker (rerank.Reranker over invoke.Rerank; Document re-derived from input by index, equivalent to the vendor echo) + invokeASR over invoke.Transcribe. - GetRerankModel branches by provider: lkeap/volcengine keep the v1 SDK clients (TC3/IAM signatures — P5), everything else rides BuildModelConfig + invoke.Rerank; the trimmed models/rerank factory rejects any other provider loudly. checkRerankModelConnection mirrors the branch. - TestASRModel goes through BuildModelConfig + invoke.Transcribe (silent-WAV regression preserved). - models/rerank keeps only lkeap/volcengine clients + factory + decorators + transport; deletion completes in P5 with the signature adapters.
…AC signer to invoke, ollama up one level P4 清场批次 1/4。slices.go 泛型工具归 internal/utils（与同层通用工具同居）； WeKnoraCloud HMAC Sign 迁 invoke/signer.go（端点协议知识与 weknoracloud 适配器 同居调用核心包，docparser 转发路径同源复用）；ollama 子包升一级供模型管理 UI 独立使用（不进调用链）。零行为变更；TestSSRFSafeURL 失败为存量 fake-IP DNS 环境族。
… is executor semantics P4 清场批次 2/4。Redis 分布式信号量（Lua 原子准入 + 租约心跳 + fail-open）、 本地信号量、governor 单例与 RuntimeStats /system 暴露面全部原样迁入 invoke 包； executor.go 直调 GateNamedN，container/system_setting/handler 三处调用方改 invoke.*。零行为变更；handler 失败集仍为已知存量 1 项。
… dispatch deleted P4 清场批次 3/4。openai 家族（含 weknoracloud/volcengine 经 composite 提升）与 anthropic 补第五分面 BuildListRequest/ParseListResponse，URL 推导自 v1 catalog/remote.go 逐字迁移（parity 由 list_test 钉住）；invoke.List 入口接管 探测编排——SSRF 门、8s 探测超时（env 可调）、500 上限，执行器统一出口。 azure 部署模式 Build 时显式拒绝（DRIFT：v1 发注定 404 的请求后降级，理由文案 不同）；jina/weknoracloud 不实现 listing，与分片声明锁 #2 一致。 RemoteModel 补 v1 wire JSON 契约（id/display_name/owned_by；恒空的 meta 信封 删除——v1 从不填充，前端只读 id）。HTTP 契约不变。
…lds, usage chip Ruling batch (2026-09-14): Tencent#15 / D1 / azure-extra-fields / Tencent#14 FE / E9 UI. - Tencent#15: providerLogos registry gains the 'model' category — drop SVGs at assets/img/providers/{color,mono}/model/<provider>.svg and they light up in the provider select (color direct / mono masked, initial-badge fallback) and on the model-card badge (type icon stays the fallback). - D1: the editor drawer no longer closes on emit — the parent calls the exposed resetAfterSave() only after create/update resolves, so a failed save keeps every input on screen. - Azure extra fields: provider extraFields render as dynamic inputs bound to extra_config (defaults prefilled on provider switch, values merged into save payload and the connection-test request; lkeap region keeps its dedicated control). - app_secret for signed rerank (lkeap/volcengine) is required in label star + save validation, matching the rerank constructor contract. - Provider switch with no default URL for the type clears the base URL instead of keeping the previous vendor's host. - Tencent#14 FE: the type tab filter is pushed to the server (?type=), tab switches reload; full loads only refresh the shared model cache. - E9: the complete handler stores event usage on the assistant message and botmsg renders a tokens line (prompt → completion · total · cache hit) whenever usage exists.
…nd to end Report: uploading a document with parent-child chunking failed the embeddings batch insert with 'halfvec must have at least 1 dimension'. Root cause chain, three layers: - The two native DashScope embedding endpoints disagree on the response index field: text-embedding carries text_index (通用文本向量API文档), multimodal-embedding carries index (多模态向量API详情.md:219). The shared parser read only text_index, so for vision embedding models EVERY entry decoded to slot 0 — all vectors in a sub-batch overwrote one slot and the rest stayed nil (one real + N empty per batch of 5, matching the observed rows). - The sub-batch pooler validated count only, so empty vectors passed. - The postgres batch insert happily wrote 0-dimension rows, and halfvec rejects them, killing the whole statement. Fixes: the parser decodes both fields (typed pointer picks the one the wire sent) and hard-errors on unfilled slots (duplicate/missing index); the pooler rejects empty vectors as a sub-batch failure; the postgres batch save skips vector-less rows (the documented 'parents go into DB but NOT into the vector index' contract) with a warn instead of poisoning the statement. Tests pinned at all three layers.
…ame autofill Real-machine feedback (2026-09-14): - The remote model dropdown now filters by the edited model type client-side: server-side filtering only works for aliyun (capability codes), every other vendor's catalog carries no type metadata. Name heuristics: embedding/rerank/asr keep their families, chat/vllm exclude the non-chat families; creatable manual input stays as the escape hatch for anything misnamed. - Dropdown options render model ID as the primary text with the vendor display name as small gray secondary text (label keeps 'id · name' so search and the selected trigger are unchanged). - Picking a model auto-fills 显示名称 from the vendor's display name — manual edits win (prefill-source badge + manual-field tracking, same contract as the other prefilled fields).
The card row was align-items:flex-start (plus a 1px nudge on the badge), so the 36px logo sat on the title line while the body is two lines tall. Row now centers; the top-right action buttons pin themselves back to the top via align-self.
The custom two-part option layout never took effect under TDesign's options/slot interplay and the user prefers the plain form anyway — options now carry label = model ID. Type filtering and the display-name autofill (which reads the vendor display_name from data, not the dropdown text) are unchanged.
The ID-only simplification dropped BOTH the options prop and the slot, leaving the select with no items at all. Re-attach :options=remoteModelOptions (label = model ID); type filtering lives in the computed and is unaffected.
…lidates, gate was vapor
The 自定义输出维度 toggle and 向量维度 input were hard-locked for every
remote provider: defaultCapabilities synthesized EmbeddingCaps{} with
CanOverrideDimension=false and nothing ever set it true, making the
fully-wired dimensions machinery (every embedding adapter sends the
vendor dimension param) unreachable dead UI.
The toggle itself is the user's confirmation, and 测试连接 sends the
requested dimension and reports the observed one — a vendor that
ignores/rejects the param surfaces at probe time. Provider-level
false stays as an explicit escape hatch.
…ation Hydration intentionally skips the probe watch (one guard against per-field firing while the form fills), but nothing triggered the first probe afterwards — an edit-open model dropdown stayed empty until the user edited some field (e.g. cleared the model name) to wake the watch. Call probeRemoteModels() once after hydratingForm clears; the probe already carries edit-mode credential fallback (model_id) and the sequence guard dedupes any racing user-triggered probe.
…or buttons Ollama's think is a boolean wire param (adapter already passes opts.Thinking straight through), but the capability layer never declared it — the editor's thinking section was hidden for every ollama model. Declare Supported/CanDisable with an empty level set; ThinkingControls now renders toggle-only for level-less providers (levels mode shows the hint under the toggle; single mode drops the level select) instead of the old hint-with-no-toggle dead end. The session-level thinking panel stays hidden for boolean providers: the session API only carries thinking_level, which cannot express a boolean. Also flex-center the icon+text content of the ollama 刷新列表 and dimension-check buttons — a raw t-icon (svg, no text baseline) aligned against a bare text node rides the baseline and floats. The capability sanity test now accepts empty SupportedLevels as a legitimate boolean-thinking declaration while keeping the DefaultLevel ⊆ SupportedLevels invariant.
Merge origin/main (9e55f74, 56 upstream commits) into the unified invocation branch. Conflict resolution and porting decisions: - internal/models/chat/* (deleted here, upstream added anthropic tool-use + langfuse tool metadata): ported the anthropic tool-use feature into invoke/adapters/anthropic_tools.go (invoke.ToolDef / invoke.ToolCall shapes, SSE tool_stream assembly with unclosed-block drop), and the langfuse tool_names/mcp_catalog metadata into the entry-builtin startLangfuse (invoke/langfuse.go + tests). - internal/agent: took upstream's streamlined prompt policy (image requirement injection removed, sanitize orphaned tool results as escaped untrusted_tool_result, finalize reuses the live transcript); converted new upstream files (steer.go, tool_images.go) to invoke types; mcpCatalogDescriptionLen re-typed for the merged engine log. - internal/agent/mockchat_test.go: new test double porting main's mockChat onto the invoketest seam (chunk conversion keeps unset ResponseType as answer, emits non-Done tool calls). - openai stream bridge: a frame carrying finish_reason AND tool_calls no longer drops the calls (mixed-delta fall-through) — real bug the upstream MCP exposure integration test flushed out. - FE ModelEditorDialog: kept this branch's contract (credentials-spec rendering, emit-confirm + parent-driven close per D1, explicit validations); re-adopted upstream's connection-field invalidation watch; the inputModalities watch moved below the formData declaration (TDZ on mount, caught by upstream's new flow test). - FE chat: union of steer messaging (upstream) + per-request thinking level (this branch). Verification: go vet clean; diff-scoped golangci-lint 0 issues; full go test failure set identical to origin/main baseline (known fake-IP DNS SSRF + PythonVerifier env failures only); FE type-check clean, 894/894 tests, vite build ok.
Session-level thinking was a level-only override (the on/off toggle was a dead control: the QA API carried no thinking boolean). Rulings from real-machine feedback: - QA requests carry an optional thinking bool (nil = follow the agent or global default); knowledge-QA and agent paths apply it after the agent overrides fold in, so the panel toggle is a first-class per-request override. - The panel now shows the agent's own thinking setting as the initial state (enabled + configured level, falling back to the provider default_level when the agent has none or it is not offered by the selected model); untouched state sends nothing and the backend chain resolves as before. - The reset button is gone (the untouched state IS the reset); the trigger button lights up (brand color) whenever thinking is effectively on. - Boolean-thinking providers (ollama think, empty SupportedLevels) are un-hidden from the session panel — the bool now travels, so the panel renders a bare toggle there. - ThinkingControls loses the hideToggle prop (its only consumer needed the toggle hidden because the bool could not travel). Model/agent/session switches still clear the override (model switch drops a level the new model no longer offers, with the existing toast).
… list URLs
listURLFor only whitelisted /v1, /v1beta and /compatible-mode/v1 as
versioned suffixes, so an OpenAI-compatible base like Volcengine Ark's
/api/plan/v3 fell through to the bare-base branch and the probe hit
{base}/v1/models — a double-versioned path that 404s no matter what the
gateway serves (the frozen 'volcengine listing URL 404' audit finding,
reproduced here through the generic provider). Any path ending in a
bare v<digits> segment now appends /models directly; bare bases keep
the /v1/models convention. Also skip the remote-catalog probe while the
base URL is still empty (it could only ever answer 'base URL is
required' — pure log noise on every editor keystroke).
…-supply name The DashScope always-thinking predicate only matched the ZHIPU/-prefixed form (zhipu/glm-5.3), so a model record named glm-5.3 — the aliyun direct-supply spelling — fell through to the hybrid branch and the title-generation path's explicit thinking=false went out as enable_thinking=false, which the vendor rejects with 400 'enable_thinking ... restricted to True'. The field is now suppressed for any glm-5.3 spelling (bare, -flash, ZHIPU/-prefixed) and kimi-k3. The main conversation in the reported session was unaffected — the 400 came from the async title generation, which shares the session's title model.
The adapter requests extended thinking (budget_tokens from the resolved level) but the response side never delivered it: the stream bridge only handled text_delta frames, so thinking_delta — the carrier of the model's reasoning — was silently dropped, and the aggregate/non-stream paths ignored thinking content blocks. Real-machine symptom: with thinking explicitly enabled (session override on, level high) the UI never showed a thinking panel. thinking_delta frames now bridge to StreamKindThinking (entry maps them to ResponseTypeThinking), the SSE aggregate and the non-stream parser fold thinking blocks into ChatResponse.Thinking, and the llm_debug options line prints the thinking boolean — 'was the param requested at all' is the first question every no-thinking debug starts with, and ThinkingLevel alone could not answer it. Note: whether the Volcengine agent-plan Anthropic endpoint actually emits thinking blocks for deepseek models is vendor-side; with this fix whatever the vendor sends now reaches the UI.
Drop-in color SVGs for 26 providers under assets/img/providers/color/model/<provider>.svg — consumed by the providerLogos registry (import.meta.glob; mono/ falls back to color when a mono variant is absent).
CONTEXT.md and the three refactor ADRs are review material for this branch's design discussions, not deliverables — untrack them and ignore the paths so they stay on disk but out of the PR.
Contributor
Author
关于 golangci-lint CI 失败的说明本次 CI 失败由两个独立原因叠加，均不影响本 PR 的实际改动质量： 1. PR diff 超过 GitHub API 上限，action 的增量模式失效 2. 回退后报告的问题全部是存量基线，不在本 PR 改动范围内
本 PR 自身的 lint 状态： 后续：458 文件的 diff 对 reviewer 同样不友好（API 上限同样影响逐文件审阅），我们正在评估按依赖顺序拆分为多个 PR（统一调用核心 → 调用方迁移 → 前端）。欢迎对本 PR 的实际改动直接使用 diff-scoped lint 本地验证。 |
The workflow was added on the fork for local image builds and is not part of the refactor — untrack it and ignore the path so it stays out of this PR and future upstream-facing ones.
…role vocabulary
TextMessage("system", …)-style literals compile because Role is a typed
string and untyped constants convert implicitly — a v1 porting artifact
(v1 chat.Message.Role was a plain string with no constants). Typos in
literals compile too and fall through the adapters' exhaustive-enforced
switch defaults silently, so the constants turn a class of silent bugs
into compile errors. Mechanical sweep across the neutral layer
(callers + invoke's own seam tests); wire-layer protocol literals in
the adapters are untouched.
Three sites the mechanical pass missed: two multiline invoke.Message composites with aligned fields (handler/model.go, handler/session/ image_upload.go) and one multiline TextMessage call in the engine's empty-answer retry. The handler-layer types.Message literals (DB persistence role vocabulary, a plain string by design) stay.
… modalities The VLLM (vision) type is abolished: a vision model is a chat model that declares image input. Type axis = wire-protocol shape (chat / embedding / rerank / ASR, mirroring the invoke facets); input modality = capability axis within chat. Output modality is rejected as a primary key — whisper vs qwen-omni share audio-in/text-out but serve disjoint protocols (ADR 0004). Backend: - drop ModelTypeVLLM; wizard slot dispatch switches to explicit descriptor slots (LLM and multimodal slots share KnowledgeQA now); KB config assembly resolves the image-understanding model via VLMConfig.ModelID instead of a type scan - debug endpoint: image debug folds into the chat case, gated by GetSupportsVision with a 400 for image uploads to text-only models - create/update validate the closed type vocabulary and normalize chat InputModalities: text always present, values outside text/image/audio stripped (a catalog prefill must not resurrect banned values) - provider catalog drops per-type VLLM entries (all equaled the chat URL) Data migration 000096 (postgres) / 000017 (sqlite): VLLM rows become KnowledgeQA with text+image stamped — the type WAS the vision declaration, so this overrides an explicit text-only shard; other modalities (audio) are preserved. Irreversible by design; verified against all four legacy row shapes on a live jsonb schema and fresh sqlite databases. Frontend: type tabs 5→4 with vision badge derived from modalities; editor merges the vllm form (text checkbox fixed on, video removed, catalog prefill whitelist-filtered); slot pickers select by the 'vision' pseudo-type (KnowledgeQA ∧ supports_vision); debug drawer goes capability-driven; wkc vlm kind lands as KnowledgeQA + supports_vision with name-based dedup. client/cli: SDK type vocabulary converged; CLI help updated. Note: pre-000096 code reads the converged rows fine, so down is a no-op.
…ible endpoints 自定义 OpenAI 兼容接口（ProviderGeneric）不在 thinkingCapsFor 厂商表里， 落 default 返回 Supported=false，导致编辑器对自定义端点整块隐藏思考开关 与档位——而运行时走的是同一个 openai 适配器（reasoning_effort），会话级 思考早已可用。声明与运行时能力对齐：加入 standard 档位表（low/medium/ high，可关闭，默认 medium）。 用户可见后果：自定义端点的对话模型编辑器恢复思考配置区；ark plan 等 自定义协议网关（如 doubao-seed / deepseek-v4.1-flash）可在模型级配置 思考，与阿里云/火山引擎厂商条目行为一致。
…r dispatch Agent 直传图像时，附件管线产出的是 resource:// 句柄（本地资源表引用）， 此前 chatExecute 未做入口级解析，openai 线协议把 resource:// 原样发给厂商， 火山 Ark 直接 400："Only base64, http or https URLs are supported"。 按 design §6.6 的既定归宿补上入口级预处理（P1c 迁移项）：chatExecute 统一 把 resource:// / local:// / storage:// 引用经 LocalImageResolver 内联为 data URI 后再分发适配器；data:/http(s) 引用原样透传，解析失败的引用保持 原样让厂商错误保持响亮。ollama 适配器的同款私有判断收编为 invoke.IsApplicationStoredImage。anthropic/gemini 适配器目前完全没有图像 映射（静默丢图），属于另一个待补缺口，不在本修范围。 真机回归对应：智能体开启图片上传后，doubao-seed（视觉对话模型）会话直传 图片不再 400，图像以 data URI 到达厂商。
模型管理页的思考主开关此前在运行时零消费——会话/智能体层照常发送思考 参数（用户真机发现：模型关了思考，对话面板与智能体配置仍处于开启态）。 按用户拍板的语义落地"模型级开关 = 硬门"： - 后端：ModelConfig 透传记录的 Chat.ThinkingEnabled，foldChatOptions （Chat/ChatStream 单点）在开关为显式 false 时强制 Thinking=&false 并清空 档位——会话/智能体层要开也无效；nil（从未碰过编辑器开关的记录）不设门， 行为不变。显式 &false 对支持关闭参数的协议（如 dashscope enable_thinking=false）是比"省略"更强的禁用信号。 - FE 对话页：所选模型的 chat 分片 thinking_enabled === false 时，会话思考 触发按钮 + 面板整块消失（thinkingSupported 门控扩展），且已触碰的思考 状态不再随请求发送。 - FE 智能体编辑器：绑定模型主开关关闭时 caps 置 undefined，ThinkingControls 整块置灰并显示"当前模型不支持思考"（已存配置保留——智能体可换模型， 数据不删，运行时硬门兜底）。 测试：entry_seam 新增 TestFoldChatOptionsMasterSwitchOff（强制关、显式关、 免拷贝、nil 不设门四态）；全量 FE 893/893、tsc 0 错误。
并行会话收编：思考不可用（厂商未声明 / 模型主开关关闭）的 UX 从 "整块置灰 + 不支持说明"统一为"整块隐藏"——与会话面板消失语义一致； thinking.unsupportedHint 键五语言同步移除，智能体编辑器思考行随 agentThinkingAvailable 一并隐藏。已存配置仍由调用侧保留。
…odel-mgmt-unified-invocation # Conflicts: # internal/database/migration_sqlite_versioned_schema_test.go # internal/models/chat/anthropic_tools.go # internal/models/chat/ollama.go # internal/models/chat/openai_request.go # internal/models/invoke/special_tokens.go # internal/models/invoke/special_tokens_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
本 PR 重构模型接入体系：以「能力声明 + 统一调用层」取代原先分散在 chat / embedding / rerank / asr / vlm 五个包的厂商实现，让新厂商接入从"实现五个接口"降为"声明能力 + 覆盖线协议差异"。当前仅阿里云（DashScope 原生）完成适配与真机验证，其余厂商的适配完成度与待办见文末——欢迎官方在此基础上共同开发与审核。
统一调用层（
internal/models/invoke）Capabilities，JSON 形态随/models/providers下发）：思考档位、维度可覆盖、输入模态、凭证 spec、模型列表支持位等，前端据此动态渲染编辑器表单//exhaustive:enforce）internal/models/{chat,embedding,rerank,asr,vlm}全部退役，所有调用方完成迁移（agent 引擎、压缩、记忆、QA 流水线、会话）模型管理
/credentials子资源）、厂商 extraFields（进extra_config，测试连接与生产同源）、思考档位（模型记录分片 ∩ 厂商枚举）、目录预填即终态（保存后不再被目录覆盖）thinking布尔，nil=跟随智能体）顺带修复（真机排查发现）
finish_reason与tool_calls同帧时工具调用被整帧丢弃thinking_delta未透传（思考开了但 UI 不可见）finish_reason:"null"字符串掐断流/api/v3、/api/plan/v3类地址拼出双版本 URL 必 404本轮追加（2026-09-16，模型类型收敛 + 真机回归修复）
foldChatOptions单点强制，流式/非流式全覆盖）；对话页所选模型思考关闭时思考触发按钮与面板整块消失；智能体编辑器思考控件置灰说明；"思考不可用" UX 统一为整块隐藏<|、[INST]、<<SYS>>等）的出站中和，在 invoke 入口单点应用于全部消息（文本 / ReasoningContent / 工具参数），全适配器继承Type of Change
Related Issue
无对应 issue；后续协作项见文末，欢迎认领。
Testing
go test ./...（含-ldflags="-X 'google.golang.org/protobuf/reflect/protoregistry.conflictPolicy=warn'"，qdrant/milvus proto 冲突所需）：失败集与 origin/main 基线逐名一致，差异为零；基线中的环境依赖失败为 fake-IP DNS SSRF 家族（TestQueryTemplates*/TestUpdate*/TestDelete* 等，本机 DNS 代理所致）与 TestSkillPythonVerifier，与本次改动无关golangci-lint run --new-from-rev=origin/main ./...：0 issues；git diff --check通过vue-tsctype-check 0 错误；npm run test925/925（含上游合并新增用例）；vite build通过未完成与后续协作项
一、架构级待定（需与官方对齐设计后再动）
ProviderError分类（auth/provider_upstream/rate_limit/…）仅服务于重试链内部决策，面向用户与开发者的统一错误码体系待设计二、厂商适配（骨架已迁移，逐厂商补齐）
api-version仍为 2023-05-15 旧版max_tokens必填未发送；rerank 未注册chat_template_kwargs四、文档（刻意后置）
make docs）；apifox 同步；v2 迁移升级说明；Langfuse 集成文档website-docs/02-api-model-system.md已随本 PR 更新（capabilities/extraFields/catalog 端点）五、观察项 / 候选
/models路由（厂商确认 404），该网关下配置走手输模型 IDmodel_listing位存在双轨缺口Checklist
git diff --check origin/main...HEADpassesgolangci-lint run --new-from-rev=origin/main ./...)docs/, Swagger annotations, etc.)Screenshots / Recordings
真机截图随后补充在评论区（模型编辑器能力声明动态表单 / 会话思考面板 / 厂商 logo 与用量展示）。