Skip to content

feat: hybrid LLM model picker + per-model cost tracking#4

Merged
AlanY1an merged 20 commits into
mainfrom
feat/llm-model-picker
Apr 29, 2026
Merged

feat: hybrid LLM model picker + per-model cost tracking#4
AlanY1an merged 20 commits into
mainfrom
feat/llm-model-picker

Conversation

@AlanY1an

Copy link
Copy Markdown
Owner

Summary

  • Replace the admin LLM card's free-text model input with a hybrid picker (curated dropdown for official Anthropic / OpenAI providers + Custom… escape hatch + free-text fallback for non-official base_url). base_url and api_key_env are also editable inline.
  • Migrate the cost-tracking ledger from role-only rates to per-(provider, model) pricing via a vendored copy of LiteLLM's model_prices_and_context_window.json (MIT-licensed; manual refresh via scripts/refresh_litellm_prices.py). Stub provider and local-host base_urls short-circuit to $0.
  • PATCH-time validation of llm.api_key_env returns HTTP 422 with structured [{field, msg}] so the admin UI renders the error inline. Skipped for provider="stub" and local-host base_urls.

Test plan

  • `uv run pytest` (expect 1444 passed)
  • `uv run ruff check src/ tests/`
  • `uv run lint-imports`
  • `cd src/echovessel/channels/web/frontend && npx tsc -p tsconfig.json --noEmit`
  • `uv build` and confirm the wheel bundles `echovessel/core/llm/data/{litellm_prices.json,LITELLM-LICENSE}`
  • Browser: provider=anthropic, empty base_url → picker shows Sonnet / Haiku / Opus with prices inline
  • Browser: pick `Custom…`, type a non-preset model id, save, reload → value persists, dropdown shows Custom selected
  • Browser: set base_url to `https://openrouter.ai/api/v1\` → picker collapses to free-text with the "custom base_url detected" hint
  • Browser: set base_url to `http://localhost:11434/v1\` with a missing api_key_env → save returns 200 (local-host skip)
  • Browser: change api_key_env to a name not in environment → save returns 422 with inline error near the field; saving with a valid name clears it
  • Browser: temperature / max_tokens / timeout / save-button enable-on-dirty all unchanged

AlanY1an added 20 commits April 29, 2026 16:07
@AlanY1an AlanY1an merged commit 4fb821d into main Apr 29, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant