Repository navigation
Fix bilingual dictation translating/duplicating output; correct provider request fields - #16
Merged
Merged
Conversation
…op translated and duplicated output
…p bare codes to tailored instructions
…rately from the prompt
…anscribe languages and keywords
…from transcription models
…e under English, extend hallucination list
… for pairs that include Chinese
…the language-preserving instruction
…expected shape only loses detection
There was a problem hiding this comment.
🟡 Changes recommended
Pair-aware cleanup normalization is incorrectly applied to chat responses and can rewrite valid non-Chinese Han text.
2 open findings
What changed in this PR
Fixes bilingual transcription duplication/translation and aligns provider request payloads with model-specific APIs.
Changes:
- Uses single-language bilingual conditioning and language-preserving cleanup.
- Adds provider-specific vocabulary, language, and response handling.
- Improves CJK normalization, hallucination filtering, migrations, tests, and documentation.
| File | Description |
|---|---|
tests/retiredModels.test.ts |
Tests Mistral model migration. |
tests/prompts.test.ts |
Tests bilingual language instructions. |
src/utils/languageSupport.ts |
Adds bilingual and base-code handling. |
src/services/tauriApi.ts |
Passes language pairs to reasoning. |
src/models/modelRegistryData.json |
Removes unsupported Voxtral Small. |
src/hooks/useTranscriptionPipeline.ts |
Threads bilingual context through cleanup. |
src/config/retiredModels.ts |
Migrates retired Mistral model. |
src/config/prompts.ts |
Applies bilingual cleanup prompts. |
src-tauri/src/transcription/normalize.rs |
Adds pair-aware CJK normalization. |
src-tauri/src/transcription/mod.rs |
Revises bilingual Whisper conditioning. |
src-tauri/src/transcription/hallucination.rs |
Expands Chinese hallucination filtering. |
src-tauri/src/transcription/dictionary_echo.rs |
Separates prompt and vocabulary hints. |
src-tauri/src/transcription/cloud.rs |
Adds provider-specific request fields. |
src-tauri/src/commands/transcription.rs |
Threads languages and request hints. |
src-tauri/src/commands/reasoning.rs |
Applies pair-aware response normalization. |
docs/TODO.md |
Records pending live verification. |
docs/superpowers/plans/2026-10-13-bilingual-prompt-and-provider-requests.md |
Documents implementation plan. |
docs/PROGRESS.md |
Records diagnosis and evaluation results. |
docs/CHANGELOG.md |
Documents user-visible fixes. |
docs/ARCHITECTURE.md |
Updates transcription architecture. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| undefined, | ||
| temperature, | ||
| resolvedLanguage, | ||
| languagePair, |
| invoke("process_reasoning", { text, provider, model, system_prompt, api_key }) | ||
| → Context-aware dictionary aliases are supplied as explicit correction mappings | ||
| → Bilingual mode uses a code-switch-preserving language rule (never translate); | ||
| other modes use the registry instruction (bare codes like `en` map to `en-US`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


What changed
Root cause — Chinese dictation mixed with English (bilingual mode). The bilingual Whisper
promptwas a Chinese sentence followed by its English translation. Whisper treats the prompt as the preceding transcript and imitates it, writing every spoken segment twice: as an interleaved English translation (那你就记得。 Then you remember.) or a verbatim duplicate (Thank you for the email. Thank you for the email.). Bilingual mode now conditions on one language's sentence (first slot that has one), with the dictionary after a Latin-script sentence and before a CJK one.Enhancement. Bilingual cleanup uses a code-switch-preserving rule (never translate; Chinese / Japanese / English punctuation each in its own convention). Bare base codes no longer fall into the forced "respond in X regardless of what the user spoke" template:
zh→ its tailored instruction;en(en-US/en-GB) → the language-preserving auto instruction, so no spelling convention is imposed.Provider requests (audit against current provider docs):
verbose_jsononly for Whisper models —gpt-transcriberejects it (HTTP 400).gpt-transcribe:languages[](both languages in bilingual mode) + sanitizedkeywords[]; detection read leniently fromlanguages[].code.gpt-4o-transcribe-diarize: noprompt.context_bias, noprompt/response_format;voxtral-small-latest(not a transcription-endpoint model) removed and migrated tovoxtral-mini-latest.RequestHints { prompt, vocabulary }; the dictionary-echo retry drops both.Normalization. CJK small-form marks (﹐﹑﹒﹔﹕﹖﹗) → standard full-width; in a pair that includes Chinese, Han text under an English resolution keeps Chinese punctuation + Simplified (other pairs such as ja+en keep plain English handling, so kanji are never simplified); more Chinese silence-hallucination phrases.
Evidence
whisper-large-v3(TTS + noise/pauses; Chinese, English, code-switched): old prompt 12/22 clean (10 duplicated/translated/echoed); the shipped shapes 22/22.transcribe_groq+ new prompt builder) against Groq: fields accepted, detection returned, no translation/duplication.Verification
cargo test: 255 passed ·bun test: 30 passed ·bun run typecheckclean ·bun run buildOKcargo clippy --all-targets: 8/9/1 warnings vs baseline 12/13/1 (none in changed code)Not verified live
gpt-transcriberequest shape (dev machine's OpenAI key returns 401) and Mistralcontext_bias(no key). Both follow the providers' docs/SDK encoding; detection parsing is lenient so a shape surprise only loses detection. The Standard default staysgpt-4o-mini-transcribeuntil a live check (TODO).Plan:
docs/superpowers/plans/2026-10-13-bilingual-prompt-and-provider-requests.md🤖 Generated with Claude Code