feat: localize the chatbot's grounding & source metadata by thread language - #61
Merged
Conversation
…anguage Fontes dos Dados showed pt names because the GraphQL lookup only fetched the default name. Fetch nameEn/nameEs, pick by the thread's language with a pt fallback where a translation is missing, and cache per (language, table_id).
Route dataset/table/column metadata through the thread's language so the
agent grounds on en/es content (pt fallback) instead of pt-only:
- search_datasets passes locale to the (already locale-aware) /search/
endpoint, so names, descriptions, themes, tags and organizations return
localized.
- get_dataset_details / get_table_details fetch the explicit
namePt/nameEn/nameEs and descriptionPt/... columns and pick by language;
themes/tags/organizations and the usage guide (userGuide/{locale}/, pt
fallback) are localized too. Column identifiers stay pt.
- Language reaches the tools via injected RunnableConfig, the same pattern
execute_bigquery_sql already uses; the model never sees config.
- Add a shared app.i18n.localized_field helper; use explicit namePt for the
data-source name resolver instead of the ambiguous name accessor.
Match tests to the new API: GraphQL mocks return the explicit namePt/descriptionPt columns; _resolve_table_name and resolve_data_source_names take the thread language; the search mock keeps name/description since the /search/ endpoint localizes server-side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Localizes the chatbot's data metadata — both what the user sees and what the agent grounds on — to the thread's language, with a pt fallback everywhere. Complements #58, which already made the bot answer in the site language and persists
Thread.language; this PR closes the metadata side, which was still Portuguese-only.What was wrong
The agent read and displayed dataset/table metadata in Portuguese regardless of the thread language:
name.search_datasets,get_dataset_details,get_table_details) read pt-onlyname/description, so the model grounded on — and often echoed — Portuguese names, descriptions, themes, tags, organizations, and usage guides.What changed
Display — source names (
app/api/streaming/data_sources.py)namePt/nameEn/nameEsand pick by the thread's language (pt fallback where a translation is missing); cache keyed by(language, table_id).Grounding — retrieval tools (
app/agent/tools/)search_datasetspasseslocaleto the/search/endpoint, which is already locale-aware (it matches the localized text field and returnsdataset_name_{locale}/dataset_description_{locale}plus localized themes/tags/organizations, all with server-side pt fallback).get_dataset_detailsfetches the explicitnamePt/nameEn/nameEs+descriptionPt/...columns and picks by language for the dataset and its tables; themes, tags, and organizations are localized too, and the usage guide is fetched fromuserGuide/{locale}/with a pt fallback (only pt guides exist today).get_table_detailslocalizes the table name/description and column descriptions (column names stay — they are real BigQuery identifiers).Plumbing
app.i18n.localized_field(node, field, language)(used by both the resolver and the tools).RunnableConfig(config["configurable"]["language"], set alongsidethread_idwhen the run is dispatched) — the same patternexecute_bigquery_sqlalready uses, so the model never sees theconfigargument.Deliberately left Portuguese
municipio,ano, …).app/agent/prompts.py(not user-visible; steer the model).Coverage / fallback
Every path falls back to pt when a localized value is empty. On a 100-table sample: ~91% have
nameEn, ~89%nameEs; descriptions and en/es usage guides are lower and fall back to pt.Verification
python -m py_compileclean on all changed files.TableNode/DatasetNode/ColumnNode/OrganizationNode/ThemeNode/TagNodeexpose thenameEn/Es+descriptionEn/Escolumns, and that the/search/endpoint localizes results (locale=en→ "American Time Use Survey (ATUS)" / theme "Economy" vs pt "Pesquisa Americana de Uso do Tempo (ATUS)" / "Economia").🤖 Generated with Claude Code