Skip to content

ci(search): automate domain-credibility list refresh#346

Merged
quiet-node merged 7 commits into
mainfrom
feat/search-maintenance-pipeline
Jul 21, 2026
Merged

ci(search): automate domain-credibility list refresh#346
quiet-node merged 7 commits into
mainfrom
feat/search-maintenance-pipeline

Conversation

@quiet-node

@quiet-node quiet-node commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Description

Makes the search domain-credibility list self-maintaining. A weekly scheduled workflow re-fetches the upstream blocklists that feed result ranking and opens a reviewable PR when they drift. The list file gains machine-managed region markers so automation can never touch the hand-curated entries.

Key changes

  • scripts/refresh-credibility-list.ts: fetches the three upstream sources, rewrites only the sentinel-delimited autogen regions, guards against truncated or oversized responses, and reports (never removes) drop-list domains that no longer resolve.
  • .github/workflows/credibility-list-refresh.yml: weekly Monday cron plus manual dispatch; opens or updates a PR only when the list actually changed; never auto-merges.
  • src-tauri/src/websearch/credibility_domains.txt: the three upstream-derived clusters are wrapped in BEGIN/END AUTOGEN sentinels; one real refresh is applied (+7 domains). Hand-curated drop/boost/penalize entries are byte-unchanged.
  • Docs: maintenance notes in docs/built-in-web-search.md and docs/search-eval.md.

Notes

Closes #309

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…ekly

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node quiet-node changed the title Add scheduled credibility-list refresh and search eval capture workflows Make the search domain-credibility list self-maintaining Jul 20, 2026
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…ce-pipeline

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node quiet-node changed the title Make the search domain-credibility list self-maintaining feat(search): make the domain-credibility list self-maintaining Jul 20, 2026
@quiet-node quiet-node changed the title feat(search): make the domain-credibility list self-maintaining ci(search): automate domain-credibility list refresh Jul 20, 2026
@quiet-node
quiet-node merged commit f582a77 into main Jul 21, 2026
6 checks passed
@quiet-node
quiet-node deleted the feat/search-maintenance-pipeline branch July 21, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scheduled maintenance pipeline for search eval runs and credibility-list refresh

1 participant