ci(search): automate domain-credibility list refresh#346
Merged
Conversation
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…ekly Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…ce-pipeline Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Makes the search domain-credibility list self-maintaining. A weekly scheduled workflow re-fetches the upstream blocklists that feed result ranking and opens a reviewable PR when they drift. The list file gains machine-managed region markers so automation can never touch the hand-curated entries.
Key changes
scripts/refresh-credibility-list.ts: fetches the three upstream sources, rewrites only the sentinel-delimited autogen regions, guards against truncated or oversized responses, and reports (never removes) drop-list domains that no longer resolve..github/workflows/credibility-list-refresh.yml: weekly Monday cron plus manual dispatch; opens or updates a PR only when the list actually changed; never auto-merges.src-tauri/src/websearch/credibility_domains.txt: the three upstream-derived clusters are wrapped inBEGIN/END AUTOGENsentinels; one real refresh is applied (+7 domains). Hand-curated drop/boost/penalize entries are byte-unchanged.docs/built-in-web-search.mdanddocs/search-eval.md.Notes
GITHUB_TOKEN, so they do not trigger the repo's other CI; a reviewer pushes an empty commit or closes/reopens to run checks before merging.Closes #309