feat: add configurable OpenAI base URL for LiteLLM / OpenRouter / custom proxies - #36
Open
tonyfruzza wants to merge 5 commits into
Open
feat: add configurable OpenAI base URL for LiteLLM / OpenRouter / custom proxies#36tonyfruzza wants to merge 5 commits into
tonyfruzza wants to merge 5 commits into
Conversation
1. scraper.ts: Update URL match regex to accept a.co short links in addition to full amazon.com URLs 2. products.ts: Use voting-based scraper in confirmed-price flow for richer price candidate data 3. Add k8s manifests for deploy via GH Actions
Merges fix/aco-url-support into main
The Sec-* and Cache-Control headers in the axios request config trigger Amazon's bot detection, returning a 5KB blocking page instead of the full product page. This breaks ALL Amazon scraping, not just a.co links. Removed headers: Cache-Control, Pragma, Sec-Ch-Ua, Sec-Ch-Ua-Mobile, Sec-Ch-Ua-Platform, Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site, Sec-Fetch-User, Upgrade-Insecure-Requests
fix: remove bot-triggering headers from Amazon HTTP requests
|
@tonyfruzza hey you might like to contribute to our active fork of this cheers |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional OpenAI Base URL field to the AI Extraction settings page. When set, the OpenAI SDK's
baseURLis overridden so that any OpenAI-compatible endpoint can be used in place ofapi.openai.com.This enables:
Changes
database/init.sql— migration:ALTER TABLE users ADD COLUMN openai_base_url VARCHAR(512)backend/src/config/init-db.ts— migration added so freshnpm run db:initdeployments also get the columnbackend/src/models/index.ts—AISettingsinterface and all DB queries includeopenai_base_urlbackend/src/routes/settings.ts— GET/PUT/api/settings/aiincludeopenai_base_urlbackend/src/services/ai-extractor.ts— all four OpenAI call sites (extract,verify,verifyStockStatus,arbitrate) accept and forwardbaseURLfrontend/src/pages/Settings.tsx— new optional text input under the OpenAI provider sectionfrontend/src/api/client.ts—openai_base_urladded to AI settings typeBackward compatibility
The
openai_base_urlfield defaults toNULL. When null, behaviour is identical to before — the OpenAI SDK uses its defaultapi.openai.comendpoint. Existing deployments are not affected.Closes
Closes #21 (custom API URL for OpenAI-compatible proxies like OpenWebUI)
Closes #23 (OpenRouter support — set base URL to
https://openrouter.ai/api/v1, use any model name)Testing
Tested against a self-hosted LiteLLM instance routing to DeepSeek (
deepseek-v4-flash) — extraction, verification, and arbitration all function correctly.