Skip to content

fix: swap httpx transport for requests to avoid Reddit 403 - #20

Open
kragent66-glitch wants to merge 1 commit into
public-clis:mainfrom
kragent66-glitch:fix/httpx-403-requests-transport
Open

fix: swap httpx transport for requests to avoid Reddit 403#20
kragent66-glitch wants to merge 1 commit into
public-clis:mainfrom
kragent66-glitch:fix/httpx-403-requests-transport

Conversation

@kragent66-glitch

Copy link
Copy Markdown

Summary

Reddit's bot detection fingerprints the httpx TLS/HTTP stack and returns a 403 challenge page even with a valid reddit_session cookie. Replacing the httpx.Client with a requests.Session fixes it β€” urllib3/requests passes Reddit's fingerprint checks.

Closes #13 (same repro: valid cookie works with curl/urllib, httpx gets 403).

Changes

  • rdt_cli/transports.py:
    • httpx.Client β†’ requests.Session (headers + cookies copied, max_redirects=10)
    • Manual BASE_URL joining for relative URLs (requests has no base_url)
    • Explicit timeout=self.config.timeout per request (requests has no default)
    • httpx.TimeoutException/NetworkError β†’ requests.Timeout/ConnectionError
  • pyproject.toml: httpx>=0.27 β†’ requests>=2.31 (httpx no longer imported anywhere)

Why requests instead of patching httpx?

Verification

  • βœ… pytest tests/ β†’ 123 passed (66 deselected)
  • βœ… Live against Reddit with a real session:
    • GET /r/python/top.json?limit=2 β†’ returns posts (no 403)
    • GET /api/v1/me β†’ no 403, no exception

Reddit's bot detection fingerprints the httpx TLS/HTTP stack and
returns a 403 challenge page even with a valid reddit_session cookie,
while urllib3/requests passes. Replace the httpx.Client with a
requests.Session, add manual BASE_URL joining (requests has no
base_url), pass an explicit timeout, and catch requests exceptions.

Verified live: /r/python/top.json returns posts; /api/v1/me no longer
403s. Closes public-clis#13.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Valid reddit_session works with curl/urllib but httpx transport returns Reddit 403

1 participant