Repository navigation
Conversation
…esses Opt-in built-in plugin for applications which scrape urls supplied by untrusted users. It registers a beforeRequest action which rejects resource urls with an ip literal in a loopback, link-local, private, CGNAT, multicast, reserved or documentation range, adds a beforeRedirect hook which rejects redirects to such literals, and wraps the dns lookup used for the connection (dnsLookup, or dnsCache.lookup when configured) so that a hostname resolving to any blocked address fails the request. Ranges are checked with net.BlockList, which also covers IPv4-mapped IPv6 addresses. The plugin is exported from the package entry point and from lib/plugins and documented in the README plugins section. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
aivus
marked this pull request as draft
November 8, 2026 22:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opt-in built-in plugin for applications which scrape urls supplied by untrusted users, where a downloaded page can otherwise make the scraper request internal services (server-side request forgery). The default behaviour is unchanged: the scraper does not restrict destinations unless the plugin is added, because mirroring intranet, staging or localhost sites is a legitimate use.
How it works
The plugin registers a single
beforeRequestaction whichurls(the WHATWG URL parser normalizes forms such as0x7f000001,2130706433and127.1);beforeRedirecthook which rejects redirects to such literals, keeping any hooks the user configured;dnsLookup, ordnsCache.lookupwhen a dns cache is configured, elsedns.lookup) so that a hostname resolving to any blocked address fails the request. Node skips the lookup for ip literals, which is why those are checked separately. The validated address is the one the socket connects to.Ranges are checked with Node's
net.BlockList, which also handles IPv4-mapped IPv6 addresses (::ffff:a.b.c.d). No new dependency.A blocked request is reported as an ordinary resource error, so
onResourceErroris called andignoreErrorsdecides whether the scrape continues.Exports and docs
Exported as a named export from the package entry point and from
lib/plugins/index.js. Documented in the README under "Built-in plugins" with a dedicated subsection.Tests
beforeRequestaction (ip literal urls, preserved request options and hooks), for the redirect hook, and for the dns wrapper (single andall: trueresults, lookup errors,dnsCachefallback).localhost) and redirect vectors, and that a blocked initial url fails the scrape whenignoreErrorsis false.🤖 Generated with Claude Code