Repository navigation
Commit 9eddf78
A dot is not a hostname (#148)
`normalizeUrl` guarded the host with `hostname.includes('.')`, and `new URL()`
is far more permissive about hosts than DNS is: quotes, parens, `=` and `&` are
not forbidden host code points. So `version="1.0"`, `text="gg.deals`, `zombies.)`
and `z.` all parsed, all contained a dot, and all became feeds.
That is how ~3,700 rows like `https://version="1.0"/` and `https://zombies.)/`
reached the directory. The bulk-upload scanner splits pasted text on whitespace
and offers every token as a URL, so pasting raw OPML instead of a URL list fed
the markup itself in -- XML attributes, and sentences whose last word ended in a
full stop. Each one is then re-crawled forever on a cadence and fails
`blocked-host`; they were most of a 960/hour spike in that error.
The check goes in `normalizeUrl` rather than in the scanner because it is the
one gate every entry path shares -- web submit, OPML import, the queue drain and
discovery -- and because the scanner runs in the browser, where importing
`@rssamplifier/feed` fails the build on `node:dns`.
It rejects what cannot be a hostname, not what is merely unusual: IDN is already
punycode by then, a port and a dotted quad still pass, and private ranges go on
being refused later by `isPublicHost`. The second test is the half that would
break the directory if this were too strict, so it is spelled out.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent a83f1e0 commit 9eddf78
2 files changed
Lines changed: 94 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
55 | | - | |
| 55 | + | |
56 | 56 | | |
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
63 | 118 | | |
64 | 119 | | |
65 | 120 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
50 | 88 | | |
51 | 89 | | |
52 | 90 | | |
| |||
0 commit comments