api-notes.md

Upstream API notes

Every entry is marked with how it was established:

• [live] — called and inspected by hand
• [source] — read from the vendor's own client code or OpenAPI spec
• [assumed] — inferred, not yet confirmed. Verify before relying on it.

Thunderstore and Hexium share a common ancestry: Hexium is an API-compatible fork. Their /api/experimental/ surfaces match. Their listing surfaces do not.

---

Thunderstore

Base: https://thunderstore.io

Listing (per tick)

[source] From thunderstore-ui/packages/thunderstore-api:

GET /api/cyberstorm/listing/{community}/?ordering=last-updated&page=1

ordering accepts newest, most-downloaded, top-rated, last-updated. Other query params: q, section, included_categories, excluded_categories, nsfw, deprecated.

Item fields [source]: namespace, name, description, icon_url, categories[], last_updated, download_count, rating_count, size, is_deprecated, is_nsfw, is_pinned, community_identifier.

latest_version_number, install_url, download_url are marked optional in their schema. [live] (2026-09) the listing does not return the version number; categories are objects ({id, name, slug}), and the default query is nsfw=False&deprecated=False, so NSFW and deprecated packages never appear. Pinned packages sit at the head with an old last_updated. Resolve the version with:

GET /api/cyberstorm/package/{namespace}/{name}/versions/

one request per changed package, capped per tick. [live] The response is an array of {version_number, datetime_created, download_url, ...} in no guaranteed order — take the maximum datetime_created as the version and the second-largest as previousVersion (null when the package has a single version). The body is refused above 512 KB (VERSIONS_MAX_BYTES, about 2300 versions), checked while streaming and before JSON.parse; the package is then skipped like a 404 (the cursor moves past it) and one line naming the package id is logged. Measured cold, parse plus scan: ~1 ms at 512 KB, ~12 ms at 25,000 versions (5.3 MB).

[live] (2026-09-20) The adapter sends nsfw=false&deprecated=false explicitly instead of relying on the server default; both forms returned the same status and size (15,016 bytes). Flags are still checked per item and fail closed: is_nsfw counts as safe only when it is the boolean false, so a missing, null or non-boolean flag marks the package NSFW. is_deprecated is true only when it is the boolean true.

Backlog: when more than one listing page (20 items) of updates is pending, the adapter reads the pending items from the bottom of the backlog so that the oldest are handled first and the cursor never passes an unseen update. While this lasts the cursor is <iso>@<page>, where <page> is the listing page the next poll resumes at; a plain timestamp (and a missing or stale page) is accepted and means "start at page 2". Each poll reads at most thunderstoreListingPages (3) listing pages.

[live] Conditional requests: no ETag, but Last-Modified is sent and If-Modified-Since returns a real 304. Cache-Control: public, max-age=60. The adapter stores the validator in the etag slot as lm:<Last-Modified>.

This is not in the public Swagger; r2modman calls parts of the cyberstorm API undocumented. It is what the production site runs on, but treat breakage as a realistic risk and fail soft.

Changelog, per version

[live] Both work. Prefer the first — Markdown goes into Discord almost unchanged, HTML would need conversion.

GET /api/experimental/package/{namespace}/{name}/{version}/changelog/
→ { "markdown": string | null }

GET /api/cyberstorm/package/{namespace}/{name}/v/{version}/changelog/
→ { "html": string }

/readme/ exists in both shapes. This is the endpoint Gale uses.

Returns the whole CHANGELOG.md as shipped in that version's archive, not a diff. Extraction strategy in docs/spec.md.

Changelog responses are refused above 128 KB (CHANGELOG_MAX_BYTES in src/sources/budget.ts, checked while streaming and before any JSON.parse), and the extractor looks at no more than the first 128 K characters of the markdown and at most 4096 heading or fence lines. The excerpt it returns is final Discord Markdown: HTML stripped, mentions neutralised, only http(s) [text](url) links kept, and a single trailing [Full changelog](url) link inside the character budget. Discord's own Markdown parser differs from CommonMark (a code block closes at the first following fence, inline code spans lines, there are no ~~~ fences), so the excerpt models no code at all: every backtick and < is escaped, the colon of ]: is escaped, and every ]( outside a kept http(s) link is escaped, so no code span, fence, autolink, reference definition or other link target can form. A fence in the changelog therefore shows as escaped text; fences still count when the extractor looks for the section of a version. Numeric entities that decode to bidi, zero-width, filler, variation-selector, tag or control characters are dropped during entity decoding, as are the raw characters. Link targets carry %40 instead of @ where they spell @everyone or @here. A link whose label looks like a URL (https://… or www.…) is kept only when the label's host equals the target's host, otherwise it degrades to plain text. HTML stripping removes only known tag names, so <T>, Dictionary<string, int> and <https://…> stay readable (as escaped text). The renderer inserts the excerpt with only invisible character stripping, mention neutralising and a length cap.

Package website, per event

[live] (2026-09-27) Neither the listing nor /versions/ carries the author's website. The experimental package endpoint does:

GET /api/experimental/package/{namespace}/{name}/
→ { namespace, name, full_name, owner, package_url, date_created, date_updated, rating_score,
    is_pinned, is_deprecated, total_downloads,
    latest { version_number, description, icon, dependencies, download_url, downloads, date_created,
             website_url, is_active, … },
    community_listings [ … ] }

latest.website_url is an author-supplied string and may be empty. Measured: 1.2 KB for a small package, 31 KB for ebkr-r2modman (the dependency list dominates); the body is refused above 64 KiB (THUNDERSTORE_PACKAGE_MAX_BYTES) before parsing. The adapter reads it in the per-event details phase (fetchChangelog), after the changelog request and only when the package has no website yet, so a delivered Thunderstore event costs up to two subrequests (MAX_DETAIL_REQUESTS_PER_EVENT) and the tick budgets for both (docs/spec.md, "Subrequest budget"). A failing, oversized or unreadable response only leaves the website out; the changelog is unaffected. Measured (Node, network excluded, median of 30): 12 events cost 0.4 ms of CPU with 1 KB bodies and 1.9 ms with 62 KB bodies.

Full catalogue

[source] GET /c/{community}/api/v1/package-listing-index/ → gzipped JSON array of chunk URLs; each chunk is gzipped JSON of packages with date_created, date_updated, versions[]. Full dump, not incremental.

[source] GET /c/{community}/api/v1/package/ — everything in one response. Thunderstore themselves recommend against it for large communities; it has hit response size limits for e.g. riskofrain2. Do not use from a Worker.

[live] /api/experimental/package-index/ 302-redirects to an 83 MB gzipped JSON covering every game; the per-community /c/{community}/api/v1/package-listing-index/ is 13 chunks of ~1.3 MB gzip (~14 MB raw) each. Neither is workable from a Worker, so Thunderstore has no reconciliation.

---

Hexium

Base: https://{game}.hexium.gg for the listing, package index, per-package lookup and changelog. hexium.gg also serves changelogs but cannot resolve frontend/p/... (404). Gale uses the bare host for /api/experimental/package/....

Cyberstorm

[live] /api/cyberstorm/listing/{community}/ → 404. Only /api/cyberstorm/community/{community_id}/ exists. The fork took cyberstorm partially.

Listing

[live] GET /api/experimental/frontend/packages/?page=N

20 items per page. Response: { bg_image_src, categories[], community_name, has_more_pages, packages[] }.

Item fields, exact [live]:

name, full_name, owner, package_url, version_number, date_updated,
rating_score, download_count, is_pinned, is_deprecated, has_nsfw_content,
categories[], icon_url, description, uuid4

Note this does carry version_number and date_updated — no second request needed to diff.

An item without owner, name, version or a parsable date, or whose is_deprecated is not a boolean, is quarantined like a lookup with the same defect: skipped, counted in the one-line warning (listing items N) and reported as a run-log warning. A missing or non-boolean has_nsfw_content still fails closed to NSFW.

Critical: the default order is package creation descending, not last updated. Established two ways: the embedded sequential id in uuid4 (00000547-…-000000000547) decreases monotonically down the page, and the first twelve items match the website's ?sort=newest exactly while ?sort=updated produces a completely different head. A package updated at 23:34 sat in position 14 while one updated at 16:48 sat in position 11.

So this endpoint catches new packages reliably and misses updates to existing packages.

[live] ?sort=updated and ?ordering=last-updated were both tried on this endpoint. Both return exactly the same payload as no parameter at all — they are ignored, not honoured. Do not retry this.

The website does serve ?sort=updated, so the backend supports the ordering; it is simply not exposed on the API. Worth asking Hexium to expose it — a one-line change on their side that would remove the need for the index comparison entirely. Until then, updates to existing Hexium packages are found by comparing the package index with the stored versions (next section).

Update source: package index plus per-package lookup

Why not the full dump. [live] (2026-09-20 and 2026-09-26) GET /api/v1/package/ carries the full version history of every package, so it grows without bound: 4.4 MB (1113 packages) on 2026-09-20, 6.7 MB raw (~614 KB gzip; 1318 packages, 5685 versions, about 1.2 KB per version) on 2026-09-26. A scan already cost ~9 ms CPU and the body exceeded MAX_SCAN_BYTES (6 MiB), so production polls were skipped. Not viable; the adapter no longer reads it. Query parameters q, search and page are ignored by both the dump and the listing.

Package index [live] (2026-09-27) GET /api/experimental/package-index/: NDJSON, application/x-ndjson, chunked, no ETag, no Last-Modified, no Content-Length, no trailing newline. One line per package with only the latest version:

{"namespace","name","version_number","file_format","file_size","dependencies":[],"suggestions":[]}

Valheim: 1318 lines, 514,768 bytes (~390 bytes per line; longest line 4947 bytes), 1318 unique namespace-name ids, every version_number a string and every file_size an integer, no \r. It grows only with the package count. There is no date, no description, no NSFW or deprecated flag, so it says whether a package changed, never what it is. The game subdomain serves that game only; hexium.gg merges every game.

Reading rules (src/sources/hexium-index.ts):

• One pass with indexOf('\n') and one sticky regex per line; no split, no whole-body JSON.parse, nothing parsed that is not needed. Every read is bounded by its line, so cost is linear in the body size. \r\n, blank lines and a missing trailing newline are accepted.
• A line must open with {"namespace":"…","name":"…","version_number":"…" in that key order. namespace and name are 1-128 characters of [A-Za-z0-9_.-] and do not start with a dot (they go into a lookup URL path: never ./.., no /, ?, space or non-ASCII); the version is 1-64 characters without quote, backslash or control characters. file_size is read only in the live layout (,"file_format":"…","file_size":<int>), otherwise the size is unknown (null). A line that does not match, or is longer than HEXIUM_INDEX_MAX_LINE_BYTES (32 KiB), is unreadable: skipped and counted in the one-line warning, never used.
• The body is refused above HEXIUM_INDEX_MAX_BYTES (checked while streaming) and above HEXIUM_INDEX_MAX_LINES (3500); the byte cap is the line cap times 512 bytes (1.75 MiB, 1.3x the live 390 bytes per line). Every line visited counts, blank ones included, against HEXIUM_INDEX_MAX_ITERATIONS (twice the line cap), so a body of newlines, CRLF pairs or spaces stops after 7000 visits (about 1 ms cold for 1.75 MiB; before the visit cap 1.5 MiB of newlines cost 11.8 ms). A body in which no line is readable (HTML, a JSON array, a changed layout) is unusable. All of these fail soft: the poll keeps its listing result, logs one line and reports a warning (below); reconcile throws.
• Degradation is visible. The poll result carries warnings, the core copies them into the source report and the run log line (redacted, at most 5 per source, 200 characters each). Hexium warns when the index is above the cap ("updates of existing packages are not detected"), when it is unavailable or unreadable, when lookups failed or were unreadable, when candidates exceeded the per-poll lookup cap, and when listing items or index lines were skipped. Before this, an index above the cap silently reduced the source to new-package detection while the report said ok.
• Unreadable lines while seeding: a poll whose scan has more unreadable lines than max(3, 1% of the lines) commits nothing, keeps the cursor and reports "seeding paused"; the next tick retries the same slice. Unreadable lines cannot be attributed to a slice, so each slice poll sees all of them; committing anyway would leave holes that later drip out as spurious new events. A permanent defect above the threshold keeps seeding paused, loudly, until it is fixed.
• Duplicate lines of one package cause one lookup.

Cost, measured cold (fresh Node process per figure, network excluded, 390-byte synthetic lines, median of 25 runs; the live 1318-line index measures 1.9-2.7 ms for the scan with comparison): the index tick's own CPU is TextDecoder decode plus rebuilding the stored-version map plus the scan with comparison.

lines	decode	version map	scan + comparison	total (nothing changed)	total (every package changed)
1318	0.2	0.3	2.0	2.5
3000	0.3	0.5	2.5	3.3
3500	0.4	0.6	3.3	4.3	4.8
4000	0.4	0.7	3.7	4.8	5.5
5000	0.4	1.6	3.9	6.0	6.6
6000	0.5	2.2	8.1	10.8

The 10 ms invocation budget is shared with Thunderstore, D1 writes and Discord, and the listing parse and 15 lookups add about 2 ms on top, so the line cap is 3500: about 4.3-4.9 ms steady state (two measurement sessions) and 4.8 ms after a store wipe. 5000 and 6000 lines measured above 5 ms and were rejected; 4000 was rejected as too close to the limit. The scan itself is bound by per-line work, not bytes: the indexOf walk is 0.15 ms per 4000 lines, the sticky regex 0.9, and the remaining ~1.5 ms is the Map lookup of the freshly built namespace-name id.

Growth. The package count rose from 1113 (2026-09-20) to 1318 (2026-09-26): about 34 packages per day. [live] (2026-10-02) the Valheim index has 1490 lines (400 bytes per line, 596 KB), 29 per day since 2026-09-26. The 3500-line cap is therefore reached between 2026-11-30 (34 per day) and 2026-12-11 (29 per day); the warning above appears the first day the index exceeds it. Past that point the CPU budget no longer allows a full scan per tick; the durable fix is a server-side sorted or filtered listing (worth asking Hexium for, see Listing) or an incremental design that does not read the whole index every scan.

Known limitations (accepted, not fixed):

• A package whose per-package lookup lags behind the index (index lists a newer version than the lookup returns) is looked up again on every scan, one subrequest and one of the 15 window slots per scan, and produces no event until the lookup catches up. Seen live on 2026-09-27.
• Any differing version is announced as update by the core diff, including an older one served by a stale listing or lookup.
• A known package whose update the listing shows is held back, not announced, until its lookup succeeds. One whose lookup keeps failing (a deleted or unlisted package, say) therefore produces no event and costs one subrequest per tick while it stays on page 1; the index scan retries it afterwards. While the index is over its caps or unreadable, a held-back package that scrolls off page 1 stays unannounced until the index is usable again (an already-alerted degraded mode). A held-back tick keeps the stored listing validator, so a listing that honours it is fetched in full again on the next tick.
• Bursty scans (up to 15 lookups plus the listing and index reads) can leave few subrequests for changelog fetches; the changelogs of that tick are skipped and not retried.
• Spurious new events are possible after a wiped store (every package is unseen) and for a package created while seeding whose slice had already passed.

Per-package lookup [live] (2026-09-27) GET /api/experimental/package/{namespace}/{name}/, ~1.1 KB JSON, 404 {"detail":"Not found."} for an unknown package:

namespace, name, full_name, owner, package_url, date_created, date_updated,
rating_score, is_pinned, is_deprecated, total_downloads, hexium_downloads,
latest { version_number, description, icon, download_url, date_created, dependencies, … },
community_listings [ { community, categories[], has_nsfw_content, review_status } ]

is_deprecated is package-wide; has_nsfw_content and categories are per community. Rules for the snapshot built from it:

• version = latest.version_number (the truth even when the index line differs), description and iconUrl from latest, updatedAt = date_updated, sizeBytes from the index line, previousVersion unset (the core diffs against the stored version for known packages; unseen packages are new).
• The response must name the requested owner and name, otherwise it is unreadable.
• NSFW fails closed. categories and has_nsfw_content come from the community_listings entry whose community equals the configured community. No such entry (or community_listings missing or not an array) means NSFW; so does any matching entry whose has_nsfw_content is not exactly boolean false.
• is_deprecated must be boolean; anything else quarantines the candidate (not emitted, counted in the one-line warning). latest must be an object with a non-empty version_number and date_updated a parsable timestamp, otherwise the lookup is unreadable and the candidate is not emitted.
• likes = rating_score, websiteUrl = latest.website_url (see "Fields the messages use").
• package_url is used only when it starts with https://{game}.hexium.gg/mods/, otherwise the URL is built from the ids.
• Responses above HEXIUM_LOOKUP_MAX_BYTES (64 KiB) are refused before parsing. A 404, another HTTP error or a network error defers the candidate: it is never emitted without a lookup and never from the lean index data. A 429 stops the remaining lookups of that poll.

How the adapter uses them (docs/spec.md has the full algorithm):

• Every tick: listing page 1 (new packages, with flags).
• Every CADENCE.hexiumIndexEveryNthTick-th tick: read the index once and compare each line's version with the stored versions (store.getAllKnownVersions). Candidates are packages whose version differs and packages the store has never seen. At most hexiumLookupsPerPoll (15) candidates are looked up per poll, six at a time; the window of candidates advances by that amount per scan so packages that keep failing cannot starve the others; the poll is then reported incomplete and the rest are found again by the next scan.
• A known package's update is never committed without its lookup. A package the store has never seen, delivered by the listing, needs no lookup (the new event carries no download link; accepted, see "Update source"). A known package whose listing version differs from the stored one gets a full lookup on every tick, scan or not (store.getKnownVersions over the page's ids, 20 rows), at most hexiumListingLookupsPerTick (6) per tick with a window that rotates by tickIndex, so a package that keeps failing cannot starve the others. The listing item has no download_url, and the package upsert replaces download_url wholesale on a version change (not a COALESCE), so a snapshot taken from the listing would erase a good link. A package whose lookup failed, was unreadable or fell beyond the cap is dropped from the tick's result: its stored version stays, the next tick looks it up again, and the poll warns listing updates held back until their lookup succeeds: N. This also runs when the index scan is skipped or fails. On a scan tick a package that is already resolved from the listing is not looked up a second time for the index.
• Cold start: the index seeds every package as a lean snapshot (index version, size, default flags, no metadata) in hexiumSeedSlices (8) stable slices, one per poll, slice = hash of namespace-name modulo 8. Seeded rows are never emitted; any later event for them goes through a lookup that supplies real flags, and the stores' upserts keep flags sticky and never overwrite richer fields with nulls.
• Reconciliation: the same comparison over the whole index, up to hexiumLookupsPerReconcile (20) lookups per run.

Changelog, per version

[source] Same shape as Thunderstore, on https://hexium.gg:

GET /api/experimental/package/{namespace}/{name}/{version}/changelog/
GET /api/experimental/package/{namespace}/{name}/{version}/readme/

Other

[source] /api/v1/package-listing-index/ and /api/v1/package-listing-chunk/ exist, gzip. Not used.

[source] /api/experimental/frontend/p/{namespace}/{name}/ returns full package detail including versions[], last_updated, markdown.

---

Nexus Mods

Base: https://api.nexusmods.com. Auth: personal API key in the apikey header. Disabled by default — see docs/legal.md.

[source]

GET /v1/games/{domain}/mods/updated.json?period=1d      # also 1w, 1m
GET /v1/games/{domain}/mods/latest_added.json
GET /v1/games/{domain}/mods/{id}.json
GET /v1/games/{domain}/mods/{id}/changelogs.json        # keyed by version
GET /v1/games/{domain}/mods/md5_search/{md5}.json

updated.json returns ids and timestamps only — names and images need a second call per mod, which is what consumes quota. Cache mod metadata indefinitely and refresh only on version change.

changelogs.json returns an object keyed by version number mapping to an array of lines. Already split per version, unlike the Thunderstore side.

Rate limits: 2000 requests/hour, 20,000/day per key. Every response carries X-RL-* headers — read them and self-throttle rather than counting locally.

GraphQL v2 at https://api.nexusmods.com/v2/graphql has a separate rate limit pool and does not consume v1 quota. Most of it is reachable without authentication; some operations need OAuth. Useful for batch metadata lookups.

contains_adult_content must be filtered — see docs/legal.md. It fails closed: only the boolean false marks a mod safe, so a missing, null or non-boolean value marks it adult.

Timestamps (latest_file_update, updated_timestamp, created_timestamp) are epoch seconds; a value outside 1970-9999 (or non-numeric) makes that row unusable and it is skipped instead of failing the whole poll. Changelog responses are refused above 128 KB, and a changelogs.json entry that is not a list of strings is ignored line by line.

---

Fields the messages use

• Download link (PackageSnapshot.downloadUrl). Thunderstore: https://thunderstore.io/package/download/{namespace}/{name}/{version}/, built by the adapter (each segment percent-encoded; null for a ./.. segment). [live] (2026-09) it answers 302 to ccdn.thunderstore.io. Hexium: latest.download_url of the per-package lookup, kept only when it is https on hexium.gg or a subdomain (for example cdn.hexium.gg), without credentials and within 512 characters, else null. Hexium listing items and lean seed rows carry none.
• Total downloads (PackageSnapshot.downloads). Thunderstore listing download_count, Hexium lookup total_downloads, Hexium listing item download_count; accepted only as a non-negative safe integer, else null. Lean seed rows carry none.
• Likes (PackageSnapshot.likes). Thunderstore listing item rating_count [live] (2026-09-27: 639 for denikson-BepInExPack_Valheim, 1422 for ebkr-r2modman), Hexium lookup rating_score and Hexium listing item rating_score [live] (an integer; 0 for most packages, 9-11 for popular ones). Accepted only as a non-negative safe integer (count()), else null; the stored value is the latest non-null one. Hexium lean seed rows and Nexus carry none.
• Website (PackageSnapshot.websiteUrl). Hexium lookup latest.website_url [live] (for example a discord.gg invite or a GitHub repository); Thunderstore latest.website_url of the package endpoint above, fetched per delivered event. Kept only when it is an http(s) URL without credentials, at most 512 characters both as received and after URL.href normalisation (websiteUrl() in src/sources/guards.ts); an empty, non-string, relative, non-http(s) or credential-bearing value is null. The host is not restricted: it is only ever shown as a link (the renderer validates it again), never fetched. Hexium listing items and lean seed rows carry none; Nexus carries none. The stored value is the latest non-null one, and the details phase writes it with the event's changelog in one batch (Store.setEventDetails).

---

Rules for all upstreams

• Send a descriptive User-Agent: project name, version, repository URL. This is the single thing that distinguishes an integrator from a scraper when an admin reads the logs. Workers egress from shared Cloudflare IPs — getting banned harms other people.
• Store ETag per source in D1 and send If-None-Match. A 304 costs no bandwidth, no parsing and no CPU.
• Respect Cache-Control.
• Treat every upstream response as untrusted input. Descriptions and changelogs are user-submitted; sanitise before putting them in an embed.