The homepage's cache has no writer, so give it one

Following on from 054efbe, which was correct and not sufficient. Widening
INDEX_MAX_STALE_MS stopped readers blocking on an entry that existed; it did
nothing about there being no way to create one.

`remember` fills itself from readers: past the TTL a reader starts a background
`refresh()`. That only works when the computation fits in the timeout the page
is willing to wait, and this one never does -- `countFeedsByKind` is a `group by`
over 476,000 rows with no covering index, so every refresh is abandoned at 20s
and the entry is never renewed. It could only ever age out.

Which is what happened in production within minutes of 054efbe deploying. Redis
no longer had the key, so the homepage fell to the branch below the widened
ceiling: waited its 20 seconds for a computation that was always going to fail,
then rendered the empty-directory branch. A directory of 476,000 feeds reporting
that it has none -- worse than the slow page it replaced, and my fault for
changing when the entry is served without checking what writes it.

So the writer moves to the poller, where `warmStatsCache` already does exactly
this for `categoryStats` on a connection with a 150-second deadline. Same shape:
its own patient connection, never throws, logs `directory-warmed` /
`directory-warm-skipped`. It runs after the category warm rather than beside it
so the two whole-table scans queue instead of competing, and it is not gated on
the category warm succeeding -- the keys fail independently.

The web side is untouched. `remember` finds a warm entry and serves it, and its
own doomed refreshes become harmless rather than load-bearing.

Verified: suite green (11 packages, fail 0), `pnpm build` compiles, and
`warmDirectoryCache` resolves through the package entry point. The key name and
the `{rows, total, byKind}` shape are a contract with
apps/web/src/lib/directory.js that no test covers -- a drift there is a silently
empty homepage -- so it is checked against the live poller log and the page
after deploy rather than only in CI.

Still does not restore the crawler: the write path stays wedged until the Turso
quota is raised or resets on 1 Sep.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: ralyodio

apps/poller/src/index.js

@@ -7,6 +7,7 @@ import {
   alerts,
   takeWriteTally,
   warmStatsCache,
+  warmDirectoryCache,
 } from '@rssamplifier/db';
 import {
   crawlDue,
@@ -872,6 +873,21 @@ async function statsTick() {
   warming = true;
   try {
     const result = await warmStatsCache({ log });
+
+    // The homepage's counts, warmed on the same tick and for the same reason.
+    // Its `remember` entry can only ever be written from here: a reader's own
+    // background refresh is capped at the 20 seconds the page will wait, and
+    // `countFeedsByKind` needs far longer, so without this the entry ages out
+    // and the directory renders itself as empty.
+    //
+    // Deliberately after the category warm rather than beside it: both are
+    // whole-table reads, and running them in sequence keeps one long scan on
+    // this connection at a time instead of two competing for the same instance.
+    // Not gated on `result.ok` either — the two keys fail independently, and a
+    // category breakdown that timed out says nothing about whether the counts
+    // will.
+    await warmDirectoryCache({ log });
+
     if (!result.ok) return;
   } finally {
     warming = false;

packages/db/index.js

@@ -1,7 +1,7 @@
 export { connect, newId, nowIso } from './src/client.js';
 export { createWriteWorker, WRITE_QUEUE, takeWriteTally } from './src/writeQueue.js';
 export { remember, redisClient, primeCache, resetCacheState } from './src/cache.js';
-export { warmStatsCache } from './src/statsWarmer.js';
+export { warmStatsCache, warmDirectoryCache } from './src/statsWarmer.js';
 export { migrate } from './src/migrate.js';
 export * as q from './src/queries.js';
 export * as accounts from './src/accounts.js';

packages/db/src/statsWarmer.js

@@ -48,6 +48,9 @@ const WARM_TIMEOUT_MS = 150_000;
 /** How much growth history the breakdown carries; matches the web's GROWTH_DAYS. */
 const GROWTH_DAYS = 30;
 
+/** How many blogs the directory index lists; matches the web's `INDEX_LIMIT`. */
+const INDEX_LIMIT = 60;
+
 /**
  * Compute the category breakdown on a patient connection and cache it.
  *
@@ -88,3 +91,71 @@ export async function warmStatsCache(opts = {}) {
     try { db?.close(); } catch { /* closing a broken client is not a failure */ }
   }
 }
+
+/**
+ * The same treatment for the homepage's three reads.
+ *
+ * ## Why this has to exist, and not just a wider staleness window
+ *
+ * `remember` fills its own cache from readers: past the TTL a reader triggers
+ * `refresh()`, which recomputes behind them. That works only if the computation
+ * can finish inside the timeout the *page* is willing to wait — and for this key
+ * it cannot. `countFeedsByKind` is a `group by category` over 476,000 rows with
+ * no covering index, so every refresh is abandoned at 20 seconds and the entry
+ * is never renewed.
+ *
+ * Which leaves the entry ageing out with nothing able to replace it. Widening
+ * `INDEX_MAX_STALE_MS` bought time and not a fix: once Redis dropped the key the
+ * homepage had nothing to serve, waited its 20 seconds for a computation that
+ * was always going to fail, and rendered the empty-directory branch — a
+ * directory of 476,000 feeds reporting itself as having none. That is worse than
+ * the slow page it replaced, and it is what production did on 2026-08-25 within
+ * minutes of the staleness change deploying.
+ *
+ * So the writer moves here, where a 150-second deadline is allowed and the read
+ * actually completes. The web side keeps `remember` exactly as it is: it finds a
+ * warm entry and serves it, and its own doomed refreshes become harmless.
+ *
+ * Priming on the same tick as the category breakdown is deliberate — both are
+ * whole-table reads and running them together keeps the number of full scans
+ * this database sees to one burst an hour rather than two.
+ *
+ * @param {{ log?: (event: string, fields?: object) => void, client?: any }} [opts]
+ * @returns {Promise<{ ok: boolean, ms: number, cached: boolean, error?: string }>}
+ */
+export async function warmDirectoryCache(opts = {}) {
+  const started = Date.now();
+  const log = opts.log ?? (() => {});
+
+  /** @type {import('@libsql/client').Client|null} */
+  let db = null;
+  try {
+    db = connect({ timeoutMs: WARM_TIMEOUT_MS, queue: false });
+
+    // Shape and key must match `apps/web/src/lib/directory.js` exactly: it
+    // destructures all three, and a mismatch here is an empty homepage that no
+    // error anywhere would explain.
+    const [rows, total, byKind] = await Promise.all([
+      q.listFeeds(db, { limit: INDEX_LIMIT }),
+      q.countFeeds(db),
+      q.countFeedsByKind(db),
+    ]);
+    const cached = await primeCache('directoryIndex', { rows, total, byKind }, {
+      client: opts.client,
+    });
+
+    const ms = Date.now() - started;
+    log('directory-warmed', { ms, total, rows: rows?.length ?? 0, cached });
+    return { ok: true, ms, cached };
+  } catch (error) {
+    const ms = Date.now() - started;
+    // `…-skipped` rather than `…-error`, for the reason above: the page has a
+    // cached answer and the operational-error panel treats a trailing "error"
+    // as an alarm.
+    const reason = error instanceof Error ? error.message : String(error);
+    log('directory-warm-skipped', { ms, reason });
+    return { ok: false, ms, cached: false, error: reason };
+  } finally {
+    try { db?.close(); } catch { /* closing a broken client is not a failure */ }
+  }
+}