v0.34.20

Changed

• Rate-limited (429/403/503) tasks now retry up to 8 times before hard-failing,
  raised from 3, so a domain that throttles hard is paced-and-deferred rather
  than failed mid-job. Tunable via GNH_RATE_LIMIT_MAX_RETRIES (previously
  documented but never wired).
• Every job now starts paced instead of bursting the full per-job concurrency at
  a domain. Previously only a never-crawled domain got a warm-up delay; a domain
  with a learned (or zero) adaptive delay seeded un-paced and could trigger an
  instant 429 storm. Job starts are now floored to a minimum seed delay, so a
  previously-crawled domain ramps up from a few concurrent requests and widens
  on success. Tunable via GNH_PACER_START_FLOOR_DELAY_MS (0 restores the
  prior behaviour).
• Dashboard and Webflow job lists now refresh via adaptive polling (500 ms while
  a job is active, 1 s when idle) instead of Supabase Realtime. public.jobs
  was removed from the supabase_realtime publication because decoding its WAL
  on every job-counter change was ~14% of total database time, billed
  continuously regardless of how many dashboards were subscribed.
  notifications realtime is unchanged. Reverse with
  ALTER PUBLICATION supabase_realtime ADD TABLE public.jobs;.

Removed

• Redundant idx_domain_hosts_domain_id index (0 recorded scans; the composite
  (domain_id, host) unique index already serves every domain_id-prefix
  lookup), removing write overhead on every domain_hosts upsert.

v0.34.19

Changed

• Pin all third-party GitHub Actions to commit SHAs and set
  persist-credentials: false on checkout steps, completing the workflow
  hardening deferred from PR #392.

v0.34.18

Changed

• Deploy and review-app images now build on the Blacksmith CI runner via
  useblacksmith/build-push-action, with BuildKit cache mounts for the Go
  module and build caches, instead of Fly's shared remote builder. This removes
  the recurring error releasing builder: deadline_exceeded flake and makes
  dependency compiles incremental across runs, cutting build time substantially.

CLI 0.1.20

CLI release cli-v0.1.20

v0.34.17

Fixed

• WAF circuit breaker no longer trips on recoverable Cloudflare Cf-Mitigated
  values (challenge, jschallenge, managed_challenge, rate_limited); the
  403/429 status code still drives pacer back-off. Only block (and unknown
  values) trips the breaker.

Added

• Pacer warm-up floor: never-crawled domains seed adaptive_delay_ms to
  GNH_PACER_WARMUP_DELAY_MS (default 2000) instead of 0, so the per-domain
  inflight cap is active from the first dispatch. Steps down via the existing
  success path.

v0.34.16

Changed

• Retired the legacy task-html Supabase Storage bucket. Page HTML has been
  written directly to Cloudflare R2 since 2026-04-25, so the bucket was no
  longer referenced by any code path but had retained the objects written during
  the four-week window when it was the hot store. The accumulated bytes pushed
  the Supabase project past its 100 GB allowance and triggered connection-slot
  restrictions on the pooler, surfacing as pgconn.ConnectError events in
  Sentry (HOVER-JG). The migration drops only the service-role RLS policy on
  storage.objects. Removal of the bucket row itself cannot be done via SQL
  (Supabase blocks direct deletes from storage.buckets with SQLSTATE 42501)
  and must be performed via the Supabase Storage dashboard or API as a manual
  operational step, after the bucket has been emptied.
• Cleared dangling task-html pointers on the tasks table. Rows written
  between 2026-03-21 and 2026-04-25 had html_storage_bucket = 'task-html' and
  a html_storage_path referencing the now-removed bucket. Both columns are
  NULLed for those rows; the remaining HTML metadata columns
  (html_content_type, html_content_encoding, html_size_bytes,
  html_compressed_size_bytes, html_sha256, html_captured_at) are kept for
  historical analysis. The html_storage_* columns remain in active use for
  newer rows, which point at the Cloudflare R2 bucket.

v0.34.15

Fixed

• fly-autoscaler no longer logs
  metrics collection failed: empty prometheus result once a minute on both
  hover-autoscaler-worker and hover-autoscaler-analysis. The broker gauges
  (bee_broker_stream_length, bee_broker_scheduled_zset_depth) are
  synchronous OTel Int64Gauges, which only emit when Record() lands inside a
  collect interval; during idle the series goes stale in Fly's managed
  Prometheus and the autoscaler's PromQL returns no result. The autoscaler
  queries now wrap with or on() vector(0) so an empty result collapses to zero
  rather than erroring. Scaling behaviour is unchanged at idle (the existing
  max(1, …) floor already kept a single machine running). Trade-off documented
  inline: a true Redis outage now reads 0 instead of producing a series gap,
  so the autoscaler scales to MIN=1 rather than holding count — acceptable
  because idle workers can't crawl during an outage anyway and restart cleanly
  once Redis recovers. The full fix (async observable gauges) is tracked in a
  follow-up issue.

Security

• Bump github.com/jackc/pgx/v5 from v5.7.6 to v5.9.2 to resolve a
  memory-safety vulnerability (Dependabot alert #54).
• Bump @webflow/webflow-cli from ^1.12.4 to ^1.21.0 in
  webflow-designer-extension-cli/ to clear transitive dev-dep vulnerabilities
  (axios, follow-redirects, fast-uri, babel, postcss). Webflow extension is
  dev-only tooling and does not ship to production.

v0.34.14

Security

• Bump github.com/jackc/pgx/v5 from v5.7.6 to v5.9.2 to resolve a
  memory-safety vulnerability (Dependabot alert #54).
• Bump @webflow/webflow-cli from ^1.12.4 to ^1.21.0 in
  webflow-designer-extension-cli/ to clear transitive dev-dep vulnerabilities
  (axios, follow-redirects, fast-uri, babel, postcss). Webflow extension is
  dev-only tooling and does not ship to production.

v0.34.13

Fixed

• App, worker, and analysis binaries no longer Fatal on the first Redis PING
  failure at startup. The ping is now wrapped in a bounded retry loop (30 s
  total, 3 s per attempt, capped exponential backoff) so the binary rides out
  the Upstash-on-Fly cold-start window that briefly closes connections with EOF
  on freshly-provisioned review apps. Production behaviour is unchanged — a
  healthy Redis still succeeds on the first attempt and persistent
  misconfiguration still fails fast. Resolves the recurring EOF burst on every
  PR preview deploy (Sentry: HOVER-JX, HOVER-MD, HOVER-JZ).

v0.34.12

Changed

• JobManager.GetRobotsRules now caches results per normalised domain (1h
  positive TTL, 60s negative TTL), and collapses concurrent misses onto a single
  origin fetch via singleflight. A long crawl previously refetched /robots.txt
  every five minutes (stream worker's job-info TTL) and a 429 on /robots.txt
  returned on the next read; both are now bounded.