Skip to content

fix(core): UTC stamps read as local time, host-dependent sitemap MIME, unclamped haversine, uncatalogued rule - #60

Open
VarunGore36 wants to merge 5 commits into
Berry-Mappemonde:mainfrom
VarunGore36:fix/utc-timestamps-and-platform-fixes
Open

VarunGore36 wants to merge 5 commits into
Berry-Mappemonde:mainfrom
VarunGore36:fix/utc-timestamps-and-platform-fixes

Conversation

@VarunGore36

@VarunGore36 VarunGore36 commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Five defects found while auditing the backend against its tests and contracts. Each commit is self-contained and comes with regression tests.

1. …Z timestamps were read as machine-local time (61dd17d)

generated_at / checked_at / enriched_at are written in UTC (time.strftime(…, time.gmtime())), but the three readers converted them with time.mktime(), which interprets the parsed struct as local time:

  • app/services/poe_pipeline.py:is_stale — 180-day staleness badge
  • app/services/marina_enrich.py:is_stale — 365-day enrichment window
  • app/routers/formalities.py:_ts_of — auto-refresh cycle

On a machine at UTC+5:30 the windows slid 5 h 30 into the future (a zone generated today already looked stale); west of UTC they slide the other way (a two-day-old zone looks fresh and is never refreshed). calendar.timegm() reads the same strings as the UTC instants they are. routers/marinas.py already compared the UTC strings directly — the correct pattern was next door.

Tests: backend/tests/test_stale_timestamps_utc.py — exact epoch, staleness boundary ± 1 day, four timezones (UTC, Asia/Kolkata, America/Los_Angeles, Pacific/Kiritimati).

2. sitemap.xml inherited the host MIME table (36446d5)

The SPA route served build files through FileResponse without a media type, so the Content-Type came from the host's mimetypes database: application/xml on the CI runners, text/xml on machines whose /etc/mime.types lists xml that way (macOS). test_spa_static_head asserts application/xml for Search Console and failed there. The type is now pinned in app/main.py.

3. marina_build.haversine_nm raises ValueError: math domain error (5512dc7)

Third haversine in the repo, third time lucky: geo.haversine_km was fixed for near-antipodal points (a > 1.0 at floating-point rounding) and climatology_common.haversine_nm already guards asin(sqrt(a)), but this one did not. Same known trigger pair, same crash. One min(1.0, …), plus a regression test over both sibling implementations.

4. formalities.catalog_min_names was read but never catalogued (dbe99c8)

extract.py:catalog_is_sufficient resolves "enough names for a list without coordinates" via get_rule("formalities.catalog_min_names", 8) while the rule did not exist in backend/data/run_rules.json: not adjustable by a run, absent from every params.rules snapshot — against the catalogue principle ("a number has an interval and a why"). Entry added with the value already in use (8, interval 4–20).

5. test_memoire_grilles could not run on macOS (a180f89)

Its child process read /proc/self/status for an informational RSS printout and aborted before reaching it — the ceiling assertions (resident bytes, land mask, app import) are portable and ran either way. Now falls back to getrusage (ru_maxrss is bytes on macOS, KiB on Linux).

Verification

  • backend: full pytest suite with the CI recipe (Mongo seeded, API up, requirements.txt env) — 1331 + 143 passed. The only remaining reds are 4 assertions on osm_confidence counts that require the CI's scripts/enrich_ci_osm.py Overpass step (data state, not code).
  • frontend: 63/63 Jest, production build clean.
  • infra/vps/ci (37), scripts/satellite (5), scripts/audit_tags: green.

Not in this PR

The macOS health probe (#40), the SearXNG default-port collision (#39) and the 40k LLM reader context (#37) are already proposed in their own PRs — deliberately not repeated here. The branch is based on main at d85f4d9 so the diff shows only these five fixes.

generated_at / checked_at / enriched_at are written in UTC
(time.strftime(..., time.gmtime())), but the three readers converted them
with time.mktime(), which interprets the parsed struct as *local* time. The
staleness windows (formalities 180 d, marinas 365 d) and the auto-refresh
cycle then slid by the machine timezone offset (±14 h): a zone generated
today could be 'stale' east of UTC, a freshly enriched marina could be
re-enriched, and a two-day-old zone could look fresh west of UTC.

calendar.timegm() reads the same strings as the UTC instants they are.
Regression tests pin the epoch and the boundaries under four timezones.
… table

The SPA route served build files through FileResponse without a media type,
so the Content-Type came from the host's mimetypes database: application/xml
on the CI runners, text/xml on machines whose /etc/mime.types lists xml that
way (macOS). test_spa_static_head asserts application/xml for sitemap.xml
(Search Console) and failed there. Pin the type instead of inheriting the
platform's.
test_import_application_reste_sous_le_plafond spawns a child that reads
/proc/self/status for its RSS printout and raised SystemExit when the file
did not exist — so the test could not run at all on macOS, where there is no
/proc. The ceiling assertions (resident_bytes, is_land, app import) are
portable and ran either way; only the printout needs a getrusage fallback
(ru_maxrss is bytes on macOS, KiB on Linux).
catalog_is_sufficient réglait « assez de noms pour une liste sans
coordonnées » via get_rule('formalities.catalog_min_names', 8) alors que la
règle n'existait pas dans le catalogue : valeur non réglable par une run,
absente du snapshot params.rules, en contradiction avec le principe du
catalogue (un nombre a un intervalle et un pourquoi). Entrée ajoutée avec
la valeur déjà utilisée (8), intervalle 4-20.
Third haversine in the repo, third time lucky: geo.haversine_km was fixed
for near-antipodal points (a > 1.0 at floating-point rounding, asin(sqrt(a))
raising ValueError) and climatology_common.haversine_nm already guards it,
but marina_build.haversine_nm did not. Same trigger pair, same math domain
error. One min(1.0, ...) and a regression test over both sibling
implementations.
@VarunGore36
VarunGore36 force-pushed the fix/utc-timestamps-and-platform-fixes branch from 286d012 to 5512dc7 Compare October 28, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant