Skip to content

[quality] data/kubecon-na-2026-end-user-talks.json has no integrity test, so a corrupt snapshot passes CI #1382

Closed

Description

@hivecommons-hive

Finding

data/kubecon-na-2026-end-user-talks.json is the only committed dataset under
data/ with no integrity gate of its own.

Every other dataset has one or both of:

  • a scripts/validate-*.mjs wired into the validate job in
    .github/workflows/ci.yml (all 13 validate:* npm scripts run there), or
  • a tests/*-data.test.mjs that asserts the committed file's own invariants
    (tests/awards-data.test.mjs, tests/members-data.test.mjs,
    tests/community-roster-data.test.mjs, tests/milestones-data.test.mjs,
    tests/community-people-data.test.mjs).

The talks snapshot has neither. The two tests that read it are rendering
contracts, not data contracts:

  • tests/end-user-talk-schedule.test.mjs renders
    src/components/EndUserTalkSchedule/index.js and compares the output to
    expectations it derives from the same file. Both sides move together, so a
    corrupt snapshot stays green.
  • tests/collect-end-user-talks.test.mjs exercises
    scripts/collect-end-user-talks.mjs against a synthetic Sessionize fixture.
    It proves the collector is correct; it never looks at the committed output.

So the following all pass CI today, unchanged:

  • duplicate day.id values — the component emits id={day.id} on each <h5>
    and href={'#' + day.id} in the nav, so a duplicate silently produces
    duplicate DOM ids and jump links that land on the wrong day. The existing
    test compares nav hrefs to day ids, and both are derived from the same
    array, so it cannot see this.
  • duplicate talk.id values across the file
  • a slot whose start/end fall on a different calendar date than the day
    that contains it
  • end at or before start
  • startLabel/endLabel that disagree with start/end
  • days or slots out of chronological order, which the component renders in
    array order
  • a generatedAt in the future, or a talkCount that disagrees with the
    number of talks actually present

The file is machine-generated and carries "Do not hand-edit.", but it is
committed, reviewed and refreshed by hand (npm run collect:end-user-talks is
not run by any workflow), so a bad or partial snapshot reaches main through
the ordinary PR path.

Coverage evidence

This is not a line-coverage gap, and no coverage number will surface it:

  • Unit: npm run test:unit:coverage (node 26.10.0, this checkout at
    f5586b0) reports src files | 100.00 | 100.00 | 9405/9405 lines | 2802/2802 regions.
  • End-to-end: e2e-coverage artifact 11783589900 from run
    38334322389
    (report.txt, status passed, run id 38334322389-1) reports
    src/components/EndUserTalkSchedule/index.js | 100.00 | 100.00.

Both lanes execute every line of the component against the committed data, and
neither can tell a correct snapshot from a corrupt one. The gap is in asserted
behaviour.

Recommendation

Add tests/end-user-talks-data.test.mjs, following the
tests/awards-data.test.mjs pattern: load the committed JSON and assert the
invariants scripts/collect-end-user-talks.mjs guarantees by construction —

  • top-level shape: description, sourceUrl, apiUrl, category,
    generatedAt, timezone are non-empty strings; days is a non-empty
    array
  • generatedAt parses and is not in the future
  • talkCount equals the number of talks reachable through
    days[].slots[].talks[]
  • day.date is a YYYY-MM-DD calendar date; day.id and day.date are
    each unique across the file
  • day.id is the collector's derivation of day.label (lowercase, commas
    dropped, spaces to hyphens), and day.label is the UTC long-form
    rendering of day.date
  • days are in ascending date order, and each day's slots are in ascending
    start order
  • every slot start/end is an event-local YYYY-MM-DDTHH:MM:SS stamp
    beginning with its day's date, with end strictly after start
  • startLabel/endLabel match the collector's 12-hour rendering of
    start/end
  • (start, end) is unique within a day — the collector folds equal pairs
    into one slot
  • every slot has at least one talk; talk.title, talk.speakers and
    talk.room are non-empty strings; talk.id is unique across the file

Deliberately out of scope, to stay disjoint from #1375: nothing about
talk.url's scheme, host or shape. That file's link safety is that PR's
ground.

Priority

  • Impact: medium
  • Effort: low

Filed by quality agent (hold-gated mode)

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed
    on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions