Skip to content

M3/E5/#15 — Ingest contract conformance - #18

Merged
Rexy-5097 merged 1 commit into
mainfrom
feat/m3-e5-issue-15-ingest-contract
Jul 30, 2026
Merged

Rexy-5097 merged 1 commit into
mainfrom
feat/m3-e5-issue-15-ingest-contract

Conversation

@Rexy-5097

Copy link
Copy Markdown
Owner

Canonical identifier: Milestone M3 · Epic E5 · Issue #15 — ingest contract conformance
Dependencies: #11 ✔ (6c26b33)
Selection: eligibility recomputed from origin/main:docs/tis/TIS-v1.0.md §15 — complete {1…14}, eligible {#15, #22, #23, #25, #26, #27, #39}; lowest Order selects #15.

First issue of Milestone M3, and the first code inside a bounded context.

What an acquisition channel must provide, and may never let out

SourceAdapter            descriptor() -> SourceDescriptor
                         acquire(selector) -> RawArtifact + AcquisitionProvenance
verify_conformance       what an adapter must pass before its data enters the platform
boundary.stamp           the single sanctioned clock read in the system
Credential               a secret that cannot be printed or serialised
assert_credential_free   refuse to let one cross the boundary

No channel is implemented. The ISSDC-PRADAN adapter is #16; the SoLEXS and HEL1OS parsers are #17 and #18; the bitemporal write path is #19.

A Protocol, not a framework

ADR-0003 is explicit that it does not authorise "a source-plugin registry, a dispatch layer, or a configurable adapter framework", and ADR-0025 lists generic base classes in the same category. So: a typing.Protocol inherited from by nothing, constructing nothing, dispatching nothing — plus a checker that takes an adapter the caller already holds. A test asserts contract.py never grows register / REGISTRY / get_adapter / dispatch.

Under ADR-0025's test this is a free seam: stating what a channel must provide would still be the correct design with exactly one source forever, because the parser downstream has to trust its input regardless of which channel produced it.

Three rules, each made structural rather than remembered

Credentials never leave (STD-19, E5 §13). Credential redacts itself in repr, str and format — the routes secrets actually escape by. The integration test writes through six of them to a real log file and greps the file, not the object.

A stronger result fell out than expected: SourceDescriptor has no field a credential could occupy. Every field is a typed domain value; there is no dict, no Any, no extras. A secret cannot be placed in a conforming descriptor at all, so the runtime scan is the backstop for the duck-typed case rather than the primary defence. The test that assumed otherwise was corrected to assert what actually holds.

Tier 0 is referenced, never redistributed (ADR-0023, STD-23, E5 §11(iv), §12). RawArtifact has a digest, a size, a way to re-acquire, and no field that could hold the bytes — a test pins the field set. An artifact that can hold a whole-day archive will hold one, and nobody would see it in review; the object would just be larger.

Both times, and the second never fabricated (ADR-0004, ADR-0022, E5 §11(i)(ii)). boundary.stamp is the exception TIS §0.4 grants, in one named module so grep -rn 'ingest.boundary' enumerates every clock read in the repository. Nothing here takes a row and returns a time for it: __all__ is exactly {stamp, stamp_from_epoch, monotonic}, asserted.

The #13 mechanism fired, exactly as built

Adding the first module to contexts/ingest turned the gate red on its own, before any policy edit:

contexts.ingest: declared unpopulated but contains 7 module(s). Update the policy so the rule binds.

populated flipped False → True. That is the staleness check M2/E4/#13 shipped doing the one job it was written for, one issue later, without anyone having to remember it existed.

Feature-level integration — the first real data path

tests/integration/test_ingest_contract_with_platform.py follows scientific data from a channel to a contract-valid, provenance-linked Observation across all six components: the kernel mints the digest, ingest stamps the time, the domain builds the row, the contracts validate it, the manifest records where the Tier 0 bytes stay, and the import rules say who may do it.

Both bitemporal paths are followed, because the difference between them is the whole of ADR-0022: a new acquisition carries both times; a historical row carries valid_time and a null ingest_time, has no acquisition to describe, and nothing anywhere would fill it in. A fabricated one passes the schema and fails the invariant — the division of labour between the layers, on the case that motivated the ADR.

Verification

unit (kernel + domain + contexts)  372 passed   (+50)
tests/architecture                 562 passed
tests/integration                   97 passed   (+26)
research/tests/v2                  188 passed
links / imports / contracts gates  clean
imports gate: 37 modules, 168 imports, 9 policies

clean export (git archive HEAD):
  7 ingest modules · unit 372 · architecture 562 · integration 73 passed
  + 24 pre-existing errors (IMPL-012, reported at #13)

CI impact

contexts added to the unit job, named as a root so a new context's tests run the day they are written rather than the day someone remembers this file. Reported as EPD-007. No new job, no new required context — still eight.

🤖 Generated with Claude Code

Canonical: Milestone M3 · Epic E5 · Issue #15. Deps: #11 (complete).
Eligibility recomputed by parsing origin/main:docs/tis/TIS-v1.0.md §15 — complete {1..14},
eligible {#15, #22, #23, #25, #26, #27, #39}; lowest Order selects #15.

First issue of Milestone M3, and the first code in a bounded context.

WHAT AN ACQUISITION CHANNEL MUST PROVIDE, AND MAY NEVER LET OUT

  SourceAdapter          descriptor() -> SourceDescriptor
                         acquire(selector) -> RawArtifact + AcquisitionProvenance
  verify_conformance     what an adapter must pass before its data enters the platform
  boundary.stamp         the single sanctioned clock read in the system
  Credential             a secret that cannot be printed or serialised
  assert_credential_free refuse to let one cross the boundary

No channel is implemented. The ISSDC-PRADAN adapter is #16; the SoLEXS and HEL1OS
parsers are #17 and #18; the bitemporal write path is #19.

A PROTOCOL, NOT A FRAMEWORK

ADR-0003 is explicit that it does NOT authorise "a source-plugin registry, a dispatch
layer, or a configurable adapter framework", and ADR-0025 lists generic base classes in
the same category. So: a typing.Protocol that is inherited from by nothing, constructs
nothing and dispatches nothing, plus a checker taking an adapter the caller already
holds. A test asserts contract.py never grows register/REGISTRY/get_adapter/dispatch.

Under ADR-0025's test this is a free seam — stating what a channel must provide would
still be right with exactly one source forever, because the parser downstream has to
trust its input regardless of which channel produced it.

THREE RULES, EACH MADE STRUCTURAL RATHER THAN REMEMBERED

Credentials never leave (STD-19, E5 §13). Credential redacts itself in repr, str AND
format, closing the routes secrets actually escape by. The integration test writes to a
real log file through six of them and greps the file — not the object, the file.

A stronger result than expected fell out: SourceDescriptor has no field a credential
could occupy. Every field is a typed domain value; there is no dict, no Any, no extras.
A secret cannot be placed in a conforming descriptor at all, so the runtime scan is the
backstop for the duck-typed case rather than the primary defence. The test that assumed
otherwise was corrected to assert what actually holds.

Tier 0 is referenced, never redistributed (ADR-0023, STD-23, E5 §11(iv), §12).
RawArtifact has a digest, a size, a way to re-acquire, and NO field that could hold the
bytes — a test pins the field set. An artifact that can hold a whole-day archive will
hold one, and nobody would see it in review; the object would just be larger.

Both times, the second never fabricated (ADR-0004, ADR-0022, E5 §11(i)(ii)).
boundary.stamp is the exception TIS §0.4 grants, in one named module so that
`grep -rn 'ingest.boundary'` enumerates every clock read in the repository. Nothing here
takes a row and returns a time for it: __all__ is exactly {stamp, stamp_from_epoch,
monotonic}, asserted, and E5 §11(ii) forbids the rest.

THE #13 MECHANISM FIRED, EXACTLY AS BUILT

Adding the first module to contexts/ingest turned the gate red on its own:

  contexts.ingest: declared unpopulated but contains 7 module(s).

populated flipped False -> True. That is the staleness check M2/E4/#13 shipped doing the
one job it was written for, one issue later, without anyone remembering it existed.

FEATURE-LEVEL INTEGRATION — THE FIRST REAL DATA PATH

tests/integration/test_ingest_contract_with_platform.py follows scientific data from a
channel to a contract-valid, provenance-linked Observation across all six components:
kernel mints the digest, ingest stamps the time, domain builds the row, contracts
validate it, the manifest records where the Tier 0 bytes stay, and the import rules say
who may do it.

Both bitemporal paths are followed, because the difference between them is the whole of
ADR-0022: a new acquisition carries both times; a historical row carries valid_time and
a null ingest_time, has no acquisition to describe, and nothing anywhere would fill it.
A fabricated one passes the schema and fails the invariant — the division of labour
between the layers, on the case that motivated the ADR.

VERIFICATION

  unit (kernel + domain + contexts)  372 passed  (+50)
  tests/architecture                 562 passed
  tests/integration                   97 passed  (+26)
  research/tests/v2                  188 passed
  links / imports / contracts gates  clean
  imports gate: 37 modules, 168 imports, 9 policies

CI: `contexts` added to the unit job, named as a root so a new context's tests run the
day they are written. Reported as EPD-007. No new job, no new context; still eight.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Rexy-5097
Rexy-5097 merged commit 1d65c63 into main Jul 30, 2026
8 checks passed
@Rexy-5097
Rexy-5097 deleted the feat/m3-e5-issue-15-ingest-contract branch July 30, 2026 12:55
Rexy-5097 pushed a commit that referenced this pull request Sep 11, 2026
Implements `SPEC-parsers` §2.5–§2.9 and §4 for HEL1OS: band light curves,
PHA spectra, housekeeping, per-detector GTI, an event reader, and orbit
identity with version precedence. No merge API, no coverage map, no
aggregation and no dual write — those belong to the write path (#19).

GOVERNANCE FIRST. Verifying against all 391 orbits proved three contract
checks undecidable at float64 resolution: R-1 H3 rejected 647 of 1,564
spectra products and §2.8's header-span check rejected 94 of 391
housekeeping products, every one of them by one or two representable
steps. `SPEC-parsers` r7 therefore defines `col_span`, adds the §5.1
time-representation allowance ε_t = 1 ms — the value CONTRADICTION-006
Defect A approved in code and never recorded in the contract — and gives
the §2.5 and §2.8 time checks the F-06 id they lacked. Recorded as
CONTRA-007, CLOSED by r7.

§2.5's `mjd` non-decreasing rule is falsified by every event HDU in the
archive (1,564 of 1,564; 37.9 M backward steps; largest 1,156 s). It is
NOT amended: the parser enforces it and fails closed, and the
falsification is recorded OPEN as CONTRA-008 for the maintainer to rule
on.

D6: the missing §2.5 row checks (span, V-EVT-2, `ener>0`), the declared
`ener` and `CTR` units, CZT `pix`/`offsetchn`, and every §2.8 decisive
column with its declared unit and archive dtype. Only `czt1temp` and
`czt2temp` are required finite, as §2.8 states.

D3: `_fits.py` moves from `parsers/solexs/` to `parsers/` so both
instruments share the no-default FITS accessors without either importing
the other, and gains `declared_unit()`. SoLEXS behaviour is unchanged and
regression-tested.

Verified with the real 391-orbit corpus: links, contracts, imports,
architecture 563, unit 583, integration 146. Clean export from the staged
tree: unit 581 (2 skipped), integration 100 (22 skipped), every real-data
test skipping rather than failing. Archive-wide, 389 of 391 orbits now
parse end to end; the 2 that do not are the known duplicate-HK archive
defects, still terminating at F-16.

Co-authored-by: slazyverse <188746735+slazyverse@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant