Repository navigation
M3/E5/#15 — Ingest contract conformance - #18
Merged
Merged
Conversation
Canonical: Milestone M3 · Epic E5 · Issue #15. Deps: #11 (complete). Eligibility recomputed by parsing origin/main:docs/tis/TIS-v1.0.md §15 — complete {1..14}, eligible {#15, #22, #23, #25, #26, #27, #39}; lowest Order selects #15. First issue of Milestone M3, and the first code in a bounded context. WHAT AN ACQUISITION CHANNEL MUST PROVIDE, AND MAY NEVER LET OUT SourceAdapter descriptor() -> SourceDescriptor acquire(selector) -> RawArtifact + AcquisitionProvenance verify_conformance what an adapter must pass before its data enters the platform boundary.stamp the single sanctioned clock read in the system Credential a secret that cannot be printed or serialised assert_credential_free refuse to let one cross the boundary No channel is implemented. The ISSDC-PRADAN adapter is #16; the SoLEXS and HEL1OS parsers are #17 and #18; the bitemporal write path is #19. A PROTOCOL, NOT A FRAMEWORK ADR-0003 is explicit that it does NOT authorise "a source-plugin registry, a dispatch layer, or a configurable adapter framework", and ADR-0025 lists generic base classes in the same category. So: a typing.Protocol that is inherited from by nothing, constructs nothing and dispatches nothing, plus a checker taking an adapter the caller already holds. A test asserts contract.py never grows register/REGISTRY/get_adapter/dispatch. Under ADR-0025's test this is a free seam — stating what a channel must provide would still be right with exactly one source forever, because the parser downstream has to trust its input regardless of which channel produced it. THREE RULES, EACH MADE STRUCTURAL RATHER THAN REMEMBERED Credentials never leave (STD-19, E5 §13). Credential redacts itself in repr, str AND format, closing the routes secrets actually escape by. The integration test writes to a real log file through six of them and greps the file — not the object, the file. A stronger result than expected fell out: SourceDescriptor has no field a credential could occupy. Every field is a typed domain value; there is no dict, no Any, no extras. A secret cannot be placed in a conforming descriptor at all, so the runtime scan is the backstop for the duck-typed case rather than the primary defence. The test that assumed otherwise was corrected to assert what actually holds. Tier 0 is referenced, never redistributed (ADR-0023, STD-23, E5 §11(iv), §12). RawArtifact has a digest, a size, a way to re-acquire, and NO field that could hold the bytes — a test pins the field set. An artifact that can hold a whole-day archive will hold one, and nobody would see it in review; the object would just be larger. Both times, the second never fabricated (ADR-0004, ADR-0022, E5 §11(i)(ii)). boundary.stamp is the exception TIS §0.4 grants, in one named module so that `grep -rn 'ingest.boundary'` enumerates every clock read in the repository. Nothing here takes a row and returns a time for it: __all__ is exactly {stamp, stamp_from_epoch, monotonic}, asserted, and E5 §11(ii) forbids the rest. THE #13 MECHANISM FIRED, EXACTLY AS BUILT Adding the first module to contexts/ingest turned the gate red on its own: contexts.ingest: declared unpopulated but contains 7 module(s). populated flipped False -> True. That is the staleness check M2/E4/#13 shipped doing the one job it was written for, one issue later, without anyone remembering it existed. FEATURE-LEVEL INTEGRATION — THE FIRST REAL DATA PATH tests/integration/test_ingest_contract_with_platform.py follows scientific data from a channel to a contract-valid, provenance-linked Observation across all six components: kernel mints the digest, ingest stamps the time, domain builds the row, contracts validate it, the manifest records where the Tier 0 bytes stay, and the import rules say who may do it. Both bitemporal paths are followed, because the difference between them is the whole of ADR-0022: a new acquisition carries both times; a historical row carries valid_time and a null ingest_time, has no acquisition to describe, and nothing anywhere would fill it. A fabricated one passes the schema and fails the invariant — the division of labour between the layers, on the case that motivated the ADR. VERIFICATION unit (kernel + domain + contexts) 372 passed (+50) tests/architecture 562 passed tests/integration 97 passed (+26) research/tests/v2 188 passed links / imports / contracts gates clean imports gate: 37 modules, 168 imports, 9 policies CI: `contexts` added to the unit job, named as a root so a new context's tests run the day they are written. Reported as EPD-007. No new job, no new context; still eight. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rexy-5097
pushed a commit
that referenced
this pull request
Sep 11, 2026
Implements `SPEC-parsers` §2.5–§2.9 and §4 for HEL1OS: band light curves, PHA spectra, housekeeping, per-detector GTI, an event reader, and orbit identity with version precedence. No merge API, no coverage map, no aggregation and no dual write — those belong to the write path (#19). GOVERNANCE FIRST. Verifying against all 391 orbits proved three contract checks undecidable at float64 resolution: R-1 H3 rejected 647 of 1,564 spectra products and §2.8's header-span check rejected 94 of 391 housekeeping products, every one of them by one or two representable steps. `SPEC-parsers` r7 therefore defines `col_span`, adds the §5.1 time-representation allowance ε_t = 1 ms — the value CONTRADICTION-006 Defect A approved in code and never recorded in the contract — and gives the §2.5 and §2.8 time checks the F-06 id they lacked. Recorded as CONTRA-007, CLOSED by r7. §2.5's `mjd` non-decreasing rule is falsified by every event HDU in the archive (1,564 of 1,564; 37.9 M backward steps; largest 1,156 s). It is NOT amended: the parser enforces it and fails closed, and the falsification is recorded OPEN as CONTRA-008 for the maintainer to rule on. D6: the missing §2.5 row checks (span, V-EVT-2, `ener>0`), the declared `ener` and `CTR` units, CZT `pix`/`offsetchn`, and every §2.8 decisive column with its declared unit and archive dtype. Only `czt1temp` and `czt2temp` are required finite, as §2.8 states. D3: `_fits.py` moves from `parsers/solexs/` to `parsers/` so both instruments share the no-default FITS accessors without either importing the other, and gains `declared_unit()`. SoLEXS behaviour is unchanged and regression-tested. Verified with the real 391-orbit corpus: links, contracts, imports, architecture 563, unit 583, integration 146. Clean export from the staged tree: unit 581 (2 skipped), integration 100 (22 skipped), every real-data test skipping rather than failing. Archive-wide, 389 of 391 orbits now parse end to end; the 2 that do not are the known duplicate-HK archive defects, still terminating at F-16. Co-authored-by: slazyverse <188746735+slazyverse@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canonical identifier: Milestone M3 · Epic E5 · Issue #15 — ingest contract conformance
Dependencies: #11 ✔ (
6c26b33)Selection: eligibility recomputed from
origin/main:docs/tis/TIS-v1.0.md§15 — complete{1…14}, eligible{#15, #22, #23, #25, #26, #27, #39}; lowest Order selects #15.First issue of Milestone M3, and the first code inside a bounded context.
What an acquisition channel must provide, and may never let out
No channel is implemented. The ISSDC-PRADAN adapter is #16; the SoLEXS and HEL1OS parsers are #17 and #18; the bitemporal write path is #19.
A Protocol, not a framework
ADR-0003 is explicit that it does not authorise "a source-plugin registry, a dispatch layer, or a configurable adapter framework", and ADR-0025 lists generic base classes in the same category. So: a
typing.Protocolinherited from by nothing, constructing nothing, dispatching nothing — plus a checker that takes an adapter the caller already holds. A test assertscontract.pynever growsregister/REGISTRY/get_adapter/dispatch.Under ADR-0025's test this is a free seam: stating what a channel must provide would still be the correct design with exactly one source forever, because the parser downstream has to trust its input regardless of which channel produced it.
Three rules, each made structural rather than remembered
Credentials never leave (STD-19, E5 §13).
Credentialredacts itself inrepr,strandformat— the routes secrets actually escape by. The integration test writes through six of them to a real log file and greps the file, not the object.A stronger result fell out than expected:
SourceDescriptorhas no field a credential could occupy. Every field is a typed domain value; there is no dict, noAny, no extras. A secret cannot be placed in a conforming descriptor at all, so the runtime scan is the backstop for the duck-typed case rather than the primary defence. The test that assumed otherwise was corrected to assert what actually holds.Tier 0 is referenced, never redistributed (ADR-0023, STD-23, E5 §11(iv), §12).
RawArtifacthas a digest, a size, a way to re-acquire, and no field that could hold the bytes — a test pins the field set. An artifact that can hold a whole-day archive will hold one, and nobody would see it in review; the object would just be larger.Both times, and the second never fabricated (ADR-0004, ADR-0022, E5 §11(i)(ii)).
boundary.stampis the exception TIS §0.4 grants, in one named module sogrep -rn 'ingest.boundary'enumerates every clock read in the repository. Nothing here takes a row and returns a time for it:__all__is exactly{stamp, stamp_from_epoch, monotonic}, asserted.The #13 mechanism fired, exactly as built
Adding the first module to
contexts/ingestturned the gate red on its own, before any policy edit:populatedflippedFalse→True. That is the staleness check M2/E4/#13 shipped doing the one job it was written for, one issue later, without anyone having to remember it existed.Feature-level integration — the first real data path
tests/integration/test_ingest_contract_with_platform.pyfollows scientific data from a channel to a contract-valid, provenance-linked Observation across all six components: the kernel mints the digest, ingest stamps the time, the domain builds the row, the contracts validate it, the manifest records where the Tier 0 bytes stay, and the import rules say who may do it.Both bitemporal paths are followed, because the difference between them is the whole of ADR-0022: a new acquisition carries both times; a historical row carries
valid_timeand a nullingest_time, has no acquisition to describe, and nothing anywhere would fill it in. A fabricated one passes the schema and fails the invariant — the division of labour between the layers, on the case that motivated the ADR.Verification
CI impact
contextsadded to theunitjob, named as a root so a new context's tests run the day they are written rather than the day someone remembers this file. Reported as EPD-007. No new job, no new required context — still eight.🤖 Generated with Claude Code