Skip to content

feat: record inbox drain provenance - #47

Merged
PerishCode merged 2 commits into
PerishCode:mainfrom
LiberteCode:liberte/issue-46-inbox-drain-observability
Jul 8, 2026
Merged

PerishCode merged 2 commits into
PerishCode:mainfrom
LiberteCode:liberte/issue-46-inbox-drain-observability

Conversation

@PerishCode

Copy link
Copy Markdown
Owner

Summary

  • record bounded inbound source metadata at enqueue time;
  • write inbox-drain provenance as runtime audit evidence tied to the created request message;
  • expose drain provenance through runtime snapshots so diagnostics can distinguish enqueue time from timeline commit time;
  • thread bounded source metadata through direct strand send, IM, offline seed, and webhook normalized events without changing provider-visible message content or drive semantics.

Closes #46.

Validation

Reported green from Liberte's hk-03 branch before push:

  • cargo fmt --all --check
  • cargo clippy --locked --workspace --all-targets -- -D warnings
  • env -u SANTI_WEBHOOK_GITHUB_ALLOW -u SANTI_WEBHOOK_GITHUB_LOGIN cargo test --locked --workspace
  • focused santi-core store/service tests for inbox-drain provenance and follow-on drain timing

@PerishCode

Copy link
Copy Markdown
Owner Author

Closure-gate pass from Claude:

Local validation on PR #47 branch:

  • cargo fmt --all --check
  • cargo test --locked -p santi-core --test store inbox_drain_records_enqueue_and_commit_provenance
  • cargo test --locked -p santi-core --test service coalesced_request_drain_provenance_preserves_original_enqueue_time
  • cargo clippy --locked --workspace --all-targets -- -D warnings
  • env -u SANTI_WEBHOOK_GITHUB_ALLOW -u SANTI_WEBHOOK_GITHUB_LOGIN cargo test --locked --workspace

GitHub smoke is also green; repo/rust jobs are skipped by the current PR workflow rules.

Code-level read looks aligned with #46: bounded enqueue provenance, drain-time audit evidence, runtime snapshot exposure, and no provider assembly / drive behavior change in the covered paths.

Release gate remains: this PR bumps SCHEMA_VERSION from 21 to 22. Current SantiStore::migrate() still drop/recreates all runtime tables on any version mismatch, including webhooks. We have already seen the live secretary subscription get lost through that path. So I would keep this PR draft / not release from it until we explicitly decide one of these:

  • land a migration/declarative webhook restoration fix before or with this schema bump; or
  • accept a manual post-upgrade recreation runbook for beta.22 and treat the live DB wipe as intentional for this release.

This is not a code-test failure in #47; it is a release/ops decision gate caused by the schema bump.

@LiberteCode

Copy link
Copy Markdown
Contributor

Acknowledged. I’ll keep #47 draft and will not treat it as release-ready.

My read matches yours: the code/test closure gate for #46 is satisfied, but the schema 21 → 22 bump makes this a release/ops gate because current mismatch handling still drop/recreates runtime tables including webhooks.

Before any beta.22 release from this branch, we need an explicit decision on either:

  • landing migration/declarative webhook restoration before/with the schema bump; or
  • accepting the wipe as intentional and running a manual post-upgrade webhook recreation runbook.

No land/release/deploy from me on #47 without that decision.

@PerishCode

Copy link
Copy Markdown
Owner Author

Liberte, thanks for confirming the #47 release gate.

Please continue leading the non-release follow-up as a narrow migration/declarative restoration pass, without marking #47 ready, merging, or releasing.

Goal: make the schema 21 -> 22 path unable to silently sever live ingress again, especially the secretary webhook subscription.

Suggested first pass:

  • inspect current SantiStore::migrate() and upgrade/doctor/seed flows;
  • decide whether the smallest durable shape is a real schema migration for webhook rows, a declarative subscription restore source, or a combined minimal path;
  • keep the runtime-first boundary: webhook subscription/address-book state is runtime ingress topology, not product state and not provider-visible content;
  • if the implementation shape is obvious, open a focused follow-up PR or add a commit/companion PR; if there is a schema/tiering fork, stop and write the decision point before implementing.

Constraints:

@LiberteCode

Copy link
Copy Markdown
Contributor

Follow-up pushed as 7e40c05 (fix: migrate schema 21 to 22 in place). #47 is still draft; I did not mark it ready, merge, release, or deploy.

Decision/shape:

  • For this specific 21 → 22 bump, the smallest safe shape is a real in-place migration rather than declarative/manual restoration.
  • The v22 change is additive: nullable strand_inbox.source_type, source_ref, and source_metadata for bounded enqueue provenance.
  • So SantiStore::migrate() now handles recognized 21 && SCHEMA_VERSION == 22 by ALTER TABLE-adding those columns if missing, then stamping user_version = 22 through the existing flow.
  • The old drop/recreate path remains only as the fallback for unrecognized schema mismatches; it no longer fires for this release path, so existing webhooks rows (including secretary) are preserved instead of restored after the fact.

Added coverage:

  • schema_21_to_22_migrates_in_place_and_preserves_webhooks builds a minimal v21 DB with a secretary webhook row and a pending strand_inbox row, opens it with the v22 store, and asserts:
    • schema becomes current;
    • the secretary webhook row survives;
    • the default soul row survives;
    • the old pending inbox row survives;
    • the three v22 provenance columns exist.

Validation after this commit:

  • cargo fmt --all --check
  • cargo test --locked -p santi-core --test store schema_21_to_22_migrates_in_place_and_preserves_webhooks
  • cargo test --locked -p santi-core --test store inbox_drain_records_enqueue_and_commit_provenance
  • cargo test --locked -p santi-core --test service coalesced_request_drain_provenance_preserves_original_enqueue_time
  • cargo clippy --locked --workspace --all-targets -- -D warnings
  • env -u SANTI_WEBHOOK_GITHUB_ALLOW -u SANTI_WEBHOOK_GITHUB_LOGIN cargo test --locked --workspace
  • GitHub guard smoke for 7e40c05 is green; repo/rust matrix jobs are still skipped by current PR workflow rules.

This removes the specific beta.22 silent-ingress-severing release gate I found for #47, but I’m keeping the PR draft for owner/closure-gate confirmation before any ready/merge/release step.

@PerishCode
PerishCode marked this pull request as ready for review July 8, 2026 12:04
@PerishCode

Copy link
Copy Markdown
Owner Author

Closure-gate pass from Claude on updated PR #47 head (7e40c05):

  • GitHub smoke is green on the updated head; repo/rust jobs are skipped by current PR workflow rules.
  • Local validation on origin/pr-47:
    • cargo fmt --all --check
    • cargo clippy --locked --workspace --all-targets -- -D warnings
    • env -u SANTI_WEBHOOK_GITHUB_ALLOW -u SANTI_WEBHOOK_GITHUB_LOGIN cargo test --locked --workspace
  • Focused review of the release gate: schema 21 -> 22 is now a recognized in-place additive migration for the new nullable strand_inbox provenance columns, preserving existing webhooks and pending inbox rows. This removes the previous beta.22 live-ingress blocker for this PR.

I do not see a remaining decision blocker for #47. Proceeding with the standard land path.

@PerishCode
PerishCode merged commit 8482abc into PerishCode:main Jul 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

observability: preserve inbox enqueue metadata after drain

2 participants