📋 Pre-flight Checks
Version: engram v2.0.0-rc.9 (Linux x86_64, WSL2)
📝 Bug Description
Two defects in the same surface, found on a store whose cloud backend was decommissioned months ago. sync_enrolled_projects held 4 projects enrolled since 2026-07-21; the server they replicate toward has been off since. Neither defect is the unbounded-queue problem of #1104 — that one assumes enrollment never happened. Here enrollment did happen, and the product offers no exit and reports the wrong state.
1. cloud enroll has no inverse
$ engram cloud
usage: engram cloud <subcommand> [options]
supported subcommands: status, enroll, config, serve, upgrade, repair, bootstrap
A project can be enrolled for cloud replication and can never be unenrolled. There is no unenroll, no disable, and no documented flag on any of the seven subcommands that reverses it.
The consequence is not cosmetic. While enrolled, mutations keep being enqueued against a dead target: 31,203 pending rows, 45.9 MB — larger than observations (25.5 MB), the data the queue exists to replicate. Growth continued at ~200–300 rows/day right up to the moment of writing.
The only way out was to delete rows from sync_enrolled_projects by hand, against the SQLite store. That is not a recovery path a user should have to find, and it silently bypasses whatever invariants the product maintains around enrollment.
Expected: engram cloud unenroll <project> (or an equivalent documented verb) that stops enqueueing for that project.
2. cloud status reports not configured while replication is actively running
$ engram cloud status
Cloud status: not configured
At the same moment, sync_state showed live push attempts against a real endpoint:
| column |
value |
target_key |
cloud |
lifecycle |
pending |
last_enqueued_seq |
31203 |
last_acked_seq |
21633 |
consecutive_failures |
109 |
reason_code |
transport_failed |
last_error |
push: transport push project "<redacted>": cloud: mutation push: Post "https://<redacted>/sync/mutations/push": context deadline exceeded |
updated_at |
minutes before the status call |
last_acked_seq = 21633 proves this target successfully replicated in the past — it is not an unconfigured store that never had a backend. Four additional targets exist (cloud:<redacted> ×4) with their own pending counters.
~/.engram/cloud.json is {"server_url": "", "token": ""}, which is presumably what status reads. But the enrollment rows and the sync targets are what actually drive the worker, and they disagree with the file.
Expected: cloud status reflects the state that governs behaviour — enrolled projects, live targets, pending backlog, consecutive failures and last error — rather than an empty config file. A user asking "is cloud on?" was told "no" while 109 failed pushes said otherwise.
Why they belong together
Both stem from enrollment being writable-once and invisible to the status surface. A user who enrolled, lost the backend, and asks the product what is going on gets not configured and no verb to stop it — while the queue quietly becomes the largest object in the database.
Impact
Silent unbounded growth toward a dead endpoint, with the diagnostic surface actively denying it is happening, and no supported remediation. Related to but not covered by #1104, whose store was never enrolled and whose status output was therefore correct.
Reproduction
engram cloud config --server <url> and engram cloud enroll <project> against a reachable backend; let it sync so last_acked_seq > 0.
- Take the backend down permanently and clear
cloud.json.
- Keep using engram normally.
engram cloud status → not configured, while sync_mutations keeps growing and sync_state.consecutive_failures climbs.
- Look for a way to unenroll. There is none.
📋 Pre-flight Checks
status:approvedbefore a PRVersion: engram v2.0.0-rc.9 (Linux x86_64, WSL2)
📝 Bug Description
Two defects in the same surface, found on a store whose cloud backend was decommissioned months ago.
sync_enrolled_projectsheld 4 projects enrolled since 2026-07-21; the server they replicate toward has been off since. Neither defect is the unbounded-queue problem of #1104 — that one assumes enrollment never happened. Here enrollment did happen, and the product offers no exit and reports the wrong state.1.
cloud enrollhas no inverseA project can be enrolled for cloud replication and can never be unenrolled. There is no
unenroll, nodisable, and no documented flag on any of the seven subcommands that reverses it.The consequence is not cosmetic. While enrolled, mutations keep being enqueued against a dead target: 31,203 pending rows, 45.9 MB — larger than
observations(25.5 MB), the data the queue exists to replicate. Growth continued at ~200–300 rows/day right up to the moment of writing.The only way out was to delete rows from
sync_enrolled_projectsby hand, against the SQLite store. That is not a recovery path a user should have to find, and it silently bypasses whatever invariants the product maintains around enrollment.Expected:
engram cloud unenroll <project>(or an equivalent documented verb) that stops enqueueing for that project.2.
cloud statusreportsnot configuredwhile replication is actively runningAt the same moment,
sync_stateshowed live push attempts against a real endpoint:target_keycloudlifecyclependinglast_enqueued_seqlast_acked_seqconsecutive_failuresreason_codetransport_failedlast_errorpush: transport push project "<redacted>": cloud: mutation push: Post "https://<redacted>/sync/mutations/push": context deadline exceededupdated_atstatuscalllast_acked_seq = 21633proves this target successfully replicated in the past — it is not an unconfigured store that never had a backend. Four additional targets exist (cloud:<redacted>×4) with their own pending counters.~/.engram/cloud.jsonis{"server_url": "", "token": ""}, which is presumably whatstatusreads. But the enrollment rows and the sync targets are what actually drive the worker, and they disagree with the file.Expected:
cloud statusreflects the state that governs behaviour — enrolled projects, live targets, pending backlog, consecutive failures and last error — rather than an empty config file. A user asking "is cloud on?" was told "no" while 109 failed pushes said otherwise.Why they belong together
Both stem from enrollment being writable-once and invisible to the status surface. A user who enrolled, lost the backend, and asks the product what is going on gets
not configuredand no verb to stop it — while the queue quietly becomes the largest object in the database.Impact
Silent unbounded growth toward a dead endpoint, with the diagnostic surface actively denying it is happening, and no supported remediation. Related to but not covered by #1104, whose store was never enrolled and whose
statusoutput was therefore correct.Reproduction
engram cloud config --server <url>andengram cloud enroll <project>against a reachable backend; let it sync solast_acked_seq > 0.cloud.json.engram cloud status→not configured, whilesync_mutationskeeps growing andsync_state.consecutive_failuresclimbs.