fix(synccompactor): use ResumeSync when reopening the merged sync - #1103
Conversation
Incremental grant expansion reopens the compacted sync after the merge so it can write derived grants. It did so via StartOrResumeSync, which on Pebble resumes only when the id lookup returns no error at all — any other outcome falls through to StartNewSync, whose ResetForNewSync excises the record range holding everything the merge just wrote. A transient lookup failure on a sync that demonstrably exists would therefore discard the compacted output and report success. ResumeSync has no such fallback: a lookup error returns an error and leaves the destination untouched, so the existing plain-error path still falls back to full expansion against a consistent store. Tests cover the reopen contract at the engine level: resume fails closed on an unresolvable id, resume of an ended sync accepts writes (the path the compactor depends on), and StartOrResumeSync's destructive fallback is pinned as current-not-desired behavior. Co-authored-by: c1-squire-dev[bot] <c1-squire-dev[bot]@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| // adapter.go's claim to mirror the SQLite cascade. If Pebble is ever aligned, | ||
| // invert the assertions below — a failure here is that alignment landing, not a | ||
| // regression. | ||
| func TestStartOrResumeSyncUnknownIDWipesRecords(t *testing.T) { |
There was a problem hiding this comment.
🟡 Suggestion (medium confidence): This test pins a real data-loss hazard that the PR fixes at one call site rather than at the source. Engine.StartOrResumeSync (pkg/dotc1z/engine/pebble/adapter.go:176-180) still treats any GetSyncRunRecord error on an explicit non-empty id — including a transient read failure — as "nothing to resume" and falls through to StartNewSync, whose ResetForNewSync wipes the sync-scoped keyspace. SQLite's C1File.StartOrResumeSync (pkg/dotc1z/sync_runs.go:704) fails closed with NotFound in the same situation. Since no production caller passes a non-empty id to StartOrResumeSync today (only pkg/sync/syncer.go:839, with ""), aligning Pebble to fail closed would be behavior-preserving and would remove the class of bug instead of one instance. The PR body already flags this as out of scope — noting it so the follow-up is tracked.
General PR Review: fix(synccompactor): use ResumeSync when reopening the merged syncBlocking Issues: 0 | Suggestions: 1 | Threads Resolved: 0 Review SummaryThe full PR diff (one call-site change in Security IssuesNone found. Correctness IssuesNone found. Suggestions
Prompt for AI agents |
Follow-up to #1013, blocker #4 from the approval review.
The problem
After the merge, incremental grant expansion reopens the compacted sync so it can write derived grants. It did that with
StartOrResumeSync:StartNewSynccallsResetForNewSync, which excises the key range fromtypeResourceTypetotypeEngineMeta— everything the merge just wrote.So a transient failure looking up a sync id that demonstrably exists (the merge produced it moments earlier) discards the compacted output and returns an empty artifact, with the compaction reporting success.
The fix
Call
ResumeSyncdirectly. It is already onconnectorstore.Writer, and it has no "start a new one" branch: a lookup error returns an error and leaves the destination untouched.That keeps the existing error classification correct. The failure stays a plain error rather than
errIncrementalFatal, andResumeSyncreturns before binding anything, so the store is still in the ended state the full-expansion fallback expects — norestoreEndedSyncneeded.Tests
New
pkg/dotc1z/engine/pebble/resume_sync_test.go:TestResumeSyncUnknownIDFailsClosed— unresolvable id errors, records survive.TestResumeSyncOnEndedSyncAllowsWrites— resuming an ended sync succeeds and accepts writes. This is the contract the compactor actually depends on (the merge leaves the sync ended), and nothing pinned it before.TestStartOrResumeSyncUnknownIDWipesRecords— pins the destructive fallback so the reason for this change stays visible.Note for a separate issue
That last test documents current behavior, not desired behavior. SQLite's
C1File.StartOrResumeSync(pkg/dotc1z/sync_runs.go:704) returnsNotFoundrather than starting a new sync when an explicit id fails to resolve, so the engines diverge — despiteadapter.go's comment claiming the Pebble path mirrors the SQLite cascade. Out of scope here, since incremental expansion is Pebble-only, but worth aligning. After this change no production caller passes a non-empty id toStartOrResumeSyncat all.🤖 Generated with Claude Code