Repository navigation
Return read errors with VStream schema guidance - #89
Merged
Merged
Conversation
nickvanw
force-pushed
the
fix/vstream-schema-fail-fast
branch
from
June 16, 2026 05:36
31fb5dd to
79b18f1
Compare
nickvanw
changed the title
nickvanw
force-pushed
the
fix/vstream-schema-fail-fast
branch
from
June 16, 2026 05:38
79b18f1 to
8069146
Compare
This was referenced Jun 16, 2026
maxenglander
approved these changes
Jun 16, 2026
nickvanw
force-pushed
the
fix/vstream-schema-fail-fast
branch
from
June 16, 2026 05:58
8069146 to
17a8e22
Compare
nickvanw
force-pushed
the
fix/vstream-schema-fail-fast
branch
from
June 16, 2026 06:12
17a8e22 to
1081465
Compare
This was referenced Aug 3, 2026
orware
added a commit
that referenced
this pull request
Aug 3, 2026
IsBinlogsExpirationError drops the saved position and lets the next iteration run a historical sync, so the connection recovers on its own. The schema-incompatibility path added in #89 returns an error instead, so a deploy that changes the layout of a replicated table stalls the connection until someone triggers a historical re-sync by hand. Until they do, every sync retries from the same position and fails on the same binlog event. Because a Read() error returns out of the whole Sync handler rather than skipping the affected table, one table in this state stalls every table in the connection. Unlike expired binlogs the operator has a real choice here, since the data is still readable once the projection is rebuilt, and an automatic historical sync costs monthly active rows. So this is opt-in via a new auto_resync_on_schema_change configuration field, defaulting to false, which preserves the existing behaviour exactly. With it enabled the cursor is reset and the stream recovers unattended. The cursor carries SCHEMA_INCOMPATIBILITY_ERROR rather than reusing BINLOG_EXPIRATION_ERROR, so the two causes stay distinguishable downstream and an unexpected historical sync remains attributable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #77.
Supersedes #78 and #87.
Notes
This intentionally does not auto-reset state or perform a historical re-sync. If the VStream schema incompatibility condition is hit, continuing incremental replication cannot make progress safely and the user/operator should trigger a Fivetran historical re-sync. Generic non-timeout read errors now fail the sync as well, preserving the underlying error instead of silently dropping it.
Tests