feat: resume saved collection pagination

[eferm]

Summary

• add a --pagination-token option to sync likes and sync bookmarks
• seed the existing sync plan with the supplied X pagination cursor
• require explicit xurl mode, preserve existing no-token cache keys, and isolate resumed segments in the live cache
• document bounded, resumable saved-collection backfills

Verification

• ./scripts/bun-canary.sh run --bun check
• ./scripts/bun-canary.sh ./scripts/run-vitest.mjs run src/lib/timeline-collections-live.test.ts (25 tests)
• ./scripts/bun-canary.sh run --bun test (1,351 tests)
• ./scripts/bun-canary.sh run --bun build
• BIRDCLAW_BACKUP_AUTO_SYNC=0 ./scripts/bun-canary.sh src/cli.ts sync likes --help

Live behavior proof

Executed from commit 52820c66141927eebb92427bd70301c58e3709bb on August 18, 2026. The account identifier, tweet IDs, and opaque pagination cursors are redacted. Both requests used --refresh and reported source: "xurl".

First page:

$ BIRDCLAW_BACKUP_AUTO_SYNC=0 ./scripts/bun-canary.sh src/cli.ts --json sync likes \
    --account '<redacted-account>' --mode xurl --limit 5 --max-pages 1 --refresh
{
  "source": "xurl",
  "fetched": 5,
  "pages": 1,
  "tweetIds": "<5 tweet IDs redacted>",
  "nextToken": "<redacted non-null cursor 1>"
}

Resumed page using that response cursor:

$ BIRDCLAW_BACKUP_AUTO_SYNC=0 ./scripts/bun-canary.sh src/cli.ts --json sync likes \
    --account '<redacted-account>' --mode xurl --limit 5 --max-pages 1 --refresh \
    --pagination-token '<redacted non-null cursor 1>'
{
  "source": "xurl",
  "fetched": 5,
  "pages": 1,
  "tweetIds": "<5 tweet IDs redacted>",
  "nextToken": "<redacted non-null cursor 2>"
}

Comparison of the unredacted local responses:

{
  "firstPageCount": 5,
  "resumedPageCount": 5,
  "overlapCount": 0
}

The resumed request returned five additional tweets with no overlap and a non-null continuation cursor, demonstrating that the supplied cursor was honored and the walk can continue.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

[clawsweeper]

Codex review: needs maintainer review before merge. Reviewed August 29, 2026, 9:14 AM ET / 13:14 UTC.

ClawSweeper review

What this changes

The PR adds an Xurl-only --pagination-token option for resuming Likes and Bookmarks syncs, keeps resumed responses in separate cache entries, and documents bounded backfills.

Merge readiness

⚠️ Ready for maintainer review - 2 items remain

The PR is still necessary: current main does not expose this resume cursor. The implementation follows Birdclaw’s existing pagination plan and has credible live Xurl proof, but the repository owner should explicitly accept the new public cursor-and-cache contract before merge.

Priority: P2
Reviewed head: a9a2432f3ebd2b17d2db7f444b776e11a3ce3eea
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🦞 diamond lobster (5/6)	A focused, documented feature with targeted tests and credible real Xurl behavior proof; only public-contract acceptance remains.
Proof confidence	🦞 diamond lobster (5/6)	Sufficient (terminal): The changed production owner passes the CLI cursor into the Xurl page walker; the supplied redacted terminal trace shows a real first request followed by a cursor-resumed request that returned five additional non-overlapping tweets and another continuation cursor.
Patch quality	🦞 diamond lobster (5/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (terminal): The changed production owner passes the CLI cursor into the Xurl page walker; the supplied redacted terminal trace shows a real first request followed by a cursor-resumed request that returned five additional non-overlapping tweets and another continuation cursor.
Evidence reviewed	6 items	Current main lacks the capability: The fetched current-main version has no pagination-token option in the saved-collection sync options, so the central requested behavior is not already implemented. Cursor path and cache isolation: The PR head seeds the established page walker with the supplied cursor, restricts the option to Xurl mode, and includes the cursor in the cached-response key. Existing pagination mechanism: The shared sync plan already accepts and normalizes an initial cursor before issuing its first page request.
Findings	None	None.
Security	None	None.

How this fits together

Birdclaw sync commands fetch X Likes and Bookmarks into the local SQLite archive. This change carries a supplied cursor from the CLI into the shared page walker, then to Xurl requests and the local response cache.

flowchart LR
  A[Sync CLI command] --> B[Cursor option]
  B --> C[Xurl mode validation]
  C --> D[Shared page walker]
  D --> E[Xurl collection request]
  E --> F[Local tweet archive]
  E --> G[Cursor-specific cache]

Loading

Decision needed

Question	Recommendation
Should Birdclaw expose a user-supplied X pagination cursor as a public saved-collection sync option and retain it in a local cache key?	Accept the resumable CLI contract: Keep the explicit Xurl-only option and cursor-specific cache entries so bounded backfills can resume without reusing the head response.

Why: This is a new public CLI capability with durable local-state behavior, so implementation correctness alone cannot determine the intended product contract.

Before merge

• Complete next step (P2) - The code and proof are ready for normal review, but the repository owner must accept the new public cursor and cache-persistence contract.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Change surface	production +18/-2, tests +73, docs +7 across 5 files	The production change is small and is paired with focused cache and pagination coverage.

Technical review

Best possible solution:

If accepted, merge the documented Xurl-only cursor contract; otherwise keep continuation state internal rather than adding a parallel pagination path.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction: the PR’s supplied terminal trace does demonstrate the new real Xurl resume flow with non-overlapping pages.

Is this the best way to solve the issue?

Yes technically: it reuses the established cursor-capable sync plan, preserves Xurl-only behavior, and prevents head-cache reuse for resumed requests.

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against c184cf5f1b5e.

Labels

Label changes:

• add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

• P2: This is a bounded optional sync capability with limited blast radius that needs owner approval.
• rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The changed production owner passes the CLI cursor into the Xurl page walker; the supplied redacted terminal trace shows a real first request followed by a cursor-resumed request that returned five additional non-overlapping tweets and another continuation cursor.
• proof: sufficient: Contributor real behavior proof is sufficient. The changed production owner passes the CLI cursor into the Xurl page walker; the supplied redacted terminal trace shows a real first request followed by a cursor-resumed request that returned five additional non-overlapping tweets and another continuation cursor.

Evidence

What I checked:

• Current main lacks the capability: The fetched current-main version has no pagination-token option in the saved-collection sync options, so the central requested behavior is not already implemented. (src/lib/timeline-collections-live.ts:39, c184cf5f1b5e)
• Cursor path and cache isolation: The PR head seeds the established page walker with the supplied cursor, restricts the option to Xurl mode, and includes the cursor in the cached-response key. (src/lib/timeline-collections-live.ts:137, a9a2432f3ebd)
• Existing pagination mechanism: The shared sync plan already accepts and normalizes an initial cursor before issuing its first page request. (src/lib/sync-plan.ts:44, a9a2432f3ebd)
• Product direction: The repository VISION.md explicitly calls for bounded, observable, resumable live reads and CLI/JSON access for new data surfaces. (VISION.md:9, a9a2432f3ebd)
• Real behavior proof: The supplied PR body records real authenticated Xurl CLI runs: a five-item first page followed by a five-item non-overlapping resumed page with a new continuation cursor. (52820c661419)
• Feature history: Current-area history shows Peter Steinberger’s pagination centralization and original saved-collection support preceding this PR. (src/lib/timeline-collections-live.ts:137, 17ef75f11928)

Likely related people:

• steipete: Peter Steinberger introduced Likes/Bookmarks support and most recently centralized the live pagination policy used by this PR. (role: feature owner and recent area contributor; confidence: high; commits: aba65a1181e1, 17ef75f11928; files: src/lib/timeline-collections-live.ts, src/lib/sync-plan.ts, src/cli/register-sync.ts)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (20 earlier review cycles; latest 8 shown)

• reviewed 2026-08-23T10:10:38.187Z sha a3f96fa :: needs maintainer review before merge. :: none
• reviewed 2026-08-23T17:10:32.653Z sha a3f96fa :: needs maintainer review before merge. :: none
• reviewed 2026-08-23T23:58:35.023Z sha a3f96fa :: needs maintainer review before merge. :: none
• reviewed 2026-08-24T12:01:34.448Z sha a3f96fa :: needs changes before merge. :: none
• reviewed 2026-08-24T12:43:12.812Z sha a3f96fa :: needs changes before merge. :: none
• reviewed 2026-08-24T20:00:21.394Z sha a3f96fa :: needs maintainer review before merge. :: none
• reviewed 2026-08-24T21:41:57.776Z sha a9a2432 :: needs maintainer review before merge. :: none
• reviewed 2026-08-25T11:05:30.763Z sha a9a2432 :: needs maintainer review before merge. :: none

[eferm]

@clawsweeper re-review

[clawsweeper]

🦞👀
Exact review queued.

Re-review progress:

• State: Complete
• Detail: The durable review result and its route handoff completed.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/32178994728
• Updated: 2026-08-18T19:51:59.317Z

[eferm]

@clawsweeper re-review

[clawsweeper]

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

• State: Waiting
• Detail: A newer exact-review queue owner superseded this run. This run completed as a no-op.
• Run: https://github.com/openclaw/clawsweeper/actions/runs/32179784384
• Updated: 2026-08-18T20:01:59.207Z