Skip to content

fix(video): allow slow Seedance 2.5 jobs to finish - #110

Merged
VickyXAI merged 1 commit into
BlockRunAI:mainfrom
KillerQueen-Z:fix/video-poll-timeout
Aug 28, 2026
Merged

VickyXAI merged 1 commit into
BlockRunAI:mainfrom
KillerQueen-Z:fix/video-poll-timeout

Conversation

@KillerQueen-Z

Copy link
Copy Markdown
Contributor

Summary

  • extend the client-side video polling budget from 5 minutes to 9 minutes
  • start the budget before creating the 600-second payment authorization, preserving roughly one minute of safety margin
  • update the MCP tool description and add regression coverage for the timeout boundary

Why

Production tests of Seedance 2.5 30-second clips completed in 479s, 275s, and 400s. All three succeeded after increasing the local MCP budget; two exceeded the previous 300-second hard cap. The gateway already uses async submit plus short polling requests, so the old failure was client-side.

Validation

  • npm test — 342 passed
  • npm run typecheck
  • npm run build
  • three live Seedance 2.5 30-second generations completed successfully

No gateway protocol or settlement behavior changes.

@VickyXAI VickyXAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for chasing this down with real production numbers — the 479s/400s/275s measurements are exactly the right evidence, and the diagnosis that the failure was client-side is correct. One problem with the margin arithmetic before this lands.

The 60s margin doesn't hold for the last poll

The 540s budget sits 60s inside the 600s authorization only at loop entry. The loop body then adds two costs that aren't in the budget:

video.ts:541   while (Date.now() - startedAt < VIDEO_TOTAL_BUDGET_MS)  // may enter at 539.999s
video.ts:542     await new Promise(r => setTimeout(r, POLL_INTERVAL_MS));   +5s   -> 545s
video.ts:547     }, 90_000);   // poll fetch timeout                        +90s  -> 635s

validBefore is startedAt + 600s (@blockrun/llm/dist/index.js:109: const validBefore = now + (options.maxTimeoutSeconds || 300)), with maxTimeoutSeconds: Math.max(..., 600) at video.ts:487. And fetchWithTimeout deliberately never clears its abort timer (src/utils/http.ts:6-12), so 90s bounds headers plus body read.

So a poll can be in flight at 635s — 35s after the authorization expires, rather than 60s before it.

This already shipped once, on the sibling tool

CHANGELOG.md:1351:

fix(music) — keep the payment authorization valid through the poll window. ... A slow MiniMax track completing after validBefore failed settlement server-side and surfaced as a wrong "fund your wallet" error for a track that actually generated.

That fix raised music to 600s to stay inside the window. This change pushes video back out past it:

budget +interval +poll timeout worst case vs 600s auth
video (before this PR) 300s 5s 90s 395s +205s safe
music (music.ts:164-170) 240s 5s 90s 335s +265s safe
video (this PR) 540s 5s 90s 635s -35s expired

There's a secondary effect worth noting: video.ts:571-576 books local spend the instant it observes status === "completed", without checking that settlement succeeded. That was safe before because expiry was unreachable; in the 600–635s tail it would record a ledger charge for USDC that never moved — the mirror image of the bug the comment directly above it describes fixing.

Lowering the budget won't work

Worth flagging because it's the tempting fix: staying inside 600s unclamped requires budget <= 445s, which fails the 479s job this PR exists to support. The budget number isn't the lever.

Suggested fix: clamp the last poll to the remaining budget

Bounds worst-case wall time at exactly 540s, so the 60s margin becomes real and the 479s job still finishes:

export const VIDEO_POLL_TIMEOUT_MS = 90_000;
export const VIDEO_PAYMENT_AUTH_SECONDS = 600;

// How long the next poll may block, given the budget that is left. Returns 0
// when the budget is spent, which the caller treats as "stop".
export function pollTimeoutFor(deadlineMs: number, nowMs: number): number {
  const remainingMs = deadlineMs - nowMs;
  if (remainingMs <= 0) return 0;
  return Math.min(VIDEO_POLL_TIMEOUT_MS, remainingMs);
}
const deadline = startedAt + VIDEO_TOTAL_BUDGET_MS;

while (Date.now() < deadline) {
  await new Promise((r) => setTimeout(r, POLL_INTERVAL_MS));

  // Checking the deadline only at the top of the loop bounds when a poll may
  // START, not when it finishes — an unclamped 90s poll entered just under the
  // wire stays in flight well past validBefore.
  const pollTimeoutMs = pollTimeoutFor(deadline, Date.now());
  if (pollTimeoutMs === 0) break;

  const pollResp = await fetchWithTimeout(pollAbsoluteUrl, {
    method: "GET",
    headers: { "PAYMENT-SIGNATURE": paymentPayload },
  }, pollTimeoutMs);

Also worth using VIDEO_PAYMENT_AUTH_SECONDS at video.ts:487 — right now 600 is a magic literal there, while the budget it has to be reconciled against is an exported constant.

Extracting pollTimeoutFor rather than inlining the clamp is deliberate: inline, it sits inside the handler behind a 402 + payment + submit chain that test/video-models.test.ts blocks at the network by design (the NETWORK_ESCAPE sentinel), so its removal would go uncaught.

On the new test

assert.equal(VIDEO_TOTAL_BUDGET_MS, 9 * 60 * 1000);
assert.equal(10 * 60 * 1000 - VIDEO_TOTAL_BUDGET_MS, 60 * 1000);

Two issues: it restates the authorization window as a literal instead of deriving it from the 600 the request actually sends, so changing that value leaves the test green while the invariant breaks; and it omits POLL_INTERVAL_MS and the poll timeout — the two terms that actually blow the margin. As written it asserts 600000 - 540000 === 60000, which certifies a margin that doesn't exist.

Suggested replacement — asserts both the capability and the invariant, and sweeps the property that matters:

test("video polling allows slow 30s jobs while staying inside payment authorization", () => {
  // The capability this budget buys: production jobs measured at 479s/400s/275s.
  const SLOWEST_OBSERVED_MS = 479_000;
  assert.ok(VIDEO_TOTAL_BUDGET_MS > SLOWEST_OBSERVED_MS);

  const authMs = VIDEO_PAYMENT_AUTH_SECONDS * 1000;
  assert.ok(VIDEO_TOTAL_BUDGET_MS + 60_000 <= authMs);

  // The clamp is load-bearing: unclamped, worst case overruns the window.
  assert.ok(VIDEO_TOTAL_BUDGET_MS + VIDEO_POLL_TIMEOUT_MS > authMs);
});

test("the last poll is clamped to the remaining budget, never left in flight past the deadline", () => {
  const start = 1_000_000;
  const deadline = start + VIDEO_TOTAL_BUDGET_MS;

  assert.equal(pollTimeoutFor(deadline, start), VIDEO_POLL_TIMEOUT_MS);
  assert.equal(pollTimeoutFor(deadline, deadline - 1_000), 1_000);
  assert.equal(pollTimeoutFor(deadline, deadline), 0);

  // A poll started at any reachable instant finishes on or before the deadline.
  for (let elapsed = 0; elapsed <= VIDEO_TOTAL_BUDGET_MS; elapsed += 4_999) {
    const now = start + elapsed;
    assert.ok(now + pollTimeoutFor(deadline, now) <= deadline);
  }
});

Verified locally

I applied the above on top of your branch: 343/343 pass, tsc --noEmit clean, npm run build clean. I also confirmed the new test is load-bearing — reverting the Math.min to a bare VIDEO_POLL_TIMEOUT_MS produces not ok 15 - the last poll is clamped..., so it catches the regression it's meant to catch.

One last thing: the description's "9 min hard cap" becomes accurate on its own once the clamp lands, so no change needed there.

Happy to push the commit to this branch if you'd rather not hand-apply it — just say the word.

@VickyXAI

Copy link
Copy Markdown
Contributor

Merging this as-is — the budget extension is right and the production data backs it. I'm landing the polling-window clamp from my review as an immediate follow-up PR rather than asking you to round-trip it.

To be explicit about the risk window: nothing ships from this merge. publish.yml only fires on pushes to main touching package.json, server.template.json, or the workflow, and this PR touches neither, so no npm user is exposed to the 635s-vs-600s overrun in the interim.

Thanks for the live Seedance measurements — that's what made the client-side diagnosis unambiguous.

@VickyXAI
VickyXAI merged commit f275463 into BlockRunAI:main Aug 28, 2026
1 check passed
VickyXAI added a commit that referenced this pull request Aug 28, 2026
…t the poll budget (#113)

blockrun_video got this in #111; music has the same shape and the same latent
bug, one budget bump away from firing.

music measures its 240s poll budget from AFTER submit, but validBefore is
counted from signing — and submit is allowed 95s. So the real worst case from
the authorization's own origin is 95 + 240 + 5 + 90 = 430s. That fits inside
600s today, which is exactly why it looks fine and why the next person to raise
MUSIC_POLL_BUDGET_MS the way #110 raised video's would walk straight into it:
at 420s the sum is 610s and settlement starts failing for tracks that rendered.

Naively copying video's fix (stamp startedAt before signing) would have been
worse than the bug: the 240s budget would then absorb up to 95s of submit,
leaving as little as 145s of polling for MiniMax tracks the code documents as
taking 60-180s. So the loop takes the EARLIER of two deadlines instead — the
poll budget, and the authorization minus settle-side slack. A fast submit keeps
the full 240s window; a slow one shortens it rather than silently pushing polls
past validBefore.

- share pollTimeoutFor from src/utils/poll.ts, used by both tools
- deliberately not in http.ts: four test files replace that module with a
  network sentinel, so a pure function there forces them to stub arithmetic
- video.ts: repair the indentation break at the spend-booking site and drop the
  redundant inner !spendBooked guard already implied by the enclosing condition
- cover the clamp in test/poll.test.ts and the music invariant in music-cost

Co-authored-by: VickyXAI <vicky.fuyu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants