Skip to content

writeBatch's open-retry loop is unbounded, so a persistently failing volume spins through sequence numbers #752

Description

@kriszyp

Noticed while fixing #748; pre-existing, not introduced by that change.

TransactionLogStore::writeBatch retries in two nested loops with no bound:

while (!batch.isComplete()) {
  while (logFile == nullptr) {
    logFile = this->getLogFile(currentSequenceNumber);
    try { if (!logFile->isOpen()) logFile->open(latestTimestamp); }
    catch { writeFailures++; logFile = nullptr; }
    if (logFile == nullptr || maxFileSize > 0) rotateToNextSequence(logFile);
  }
  ...
}

Every open failure advances the sequence number and tries again. Under a condition that fails every open — a full volume, an exhausted quota, a directory that lost write permission — there is nothing to stop it: each iteration attempts to create another log file, fails, and burns a sequence number. The commit never returns an error to the caller, it just spins.

The same loop is the fallback for a file that reports no write progress, so #748's new "retire this file and rotate" path reaches it too, as does a short header write. Those are additional routes into the loop rather than new failure modes — openFile() already threw under ENOSPC before any of that work.

Worth deciding: a retry bound (or a bound on consecutive open failures) after which writeBatch throws, so a full disk surfaces as a rejected commit instead of a spin. The tricky part is picking a bound that cannot reject a commit that a legitimate rotation would have satisfied — a batch genuinely can span several files.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions