Support combining .atomic and .withoutOverwriting in Data.write(to:options:)

[maxches99]

Support combining .atomic and .withoutOverwriting in Data.write(to:options:).

Motivation:

Data.write(to:options:) traps with fatalError("withoutOverwriting is not supported with atomic") when [.atomic, .withoutOverwriting] are passed together.

To be precise about what this PR does and does not argue, after the review discussion below: the combination is documented as unsupported ("You can't combine this constant with atomic because atomic allows the system to overwrite the original file"), and for a pair of options chosen statically in source, an immediate hard failure is a reasonable contract for a documented-invalid combination — the Objective-C implementation raises NSInvalidArgumentException for the same reason. So this is no longer a proposal to soften the trap into a thrown error.

What it proposes is to make the combination supported, which removes the trap as a side effect rather than weakening it. The combination is well defined and implementable: write to a temporary file, then move it into place with an exclusive rename. Both guarantees hold at once — the destination is never observed half-written, and an existing file is never replaced.

See also #2011, which targets the same issue.

Resolves #1098

Modifications:

The data is written to a temporary file first, as it already is for an atomic write, and the temporary file is then moved to the destination in a way that fails rather than replacing an existing file:

• Darwin: renameatx_np(2) with RENAME_EXCL.
• Where that is unsupported by the file system (ENOTSUP, or EINVAL for the unrecognized flag), and on all other POSIX platforms: linkat(2) followed by unlinkat(2). link(2) atomically fails with EEXIST when the destination exists. (renameat2(RENAME_NOREPLACE) is Linux-specific and is not declared by every C library supported here.)
• Windows: FileRenameInfoEx without FILE_RENAME_FLAG_REPLACE_IF_EXISTS, and MoveFileExW without MOVEFILE_REPLACE_EXISTING in the cross-volume fallback.

There is no non-atomic fallback. Compared to the previous revision of this PR, the lstat + rename path for file systems that support neither an exclusive rename nor hard links is gone; such a write now fails with CocoaError(.featureUnsupported). Silently weakening one of the two guarantees the caller asked for seems worse than reporting that they cannot both be provided.

For the same reason, none of the pre-existing fallbacks in the atomic path are reachable with .withoutOverwriting, since each of them replaces the destination:

• the EINVAL rename-swap for DOS file systems,
• the EBUSY retry that rewrites the file non-atomically,
• the ERROR_ACCESS_DENIED read-only-attribute retry on Windows (without REPLACE_IF_EXISTS the destination is never deleted, so there is nothing to clear).

Rebased onto #2161, which rewrote the atomic write path to use directory file descriptors; the rename helper is renameat-shaped accordingly.

Result:

Data.write(to:options:) with [.atomic, .withoutOverwriting] behaves as follows:

• destination does not exist → the write completes, atomically;
• destination exists → CocoaError(.fileWriteFileExists), consistent with non-atomic .withoutOverwriting; the existing file is untouched and the temporary file is cleaned up;
• the file system provides neither an exclusive rename nor hard links → CocoaError(.featureUnsupported), with the temporary file cleaned up.

On the deployment-target question raised in review: I do not have a better answer than was already suggested. Code built against a newer SDK but running on an older OS still traps, and there is no availability annotation that can express "this combination of options became valid in version X". The doc comment on write(to:options:) now states both the .featureUnsupported case and the trap on older releases, which at least documents the pitfall rather than leaving it to be discovered at runtime.

Testing:

atomicWriteWithoutOverwriting in the Data I/O suite covers:

• writing to a nonexistent destination succeeds and round-trips the contents;
• writing to an existing destination throws .fileWriteFileExists, leaves the existing contents untouched, and leaves no temporary file behind;
• the write succeeds again once the destination is removed.

The Data I/O suite passes on macOS, both normally (exercising renameatx_np) and with the renameatx_np branch compiled out, so that the linkat path used on non-Darwin platforms is exercised locally as well. The .featureUnsupported path is not covered by a test: it needs a file system with neither an exclusive rename nor hard links, which the test suite cannot rely on being mounted.

[maxches99]

I opened this before noticing #2011, which targets the same issue — apologies for the duplicate effort. Having read that discussion, two things need correcting here, and one design question is worth settling before this goes further.

Correcting my own motivation. I wrote that "the documentation does not state that the two options are mutually exclusive". That is true of the doc comment in this repository, but not of the published API documentation, which @jmschonfeld quoted on #2011:

You can't combine this constant with atomic because atomic allows the system to overwrite the original file.

So the combination is documented as unsupported, and my "source-breaking regression" framing was wrong. I have updated the PR description accordingly. What I think still stands on its own is the narrower point: fatalError is a poor response even to a documented-invalid combination, since it takes down the process instead of letting the caller handle it.

On @kperryua's points, mapped onto what this PR actually does:

• Exclusive rename: this uses renamex_np with RENAME_EXCL on Darwin and the equivalent flags on Windows, which matches what you described as the modern silver bullet.
• link() and Sandbox friction: link() is only reached on non-Darwin platforms here — the #if canImport(Darwin) branch takes renamex_np and never falls through to it — so the Darwin Sandbox problem you flagged should not apply. On Linux I avoided renameat2(RENAME_NOREPLACE) because it is not exposed by every supported C library, but I am happy to call it through syscall(2) with a fallback if you would rather have RENAME_NOREPLACE than link() there.
• File systems without support: this is the open question I would like a decision on. You framed it as "we'd have to decide whether we're willing to start dynamically throwing feature-unsupported errors on file systems where the behavior is not supported". This PR currently makes the other choice: on ENOTSUP/EINVAL it degrades to an lstat followed by a plain rename, which is racy and quietly gives up the exclusivity the caller asked for. Silently weakening a safety guarantee seems worse than reporting it, so I would rather throw CocoaError(.featureUnsupported) there — but that is a semantic call for the workgroup, not mine, and I will implement whichever you prefer.
• Deployment target: I have not addressed this. Code built against a Foundation with this change would still hit the fatalError when run against an older OS on Apple platforms. If that needs gating on deployment target, I would appreciate a pointer to the pattern you want used, since I did not find an existing example of it in this area.

A cheaper alternative, if implementing the semantics is not something the workgroup wants to commit to right now: narrow this to replacing the fatalError with a thrown CocoaError. That removes the crash — which is the part of #1098 that is unambiguously a bug — without taking any position on what the combination should mean. I am happy to reduce the PR to that if it is more likely to land.

Either way, if #2011 is the preferred vehicle I am glad to close this and move the Windows handling and tests over there instead.

[kperryua]

fatalError is a poor response even to a documented-invalid combination

This is in fact a common way of handling this. Method options are often chosen statically by the developer in code, and as documented, the two options together are illegal. With that contract, that code should result in an immediate and unmistakable failure, not an error that could be grouped together with other various runtime errors. In the Objective-C implementation of this, an NSInvalidArgumentException is thrown—not for the intention of it being handled of course, but to be a fatal condition indicating the programmer's error.

link() and Sandbox friction

Saving the link() workaround for Linux is fine and does avoid the sandbox issue I was referring to.

File systems without support
Silently weakening a safety guarantee seems worse than reporting it, so I would rather throw CocoaError(.featureUnsupported) there

Yes, this is the safer option. It's unacceptable—and potentially represents a security issue—to break an atomicity guarantee and it builds in a TOCTOU race to the algorithm.

It may be acceptable if RENAME_EXCL returns ENOTSUP to fall back to the link() based implementation. If it runs into a Sandbox violation as mentioned above—so be it. That's just another kind of error that is maybe more informative than "not supported", and enables a wider variety of scenarios to work.

Deployment target

I don't have a good suggestion for this yet. The behavior before and after the release should definitely be explained in the documentation at the very least, but that's not a very strong way to help developers avoid the pitfall here.

I would also appreciate if any changes here were held until after an upcoming re-application of #2077, which heavily modifies this code. This should hopefully happen this week.

[maxches99]

Thanks — that all makes sense, and I'll drop the "fatalError is a poor response" framing: for a statically-chosen, documented-illegal option combination, a hard failure is the right contract, and the NSInvalidArgumentException precedent settles it. So I'm no longer proposing the narrower "throw instead of trap" variant; the only justification left for this PR is making the combination actually supported, which removes the trap as a side effect rather than softening it.

On the fallbacks, I'll restructure to: renamex_np(RENAME_EXCL) → on ENOTSUP, fall back to the link()/unlink() path on all platforms including Darwin → CocoaError(.featureUnsupported) if that also fails. The non-atomic lstat+rename path goes away entirely, so there's no configuration where the atomicity guarantee is silently weakened.

On the deployment target, I don't have a better answer than you do. I'll at least document the pre/post-release behavior in the doc comments as part of this change, with the caveat that it doesn't do much to keep anyone out of the pitfall.

Happy to hold until #2077 is re-applied; I'll rebase on top of it once it lands and update this PR then.

[maxches99]

Pushed the restructured version, rebased on top of #2161 now that it has landed. Summary of what changed since your review:

• Framing. Dropped the "fatalError is a poor response" argument from the description, as agreed: for a statically chosen, documented-illegal pair of options an immediate hard failure is the right contract. The only justification left for the PR is making the combination supported, which removes the trap as a side effect rather than softening it.
• Fallbacks. Now renameatx_np(RENAME_EXCL) → linkat/unlinkat → CocoaError(.featureUnsupported). The non-atomic lstat + rename path is gone, and none of the pre-existing fallbacks (the EINVAL swap, the EBUSY non-atomic retry, the Windows read-only retry) are reachable in this mode, so there is no longer any configuration in which either guarantee is silently weakened.
• link() on Darwin — worth a second look. You said keeping the link() workaround for Linux only is fine and avoids the sandbox friction. I extended it to Darwin so that a volume reporting ENOTSUP for RENAME_EXCL still gets an atomic exclusive move instead of an outright failure, but that does put link() back in the Darwin path — only on such volumes, and only as the last step before failing with .featureUnsupported. One consequence worth naming: a sandbox denial surfaces as EPERM from linkat, which this code cannot distinguish from "this file system has no hard links", so it would be reported as .featureUnsupported. If you would rather Darwin skip link() entirely and go straight to .featureUnsupported, that is a one-line change — happy to make it.
• Deployment target. No better answer than before. Code built against a newer SDK still traps when run on an older OS, and no availability annotation can express "this combination became valid in version X". The doc comment on write(to:options:) now states both the .featureUnsupported case and the trap on older releases, so the pitfall is at least documented.

Verification: the Data I/O suite passes on macOS both normally and with the renameatx_np branch compiled out, so the linkat path that non-Darwin platforms take is exercised locally too — APFS always takes RENAME_EXCL otherwise.