Prefer posix_spawn over fork/exec on every Unix platform

[jakepetroules]

Important

The second commit, [DO NOT MERGE] Run the test suite on OpenBSD, is temporary. It exists so this PR can be seen green on every platform the package supports, and must be reverted before merging.

Subprocess used posix_spawn only on Darwin and fork/exec everywhere else, in two near-duplicate implementations that each owned the candidate-path loop, the transient-failure retry, the file-descriptor bookkeeping and the error mapping.

Both now live in one shared implementation in Subprocess+Spawn.swift, which prefers posix_spawn and falls back to fork/exec — or, on Darwin, a pre-fork plus posix_spawn(POSIX_SPAWN_SETEXEC), so Darwin still never reaches a raw execve — only for configurations posix_spawn cannot express.

When the fallback path is still required

Capabilities are probed once per process at runtime, not at compile time, since a binary built against one C library is expected to run against another.

1. userID, groupID or supplementaryGroups is set. No platform has a posix_spawn attribute for setuid, setgid or setgroups.
2. The C library offers no way to close every inherited descriptor. Closing them by enumerating this process's own table would leak any descriptor another thread opens between the enumeration and the spawn, so that is not an acceptable substitute.
3. createSession without POSIX_SPAWN_SETSID (FreeBSD, OpenBSD).
4. createSession together with processGroupID. glibc calls setsid then setpgid, Bionic the reverse, where setsid then fails EPERM because the child already leads a process group. The fallback path controls the order explicitly and reproduces the previous behavior.
5. workingDirectory without an fchdir file action, or without a way to open a descriptor for it (OpenBSD, Android below API 34). Both halves are tested because they are established differently — one by dlsym at runtime, one by which of O_SEARCH/O_PATH the headers define — and a working directory silently ignored would be the worst available outcome.

Effect

Linux with glibc 2.34+, Android 13+ and FreeBSD 13.1+ stop forking for the common case: posix_spawn on glibc clones with CLONE_VM and CLONE_VFORK, so there is no page-table copy and no fork() in a multithreaded parent. musl, older glibc, older Android and OpenBSD keep fork/exec. On Darwin, createSession no longer forks, since POSIX_SPAWN_SETSID works there.

Why the C shim grew

Swift no longer names posix_spawn_file_actions_t or posix_spawnattr_t. glibc and musl typedef structs, Darwin and the BSDs typedef pointers, and Bionic annotates its mutators _Nonnull, so no single Swift spelling compiles everywhere; the shim wraps every entry point behind an opaque handle. The public PlatformSpawnAttributes and PlatformSpawnFileActions name the form each platform's C library accepts, and an unavailable function calls the library's mutators through them so a mistake there is a build failure rather than a public API nobody can use. Darwin declares neither: its preSpawnProcessConfigurator predates them and keeps its inout signature, so code that configures the spawn on Darwin and elsewhere still has to branch on #if canImport(Darwin).

Those typealiases have to be public, and on every platform whose C library declares the two types as structs, Swift resolves them through the shim rather than through the libc overlay — even where that overlay is itself imported publicly. The shim is therefore imported publicly off Darwin, which is also why OpenBSD already carried such an import; the // FIXME: Why is this necessary only on OpenBSD? it was added under is now answered.

Working directory

A working directory is now applied through an fchdir file action on a descriptor opened in the parent, which replaces the TOCTOU-prone guess about whether a spawn failure meant a missing executable or a missing directory. The open flags come from the shim: chdir needs only execute permission, so O_RDONLY would reject a directory that is a perfectly legal working directory.

Only a real O_SEARCH (Darwin, FreeBSD) actually checks that permission, though. The kernel zeroes acc_mode for O_PATH, so on glibc, musl and Bionic a directory with mode 000 opens successfully and the child's fchdir then fails EACCES — one of the errnos meaning "try the next candidate path", so it would surface as executableNotFound, the very misdiagnosis this replaces. Searchability is therefore checked explicitly, against the returned descriptor rather than the path, so the object checked is the object fchdir will use and no TOCTOU window is added. Whether that check can ask about effective IDs is another C library question answered in C: AT_EACCESS where it works, 0 on Bionic, which rejects every nonzero faccessat flag and has no set-uid programs for the flag to matter to. Where no search-only open exists the directory is validated without being opened.

Ordering that is load-bearing

File actions are appended in the order the child must run them, which matters in three places. The fchdir action goes first, because the descriptor it names can itself be 0, 1 or 2 — it is opened after the pipes, so it takes the lowest free number, and a caller whose standard descriptors are closed leaves low numbers free — and a dup2 onto that number would replace the directory before the child reached it. Parent-end closes precede the dup2s, or a close of a parent end that happens to occupy 0, 1 or 2 would close the child's standard stream instead. The close-all action goes last, after everything that still needs a descriptor.

The child's signal dispositions are reset through POSIX_SPAWN_SETSIGDEF with every signal except SIGKILL and SIGSTOP. Excluding those two is required rather than tidiness: Bionic resets each member of that set unconditionally, and Linux rejects sigaction() on them with EINVAL even for SIG_DFL, whereupon Bionic _exit(127)s a child that posix_spawn still reports as successfully launched. Neither signal can be caught or ignored, so neither can have a disposition needing a reset on any platform.

Apple platforms other than macOS

posix_spawn_file_actions_addfchdir_np is referenced under TARGET_OS_OSX rather than TARGET_OS_MAC, which is every Apple platform: the SDK marks it __API_UNAVAILABLE(ios, tvos, watchos, visionos), and that is a hard error rather than a deprecation warning. Mac Catalyst is inside that exclusion, since it reports as iOS even though it runs on macOS. Leaving the capability unset there routes a working directory through rule 5 to the fallback path.

Other observable changes

A setpgid or setsid failure is reported rather than silently ignored where the C library applies it, as part of posix_spawn; the fork/exec path still discards both results, which is now documented on processGroupID. Working-directory errors carry the errno that actually occurred, and on FreeBSD a process descriptor is available only on the fork/exec path until posix_spawnattr_setprocdescp_np can be required. Both FreeBSD 15.1 follow-ups are recorded as FIXMEs.

Testing

A test-only task-local forces the fallback path, so every spawn-sensitive test runs under both paths on every platform, instead of macOS never exercising fork and modern glibc never exercising fork/exec. Which path a spawn is expected to take is derived from the same rule the implementation consults rather than from that override alone, because where posix_spawn cannot express the request the unforced case legitimately takes the fallback too.

CI is what validates this change: its entire behavioral effect lands on platforms that cannot be exercised on a development Mac. The matrix covers glibc across five distros on 6.2 and nightly, the al2 4.18 and 5.10 kernels (glibc 2.26, so the fallback path), Android on NDK r27d and r29, FreeBSD 14.3, musl static-SDK builds, Windows, macOS, the CMake build that does not glob sources, and — via the temporary job above — OpenBSD.

Found but not fixed here

A review of this change turned up three pre-existing defects, all of which behave identically before and after it. They are out of scope — this PR is already large, and the remedies touch the fork/exec path — but they should not be lost:

• A child's pipe end landing on descriptor 0, 1 or 2 is clobbered. Both paths bind the child's ends in the fixed order dup2(inputRead, 0), dup2(outputWrite, 1), dup2(errorWrite, 2). If outputWrite is itself descriptor 0, the first dup2 overwrites it and the second then copies inputRead onto 1, so the child's stdout becomes the read end of its own stdin pipe; the child dies on the redirection rather than producing wrong output. This needs no caller-supplied descriptors, only a caller whose standard descriptors are closed — a daemon, or a process re-exec'ing itself. The fix is to relocate any child end numbered below 3 with F_DUPFD_CLOEXEC in the parent, which would cover both paths at once. testStandardStreamsSurviveLowParentDescriptors does not catch it: it uses pipes for all three streams, where the parent end always takes the lower number.

• supplementaryGroups: [] silently retains the parent's groups. Both the rule that routes privileged configurations to the fork/exec path and the shim itself test count > 0, so an empty array — the natural way to ask that inherited supplementary groups be dropped — results in no setgroups call at all. setgroups(0, NULL) is the meaningful call and is never made.

• A child can inherit the entire descriptor table if /dev/fd cannot be opened. Where close_range(…, CLOSE_RANGE_CLOEXEC) is unavailable (kernels before 5.9, which includes the al2-4.18 CI leg), the forked child falls back to enumerating /dev/fd, and if that open fails it simply returns, leaking every descriptor into the child with no error reported. Reachable in a FROM scratch container or a chroot with no /proc — a normal deployment for a statically linked musl binary, which is also a configuration where rule 2 forces this path for every spawn. glibc's own posix_spawn fails the spawn in the equivalent situation rather than proceeding. This change reduces exposure, since glibc ≥ 2.34, Android ≥ 13 and FreeBSD ≥ 13.1 all move off that path, but it is still the designated safe fallback.

Fixes #172