feature: Add Job API + CLI support for setting the working directory from polyglot hooks

[francoiscampbell]

Description

Today only wrapped POSIX shell hooks can change the working directory for subsequent hooks/commands — the hook wrapper exports BUILDKITE_HOOK_WORKING_DIR=$PWD after sourcing the hook and the executor chdirs to it.

Binary and polyglot (non-shell) hooks can't do this. They run via runUnwrappedHook in a child process, so any chdir() inside the child is lost.

This PR gives unwrapped hooks a first-class way to change the working directory for subsequent phases, via a new Job API endpoint and a matching buildkite-agent workdir set <path> CLI command.

How it works:

• The CLI resolves relative paths against its own cwd (the hook's actual working directory) and validates the path exists and is a directory.
• The Job API endpoint (PUT /workdir) accepts only absolute paths and records the request as pending (it does not stat the filesystem).
• Because the Job API handler runs in the server goroutine while shell.wd is single-writer, the endpoint only records the request. The executor consumes the pending workdir after the hook process exits and applies it through the existing applyEnvironmentChanges → shell.Chdir path — single-threaded, consistent with the wrapped-hook path.
• The change persists across subsequent hooks and the command phase via shell.wd; it is not reset between hooks.

Scope is intentionally the unwrapped hook path. Wrapped POSIX shell hooks continue to use the wrapper's $PWD capture and are unaffected.

Context

Internal Linear ticket (Buildkite).

Changes

• Job API: new PUT /api/current-job/v0/workdir endpoint (jobapi/workdir.go), WorkdirSetRequest/WorkdirSetResponse payloads, Server.pendingWorkdir + take-on-read TakePendingWorkdir, and Client.SetWorkdir.
• Executor: retain the *jobapi.Server on the Executor; runUnwrappedHook now consumes any pending workdir and applies it after the hook exits. New hook.EnvChangesForWorkdir constructor for workdir-only changes.
• CLI: new workdir parent command with a set subcommand (leaving room for a future get).

$ buildkite-agent workdir --help
Usage:

  buildkite-agent workdir <command> [options...]

Available commands are:

  set   Sets the working directory for subsequent phases of the job
  help  Shows a list of commands or help for one command

$ buildkite-agent workdir set --help
Usage:

    buildkite-agent workdir set <path>

Description:

Sets the working directory for subsequent phases of the current job. The change
persists across later hooks and the command phase.

This is intended for binary and polyglot (non-shell) hooks, which run in a child
process whose own directory changes are otherwise lost. Wrapped POSIX shell
hooks can change the working directory simply by `cd`-ing.

Relative paths are resolved against the current working directory of this command
(i.e. the hook's actual working directory). The path must exist and be a
directory.

Testing

• Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
• Code is formatted (with go tool gofumpt -extra -w .)

New tests:

• jobapi/workdir_test.go: absolute path → 200 + pending workdir set; relative → 422; empty → 422; TakePendingWorkdir returns once then clears.
• clicommand/workdir_set_test.go: relative paths resolved against cwd; nonexistent path and non-directory both error before any API call.
• internal/job/integration/hooks_integration_test.go (TestBinaryHookCanSetWorkdir): a binary pre-command hook sets the workdir via the Job API; asserts both the command and post-command hooks run in the requested directory, proving the change persists across hooks.

Disclosures / Credits

Claude Code (Opus) implemented this change from a written plan I authored and reviewed, including the tests; I reviewed and verified the result.