Skip to content
99 changes: 99 additions & 0 deletions reference/components/javascript-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,3 +194,102 @@ Returns the outgoing `Response` object for the current request, or `undefined` i
### Current Working Directory

Harper has a multi-threaded server architecture and uses the harper data root path as the current working directory. Components should not and cannot change the current working directory, and must not use `process.chdir()` or any package that does.

## Child Processes

Harper substitutes its own `child_process` module into components that need to launch a helper binary or a sidecar process. The substitute adds two things on top of Node's API: a command allowlist, and a node-wide single-process lock keyed by a name you supply.

```javascript
import { spawn } from 'node:child_process';

const agent = spawn('datadog-agent', ['run'], {
name: 'datadog-agent',
version: 3,
});
```

### Which imports get the substitute

The substitution happens in Harper's module loader, so it only reaches code that loader handles:

| How the module is reached | What you get |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- |
| `import` from component source under `applications.moduleLoader: vm-current-context` (the default) or `vm` | Harper's constrained module |
| Dynamic `import('node:child_process')` from component source under those same loaders, including from CommonJS | Harper's constrained module |
| `require('node:child_process')` | Node's unmodified module |
| Any import under `applications.moduleLoader: compartment` | Node's unmodified module |
| Any import under `applications.moduleLoader: native` | Node's unmodified module |
| A dependency loaded by the native loader — with the default `applications.dependencyLoader: auto` that is any package which does not list `harper` as a dependency | Node's unmodified module |

Two consequences are worth planning around. A supervisor factored into an npm package that does not depend on `harper` receives the real `child_process`: no allowlist, no lock, and one child per worker thread rather than one per node. And under `compartment` the substitute is bypassed entirely — that loader resolves built-ins through Node directly, so the allowlist, the mandatory `name`, the single-process lock, and the `execSync` block all disappear together. Keep process-spawning code in component source, reached with `import`, under one of the VM loaders.

### Which functions are usable

| Function | Status |
| ---------- | ---------------------------------------------------------------------------------------------- |
| `spawn` | Supported. Subject to the allowlist and the `name` lock. |
| `execFile` | Supported. Subject to the allowlist and the `name` lock. |
| `fork` | Supported and exempt from the allowlist, since it launches Node itself. Still requires `name`. |
| `exec` | Not usable. See below. |
| `execSync` | Always throws. Harper does not permit synchronous spawning. |

Nothing else Node's `child_process` exports is present. `spawnSync`, `execFileSync`, `ChildProcess`, and the rest are absent from the substituted module, so a named import of one fails when the module is linked rather than at the call site.

The substitute takes `(command, args, options, callback)` positionally for every wrapped function, but Node's `exec` signature is `exec(command[, options][, callback])`. So `exec('ffmpeg -version', { name: 'ffmpeg' })` puts the options object in the `args` slot and throws for a missing `name`, while shifting it into the `options` slot to satisfy that check produces a call Node's own `exec` rejects with `ERR_INVALID_ARG_TYPE`. Use `execFile` (whose signature does line up) or `spawn` instead.

### Allowlist

Every call except `fork` is checked against [`applications.allowedSpawnCommands`](../configuration/options.md#applications) before anything else:

- The check is an exact match on the first space-separated token of `command`. `'node'` matches an allowlisted `node`; `'/usr/local/bin/node'` does not, and a path containing a space can never match.
- The list is read once at startup. Changing it requires a restart.
- Because only the first token is matched, the allowlist is not an argument or injection barrier. If you pass `shell: true`, everything after the first token reaches a shell unchecked — treat the command string as trusted input regardless of the allowlist.

### One process per node: the `name` option

Harper runs a pool of worker threads, and component code runs on each of them. Without coordination, a component that spawns a sidecar would start one per thread. Harper prevents that with a PID-file lock, which is why `name` is mandatory:

- **`name` (string, required).** Spawning without it throws, on `fork` as well as the allowlisted functions.
- The lock file is `<rootPath>/pids/<name>.pid`. Line 1 is the child's PID; line 2, when `version` was passed, is the version.
- Exactly one caller wins the lock and spawns a real child process. Every other caller — other threads, and later calls with the same name — receives an `ExistingProcessWrapper` for the already-running process.
- The name is the whole key. It is not namespaced per component and it is interpolated into the path unsanitized, so two independently installed components that both pick `agent` share one lock and adopt each other's process, and a name containing `../` places the lock file outside `<rootPath>/pids/` entirely. Use a literal, path-safe name prefixed with your component's name — never one derived from configuration or any other input.
- When the real child exits, the thread that spawned it removes the PID file, so the next spawn call starts a fresh process.
- If a PID file survives an unclean shutdown, Harper recovers it by checking whether the recorded PID is still alive: if it is not, the stale file is removed and the next caller spawns normally. If the operating system has recycled that PID onto an unrelated process, Harper treats that process as the sidecar: it adopts it when you pass no `version`, and sends it `SIGTERM` when the `version` you pass does not match the recorded one. Another reason to keep the name unique.

### Replacing a running process: the `version` option

<VersionBadge version="v5.0.2" />
Comment thread
kriszyp marked this conversation as resolved.

`version` lets a component replace a process it started earlier — after an upgrade, for example — instead of adopting it.

- **`version` must be an integer.** Line 2 of the PID file is parsed with `parseInt`, and the comparison is a strict `!==` against the value you pass. A string `'3'` never equals the parsed `3`, so every call kills the running child and respawns it, forever. There is no validation of this; pass a number.
- The comparison is equality, not ordering: a version lower than the recorded one also triggers replacement.
- On a mismatch Harper signals the running process (`SIGTERM`), removes the PID file, and re-acquires the lock so the new version spawns.
- Omitting `version` means "adopt whatever is running under this name".

The handoff is not graceful, and replacement is the least robust part of this contract. Harper does not wait for the outgoing process to exit before starting the replacement, so a sidecar that holds a listening socket or an exclusive file lock must tolerate an overlapping predecessor. The outgoing process's exit handler also removes the PID file by path rather than by PID, so it can delete the lock the replacement just wrote — after which a later spawn call sees no lock and starts a second process alongside it. Prefer restarting the component (or the node) over relying on in-place version replacement for anything that cannot tolerate a duplicate.

### What `spawn` returns

The lock winner gets a real [`ChildProcess`](https://nodejs.org/api/child_process.html#class-childprocess). Everyone else gets an `ExistingProcessWrapper`, which is deliberately narrow:

| Member | Notes |
| --------------- | ----------------------------------------------------------- |
| `pid` | PID of the process that is actually running |
| `kill(signal?)` | Signals that process; returns `false` if it is already gone |
| `unref()` | Stops the liveness poll — see below |
| `'exit'` event | Emitted with `(null, null)` once the process is gone |

It does **not** have `stdout`, `stderr`, `stdin`, or `spawnargs` — those properties are simply absent, so reading one yields `undefined` and using it (`child.stdout.on(...)`) throws a `TypeError` on exactly the threads that lost the race, which is most of them. A component that reads the child's output must do so only on the thread that owns the real `ChildProcess`. Because `spawnargs` reads as `undefined` rather than throwing, it is the practical way to tell the two apart. It tests an internal detail rather than a discriminator Harper promises: it works only because the wrapper does not define `spawnargs` today, and a release that added the property would silently send every losing thread down the real-`ChildProcess` branch. Re-check it when you upgrade.

```javascript
const child = spawn('datadog-agent', ['run'], { name: 'datadog-agent', version: 3 });

if (child.spawnargs) {
child.stdout.on('data', (chunk) => logger.info(chunk.toString()));
} else {
child.unref();
}
```

The wrapper detects the process going away by polling it once a second with a `setInterval` that Harper does not unref, so a thread that never calls `unref()` keeps its event loop alive and delays shutdown. That same interval is the only source of the `'exit'` event, so the two members are mutually exclusive: once you call `unref()`, the wrapper will never emit `'exit'`. Pick one — watch the process, or release the timer.
10 changes: 5 additions & 5 deletions reference/configuration/options.md
Original file line number Diff line number Diff line change
Expand Up @@ -346,18 +346,18 @@ Added in: v5.0.0

```yaml
applications:
lockdown: freeze
moduleLoader: vm
lockdown: freeze-after-load
moduleLoader: vm-current-context
dependencyLoader: auto
allowedSpawnCommands:
- npm
- node
```

- `lockdown` — Indicates if intrinsic/built-in objects should be locked down/frozen. This provides additional security and protection against prototype pollution attacks. The options can be `freeze` (default, which freezes the important built-in objects, without interfering with most packages), 'none', or 'ses' (lockdown provided by `ses` package, which is more strict).
- `moduleLoader` — The method used to load modules (and isolate the application). The default is `vm`, which uses Node's VM to load modules. This can also be set to `native` (use standard Node module loader), or `compartment`, which uses the `ses` implementation of the proposed `Compartment` functionality.
- `lockdown` — Indicates if intrinsic/built-in objects should be locked down/frozen. This provides additional security and protection against prototype pollution attacks. The default is `freeze-after-load`, which freezes the important built-in objects once all components have loaded, so component initialization can still modify them. This can also be set to `freeze` (freeze before any application code loads), `none`, or `ses` (lockdown provided by the `ses` package, which is more strict). See [Intrinsic Lockdown](/release-notes/v5-lincoln/v5-migration#intrinsic-lockdown).
- `moduleLoader` — The method used to load modules (and isolate the application). The default is `vm-current-context`, which uses Node's VM module loader in Harper's own context so applications share JavaScript intrinsics. This can also be set to `vm` (VM loader with a separate context and its own intrinsics per application), `native` (standard Node module loader), or `compartment`, which uses the `ses` implementation of the proposed `Compartment` functionality. See [Module Loader Modes](/release-notes/v5-lincoln/v5-migration#module-loader-modes).
Comment thread
kriszyp marked this conversation as resolved.
- `dependencyLoader` — The application module loader can be used to load packages/dependencies (installed as `dependencies` from the package.json). The default is 'auto', which only use the VM module loader if the package specifies `harper` as a dependency. This can also be set to `app` to always use the application module loader or `native` to always native module loader for packages.
- `allowedSpawnCommands` - This lists the specific commands that can be spawned by the application (using `child_process`'s `spawn()`, `exec()`, and `execFile()` functions). You can add commands that you are application will need to launch (this is to protect against malicious code spawning processes).
- `allowedSpawnCommands` - This lists the specific commands that can be spawned by the application (using `child_process`'s `spawn()` and `execFile()` functions). You can add commands that your application will need to launch (this is to protect against malicious code spawning processes). Only the first token of the command is matched, spawning also requires a mandatory `name` option, and the call is subject to a node-wide single-process lock — see [Child Processes](../components/javascript-environment.md#child-processes) for the full contract.

## Component Configuration

Expand Down
4 changes: 2 additions & 2 deletions release-notes/v5-lincoln/v5-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,8 @@ Automatic context tracking can greatly simplify code and automatically handling
## Spawning new processes (via `node:child_process`)

The ability to spawn new processes is a dangerous pathway for exploitation and security vulnerabilities. Additionally, spawning processes from multiple threads presents unique challenges and hazards. In Harper version 5, spawning new processes (through node's `child_process` module) is more tightly controlled and managed.
First, any `spawn`, `exec`, or `execFile` may only spawn executables or commands that have been registered in the `applications.allowedSpawnCommands` configuration. This provides a much more secure evironment, preventing malicious intrusions.
Second, it is common to attempt to use spawn child processes with the expectations of code that is written to run in a single thread for an indefinite period of time. However, Harper runs multiple threads that may frequently be restarted. When attempting to start/run a supporting process, spawning every time a module loads leads multiplication of processes and orphaned processes. Harper now manages the spawning process to ensure a single process is spawned. To ensure that only a single process is started, the `spawn`, `exec`, etc. functions require a `name` property in the `options` argument, to create a named process that other threads can check and omit starting a new process if one is already started. If you really want to start a separate process from a previously started process, a new `name` must be provided.
First, any `spawn` or `execFile` may only spawn executables or commands that have been registered in the `applications.allowedSpawnCommands` configuration. This narrows what component code can launch. It is not a complete barrier — only the first token of the command is matched, and the substitution reaches only code the VM module loaders handle. (`exec` is not usable through the substituted module, and `execSync` always throws.) See [Child Processes](/reference/v5/components/javascript-environment#child-processes) for the full contract and its limits.
Second, it is common to attempt to spawn child processes with the expectations of code that is written to run in a single thread for an indefinite period of time. However, Harper runs multiple threads that may frequently be restarted. When attempting to start/run a supporting process, spawning every time a module loads leads to the multiplication of processes and orphaned processes. Harper now manages the spawning process to ensure a single process is spawned. To ensure that only a single process is started, the `spawn`, `execFile`, and `fork` functions require a `name` property in the `options` argument, to create a named process that other threads can check and omit starting a new process if one is already started. If you really want to start a separate process from a previously started process, a new `name` must be provided. The complete contract — allowlist matching, the PID-file lock, the `version` replacement option, and what a losing caller receives instead of a `ChildProcess` — is documented under [Child Processes](/reference/v5/components/javascript-environment#child-processes).

## Response Objects

Expand Down
Loading