fix(cli): suggest supervise for a registered remote mesh

.changeset/safe-remote-supervise.md

@@ -0,0 +1,5 @@
+---
+"@cotal-ai/cli": patch
+---
+
+Suggest supervise and delivery recovery commands when `up` refuses a registered remote mesh, and clarify that removing a mesh only drops this machine's registry route.

docs/cli.md

@@ -437,15 +437,18 @@ user-auth space cannot be registered by hand, because its IdP pins are trust tha
 address, or credentials that mesh will not accept, fails here instead of at the first `spawn`;
 `--force` records without verifying (and replaces an existing record).
 
-`meshes rm` drops records — it never stops a mesh. For a mesh running on this machine `cotal down`
+`meshes rm` drops this machine's registry route to the named mesh — it never stops a mesh. For a mesh running on this machine `cotal down`
 is the right verb, and `rm` says so unless you pass `--force`. A record you added by hand is only
 removed by something that names it — `meshes rm`, or an `add --force` replacement — or by a
 `cotal up` that actually starts the broker for that same space, server and root, which becomes that
 mesh and so takes the record over (a `cotal up` for that space anywhere else refuses instead).
 Nothing that merely *infers* a record is stale touches it: an
 unreachable broker is listed `offline` and stays, and `cotal down` / `cotal clean all` leave it
 alone even when it shares a root with the project they are tearing down, because nothing on this
-machine could write it back.
+machine could write it back. When `up` refuses a hand-registered non-local broker, recover that
+mesh's control plane with `cotal supervise --space <space> --server <registered-url>` and, where
+durable delivery is needed, `cotal deliver --space <space> --server <registered-url>`; use `meshes
+rm` only for a record you know is stale.
 
 `use <space>` sets that default; the selection applies from every directory,
 including inside another mesh's project. `status` is a read-only report: machine prerequisites

docs/run-a-mesh.md

@@ -176,12 +176,15 @@ can be dropped on a hunch — a failed liveness probe, a `cotal down` in its pro
 `cotal up` writes the record straight back. One you registered by hand cannot be reconstructed, so
 nothing removes it by inference: an unreachable broker is shown as `offline` in `cotal meshes`, and
 `cotal down` / `cotal clean all` leave it alone even when `--root` pointed at the project they are
-tearing down. A `cotal up` for that space refuses outright (naming `cotal meshes rm`) unless it is
-that same endpoint: finding a broker already answering there is a refresh that starts nothing and
-leaves the record's provenance alone, while actually starting the broker for that space, server and
-root makes this machine the one running it, so the record becomes an ordinary local one that
-`cotal down` clears. `cotal meshes rm` drops it and re-registering with `--force` replaces it. `rm`
-only forgets a mesh — to stop one running here, use `cotal down`.
+tearing down. A `cotal up` for that space refuses outright unless it is that same endpoint: for a non-local
+registered broker it names `cotal supervise --space <space> --server <registered-url>` to recover
+the manager and `cotal deliver --space <space> --server <registered-url>` for durable delivery.
+It reserves `cotal meshes rm` for a record you know is stale. Finding a broker already answering at
+the same endpoint is a refresh that starts nothing and leaves the record's provenance alone, while
+actually starting the broker for that space, server and root makes this machine the one running it,
+so the record becomes an ordinary local one that `cotal down` clears. `cotal meshes rm` drops this
+machine's registry route and re-registering with `--force` replaces it. `rm` only forgets a mesh —
+to stop one running here, use `cotal down`.
 
 ## Watching
 

implementations/cli/smoke/fixtures/remote-supervise-remedy.mutations.json

@@ -0,0 +1,12 @@
+{
+  "command": "pnpm smoke:meshes-registry",
+  "mutations": [
+    {
+      "name": "the remote registered-mesh branch falls back to record removal instead of naming the control-plane recovery commands",
+      "file": "implementations/cli/src/commands/up.ts",
+      "find": "`To recover its control plane, run \\`cotal supervise --space ${space} --server ${existing.server}\\`; for durable delivery, run \\`cotal deliver --space ${space} --server ${existing.server}\\`. ` +",
+      "replace": "`Remove its record with \\`cotal meshes rm ${space}\\`. ` +",
+      "expectRed": "a remote registered space directs the operator to supervise and deliver on its recorded broker"
+    }
+  ]
+}

implementations/cli/smoke/meshes-registry.smoke.ts

@@ -24,7 +24,7 @@ import { strict as assert } from "node:assert";
 import { spawn } from "node:child_process";
 import { existsSync, mkdirSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from "node:fs";
 import { createServer } from "node:net";
-import { tmpdir } from "node:os";
+import { networkInterfaces, tmpdir } from "node:os";
 import { dirname, join } from "node:path";
 
 // Sandbox the machine-home BEFORE anything reads the registry — homeCotalDir() reads COTAL_HOME per
@@ -98,7 +98,15 @@ function projectRoot(label: string): string {
 const DEAD = `nats://127.0.0.1:${await freePort()}`; // nothing listens there
 const brokerPort = await freePort();
 const LIVE = `nats://127.0.0.1:${brokerPort}`;
-const broker = spawn("nats-server", ["-a", "127.0.0.1", "-p", String(brokerPort)], { stdio: "ignore" });
+// Drive the remote-registration remedy through a real non-loopback route. The broker is still a
+// smoke fixture, but the registry sees the same address class an operator names for a mesh hosted
+// elsewhere, rather than a loopback alias.
+const remoteAddress = Object.values(networkInterfaces()).flat().find((a) =>
+  a?.family === "IPv4" && !a.internal,
+)?.address;
+assert.ok(remoteAddress, "the smoke host has no non-loopback IPv4 address");
+const REMOTE_LIVE = `nats://${remoteAddress}:${brokerPort}`;
+const broker = spawn("nats-server", ["-a", "0.0.0.0", "-p", String(brokerPort)], { stdio: "ignore" });
 // A second broker that actually ENFORCES something, so the guided flow's auth branches (the
 // "this folder holds no credentials" recovery) are reachable at all. Password auth is enough:
 // probeEnforcement only asks whether a bare connect is refused.
@@ -276,6 +284,21 @@ try {
   check("`up` refuses to reclaim a registered space rather than deleting it", claimError !== undefined, claimError?.message);
   check("…and the registration survives the refusal", findMesh("claimed") !== undefined, loadMeshes());
   check("…naming `cotal meshes rm` as the way through", claimError?.message.includes("cotal meshes rm claimed") === true, claimError?.message);
+
+  // This is the #757 reproduction: write an operator registration for a broker reached through a
+  // real non-loopback address, then drive `up`'s actual claim guard. A registry entry is the route
+  // used by supervise, attach, and read commands, so telling its operator only to remove it loses
+  // the control path they came to recover.
+  check("the remote broker route is live before it is registered", await isReachable(REMOTE_LIVE), REMOTE_LIVE);
+  recordMesh({ space: "claimed-remote", server: REMOTE_LIVE, root, mode: "open", origin: "manual", ts: new Date(0).toISOString() });
+  let remoteClaimError: Error | undefined;
+  await claimSpace("claimed-remote", LIVE, localRoot).catch((e: Error) => void (remoteClaimError = e));
+  check("a remote registered space directs the operator to supervise and deliver on its recorded broker",
+    remoteClaimError?.message.includes(`cotal supervise --space claimed-remote --server ${REMOTE_LIVE}`) === true &&
+      remoteClaimError.message.includes(`cotal deliver --space claimed-remote --server ${REMOTE_LIVE}`) === true &&
+      remoteClaimError.message.includes("only if that record is stale") === true,
+    remoteClaimError?.message);
+  check("…and the remote registration survives that control-plane remedy", findMesh("claimed-remote")?.server === REMOTE_LIVE, findMesh("claimed-remote"));
   // A LIVE registered holder must reach the SAME refusal. Deciding liveness first sent the operator
   // to `cotal down`, which cannot stop a mesh this machine does not run.
   recordMesh({ space: "claimed-live", server: LIVE, root, mode: "open", origin: "manual", ts: new Date(0).toISOString() });
@@ -290,6 +313,7 @@ try {
   check("a dead `up` holder is still reclaimed (unchanged)", findMesh("reclaimable") === undefined, loadMeshes());
   removeMesh("claimed");
   removeMesh("claimed-live");
+  removeMesh("claimed-remote");
 
   // PROVENANCE IS NOT DOWNGRADED BY A REFRESH. Several `up` paths re-record a mesh they did not
   // start (the "a broker is already on this port" branch concludes it is up from reachability
@@ -339,6 +363,7 @@ try {
   setCurrent("remote-dead");
   const removed = await run(["rm", "remote-dead"]);
   check("rm drops the record", removed.code === 0 && findMesh("remote-dead") === undefined, loadMeshes());
+  check("rm says it removed this machine's route to the mesh", removed.out.includes("removed this machine's registry route to that mesh"), removed.out);
   check("rm releases a current that pointed at it", getCurrent() === undefined, getCurrent());
   check("rm says the default is gone", removed.out.includes("no default mesh now"), removed.out);
 

implementations/cli/src/commands/meshes.ts

@@ -199,8 +199,8 @@ function take<T>(r: Check<T>): T {
 
 // ---- rm -------------------------------------------------------------------------------------
 
-/** `cotal meshes rm <space> …` — drop records. This never stops a mesh: it removes what THIS
- *  machine remembers about one. For a mesh running here that distinction is a footgun (the broker
+/** `cotal meshes rm <space> …` — drop this machine's registry route. This never stops a mesh: it
+ *  removes what THIS machine remembers about one. For a mesh running here that distinction is a footgun (the broker
  *  keeps running with nothing pointing at it), so those are refused in favour of `cotal down`. */
 async function removeMeshes(names: string[], v: Values): Promise<void> {
   if (names.length === 0) {
@@ -243,7 +243,7 @@ async function removeMeshes(names: string[], v: Values): Promise<void> {
       clearCurrent();
       clearedCurrent = true;
     }
-    console.log(c.green(`✓ unregistered "${space}"`), c.dim(m.server));
+    console.log(c.green(`✓ unregistered "${space}"`), c.dim(`${m.server} (removed this machine's registry route to that mesh)`));
   }
   if (clearedCurrent) console.log(c.dim("there is no default mesh now - `cotal use <name>` to set one"));
   if (failed) process.exit(1);

implementations/cli/src/commands/up.ts

@@ -2117,7 +2117,8 @@ function realpathSafe(p: string): string {
 
 /** A space name maps to one mesh in the registry (the key `--space`/`use`/`down` act on). Before
  *  starting a broker, refuse to reuse a space already claimed by a DIFFERENT live mesh — a stale/dead
- *  holder is reclaimed. Re-`up`ping the same mesh (same server + root) is a refresh (port-reachable
+ *  holder is reclaimed. A non-local operator registration instead directs recovery to its existing
+ *  broker; removing that record is reserved for an operator-confirmed stale route. Re-`up`ping the same mesh (same server + root) is a refresh (port-reachable
  *  path). NOTE: this is a best-effort sequential guard — two `cotal up --space X` racing from
  *  different roots within the same instant can both pass the check before either records; that
  *  concurrent case is out of scope (a single-operator CLI action), not synchronized with a lock. */
@@ -2129,8 +2130,18 @@ export async function claimSpace(space: string, server: string, root: string): P
   // reclaimed: unreachable is not proof the mesh is gone (the record describes a broker on another
   // machine), the reclaim runs BEFORE this launch starts anything, and `cotal down` — what the
   // liveness branch below would advise — cannot stop a mesh this machine does not run.
-  if (existing.origin === "manual")
+  if (existing.origin === "manual") {
+    const host = new URL(existing.server).hostname.replace(/^\[|\]$/g, "");
+    const local = host === "localhost" || host === "::1" || host === "0:0:0:0:0:0:0:1" || /^127(?:\.\d{1,3}){3}$/.test(host);
+    if (!local) {
+      throw new Error(
+        `space "${space}" is registered to a mesh at ${existing.server} (${existing.root}) - it was registered by hand, so \`cotal up\` neither takes it over nor reclaims the name. ` +
+        `To recover its control plane, run \`cotal supervise --space ${space} --server ${existing.server}\`; for durable delivery, run \`cotal deliver --space ${space} --server ${existing.server}\`. ` +
+        `Use \`cotal meshes rm ${space}\` only if that record is stale, or start this one under a different \`--space\``,
+      );
+    }
     throw new Error(`space "${space}" is registered to a mesh at ${existing.server} (${existing.root}) - it was registered by hand, so \`cotal up\` neither takes it over nor reclaims the name: \`cotal meshes rm ${space}\` to drop that record first, or start this one under a different \`--space\``);
+  }
   if (await isReachable(existing.server)) {
     throw new Error(`space "${space}" is already in use by a mesh at ${existing.server} (${existing.root}) - pick a different \`--space\`, or \`cotal down\` it first`);
   }