Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/INTERACTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ The cat's answer/narration renders in a tethered speech bubble
(`reactions/speechBubble.ts`), fed by the same CaptionSink seam that feeds the
dev caption panel. When — and only when — memory was actually RECALLED during the
turn (a same-run `status` reporting a recalled fact OR episode, `factCount>0 ||
episodeCount>0`), the bubble appends a subtle, truthful receipt: "used what I
episodeCount>0`), the bubble appends a subtle, truthful receipt: "checked what I
remember about your setup" (`MEMORY_RECEIPT_TEXT`). (It attests recall, not that a
specific fact provably shaped the answer — it never claims more than the pipeline
can verify.) The gate is strict: the receipt attaches ONLY
Expand Down
9 changes: 8 additions & 1 deletion docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,14 @@ first-turn/memory validation** — otherwise it is deferred (§7 rule 4); C is s
Even then, prompt-capture is **necessary, not
sufficient** — it proves memory *shaped the task*, not that the executor *honored* it; so keep the
**diff-reflects-preference output check** as the real-honoring proof (judged across runs, given
stochasticity), and keep **real Codex/Claude as a separate auth/readiness check**. **The manual check
stochasticity), and keep **real Codex/Claude as a separate auth/readiness check**.
**UPDATE (2026-07-04): the real-honoring check is now AUTOMATED** — `npm run verify:memory-steered-real`
(`src/core/memory2/memorySteered.live.test.ts`, opt-in/non-CI) runs real qwen2.5:3b + real codex over
N treatment/control runs and asserts the recalled marker reaches `args.task` AND that the created `.py`
file HONORS the convention (marker as its header line — not merely present somewhere), ≥60% treatment,
0 control. First measured run: **5/5 args.task · 5/5 marker-in-file · 0/5 control** — the moat mechanism
holds for a well-formed, relevant preference (the guard's stricter per-header assertion re-measures it). (The *extraction* hop — conversation→stored fact,
the ~40% behavioral-eval weak link — remains separate and is mitigated by the executor-facts confirm gate.) **The manual check
is auditable — it produces an evidence packet:** {build id; both memory-root paths + the empty-control
proof; the exact prompt; the event stream; the generated `args.task`; the resulting diff; observer +
date}. **Packet format (privacy-safe):** the **raw** packet (full DECIDE prompt — recalled
Expand Down
2 changes: 1 addition & 1 deletion docs/VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ Manual full-window checklist:
| 2 | Submit a non-empty prompt with project + brain ready | Start disables, typed text stays visible, Stop enables after readiness accepts the turn, status says `Thinking... click Stop if you need to pause.` |
| 3 | Click Stop before `run.started` | Stop says `Stopping...`, status says `Stopping...`, and the turn ends with neutral `Stopped.` copy |
| 4 | Click Stop after `run.started` on an active executor task | Stop says `Stopping...`, the executor is aborted, no completed file change lands, and the turn ends with neutral `Stopped.` copy |
| 5 | Let a successful turn finish | Status shows a compact receipt such as `Done. Memory checked.` or `Done. Changed 1 file. Memory used.` |
| 5 | Let a successful turn finish | Status shows a compact receipt such as `Done. Memory checked.` or `Done. Changed 1 file. Memory recalled.` |
| 6 | Submit while project selection or local brain readiness blocks dispatch | Start remains enabled, Stop remains disabled, and the draft stays in the input |

## On-screen floating Ask + Stop
Expand Down
2 changes: 1 addition & 1 deletion docs/plans/floating-redesign.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ memory/job is the payload.
4. **Modest transparent click-through window** (NOT full-screen). Grow enough for bloom room (~360–480 wide
× ~360–440 tall — final size pinned in S1 against the cat sprite + tether math); drag = move the window
(existing path, unchanged). The ~80% dead space passes clicks through to the app underneath.
5. **Truthful memory receipt.** On a successful turn, a speech bubble may say "used what I remember about
5. **Truthful memory receipt.** On a successful turn, a speech bubble may say "checked what I remember about
your setup" ONLY when memory was actually recalled that turn (a same-run status with a recalled fact or
episode; it attests recall, not that a fact provably shaped the answer) — never on every turn (that would
be a lie that erodes trust). Bare no-payload success shows nothing (restraint; matches today's no-Done-
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"verify:packaged-sdk-executor": "node scripts/smoke-packaged-sdk-executor.mjs",
"verify:packaged-model-setup": "node scripts/smoke-packaged-model-setup.mjs",
"verify:memory-steered": "node scripts/smoke-memory-steered.mjs",
"verify:memory-steered-real": "RORO_STEERING_LIVE=1 OLLAMA_AVAILABLE=1 OLLAMA_MODEL=qwen2.5:3b OLLAMA_EMBED_MODEL=nomic-embed-text vitest run src/core/memory2/memorySteered.live.test.ts",
"verify:release-artifact": "node scripts/verify-release-artifact.mjs --mode default",
"verify:release-artifact:dmg": "node scripts/verify-release-artifact.mjs --mode default --require-dmg",
"verify:release-artifact:signed": "node scripts/verify-release-artifact.mjs --mode signed",
Expand Down
8 changes: 7 additions & 1 deletion scripts/smoke-memory-steered.mjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
// scripts/smoke-memory-steered.mjs — the automated "memory steered the coding work" proof.
// scripts/smoke-memory-steered.mjs — the DETERMINISTIC plumbing + trace-capture proof of memory steering.
//
// This is one of TWO complementary steering proofs. THIS one runs the REAL packaged app end-to-end (preload
// bridge → IPC → orchestrator → RORO_TRACE_DECIDE capture over CDP) with a FAKE brain + FAKE codex, so it is
// deterministic and CI-safe and pins the packaged-app WIRING. Its sibling `npm run verify:memory-steered-real`
// (src/core/memory2/memorySteered.live.test.ts) proves the SEMANTIC half — that a REAL local model + REAL
// codex actually carry a recalled preference into a real diff (opt-in, non-CI). Keep BOTH: plumbing vs models.
//
// Run AFTER `npm run package`: `npm run verify:memory-steered` (or RORO_PACKAGED_APP=/abs/Roro.app).
//
Expand Down
5 changes: 3 additions & 2 deletions src/core/brain/eval/proposalFixtures.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { buildProposalPrompt } from '../../orchestrator/factProposals/prompt';
// canned model replies (grounded admitted; ungrounded/boolean/garbage dropped).

describe('proposal eval fixture digests — real-shaped, straight from the production mappers', () => {
it('codex: the captured v0.139.0 run digests to 2 commands, 1 added file, 4 messages, no finalText', () => {
it('codex: the captured v0.139.0 run digests to 2 commands, 1 added file, 4 messages, and the stitched finalText', () => {
const d = codexFixtureDigest();
expect(d.agent).toBe('codex');
expect(d.outcome).toBe('completed');
Expand All @@ -20,7 +20,8 @@ describe('proposal eval fixture digests — real-shaped, straight from the produ
expect(d.commands[1]).toContain('python3 hello.py');
expect(d.files).toEqual([{ path: '/tmp/companion_scratch/hello.py', op: 'add' }]);
expect(d.messages).toHaveLength(4);
expect(d.finalText).toBeUndefined(); // codex run.completed carries no finalText — same as production
// finalText is now the LAST agent_message, stitched in the runCodex loop (mirrored here) — same as production.
expect(d.finalText).toBe('Created [hello.py](/tmp/companion_scratch/hello.py:1).\n\nVerified with `python3 hello.py`; it prints:\n\n```text\nhi\n```');
});

it('claude: the documented 2.1.x sample digests to 1 command, 1 message, and a finalText', () => {
Expand Down
17 changes: 11 additions & 6 deletions src/core/brain/eval/proposalFixtures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import {
mapCodexThreadEvent,
stitchCodexFinalText,
mapClaudeMessage,
mapClaudeMessageBlocks,
mapClaudeStreamEvent,
Expand All @@ -29,14 +30,18 @@ const CLAUDE_FIXTURE_TASK = 'make a hello.py script and check it works';
export function codexFixtureDigest(): RunDigest {
const lines = readFileSync(join(__dirname, '../../executor/__fixtures__/codex_hello.jsonl'), 'utf8').split('\n');
const acc = createDigestAccumulator();
// Mirror production exactly: the pure mapper emits run.completed with NO finalText; runCodex's stream loop
// then stitches the LAST agent_message onto it. Replay the same stitch here (stitchCodexFinalText) so the
// deterministic eval stays representative — codex's finalText is now its final assistant message, like claude's.
const events = lines
.map((line) => line.trim())
.filter((s) => s && s[0] === '{')
.map((s) => mapCodexThreadEvent(JSON.parse(s), 'run_proposal_fixture_codex'))
.filter((ev): ev is NonNullable<typeof ev> => ev !== null);
let finalText: string | undefined;
for (const line of lines) {
const s = line.trim();
if (!s || s[0] !== '{') continue;
const ev = mapCodexThreadEvent(JSON.parse(s), 'run_proposal_fixture_codex');
if (!ev) continue;
for (const ev of stitchCodexFinalText(events)) {
acc.see(ev);
if (ev.kind === 'run.completed') finalText = ev.finalText; // codex emits none — stays undefined, like production
if (ev.kind === 'run.completed') finalText = ev.finalText;
}
return acc.finish({
runId: 'run_proposal_fixture_codex',
Expand Down
29 changes: 28 additions & 1 deletion src/core/executor/codex.ts
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,30 @@ export function codexExecArgs(opts: Pick<ExecutorRunOptions, 'repo' | 'prompt' |
* `codex exec --json --skip-git-repo-check -s workspace-write -C <repo> "<prompt>" </dev/null`
* stdin = 'ignore' (== /dev/null), stdout = JSONL parsed via readline, stderr drained.
*/
/** codex has NO single authoritative final-result string (claude uses the SDK's `m.result` — see claude.ts).
* Its LAST assistant `agent_message` is NORMALLY the final answer — a best-effort heuristic, not a guarantee
* (a run ending on a terse/mid-work message would surface that instead) — so stitch it onto run.completed as
* `finalText`: the truthful memory receipt (speechBubble.ts), the answer caption, and the "job done"
* notification read it. The receipt claims RECALL not answer content, so the heuristic never makes it lie.
* Pass-through when it's not run.completed, when finalText is already set (NEVER clobber the SDK/claude value),
* or when there was no message (leaving finalText empty correctly keeps the receipt suppressed). */
export function attachCodexFinalText(ev: ActionEvent, lastMessageText: string | undefined): ActionEvent {
return ev.kind === 'run.completed' && !ev.finalText && lastMessageText
? { ...ev, finalText: lastMessageText }
: ev;
}

/** Batch form for replay contexts (the proposal-eval fixture): track the last message, attach it. runCodex's
* streaming loop mirrors this exact rule inline (tracking `lastMessageText` + attachCodexFinalText) so the
* two paths never diverge — the proposal eval stays representative of production. */
export function stitchCodexFinalText(events: readonly ActionEvent[]): ActionEvent[] {
let last: string | undefined;
return events.map((ev) => {
if (ev.kind === 'message' && ev.text) last = ev.text;
return attachCodexFinalText(ev, last);
});
}

export async function* runCodex(
opts: ExecutorRunOptions,
): AsyncIterable<ActionEvent> {
Expand Down Expand Up @@ -268,6 +292,7 @@ export async function* runCodex(
child.on('close', (code, signal) => resolve({ code, signal }));
});
let emittedTerminal = false;
let lastMessageText: string | undefined; // the last agent_message → stitched onto run.completed as finalText

const rl = createInterface({ input: child.stdout!, crlfDelay: Infinity });
try {
Expand All @@ -283,8 +308,10 @@ export async function* runCodex(
}
const mapped = mapCodexThreadEvent(obj, runId);
if (mapped) {
if (mapped.kind === 'message' && mapped.text) lastMessageText = mapped.text;
if (mapped.kind === 'run.completed' || mapped.kind === 'run.failed') emittedTerminal = true;
yield mapped;
// Streaming mirror of stitchCodexFinalText: attach the last agent_message as run.completed.finalText.
yield attachCodexFinalText(mapped, lastMessageText);
}
}
} catch (e) {
Expand Down
2 changes: 1 addition & 1 deletion src/core/executor/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { CodexExecutor } from './codex';
import { ClaudeExecutor } from './claude';
import { ClaudeSdkExecutor } from './claudeSdk';

export { CodexExecutor, runCodex, mapCodexThreadEvent } from './codex';
export { CodexExecutor, runCodex, mapCodexThreadEvent, stitchCodexFinalText } from './codex';
export {
ClaudeExecutor,
runClaude,
Expand Down
53 changes: 53 additions & 0 deletions src/core/executor/runCodexStitch.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Deterministic coverage for the runCodex STREAMING stitch — the default-codex production path the receipt
// slice lights up. fixtures.test.ts drives the raw mapper (no stitch); proposalFixtures drives the BATCH
// helper; this drives the ACTUAL runCodex loop by mocking spawn with a fake child whose stdout replays the
// captured codex JSONL, and asserts run.completed.finalText === the last agent_message (no real codex spawned).
import { describe, it, expect, vi } from 'vitest';
import { Readable } from 'node:stream';
import { EventEmitter } from 'node:events';
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import type { ActionEvent } from '../../shared/events';

const h = vi.hoisted(() => ({ child: null as unknown as EventEmitter & { stdout: Readable; stderr: Readable } }));
vi.mock('node:child_process', () => ({ spawn: () => h.child }));

import { runCodex } from './codex';

function fakeChild(stdout: string): EventEmitter & { stdout: Readable; stderr: Readable } {
const child = new EventEmitter() as EventEmitter & { stdout: Readable; stderr: Readable; exitCode: number | null; signalCode: null; kill: () => boolean };
child.stdout = Readable.from(stdout);
child.stderr = Readable.from('');
child.exitCode = null;
child.signalCode = null;
child.kill = () => true;
child.stdout.once('end', () => { child.exitCode = 0; queueMicrotask(() => child.emit('close', 0, null)); });
return child;
}

async function drain(): Promise<ActionEvent[]> {
const events: ActionEvent[] = [];
for await (const ev of runCodex({ repo: '/tmp/scratch', prompt: 'create hello.py' })) events.push(ev);
return events;
}

describe('runCodex — the streaming finalText stitch (deterministic)', () => {
it('stitches the last agent_message onto run.completed.finalText over the captured v0.139.0 fixture', async () => {
h.child = fakeChild(readFileSync(join(__dirname, '__fixtures__/codex_hello.jsonl'), 'utf8'));
const events = await drain();
const done = events.find((e) => e.kind === 'run.completed') as { finalText?: string } | undefined;
expect(done).toBeTruthy();
expect(done!.finalText).toBe('Created [hello.py](/tmp/companion_scratch/hello.py:1).\n\nVerified with `python3 hello.py`; it prints:\n\n```text\nhi\n```');
});

it('leaves finalText undefined when the stream carries no agent_message (receipt stays suppressed)', async () => {
// A minimal stream: thread + turn.completed, no agent_message → no answer → no finalText.
h.child = fakeChild(
'{"type":"thread.started","thread_id":"t"}\n{"type":"turn.completed","usage":{"input_tokens":1,"output_tokens":1}}\n',
);
const events = await drain();
const done = events.find((e) => e.kind === 'run.completed') as { finalText?: string } | undefined;
expect(done).toBeTruthy();
expect(done!.finalText).toBeUndefined();
});
});
56 changes: 56 additions & 0 deletions src/core/executor/stitchFinalText.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Reproduce-first (codex finalText): codex has no single authoritative final-result string like claude's
// SDK m.result — its LAST assistant agent_message IS the answer. attachCodexFinalText / stitchCodexFinalText
// stitch that onto run.completed.finalText so the truthful receipt + captions + job notification have it.
import { describe, it, expect } from 'vitest';
import { attachCodexFinalText, stitchCodexFinalText } from './codex';
import type { ActionEvent } from '../../shared/events';

const msg = (text: string): ActionEvent => ({ kind: 'message', runId: 'r', text, ts: 0 });
const completed = (finalText?: string): ActionEvent => ({ kind: 'run.completed', runId: 'r', ok: true, ts: 0, ...(finalText ? { finalText } : {}) });

describe('attachCodexFinalText', () => {
it('attaches the last agent_message text as finalText on run.completed', () => {
const out = attachCodexFinalText(completed(), 'the answer');
expect(out.kind).toBe('run.completed');
expect((out as { finalText?: string }).finalText).toBe('the answer');
});

it('leaves finalText undefined when there was no message (empty answer → receipt stays suppressed)', () => {
expect((attachCodexFinalText(completed(), undefined) as { finalText?: string }).finalText).toBeUndefined();
expect((attachCodexFinalText(completed(), '') as { finalText?: string }).finalText).toBeUndefined();
});

it('never clobbers an already-set finalText (idempotent — claude/SDK path already carries it)', () => {
expect((attachCodexFinalText(completed('already'), 'other') as { finalText?: string }).finalText).toBe('already');
});

it('passes non-run.completed events through unchanged', () => {
const m = msg('hi');
expect(attachCodexFinalText(m, 'x')).toBe(m);
});
});

describe('stitchCodexFinalText (batch — the proposal-fixture replay path)', () => {
it('stitches the LAST message text onto the terminal run.completed', () => {
const out = stitchCodexFinalText([msg('first'), msg('the real answer'), completed()]);
const done = out.find((e) => e.kind === 'run.completed') as { finalText?: string };
expect(done.finalText).toBe('the real answer');
});

it('ignores empty message text (a blank agent_message never becomes the answer)', () => {
const out = stitchCodexFinalText([msg(''), msg('real'), completed()]);
expect((out.find((e) => e.kind === 'run.completed') as { finalText?: string }).finalText).toBe('real');
});

it('a TRAILING blank agent_message does not wipe the real answer (pins the `&& ev.text` guard)', () => {
// Without the guard, last-wins would take the trailing '' and finalText would go undefined → the answer
// would silently vanish from the receipt/caption/notification. This case fails if the guard is removed.
const out = stitchCodexFinalText([msg('the real answer'), msg(''), completed()]);
expect((out.find((e) => e.kind === 'run.completed') as { finalText?: string }).finalText).toBe('the real answer');
});

it('a run with no message yields run.completed with no finalText', () => {
const out = stitchCodexFinalText([completed()]);
expect((out[0] as { finalText?: string }).finalText).toBeUndefined();
});
});
Loading
Loading