Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,8 @@ roro-workspace-*.png
# On-device voice runtime assets (Silero/ORT wasm) + model weights (whisper STT, Kokoro TTS) — staged
# regenerable into the voice sub-package by packages/voice/scripts/stage-voice-assets.mjs. Never committed.
packages/voice/public/
src/brain/eval/latest.json
src/brain/eval/proposalLatest.json
src/core/brain/eval/latest.json
src/core/brain/eval/proposalLatest.json

# Voice runtime/model staging (dev-machine only; the app ships WITHOUT voice — packages/voice owns its own staging)
public/models/
Expand Down
44 changes: 43 additions & 1 deletion HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,14 +179,47 @@ Every turn flows through **one** path — now the typed run state machine under
- **W7 (#149):** the **Electron-free `src/core/` split** + the enforced import boundary + the four platform ports +
`userData` as a boot parameter; tests reach the platform via `installTestPorts`, not `vi.mock('electron')`.
- **W8:** this reconciliation (docs) + the packaged smoke suite consolidation (a parallel track).
- **The floating redesign — resting purity (#153–#160, 2026-07-03 → 07-04).** At rest the cat is ALL there is:
the pill is deleted; input is a hover-summoned bubble (#157) with an Ask·Memory·Project menu + tethered
popovers (#158) — the moat surface (Memory panel) is reachable *in the pet*. Plus the stage + pure tether
positioner + WorkingChip (#156), a titled dismissible AlertCard for run failures (#155), click-through proof
with a live cat-bounds hit-test (#154), and the tethered **SpeechBubble with the truthful memory receipt**
(#159) — the receipt renders ONLY on `run.completed.finalText` in the same run whose recall actually fired
(the truthful-receipt gate). `docs/INTERACTION.md` + smokes reconciled to the new reality (#160); the
executor-facts proposal IPC is exposed only under its flag (#153).
- **Earn the moat (#161, 2026-07-05).** The **codex `finalText` stitch** lights the truthful receipt on the
DEFAULT executor path (not just answers), plus honest copy and the **real-steering guard**:
`npm run verify:memory-steered-real` (opt-in, live qwen2.5:3b + real codex over N treatment/control runs)
first measured **5/5 args.task · 5/5 marker-in-file · 0/5 control** — recalled memory demonstrably steers
real coding work on the default path.
- **First founder dogfooding session (2026-07-05).** Four NAMED failure classes the green suite never met —
**deflect-on-vague** (a vague creative prompt drew a chat deflection, failing ROADMAP §6's
lands-or-clarifies), **feedback-fade** (answers/clarify fade in ≤8s with no reply affordance),
**working-illegibility** (working reads as idle; the cat can fall asleep mid-run), and
**env-overrides-config** (a stale `RORO_WORKDIR` silently outranks the configured project). Loci + status in
`LESSONS.md` (Dogfooding). Class 3's sleep-mid-run half + class 4 → the dogfood-blocker slice; class 2
(feedback-fade) and class 3's working-pose half are OPEN → the legible-moment slice; class 1 is
fixture-covered in the eval (`vague-initiative`) with the fix deferred.
- **Signal-reconcile slice (2026-07-06).** The eval instrument repaired — post-W7 both live runners crashed
ENOENT writing to stale `src/brain/eval` paths; the path family now lives in `eval/paths.ts` and is
CI-pinned (`paths.test.ts`). The deflect-on-vague class entered the DECIDE fixtures (5 `vague-initiative`
cases, lands-or-clarifies acceptance `{run_agent, clarify}`), and `baseline.json` was refreshed via
`--write-baseline` to the current runner shape (see §4).

---

## 4. The eval scorecard (`src/core/brain/eval/baseline.json`, qwen2.5:3b, temp 0 — updates only via `npm run eval:brain -- --write-baseline`; `latest.json` is the per-run scratch)
## 4. The eval scorecard (`src/core/brain/eval/baseline.json`, qwen2.5:3b; decide streams at temp 0.3, extract at temp 0 — updates only via `npm run eval:brain -- --write-baseline`; `latest.json` is the per-run scratch)
- **DECIDE 100%** (22/22) — the clarify trust gate now catches referent-less requests before the model.
- **EXTRACT (null-discipline) 100%** (10/10) — the gate holds; no profile poisoning.
- **BEHAVIORAL value-quality 40%** (2/5, the original fixture set) — a real 2× gain (#65) **and a measured model ceiling** (the 3B is genuinely weak at describing habits; noun prefs extract cleanly). The guard guarantees no *garbage* is stored — it just sometimes stores nothing.
- **W3 expansion (#145) — behavioral fixtures 5→50** (7-tag taxonomy, pairwise+prompt-4-gram disjointness CI-checked), a separate **null-discipline axis**, and the `eval:proposals` runner (live opt-in + a deterministic CI tier). Live **report-only** numbers (baseline untouched): value-quality **74.3%** on the expanded set (the original 5 reproduce 40% exactly), null-discipline **46.7%** (hard negatives are where the 3B invents facts from marker-bearing questions — the gate is the only null discipline). The panel also armed the scorer against **polarity inversion** (an `inverted` mode via `mustAlsoContainOneOf`/`mustNotContainAnyOf`) after it scored "never force push"→"force-pushes" as `ok` — see `LESSONS.md`.
- **Baseline refreshed (2026-07-06, `--write-baseline`)** — the checked-in reference now carries the current
runner shape and the full fixture set (the three bullets above are the historical pre-refresh snapshot):
**DECIDE 96.3%** (26/27 — now includes the 5 `vague-initiative` lands-or-clarifies cases, all 5 ok this
run; the one miss is `run-writetest`→answer, a temperature-0.3 snapshot artifact), **EXTRACT 100%**
(10/10), **BEHAVIORAL value-quality 74.3%** (26/35), **null-discipline 46.7%** (7/15; all 8 misses are
hard-negative `false_fact`s), plus the per-taxonomy breakdown (hard-negative 20% is the weak family;
multi-fact/supersede/marker-less 100%).

---

Expand Down Expand Up @@ -228,6 +261,15 @@ memory/embeddings). Add new expensive lessons there, not here.
2. **Produce the Developer-ID notarized build:** export `APPLE_TEAM_ID=<Apple Developer Team ID>`, `APPLE_ID`, and app-specific `APPLE_PASSWORD` in the same shell, then run `npm run verify:signing-readiness`, `npm run verify:signing-auth`, `npm run make`, `npm run verify:release-artifact:dmg`, and `npm run verify:release-artifact:signed`; validate install + memory recall on a clean second Mac.
3. **Validate Phase 2 trust on real first turns:** the Memory panel can see/fix/verify/source/forget facts, referent-less requests clarify before dispatch, the README leads with the job/privacy promise, and screen reads show a bounded one-snapshot tell.

> **Update (2026-07-06) — the corrected forward order** (absorbs the first dogfooding session, the floating
> redesign #153–#160, and earn-the-moat #161): **1) dogfood-blocker fixes** (sleep-mid-run /
> env-overrides-config / vision-timeout truthfulness — the sibling slice), **2) signing** (the Developer-ID
> notarized build + clean-Mac gate — item 2 above), **3) the legible-moment slice** (feedback-fade +
> the working pose + the receipt naming the recalled fact), **4) the
> executor-facts pilot** (toward a default-on decision), **5) self-rehearsal**, **6) demo**, **7) the
> stranger magic-moment session** — still the real lever; everything earlier exists to make it credible.
> Deflect-on-vague is measured (the `vague-initiative` eval class) with its fix deferred behind the blockers.

---

## 9. Founder decisions + open questions
Expand Down
39 changes: 39 additions & 0 deletions LESSONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,45 @@ current code — `src/renderer/bootstrap.ts` "cursor movement must NOT keep the
on every cursor sample, so the cat could never reach `asleep` (near-zero-idle violated). Gaze must
never wake the cat.

## Dogfooding

- **The first founder dogfooding session (2026-07-05) surfaced four failure classes in one sitting
that a ~1.1k-test green suite and six review rounds never met** — the suite measures what we
encoded, not what a user meets. Each is a NAMED class so fixes and fixtures can reference it:
- **deflect-on-vague** — "create something interesting" drew "Let's brainstorm a fun project idea
together" (`command=answer`): a chat deflection, failing ROADMAP §6's own lands-or-clarifies bar
(do the task or ask a *pertinent* question — `docs/ROADMAP.md` Arc A step 3). Locus: the DECIDE
contract has no vague-initiative guidance (`SYSTEM_PROMPT`, `src/core/brain/index.ts:97`) and the
deterministic clarify gate's regexes don't cover creative-vague (`src/core/brain/clarifyGate.ts:9`).
Status: **fixture-first now, fix later** — 5 `vague-initiative` DECIDE fixtures with a
lands-or-clarifies acceptance set (`src/core/brain/eval/fixtures.ts`) + CI pins (acceptance set
exactly `{run_agent, clarify}`; must not be gate-caught). **Measurement twist (2026-07-06
baseline refresh):** the current 3B lands-or-clarifies on all 5 paraphrased fixtures — and 30/30
in a 3×-repeat probe, with and without a memory section — so the live deflection has NOT
reproduced from paraphrase; the fixtures pin the class against regression while the real repro
needs a `RORO_TRACE` capture of an actual deflecting turn promoted into a fixture. *Corollary
law:* **a live failure paraphrased into a fixture may not reproduce — capture the real turn
(trace) before trusting a hand-written golden case to carry it.**
- **feedback-fade** — answers and clarify questions auto-fade in ≤8s with no reply affordance
(`fadeDwellMs` = clamp(words×220ms, 2.5s, 8s) — `src/renderer/reactions/speechBubble.ts:44`); a
clarify question that fades unanswered is a dead end. Status: OPEN — deliberately NOT in the
dogfood-blocker slice; it belongs to the legible-moment slice (persistent answer/clarify bubble
with an inline reply), the next scheduled build item.
- **working-illegibility** — a running executor is barely distinguishable from idle, and the cat
can fall ASLEEP mid-run: energy is computed purely from interaction idleness even while busy
(`src/renderer/character/avatar.ts:632`; 45s/120s thresholds — `src/renderer/character/activity.ts:13`;
no run-lifecycle event pokes the clock — only direct interaction does (petting,
`src/renderer/bootstrap.ts:241`; summoning the ask, `src/renderer/ask/askMachine.ts:43`)). Status:
the sleep-mid-run half is fixed in the sibling dogfood-blocker slice (setBusy wired into floating
runs); a distinct WORKING pose remains open for the legible-moment slice.
- **env-overrides-config** — a stale `RORO_WORKDIR` in the launching shell silently outranks the
user's persisted project choice (`tryResolveWorkdir` checks env first, no tell —
`src/core/orchestrator/workdir.ts:29`). Status: being fixed in the sibling dogfood-blocker slice.

*Generalizable law:* **every dogfooding finding becomes a NAMED failure class with a file:line
locus and either a fix or a fixture before the session's signal decays — a green suite is evidence
of encoded expectations, not of user-meeting behavior.**

## Memory / embeddings

- **Every cheap substrate stays cheap after the corpus exists — except the embedding vector space,
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,22 +80,22 @@ implementation can be mounted later behind the same interface.
flowchart LR
user["User task<br/>text by default"] --> renderer["Renderer<br/>cat, captions, controls"]
renderer --> main["Electron main<br/>typed IPC"]
main --> memoryRecall["PGlite recall<br/>pgvector (local)"]
main --> memoryRecall["memory2 recall<br/>local index over encrypted files"]
memoryRecall --> brain["Local Ollama brain<br/>qwen2.5 decision"]
brain --> executor["Executor adapter<br/>Codex or Claude"]
executor --> events["Canonical action events"]
events --> renderer
events --> memoryWrite["PGlite remember (local)"]
events --> memoryWrite["memory2 remember (local)"]
brain --> vision["Local vision<br/>optional screen read"]
```

| Subsystem | Role | Source |
| --- | --- | --- |
| Electron shell | windowing, IPC, macOS permission checks, floating mode | [`src/main/`](src/main/) |
| Character | pixel cat, state machine, lip sync facade | [`src/renderer/character/`](src/renderer/character/) |
| Brain | local Ollama reasoning/vision/embeddings | [`src/brain/`](src/brain/) |
| Memory | encrypted files-as-truth + PGlite-HNSW hybrid recall (local, owner-scoped) | [`src/memory2/`](src/memory2/) |
| Executor | Codex and Claude stream adapters | [`src/executor/`](src/executor/) |
| Brain | local Ollama reasoning/vision/embeddings | [`src/core/brain/`](src/core/brain/) |
| Memory | encrypted files-as-truth + in-memory vector index (local, owner-scoped) | [`src/core/memory2/`](src/core/memory2/) |
| Executor | Codex and Claude stream adapters | [`src/core/executor/`](src/core/executor/) |
| Voice | internal on-device VAD + STT + TTS seam (Silero / whisper / Kokoro), hidden behind dev flags and cut from v0 | [`src/renderer/voice/`](src/renderer/voice/) |
| Shared contracts | typed IPC, action events, avatar states | [`src/shared/`](src/shared/) |

Expand Down Expand Up @@ -155,7 +155,7 @@ full prompt, controls, captions, memory panel, and action timeline visible.
## Configuration

Roro's default brain and memory paths need **no app-owned cloud/model keys**:
Ollama runs locally, memory is local PGlite + pgvector, and packaged builds
Ollama runs locally, memory is a local encrypted files-as-truth store, and packaged builds
store the chosen working repo in `userData/config.json`. A local `.env` (see
[`.env.example`](.env.example)) is for development overrides, model tuning, and
optional cloud/executor paths:
Expand All @@ -174,7 +174,7 @@ RORO_WORKDIR=/absolute/path/to/scratch-git-repo
ANTHROPIC_API_KEY=... # optional, only for the Claude executor
```

Memory is local PGlite + pgvector under the app's userData dir (`RORO_DB_DIR` to
Memory is a local encrypted files-as-truth store under the app's userData dir (`RORO_DB_DIR` to
override) — no external database. See [`RUN.md`](RUN.md) for the on-device voice
dev flags, macOS permissions, and the full live-run checklist.

Expand Down Expand Up @@ -232,7 +232,7 @@ What is working:
- DMG release artifact: macOS CI builds a versioned `.dmg` and verifies it mounts with `Roro.app`
- **local Ollama brain** (decide/vision/embeddings) — verified end-to-end against a
live daemon
- **local PGlite + pgvector memory** (owner-scoped, survives restarts)
- **local encrypted files-as-truth memory** (owner-scoped, survives restarts)
- Codex and Claude executor adapters behind one event stream
- typed text path + the on-device voice control core/seam hidden behind dev flags

Expand All @@ -247,9 +247,9 @@ What needs extra setup or a real device:
```text
src/main/ Electron main process and orchestration
src/renderer/ UI, character, voice, captions, event wiring
src/brain/ local Ollama decision, vision, embeddings
src/memory2/ local encrypted files-as-truth + PGlite-HNSW memory
src/executor/ Codex and Claude adapters
src/core/brain/ local Ollama decision, vision, embeddings
src/core/memory2/ local encrypted files-as-truth + in-memory vector index
src/core/executor/ Codex and Claude adapters
src/shared/ IPC, event, memory, avatar, env contracts
RUN.md live setup and integration guide
```
24 changes: 14 additions & 10 deletions RUN.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Roro — Run & Integration Guide

Roro is **local-first**: the brain runs on a local Ollama daemon and memory is an
in-process PGlite + pgvector store. The default brain/memory path needs **no app
in-process encrypted files-as-truth store with a local vector index (no database). The default brain/memory path needs **no app
API keys** and makes no cloud model calls. The steps below bring the full app alive on your Mac.

## 1. Local brain — install Ollama + pull core models
Expand All @@ -17,7 +17,7 @@ ollama pull qwen2.5vl:7b # vision (describeScreen) — needs substantial RA
On boot, `main` runs a non-blocking brain preflight: success logs the active model;
failure (daemon down / model missing) still renders the window and surfaces a
diagnostic — it never silently falls back to the cloud. Verify end-to-end with the
opt-in smoke: `OLLAMA_AVAILABLE=1 npx vitest run src/brain/integration.test.ts`
opt-in smoke: `OLLAMA_AVAILABLE=1 npx vitest run src/core/brain/integration.test.ts`
(see [`docs/WS1-OLLAMA-INTEGRATION-TEST.md`](docs/WS1-OLLAMA-INTEGRATION-TEST.md)).

## 2. Optional `.env` — model tuning
Expand All @@ -39,16 +39,20 @@ ANTHROPIC_API_KEY=... # only if you use the Claude exec
```

> Switching the embed model changes the vector geometry. `OLLAMA_EMBED_DIM` must match
> the model's output (preflight probes it and fails loud on a mismatch), and the memory
> store's `vector(N)` column is fixed at creation — re-embedding is not automatic, so
> move/delete the memory dir when changing embedders.
> the model's output (preflight probes it and fails loud on a mismatch). Memory itself is
> files-as-truth; the `vectors.jsonl` sidecar is a derived, zero-authority cache with an
> (embedModel, dim) identity header — on an embedder change the store FAILS LOUD (identity
> refusal), and the remedy is deleting the derived `index/` subdirectory only (safe: the next
> launch re-embeds from the durable files). Never delete the memory dir itself — it holds the
> durable encrypted memories, not just an index.

## 3. Memory — local PGlite + pgvector (no setup)
## 3. Memory — local encrypted store (no setup)

Memory is an in-process PGlite database with pgvector, stored under the app's userData
Memory is encrypted files-as-truth (JSON + a rebuilt-on-launch in-memory vector index;
embeddings via local nomic-embed-text), stored under the app's userData
dir (override with `RORO_DB_DIR`). It is owner-scoped and survives restarts — **no
external database, no SQL provisioning, no keys**. A taught fact in one launch is
recalled in the next (proven by `src/main/memorySpine.crosslaunch.test.ts`).
recalled in the next (proven by `src/core/orchestrator/memorySpine.crosslaunch.test.ts`).

## 4. The avatar — a hand-built pixel-art cat (default; no assets needed)

Expand Down Expand Up @@ -106,8 +110,8 @@ npm start

Summon (Cmd+Shift+Space) → type a task in the floating Ask (or the dev prompt box) → the
cat thinks (driven by the local brain's content stream) → drives Codex in `RORO_WORKDIR`
(each action narrated + animated) → writes to local PGlite memory. Then ask
*"what did we do?"* for the pgvector recall beat.
(each action narrated + animated) → writes to local encrypted memory. Then ask
*"what did we do?"* for the semantic-recall beat.

Roro's brain and memory setup does not need an app-owned cloud key, but the coding executor is
your own CLI. Before expecting a coding turn to edit files, authenticate or configure Codex (or
Expand Down
2 changes: 1 addition & 1 deletion docs/COHORT_TRACE_TO_EVAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ in the local review packet until a future cohort repeats the pattern.
After adding redacted fixtures:

```sh
npx vitest run --no-file-parallelism src/brain/eval/fixtures.test.ts src/brain/eval/cohortTraceReview.test.ts
npx vitest run --no-file-parallelism src/core/brain/eval/fixtures.test.ts src/core/brain/eval/cohortTraceReview.test.ts
npm run eval:brain
```

Expand Down
Loading
Loading