Skip to content

Capture observed demonstrations and replay validated spans - #59

Draft
Jepson2k wants to merge 24 commits into
feat/execution-speed-overridefrom
feat/demonstration-recording
Draft

Jepson2k wants to merge 24 commits into
feat/execution-speed-overridefrom
feat/demonstration-recording

Conversation

@Jepson2k

@Jepson2k Jepson2k commented Sep 8, 2026 •

Copy link
Copy Markdown
Owner

Capture controller-reported joint/tool observations in a Demonstrations panel or standalone Python. Inspect source cadence and gaps, select an uninterrupted span, save/export original timestamps and insert an ordinary Python replay call.

Replay verifies the start, tool/TCP, source and controller session, then uses existing native joint moves and delays. It stops at every recorded waypoint and can be much slower than the demonstration. Gripper position replay is opt-in and sequential; recorded grasp flags are never treated as fresh observations. Loss of status, reference, session or tool configuration ends replay, including during a managed debug pause.

The TCP is captured at acquisition start; keep its settings fixed while recording. Existing semantic command recording retains its separate workflow.

Depends on #58 and the coordinated waldoctl, PAROL6-python-API and par6 feat/demonstration-recording PRs.

Validation: fake-serial capture/export/import, gap and wrong-start/session refusal, native planning preview and replay, gripper replay through managed pause and fault, real Chromium chart rendering, native PAR6 capture/export/import/replay, and formatting/type checks pass. Capture deadline classification has a regression test for early timer wakeups.

Companion PRs: waldoctl #33, PAROL6 #46, PAR6 #76, Waldo Commander #59.

Managed completion waits now check controller faults throughout a paused wait. The calibration reload regression closes the prior simulated page before opening another one and verifies session ownership. Seven combined pause, recording, and calibration workflows pass locally; a separate control-ownership reload check also passes. Lint, type, and formatting checks pass.

Updated CI is pending on the current revision.

Demonstration charts show readable axes and time ranges, offer an expanded view, and use From/To seconds while preserving original samples and gap validation.

Screenshots — revised interface, PAROL6 and PAR6

Captured in Chromium on September 10 from the combined feature stack; shared panels may include later PRs. PAROL6 uses fake serial; PAR6 uses par6d --sim.

PAROL6 — demonstration chart populated by recorded simulator motion.

PAROL6 — demonstration chart populated by recorded simulator motion.

PAR6 — demonstration chart populated by recorded simulator motion.

PAR6 — demonstration chart populated by recorded simulator motion.

Jepson2k and others added 21 commits September 8, 2026 06:06
# Conflicts:
#	waldo_commander/components/skill_library.py
…wn tool

replay_demonstration dereferenced rbt.tool before replaying, but a client
only learns its tool from its own select_tool(), so the documented
standalone replay (a fresh client on a controller already carrying the
recorded tool) died with the client's RuntimeError. The tool identity is
now read from the status observation the replay already requires; only
gripper replay needs the client-side tool and says so.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A demonstration could only be replayed: a dense sample-by-sample playback
that stops at every observation and stays a data file. Most recordings are
a handful of moves with pauses between them, and those are what a program
should be.

The arm holding still is the structure: a still span of at least 0.3 s ends
a move, becomes an rbt.delay, and carries any gripper position that changed
in it; the motion between two of them becomes one move_l where the tool
travelled in a straight line, or the recorded joint waypoints that hold its
path, blended so the arm does not stop at each. Each move keeps its
recorded duration. A hold at either end of the capture becomes a comment:
it is when the operator started and stopped recording.

Nothing is written on the strength of the geometry alone. Every span is
planned in the backend's preview and compared with the recording -- tool
position, tool orientation, and every joint, because a Cartesian move can
trace the recorded line through a flipped wrist and sweep the cell
differently. A straight-line candidate that fails falls back to joint
waypoints, which cannot flip; a span that fails both keeps its observations
as a replay_demonstration call over that sample range, so a program can be
part moves and part replay.

SKIP=ty: pre-existing DryRunRobotClient.skill_capabilities diagnostic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A capture whose stream stopped publishing inside the final stale window was
classified by whichever limit cut the wait short, so it ended as a clean
"duration limit" -- the file and the panel claimed a complete recording whose
last second, tens of missed publications, was never observed. How long the
wire has been quiet decides it now, against the recording's own threshold for
a gap needing reconciliation.

Selecting "Replay demonstration" in the Skills panel returned out of the form
build before the snippet and message were written, leaving the previously
selected skill's call on screen.

Export went through its own copy of the serialisation, so a schema bump would
have left exports on the old one, and it dropped the size bound the loader
enforces: a 100k-sample recording exported as a file nothing can reload. Both
go through one encoder.

SKIP=ty: pre-existing DryRunRobotClient.skill_capabilities diagnostic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
isinstance against the DryRunClient protocol resolves members statically
and never matches the preview wrapper or the skill guard; is_dry_run asks
the way a call would.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N8zt66KjkzgZLtQuSfJd9r

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants