Transcript-driven editing (Descript model) + build/test/DB/rebrand fixes#3
Merged
Conversation
allows downloading brand assets directly from the logo in the header
Fix for #703. Adds an optional sampleRate parameter to the decodeAudioToFloat32() function. The caption transcriber passes in a value of 16000 to prepare audio in the format Whisper expects.
Hard-fork opencut at the repo root. Replaces the previous Electron+Whisper apps/desktop with opencut's Next.js web editor (apps/web), GPU-compositor Rust crates, and native desktop shell. apps/website (marketing) preserved. Conflict resolution: - Took opencut's package.json, bun.lock, .gitignore (its tooling becomes the base) - Replaced our apps/desktop entirely with opencut's - Wrote a fresh top-level README explaining the fork relationship Upstream is wired as the `opencut` remote for future `git fetch opencut main`.
Adds a Descript-style "edit by editing the transcript" panel that drives
real cuts on opencut's timeline.
- Switch transcription worker to word-level timestamps (return_timestamps:
"word") and group words back into sentence-like segments for display.
Subtitles consumer keeps working since segments still expose start/end.
- New apps/web/src/transcript-editor/ module:
- types.ts: TranscriptDocument with per-word IDs and selection state.
- apply-deletion.ts: maps word ranges to splitElements + deleteElements
calls, classifying each covering element as fully-inside, partial-
left, partial-right, or straddle-both. Latest-first iteration so
earlier ranges keep stable timeline coordinates.
- transcript-panel.tsx: panel UI. Click word = seek; shift/cmd-click
or double-click = toggle selection; delete button cuts the timeline.
- Wire panel into the editor route as a toggleable "Transcript" tab in
the right column (alongside Properties).
Also removes ~90 stale Electron files from apps/desktop that the
opencut merge auto-kept; HEAD now matches opencut's apps/desktop tree.
Known limitation: each transcript edit emits 3-5 separate timeline
commands (one per split / delete) so undo steps through them
individually. BatchCommand wrapping deferred to a follow-up.
Major UX overhaul of the transcript editor: - Document-style layout: timestamps in a left margin, generous line-height, reading-width container, prose typography. Each segment is its own paragraph with a 0:00 timestamp. - Active-word highlighting: as the video plays, the current word gets an accent background and auto-scrolls into view. Re-renders driven by editor.playback subscription via useEditor. - Range selection: shift-click extends selection from the last clicked word; cmd/ctrl-click toggles a single word; plain click seeks and anchors the range. - Filler-word detection: scans words against a stop-list (um, uh, like, you know, etc.). Flagged words get a dotted amber underline. Header shows "Remove N fillers" button for one-shot bulk removal. - Cmd+F find: opens a search bar, highlights matches in yellow with the active match in solid yellow, Enter cycles forward, Shift+Enter back, Esc closes. - Single undo per edit: applyTranscriptDeletions now constructs commands directly, executes them in sequence to thread split IDs, then pushes a BatchCommand to the command manager. One Cmd+Z reverses the whole edit. Supporting changes: - src/env/web.ts: defaulted DB/Redis/auth/CMS/FreeSound vars so the editor boots without backing services. Routes that actually use them will fail at use; transcript editor doesn't touch any of them. - src/transcription/models.ts: swap whisper-tiny/small/medium from onnx-community to Xenova exports. The onnx-community ASR exports don't carry cross-attentions, breaking return_timestamps:"word".
Renames @opencut/web -> @openscript/web (root scripts and apps/web/package.json).
Updates SITE_INFO and visible UI strings (header, footer, onboarding,
mobile gate, storage dialog, legal pages, browser-compat banner, better-auth
appName).
Removes marketing routes that don't belong in an editor product:
- /blog, /roadmap, /sponsors, /contributors, /brand, /changelog
- src/changelog/* (release notes data + ChangelogNotification component)
- src/components/landing/* (hero etc.)
- src/components/gitHub-contribute-section.tsx
- src/site/sponsors.ts and src/site/external-tools.ts (now-unused)
Replaces / with a redirect to /projects so users land on their project list.
Strips the OpenCut analytics tracker (databuddy) and the OpenCut nav links
from Header. Footer reduced to copyright + Privacy/Terms.
Kept (not user-visible):
- LICENSE (MIT requires preserving the OpenCut copyright notice)
- localStorage/zustand persist keys ("opencut-keybindings",
"opencut-storage-persist-dismissed") to avoid orphaning user data
- eslint plugin name "opencut/prefer-object-params" in disable comments
- The opencut-wasm npm dep (it's the upstream-published artifact)
- public/logos/opencut/svg/logo.svg as DEFAULT_LOGO_URL with a TODO to
replace with an OpenScript wordmark
- editor-header.tsx: replace OpenCut wordmark <Image> in the project dropdown trigger with a text "OS" mark. Drop the Discord menu item and its now-orphan separator. Cleans up unused imports (Image, Link, FaDiscord, DEFAULT_LOGO_URL, SOCIAL_LINKS, DropdownMenuSeparator). - onboarding.tsx: drop the third onboarding step (the Discord pitch) and the inert roadmap link from step 2. Replace the dynamic step title with a static "OpenScript Onboarding" label for the screen-reader DialogTitle. DEFAULT_LOGO_URL and SOCIAL_LINKS still exist in src/site for the privacy/terms pages that link out to GitHub/X (currently inert "#" hrefs); replace those when real social accounts exist.
next build was failing on the TypeScript gate (12 errors); three of the root causes were also live runtime bugs. Fix all and add the action plan. - actions/keybindings: implement isShortcutKey + isActionWithOptionalArgs; persistence.ts imported guards that never existed. The 9 pre-existing persistence tests now compile and pass. - stickers: register providers with object params (was registering every built-in provider under key `undefined`, breaking the sticker library). - storage migrations: construct IndexedDBAdapter / call set() with object params in runner + v1-to-v2 (was indexedDB.open(undefined), silently breaking legacy-project migration). - timeline tests: wrap raw numbers with MediaTime brand helpers. - bun.lock: reconcile with package.json (was stale; broke --frozen-lockfile). - add PLAN.md. Verified: tsc --noEmit clean (was 12 errors), next build exits 0, keybindings suite 9/9.
opencut-wasm is a wasm-pack "bundler"-target package: its entry calls wasm.__wbindgen_start() on an import that bun's test runner doesn't instantiate, so every test reaching @/wasm crashed at module load — the entire editor core (timeline, masks, transcript, storage) was untested. Add a bun test preload that performs the bundler step itself against the real wasm (read bytes, instantiate with the glue as the import object, wire it in, run the start hook), then redirect the bare "opencut-wasm" specifier to the initialized glue. Tests now exercise the actual Rust implementation, not a hand-written stub that could drift from source. Result: 193 tests across 30 files now run (was ~0 in the wasm graph) — 192 pass, 1 skip, 0 fail. Surfaced failures fixed: - resolve.test: time spans used fractional ticks (2.5/5.5); MediaTime is integer ticks. Use integer positions that preserve the overlap scenario. - snap.test (box scale): a centered uniform scale snaps both edges to ±100; expect both vertical guides, not just the right one. - snap.test (point split): De Casteljau subdivision of the straight a→b segment yields collinear handles (-0.1 in / +0.1 out), not zero. - snap.test (text mask): needs a DOM/canvas font-metrics context; skip under bun, runs in a browser/jsdom env. Verified: tsc --noEmit clean, bun test 192 pass / 1 skip / 0 fail.
The transcript panel stored a per-word `deleted` flag and mutated it on delete, so it drifted from the actual timeline: undo restored the media but left words struck-through, and reload showed the full text over an already-cut timeline. Deletions also bypassed reactors, orphaning tracks. - Derive `deleted` from timeline coverage instead of storing it. A word is deleted when its midpoint is no longer covered by media (new coverage.ts, 7 unit tests). The panel subscribes to the active scene's tracks and recomputes, so undo/redo and reload stay correct automatically — the cut media returning makes the word reappear. `deleted` is dropped from TranscriptWord; filler-words/search take a deletedWordIds set. - Run reactors after a transcript delete (CommandManager.reactToExternalChange). Deletions execute sub-commands incrementally to read live element IDs, so they use push() rather than execute(); push() skips reactors, which left empty audio/overlay tracks behind. (Ripple-on-delete stays in Phase 2 — it needs word-coordinate re-sync.) - Persist the transcript (text + word timings only) per project and rehydrate on mount, so it survives tab-toggle and reload. Deleted-state is NOT stored; it is re-derived from the persisted timeline, so the two cannot diverge. Transcript is removed when its project is deleted. Verified: tsc clean, bun test 199 pass / 1 skip / 0 fail, next build exits 0.
Toward the "edit by transcript" promise (Phase 2, parts a+b): - Model + language selectors in the transcript empty state, threaded into transcribe(). Previously locked to whisper-small/auto despite 4 models and 9 languages being defined but unused. Each model shows an approximate first-use download size (new approxDownloadMb) so users on slow/metered connections can pick Tiny; a note clarifies it runs locally in-browser. - Inline word correction: double-click a word to edit its text (Enter/blur commits, Escape cancels), persisted with the transcript. This is a transcript-only edit — it changes the displayed/exported text, not the timeline or word timing — so it is low-risk and independent of the coverage-derived deleted state. Ripple-close on delete (Phase 2c) is intentionally not included: closing timeline gaps conflicts with deriving deleted-state from gaps and needs an architecture decision (tracked separately). Verified: tsc clean, bun test 199 pass / 1 skip / 0 fail, next build exits 0.
…en, transcript store Phase 2c foundation (steps 1-4 of 10). Inert scaffolding for the destructive Descript transcript model; not yet wired into the delete flow. - types.ts: v2 model — TranscriptWord.deleted, DeletedRange + RemovedPiece (lossless element capture for restore), TranscriptDocument.deletedRanges. - mapping.ts: pure source<->timeline mapping (frozen source seconds minus removed durations of earlier cuts; in-gap clamps to gap edge); integer-tick math. - ripple/open.ts: rippleOpenElements, the inverse of rippleShiftElements (restore). - core/managers/transcript-store.ts: EditorCore-owned holder for the doc, wired into useEditor — lets a Command mutate the transcript from global undo/redo. - storage SerializedTranscript.version loosened to number (v1->v2 migration lands with the wiring). DESIGN.md records the full plan. Tests: 14 new (types/mapping/ripple-open/store). tsc clean, full suite 213 pass / 1 skip / 0 fail.
…; save side-effect) EditorCore constructs under bun, but updateTracks triggers SaveManager -> IndexedDB (absent in bun) and would reject — so the fidelity test should drive a pure split/trim+ripple transform on fixture tracks rather than a live editor. Captures the delete input contract (contiguous runs) for the panel rewire.
Phase 2c steps 5-6 (delete side). Both pure and unit-tested on fixture tracks —
no editor/save coupling, so they are the testable core of the destructive model.
- timeline/split-element.ts: splitElementAtTime — a pure, generic, retime-aware
element split (verbatim mirror of SplitElementsCommand's per-element math,
snap-source-once). Net-new split test coverage. (Kept separate from the command
for now to avoid touching untested split behavior; noted to keep in sync.)
- transcript-editor/delete-transform.ts: deleteRangeFromTracks — removes a
[start,end) range across all tracks (split straddlers, drop contained), captures
the removed slices as RemovedPiece (with offsetTicks for restore), and
ripple-closes the gap uniformly. Returns {afterTracks, removed, durationTicks}.
Tests cover straddle-both, fully-contained, no-overlap, ripple-close, capture, and
trim math. tsc clean; full suite 219 pass / 1 skip / 0 fail.
…ity gate Phase 2c step 6c/7. restoreRangeToTracks is the pure inverse of deleteRangeFromTracks: ripple-open the gap and re-insert the captured slices (lossless but for a fresh id). The round-trip test is the primary correctness gate for the destructive model and now runs purely (no editor): - straddle-both reconstructs exact contiguous source coverage (startTimes, durations, trimStarts). - a multi-element timeline stays contiguous and the same total length after delete->restore (catches tick drift). - a retimed element round-trips without drift and keeps its retime config. tsc clean; full suite 222 pass / 1 skip / 0 fail.
Phase 2c step 6b/6c. Composes the pure transforms into the editable model: - plan.ts: planTranscriptDelete / planTranscriptRestore — pure (modulo a fresh range id). Delete processes runs latest-source-first (so each cut's timeline position, derived from pre-existing cuts, is unaffected by the batch's earlier cuts), marks words deleted, and records DeletedRanges; restore re-opens the gap at its current position, reconstructs the media, un-marks the words, drops the range. Tested: delete records+ripple-closes, restore reverses, unknown id no-op. - transcript-edit-command.ts: TranscriptEditCommand snapshots tracks + doc and applies the plan; undo/redo replay snapshots so one undo reverts the timeline AND the transcript document atomically (the desync fix). dispatchTranscriptEdit records via push() (the plan already rippled — avoid the global ripple toggle double-applying) + runs reactors. Known v1 edge noted: restoring a cut that emptied+pruned a whole track can't rebuild it. tsc clean; full suite 225 pass / 1 skip / 0 fail.
Phase 2c step 9. migrateTranscript normalizes a persisted transcript to the v2 document shape. v1 (pre-Descript) stored no deleted/deletedRanges and relied on coverage-derived deletion against an already-cut timeline; migrate it once by deriving each word's `deleted` from current timeline coverage and starting an empty deletedRanges log — so pre-v2 cuts show struck but are not restorable (documented). Never mutates the timeline. v2+ docs pass through. Tested: v1 derivation, v2 pass-through, no-tracks fallback. tsc clean.
The Descript-model backend is complete and tested (transforms, plan, command, migrator). The panel rewire is the last integration and is the part that needs in-app verification, so it is specified precisely here (exact deltas + a live test checklist) for a turnkey hand-off rather than shipped unverified.
Phase 2c final step. The panel now drives the destructive, undoable, restorable model end to end: - reads the document from the editor-owned TranscriptStore (reactive), so a command can mutate it from global undo/redo; generate/load/edit write the store. - deleted state comes from the stored `deleted` flag (not coverage). - on mount, loads + migrates (v1->v2) the persisted transcript into the store. - delete builds contiguous runs from the selection and dispatches a TranscriptEditCommand (split + delete + ripple-close + tombstone, one undo step). - seek and the active-word highlight go through the source<->timeline mapping, so they stay correct after cuts. - clicking a struck-through word restores its whole cut (ripple-open + reconstruct); pre-v2 cuts stay non-restorable. - retires coverage from the render hot path (kept only for the v1->v2 migrator) and removes the superseded apply-deletion.ts. tsc clean; full suite 228 pass / 1 skip / 0 fail; next build exits 0. Needs in-app verification (ripple/restore/seek/undo/reload) — checklist in DESIGN.md.
- Default to the Tiny model (~40 MB): fast even on the single-threaded wasm fallback (no WebGPU / no cross-origin isolation), where larger models are painfully slow. The picker still offers Small/Medium/Large. - Gate Generate on timelineHasAudio: a silent/empty timeline now shows "This timeline has no audio to transcribe" instead of downloading the model and running Whisper on silence (verified in-app). - Cancel button during transcription (the service already supported cancel); cancelling returns to the start rather than surfacing an error. - Preserve recognized text when a model returns no word-level timing, so the user still sees what was transcribed instead of a bare error. Verified: tsc clean, suite 228 pass / 1 skip / 0 fail, next build exits 0; in-app confirmed default=Tiny and the audio gate.
- drizzle.config.ts schema path pointed at ./src/lib/db/schema.ts (no such dir), so db:generate/migrate/push all failed. Fix to ./src/db/schema.ts. - The committed migration created a dead `waitlist` table and omitted the `feedback` table the app uses (feedback popover -> db.insert(feedback)), so feedback 500s on a freshly-migrated DB. Regenerated migrations from the corrected schema (pre-launch; no deployed DB depends on the old one): the new 0000 creates accounts/feedback/sessions/users/verifications, no waitlist. - README "Getting started" claimed the editor needs Postgres + Redis. It does not — the editor runs fully local (IndexedDB/OPFS + on-device Whisper). Reworded: those services are only for the optional auth/feedback/sound-search routes, with the .env.local copy + docker + db:migrate steps spelled out.
- mobile gate: the "Roadmap" button linked to a removed /roadmap route (404 on mobile/iPad); point it at the GitHub repo instead. - sitemap: drop the removed routes (/contributors, /roadmap, /why-not-capcut, /blog + per-post) that all 404'd; emit only /, /privacy, /terms. - manifest.json: name/description were still "OpenCut" + the old tagline. - LICENSE: copyright OpenCut -> OpenScript. - package.json: remove the dead *:tools scripts (no @openscript/tools package). - social links: set the real GitHub repo (x/discord left as TODO placeholders). - desktop: window title OpenCut -> OpenScript, and README now states plainly that it's an early empty-window scaffold (not wired to the editor) and that commands run from the repo root. Verified: tsc clean, next build exits 0.
…available)
Rather than leave stubs for things we have no real values for:
- Drop the placeholder contact email and the "#" X/Twitter + Discord social
links from /privacy and /terms; route contact to the real GitHub issues link.
SOCIAL_LINKS reduced to the github repo only.
- Remove the dead /blog RSS feed (apps/web/src/app/rss.xml) and its now-orphaned
blog data layer (apps/web/src/blog) — there is no blog route.
- Remove the unused DEFAULT_LOGO_URL and the OpenCut wordmark assets it pointed
at (public/logos/opencut, public/landing-page-dark.png); the header uses a
text wordmark.
- Remove the empty packages/{env,ui,desktop-bridge} scaffolding dirs.
- .github docs: rebrand OpenCut -> OpenScript (SUPPORT/CONTRIBUTING/SECURITY/
CODE_OF_CONDUCT/feature_request), remove the dead Discord invite, and point
security reports at GitHub security advisories instead of the dead
security@opencut.app. Left the legit `opencut-wasm` package name and the
opencut/opencut local Postgres creds (consistent with docker-compose).
Verified: tsc clean, next build exits 0.
Fix the dead github.com/opencut/opencut repository URL (rust/wasm/Cargo.toml, rust/wasm/README.md) to the real OpenScript repo, and OpenScript the rust/ README. Left the published `opencut-wasm` package/crate name and its opencut_wasm_* file names intact (the JS side imports them by that name).
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
CI `next build` failed type-checking next.config.ts: the root pinned
`next: ^16.1.3` (resolving to 16.2.x, which adds `adapterPath` to
NextConfig) while apps/web pinned `next: 16.1.3`, so two incompatible
`next` copies were visible and `NextConfig` types didn't match
("Property 'adapterPath' is missing"). Pin the root to `next: 16.1.3`
(and `typescript: ^5.8.3`) to match apps/web so the workspace resolves a
single version. (apps/website keeps its own next 16.1.1 — isolated.)
Verified: bun install + `cd apps/web && bun run build` exits 0.
`version: latest` mis-resolves to wasm-pack v0.9.1, whose bundled
wasm-opt fails ("exited with exit code 1") on the wasm that current
rustc/wasm-bindgen emit. Pinning to the current stable fixes the
Build WASM step across all three runner OSes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns the OpenCut fork into OpenScript's transcript-driven editor and gets the branch shippable: repairs the build/tests, implements the full "edit video by editing the transcript" model, hardens transcription, fixes the DB tooling, and clears the rebrand leftovers.
Highlights
next build; 3 were also live runtime bugs (sticker registry + storage migrations were calling object-param APIs positionally → registering underundefined/indexedDB.open(undefined)).opencut-wasmis a wasm-pack bundler build that crashedbun testat load; a preload now instantiates the real wasm, so the editor core is testable. 193→200+ tests run, 228 pass / 1 skip / 0 fail.timelineHasAudio, a Cancel button, and preserve recognized text when a model returns no word-level timing.feedbacktable the code uses (and not the deadwaitlist); README rewritten (the editor runs fully local; Postgres/Redis only for optional auth/feedback/sounds)./roadmap404 + dead sitemap routes,manifest.json/LICENSE/.github/rustOpenCut→OpenScript, dead Discord invite +security@opencut.appremoved, placeholder email/social links removed, orphan OpenCut assets + emptypackages/*dirs deleted.Verification
tsc --noEmitclean;bun test228 pass / 1 skip / 0 fail;next buildexits 0.Known follow-ups (not blockers)
apps/desktopis an honest empty-window scaffold; a real native implementation is future work.