Skip to content
BranchesTags

Repository files navigation

remotion-video-factory

Batch pipeline that turns script JSON → 9:16 vertical short videos with Remotion.

Given a structured script (scenes with narration + visual cues), the factory:

  1. Synthesises voiceover with a pluggable TTS engine (Qwen / Volcano / Edge-TTS / none).
  2. Generates subtitle timing aligned to the audio (so emphasis + motion cues land on the spoken words).
  3. Renders through a shared vertical-card template, optionally overlaying a text-emphasis layer (highlights, zoom, blur-focus) or a motion-sfx layer (full-width animated banner + sound effects + deterministic BGM).
  4. Verifies every output with ffprobe (never trusts the exit code alone).

The repo ships a zero-credential demo (demo-vertical) that renders end-to-end with no API key, no audio, and no assets — so the whole pipeline is runnable and testable out of the box.

Architecture

script JSON ──▶ tools/script-to-video ──▶ src/compositions/<id>/ (constants.ts + VideoComposition.tsx)
                                                       │
        tools/tts-* ──▶ public/assets/audio/<id>/*.mp3 │  (voiceover, optional)
        tools/timing-gen ──▶ src/generated/timing-<id>.ts (subtitle map, optional)
                                                       │
                       batch/jobs.ts (single source of truth) ◀────────┘
                          │                         │
                  src/Root.tsx (registers compositions)   batch/run.mjs (renders)
                          │                         │
                   remotion render ◀─────────────────┘
                          │
                  out/batch/*.mp4  (ffprobe-verified)

Key ideas:

  • One source of truth. batch/jobs.ts is the job list. src/Root.tsx registers Remotion compositions from it, and batch/run.mjs renders from it. Add a job once; both sides pick it up.
  • Content pack = data + glue. Each video is a directory under src/compositions/<id>/ containing constants.ts (all data: scenes, narration, palette, brand) and a ~20-line VideoComposition.tsx that wires the data into the vertical-card template. No video logic lives in the pack.
  • Pure job modules. batch/jobs.ts and batch/bases.ts must never import react/remotion — batch/run.mjs runs them in plain Node. The repo's doctor check fails the build if they do.
  • Deterministic resources. pickBgm() (FNV-1a hash → track) and palettes.ts (hash → colour set) keep output stable across runs without storing per-video config.
  • Portable env. Every path/threshold is overridable via env vars (FFMPEG_BIN, OUT_DIR, RENDER_CACHE_DIR, REMOTION_CONCURRENCY, …). No machine-specific paths are hardcoded.

Directory layout

remotion-video-factory/
├─ src/
│  ├─ Root.tsx                 # registers compositions from batch/jobs.ts
│  ├─ compositions/            # one dir per video (data + glue)
│  │  └─ demo-vertical/        # zero-credential sample (renders with no API key)
│  ├─ templates/vertical-card/ # the shared video template (scenes, theme, BgmLayer)
│  ├─ effects/text-emphasis/   # 19 emphasis effects + 10 presets (overlay layer)
│  ├─ effects/motion-sfx/      # animated banner + SFX library (overlay layer)
│  ├─ runners/                 # EmphasizedRunner / MotionSfxRunner
│  ├─ generated/               # timing-<id>.ts subtitle maps (generated, committed)
│  ├─ audio/bgmPool.ts         # deterministic BGM selection
│  ├─ theme/                   # brand + palettes (hash-based)
│  └─ layout/safeArea.ts       # safe-area helpers
├─ batch/
│  ├─ jobs.ts                  # ★ single source of truth for the job list (PURE)
│  ├─ bases.ts                 # base-video registry (PURE)
│  └─ run.mjs                  # Node renderer that drives `remotion render`
├─ tools/                      # every tool has its own README.md
│  ├─ _lib/                    # shared env.sh + cleanup.sh (protected paths)
│  ├─ doctor/                  # readiness check (read-only)
│  ├─ batch-render/            # render the whole job list
│  ├─ render-single/           # render one composition (no effect layer)
│  ├─ script-to-video/         # script JSON → content pack
│  ├─ timing-gen/              # constants.ts → timing-<id>.ts subtitle map
│  ├─ tts-qwen/                # DashScope qwen-tts voiceover
│  ├─ tts-volcano/             # Volcano Engine TTS voiceover
│  ├─ tts-edge/                # Microsoft Edge-TTS voiceover (no key)
│  └─ fetch-sfx/               # download sound-effect packs
├─ examples/
│  └─ script.example.json      # annotated script schema
├─ assets/README.md            # how to drop in bgm/sfx/fonts (repo ships none)
├─ Dockerfile                  # containerized render path
├─ .github/workflows/ci.yml    # CI: typecheck + doctor
├─ .env.example                # secret placeholders
├─ AGENTS.md                   # agent-oriented operating guide
└─ README.md

Audio / images are NOT in the repo. public/assets/{audio,bgm,sfx,images} and out/ are git-ignored. See assets/README.md for where to put your own. The demo works without any of them.

Quickstart (local)

Requirements: Node ≥ 18, FFmpeg on PATH (or set FFMPEG_BIN), and Python 3.10+ (only for the TTS / generator tools).

# 1. install deps
npm install

# 2. check the machine can render (read-only)
npm run doctor
#    or: bash tools/doctor/run.sh

# 3. render the demo (no API key needed — it has no audio/assets)
bash tools/batch-render/run.sh --only=demo-vertical

# 4. inspect output
ls -la out/batch/

The demo renders at 720×1280 (the default; faster than 1080p) and produces out/batch/demo-vertical.mp4.

Add your own video

# copy the example and edit it
cp examples/script.example.json my-script.json
# ...edit my-script.json (scenes, narration, cues)...

# generate a content pack from the script
python tools/script-to-video/main.py --script my-script.json --engine edge

# (optional) generate the subtitle timing map from the rendered audio
python tools/timing-gen/main.py demo-from-script --from-audio

# register the new job in batch/jobs.ts, then render
bash tools/batch-render/run.sh --only=demo-from-script

See tools/script-to-video/README.md, tools/timing-gen/README.md, and AGENTS.md for the full contracts.

Voiceover (TTS)

By default the demo has no audio. To add voiceover, set a TTS engine and key, then run the matching tool:

cp .env.example .env          # then fill in your key
export $(grep -v '^#' .env | xargs)   # or let env.sh source it for you

python tools/tts-edge/main.py   --composition demo-vertical   # no key needed
python tools/tts-qwen/main.py    --composition demo-vertical   # needs DASHSCOPE_API_KEY
python tools/tts-volcano/main.py --composition demo-vertical   # needs VOLCANO_API_KEY

Each tool writes public/assets/audio/<composition>/<sceneId>.mp3 — exactly the path the template reads, so no component change is needed.

Containerized / server path

# build the image (bakes in node + ffmpeg + the project)
docker build -t remotion-video-factory .

# render inside the container; mount a volume for outputs
docker run --rm \
  -e DASHSCOPE_API_KEY=$DASHSCOPE_API_KEY \
  -v "$PWD/out":/app/out \
  remotion-video-factory \
  bash tools/batch-render/run.sh --only=demo-vertical

The Dockerfile installs FFmpeg and Chromium (Remotion's headless renderer needs a browser) and runs as a non-root user. For large batches, pass --parallel=N and raise REMOTION_CONCURRENCY.

Deploy inside a fresh container (for an agent)

If an AI agent (e.g. ima's) gets a fresh Linux container with a shell + internet and must stand up the whole factory by itself, hand it this one command. It installs Node + FFmpeg + Chromium, clones the repo, installs deps, and renders the zero-credential demo to prove the pipeline works end-to-end:

bash tools/deploy-agent.sh

The script is idempotent and prints clear next steps. Override via env vars:

Env Default Purpose
REPO_URL https://github.com/awa123qwe/7.git clone URL. For a private repo, embed a read-only token: https://<TOKEN>@github.com/awa123qwe/7.git
DEPLOY_DIR /app/remotion-video-factory where to clone
BRANCH default branch branch to check out

The container must be Debian/Ubuntu (apt) and have internet for apt + npm. On first run the script pulls Node 20, FFmpeg and Chromium (~hundreds of MB of downloads). If the repo is private, the agent cannot clone it without a token — either make the repo public, or pass REPO_URL with an embedded read-only deploy token.

Tool reference

Tool Purpose Entry Details
doctor readiness check (read-only) bash tools/doctor/run.sh README
batch-render render the job list bash tools/batch-render/run.sh README
render-single render one composition (no FX) bash tools/render-single/run.sh README
script-to-video script JSON → content pack python tools/script-to-video/main.py README
timing-gen constants.ts → subtitle map python tools/timing-gen/main.py README
tts-qwen DashScope voiceover python tools/tts-qwen/main.py tools/tts-qwen/
tts-volcano Volcano voiceover python tools/tts-volcano/main.py tools/tts-volcano/
tts-edge Edge-TTS voiceover (no key) python tools/tts-edge/main.py tools/tts-edge/
fetch-sfx download SFX packs python tools/fetch-sfx/main.py tools/fetch-sfx/

All shell tools source tools/_lib/env.sh first, which locates the project, loads .env (without clobbering existing env), and sets portable defaults.

Troubleshooting

Symptom Fix
node_modules missing npm install
ffmpeg not found install FFmpeg, or export FFMPEG_BIN=/path/to/ffmpeg-dir
batch/*.ts imports react remove the import — those modules must stay pure (see doctor output)
render hangs / is very slow lower REMOTION_CONCURRENCY (4 is the 16 GB sweet spot; 8 swaps)
output missing but exit 0 the renderer trusts ffprobe, not the exit code; check out/batch/ and run doctor
TTS 403 (Volcano) trial voices are BV001/BV002/BV007; paid ids return 403
TTS empty audio engine retried; check key/quota; doctor reports key presence

License

MIT — see LICENSE.

About

6

Resources

Readme
MIT license
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages