Skip to content

feat(neuroflow): headless DICOM ingest CLI - #18

Open
cdrake wants to merge 3 commits into
mainfrom
neuroflow/headless-ingest
Open

cdrake wants to merge 3 commits into
mainfrom
neuroflow/headless-ingest

Conversation

@cdrake

@cdrake cdrake commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Replaces the five-PR stack #11–#15 with one PR: the command-line import from #15, plus only the parts of #11, #12 and #14 it needs. The window is unchanged.

What it does

bun run neuroflow-ingest runs BIDSvue's DICOM import (runImport and the post-pass) under Bun, with no window and no user input. It moves the BIDS root to <out>/bids, then writes the NeuroFlow handoff file and provenance line. A NeuroFlow workflow, or an AI agent through NeuroFlow's MCP server, can then run DICOM to BIDS unattended.

bun run neuroflow-ingest --neuroflow-session <dir> [--allow-heuristic-file]
bun run neuroflow-ingest --dicom <dir> --out <dir>
bun run neuroflow-ingest --check [--dcm2niix <path>]
  • Settings: flags first, then the session's inputs, then defaults. The settings are importer (default dcm2niix-reproin), subject, session, anonymize, shift_dates, save_derivatives, heuristic and config.
  • Importers: dcm2niix-reproin, pet2bids, heudiconv and dcm2bids.
  • dcm2niix: uses --dcm2niix (a relative path is resolved) or $DCM2NIIX, then the bundled sidecar for the host, then PATH.
  • Partial success: dcm2niix exit codes 8 and 10 continue to the post-pass, as in the app. The codes now live in one file, src/lib/import/dcm2niixPartialExit.json, which process.rs (include_str!) and the CLI both read.
  • Exit codes: 0 done, 1 import failed or its output already exists, 2 usage error.
  • --check: a preflight for NeuroFlow's Environment panel. It resolves dcm2niix as an import would, runs dcm2niix -v, prints one JSON line { ready, dcm2niix, dcm2niixVersion, problems[] }, and exits 0 if ready, 1 if not.
  • Fresh clone: the package script runs svelte-kit sync first, so $lib resolves after bun install.

No undo

A headless run is not recorded in an operations.log and cannot be reverted. Its state dir is a temp dir deleted when the run ends, including on SIGINT/SIGTERM. To redo a run, delete <out>. The CLI refuses to start if <out>/bids or the staging folder <out>/bidsvue-import already exists. This is documented in headless.ts and the README.

Heuristics run Python

heudiconv runs a heuristic .py as Python. A heuristic file from a session's context.json is refused unless --allow-heuristic-file is passed. A file named with --heuristic on the command line, or a built-in name such as reproin, needs no flag. A dcm2bids config is JSON, not code.

Files

  • src/lib/neuroflow/contract.ts (from [1/5] feat(neuroflow): session contract and Rust commands #11): the session contract's file formats (context, handoff, provenance line). It has no host dependency and is unit-tested.
  • src/lib/neuroflow/relocate.ts (from [2/5] feat(neuroflow): place an import's BIDS root at a chosen path #12): moves the single BIDS root an import produced from staging to <out>/bids. It refuses when there are several roots, a root is outside staging, or the target already exists.
  • node:fs adapters (from [4/5] refactor(test): promote the node:fs adapters out of test-only files #14):
    • mutate/testFs.ts is renamed to mutate/nodeFs.ts, and postpass/__testFs.ts to postpass/nodeFs.ts; the CLI uses them.
    • exists() uses stat, and a non-recursive remove() deletes an empty directory, both matching plugin-fs.
    • finalizeNewFileNoReplace copies with COPYFILE_EXCL and then removes the source, so it works on exFAT and SMB, where link() fails.
    • Most of the diff is test files updating their imports.
  • src/lib/neuroflow/headless.ts, nodeExecutor.ts, scripts/neuroflow-ingest.ts (from [5/5] feat(neuroflow): headless DICOM ingest CLI #15): the CLI, plus the package.json script and a README section.
  • src-tauri/src/process.rs: reads the partial-success table from the shared JSON instead of hardcoding it. Behavior is unchanged, and a unit test covers it.

Testing

  • bun run lint and bun test pass (2472 tests, 0 fail). bun run typecheck passes here except for main's @brainchop/mindgrab import, which my local node_modules doesn't have yet.
  • The partial-exit lookup was tested against the shared JSON. I couldn't run the full cargo test locally because main's mindgrab sidecar isn't staged here; CI covers it.
  • --check reports ready with dcm2niix v1.0.20260724, and reports ready: false and exits 1 with DCM2NIIX=./nope.
  • Real Siemens session (crlab, 35 NIfTIs): about 60 s, exit 0. The temp state dir was gone afterwards. A second run into the same <out> was refused with exit 1.
  • Earlier comparison against the window import (from [5/5] feat(neuroflow): headless DICOM ingest CLI #15): same files, and every JSON/TSV file byte-identical. The CLI also writes a .bidsignore that the window doesn't; I'll follow up on that separately.

Window mode, for later

#13's window mode isn't in this PR. It's on the branch neuroflow/window-ingest, which will be rebased onto this PR.

🤖 Generated with Claude Code

cdrake added a commit to cdrake/neuroflow that referenced this pull request Oct 13, 2026
…ndow-ingest

BIDSvue's NeuroFlow work is now one PR (niivue/BIDSvue#18), the headless
ingest that bidsvue-ingest runs; it is headed for main. The window's
NeuroFlow mode, which the bidsvue tool and the crlab-ingest demo need, stays
on the neuroflow/window-ingest branch for now. That branch carries both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@neurolabusc

Copy link
Copy Markdown
Contributor

Thanks — this consolidation is much easier to review than #11–#15, and the window path is untouched. Before merging, could you address:

  1. Rebase onto main. 1f4fa08 adds tests that still import nodeFsPostPassAdapter/__testFs. Rename them to nodePostPassFs/./nodeFs, and stripSbref.test.ts also needs $lib/mutate/nodeFs. Conflicts are only in src/AGENTS.md (keep main's port-sync paragraph) and bidsguessCleanup.test.ts (keep main's assertion). fix(mcp): replace deprecated fetch_update with a compare-exchange loop #16 and chore(rust): raise rust-version to 1.88, what the dependencies already need #17 are now merged.
  2. Reversibility. Either record the relocate, handoff and provenance writes through an OperationContext, or document in code and the README that the CLI has no undo. In that case the state dir should be a throwaway temp dir that's cleaned up on exit, not a stale operations.log pointing at the vanished staging dir.
  3. No-hard-link volumes. finalizeNewFileNoReplace in mutate/nodeFs.ts uses link(), which fails on exFAT and SMB. Use copyFile(src, dst, COPYFILE_EXCL) and then rm(src).
  4. Fresh clone. Make the script svelte-kit sync && bun run scripts/neuroflow-ingest.ts, or use relative imports.
  5. Refuse when <out>/bidsvue-import already exists, so a failed run doesn't feed stale staging into a rerun.
  6. Document that heuristic/config in context.json run arbitrary Python via heudiconv. Consider requiring a flag before session mode accepts a file-path heuristic.
  7. Drop or defer --workflow mode. Session mode plus --dicom/--out seem sufficient.

Nits: resolve() the --dcm2niix path before symlinking it, accept UNC paths, make "already exists" a runtime error (exit 1) instead of a usage error, and share the partial-exit codes with process.rs.

Follow-up issues, not blockers: a guard keeping node:* imports out of renderer code (e.g. one src/lib/node/ folder plus a Biome rule), and evidence for the .bidsignore difference so we can tell whether the window really drops it.

cdrake and others added 3 commits October 14, 2026 14:31
`bun run neuroflow-ingest` runs the app's DICOM import (runImport and the
post-pass) under Bun with no window and no user input, moves the BIDS root
to <out>/bids, and writes the NeuroFlow handoff file and provenance line.
This makes BIDSvue's ingest runnable as a NeuroFlow step, including from
hosts that cannot wait on an app, such as NeuroFlow's MCP server. The
window is unchanged.

It takes a NeuroFlow session (context.json), a step of a workflow document
(constants and inputs.* bindings), or plain flags. Settings come from flags,
then the step inputs, then defaults: importer dcm2niix-reproin, subject,
session, anonymize, shift_dates, save_derivatives, heuristic, config.
Importers: dcm2niix-reproin, pet2bids, heudiconv, dcm2bids. Exit codes:
0 done, 1 import failed, 2 usage error.

- $lib/neuroflow/contract: the session contract's file formats (context,
  handoff, provenance line), host-independent and unit-tested.
- $lib/neuroflow/relocate: move the single BIDS root an import produced
  from the staging folder to <out>/bids; refuses several roots, a root
  outside staging, or an existing target.
- $lib/neuroflow/headless: settings resolution, workflow step binding,
  runHeadlessIngest (refuses an existing <out>/bids before converting).
- $lib/neuroflow/nodeExecutor: spawn-based ImportExecutor. Uses the bundled
  dcm2niix sidecar for the host triple, then $DCM2NIIX / PATH; keeps the
  Rust side's partial-success exit codes (8, 10) and the dcm2bids PATH shim.
- The node:fs adapters move out of test-only files (mutate/testFs.ts ->
  mutate/nodeFs.ts, postpass/__testFs.ts -> postpass/nodeFs.ts) so the CLI
  can use them; exists() now stats and a non-recursive remove() deletes an
  empty directory, matching plugin-fs. Test imports updated.
- scripts/neuroflow-ingest.ts and the package.json script; README section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`bun run neuroflow-ingest -- --check` runs no import. It resolves dcm2niix
the way an import does (--dcm2niix, $DCM2NIIX, the bundled sidecar, PATH),
starts `dcm2niix -v`, and prints one JSON line: ready, the dcm2niix path and
version, and any problems. Exit 0 when ready, 1 when not.

NeuroFlow's Environment panel runs it as the tool's probe, so a missing or
broken dcm2niix shows up before a run instead of as the run's first error,
and NeuroFlow does not have to copy BIDSvue's dcm2niix lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- No undo, documented: the CLI's state dir is a temp dir deleted on exit
  (and on SIGINT/SIGTERM); README and headless.ts say a headless run is
  not journaled and how to redo one.
- Refuse to start when <out>/bids or <out>/bidsvue-import exists; this is
  a runtime error (exit 1), not a usage error.
- finalizeNewFileNoReplace copies with COPYFILE_EXCL and removes the
  source instead of link(), which exFAT and SMB don't support.
- The package script runs svelte-kit sync first, so $lib resolves in a
  fresh clone.
- A heuristic file from a session's context.json runs arbitrary Python
  via heudiconv; require --allow-heuristic-file for it.
- Drop --workflow mode (and --step/--input/--state-dir).
- Resolve a relative --dcm2niix path; accept UNC paths as absolute.
- Share dcm2niix's partial-success exit codes between process.rs and
  nodeExecutor.ts through src/lib/import/dcm2niixPartialExit.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cdrake
cdrake force-pushed the neuroflow/headless-ingest branch from e27c1ca to b6a4dcd Compare October 14, 2026 18:38
@cdrake

cdrake commented Oct 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review. All of it is addressed in b6a4dcd, which is rebased onto main.

  1. Rebase: done. The new tests use nodePostPassFs / ./nodeFs, and stripSbref.test.ts uses $lib/mutate/nodeFs. I kept main's port-sync paragraph in src/AGENTS.md and main's assertion in bidsguessCleanup.test.ts.
  2. Reversibility: I went with documenting it. headless.ts and the README say a headless run isn't journaled and can't be undone, and that you redo one by deleting <out>. The state dir is now a mkdtemp dir, removed in a finally and on SIGINT/SIGTERM.
  3. exFAT/SMB: finalizeNewFileNoReplace now does copyFile(src, dst, COPYFILE_EXCL) and then removes src. If removing src fails, it rolls back dst.
  4. Fresh clone: the script is now svelte-kit sync && bun run scripts/neuroflow-ingest.ts.
  5. Existing output: it refuses when <out>/bidsvue-import exists, as well as <out>/bids, before any converter runs.
  6. Heuristics: documented. A heuristic file from context.json is refused unless you pass --allow-heuristic-file. Built-in names, and a file passed explicitly with --heuristic, don't need the flag. config is JSON, so it isn't gated.
  7. --workflow: removed, along with --step, --input and --state-dir.

Nits:

  • A relative --dcm2niix is resolve()d before it's symlinked.
  • UNC paths count as absolute.
  • "Already exists" is now a runtime error (exit 1).
  • The partial-exit codes are in src/lib/import/dcm2niixPartialExit.json. process.rs reads it with include_str!, and a unit test covers the lookup.

Follow-ups: I'm happy to open issues for the node:* import guard (src/lib/node/ plus a Biome rule) and for the .bidsignore difference, if you want them tracked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants