CMR-MULTI — Approach B: MedNeXt per-view specialists

Cardiac MRI segmentation and clinical quantification for the CMR-MULTI Challenge, held with MICCAI 2026. This branch holds Approach B, one of the two backbones compared in the accompanying paper.

Seven independent specialist models, one per sequence × view, each with its own label configuration and its own 2D/3D choice driven by acquisition geometry.

Framework note. MedNeXt ships as a fork of nnU-Net v1 (package nnunet_mednext, CLI mednextv1_*) — there is no native nnU-Net v2 support. This pipeline follows v1 conventions throughout: tasks live under <nnunet_raw_data_base>/nnUNet_raw_data/TaskXXX_name/, the env vars are nnUNet_raw_data_base / nnUNet_preprocessed / RESULTS_FOLDER, and dataset.json uses the v1 schema.

Branches

branch	contents
mednext	this one, and the default branch. MedNeXt per-view specialists
nnunet	nnU-Net v2 with the residual-encoder "L" planner
main	challenge data and docs, managed with DataLad / git-annex

The dataset is not on this branch. It lives on main, managed with DataLad and git-annex:

datalad clone https://github.com/BioMeDS/cmr-multi
cd cmr-multi && datalad get data

Then point this branch's data path at that checkout (see Configuration below).

Approach B scored 0.67 against 0.6027 for Approach A on the challenge validation leaderboard. The paper attributes that margin to scar handling and ejection-fraction recovery rather than to the backbone.

Which backbone runs where

Network dimensionality is chosen per view by acquisition geometry, not by a global default:

views	configuration	backbone	why
cine_sax, lge_sax	3d_fullres	MedNeXt-L, kernel 3	deep volumetric stacks reward 3D context
the five long-axis views	2d	MedNeXt-M, kernel 3	3–7 slices only; a 3D network would model spurious through-plane correlation

A note on how this came about. Upstream MedNeXt ships 3D trainers only — every initialize_network there builds the network without passing dim, so it defaults to 3D, and the nnU-Net wrapper hardcodes conv_op = nn.Conv3d. The 3D MedNeXt configuration was tried on the long-axis views and exceeded the memory of a 12 GB card. Within the challenge submission window there was no time to write 2D trainers, so the five long-axis specialists were submitted with standard nnU-Net 2D under the same 200-epoch schedule — a mixed-backbone system.

The MedNeXt architecture was never the obstacle: MedNextV1 takes a dim argument that upstream's blocks.py plumbs down to nn.Conv2d. Only the trainers were missing. This repository adds them (see mednext-patch/), so the long-axis specialists now run on MedNeXt too and the all-MedNeXt system in the paper's title is complete. Both configurations remain reproducible — they are selected by --arm, below.

Backbone arms

Comparing MedNeXt against nnU-Net is not as simple as swapping the network. The MedNeXt trainers inherit nnUNetTrainerV2_Optim_and_LR, which also replaces nnU-Net's SGD at lr 1e-2 with AdamW at lr 1e-3. A two-way comparison therefore changes architecture and optimisation at once, and a win cannot be attributed to either.

Four arms, declared under mednext.arms in the config, separate the effects:

arm	short-axis	long-axis	optimiser (LAX)	isolates
mednext	MedNeXt-L 3D	MedNeXt-M 2D	AdamW 1e-3	the all-MedNeXt system
submitted	MedNeXt-L 3D	nnU-Net 2D	SGD 1e-2	the mixed system actually submitted
nnunet	—	nnU-Net 2D	SGD 1e-2	the long-axis baseline
adamw	—	nnU-Net 2D	AdamW 1e-3	the optimiser change alone

Read mednext against adamw to isolate architecture, and adamw against nnunet to isolate optimisation. On the long-axis views the two effects are of opposite sign on LGE and nearly cancel; completing the MedNeXt long-axis specialists changes the overall score by +0.0009.

Each arm writes to its own results directory, so arms never overwrite one another. scripts/audit_controlled.py resolves every training setting for a set of arms and reports anything differing beyond architecture — run it before trusting a comparison.

The 7 sub-problems

Label maps are defined once, in src/data/registry.py — the single source of truth. They are not duplicated in the config or in any script.

name	id	seq	view	config	output channels	labels
cine_sax	101	CINE	SAX	3d_fullres	4	BG, LV-myo, LV-cavity, RV-cavity
cine_2ch	102	CINE	2CH	2d	3	BG, LV-cavity, LV-myo
cine_4ch	103	CINE	4CH	2d	6	BG, LV-cavity, LV-myo, RV-cavity, RA, LA
lge_sax	201	LGE	SAX	3d_fullres	5	BG, LV-cavity, LV-myo, Scar, RV-cavity
lge_2ch	202	LGE	2CH	2d	4	BG, LV-cavity, LV-myo, Scar
lge_4ch	203	LGE	4CH	2d	5	BG, LV-cavity, LV-myo, Scar, RV-cavity
lge_ras	204	LGE	RAS	2d	2	BG, Right Atrium

All specialists train for 200 epochs. Fold counts as submitted: Cine SAX five folds, LGE SAX three folds (reduced from five to meet the submission deadline), and three folds for each of the five long-axis specialists. The remaining LGE folds were completed after submission; they raise Dice on every LGE view but lower the challenge score, because the scar-mass terms are dominated by one confident false positive in a seven-case cohort. The three-fold numbers are the ones the paper reports.

Training used an RTX 4090 (24 GB) for the 3D short-axis models and an RTX 3060 (12 GB) for the 2D long-axis models. No external data and no pretrained weights — every model was trained from scratch on the provided training set.

Data layout notes (verified against the real dataset)

• CINE volumes are stored 3D, not 4D: (X, Y, slices·timepoints). The third axis packs slices and cardiac phases; CINE_MULTI/sax_slice_info.json gives the timepoint count per SAX case so it can be un-flattened. See conversion.cine_mode in the config (whole_volume default, per_timepoint optional).
• Metadata workbooks: CINE_MULTI/dataset_train.xlsx (sheets SAX/2CH/4CH) and LGE_MULTI/dataset_lge.xlsx (sheets 2CH/4CH/SAX/RAS, id column 编号). The SAX sheet's LVEF column is not consistent with ejection fraction computed volumetrically from the provided annotations (no meaningful correlation, r < 0.2, on the training set). Derive EF from the masks; do not validate against that column.
• Scar (label 3) is sparse — many LGE cases legitimately contain no scar.
• Intensity-based scar refinement is disabled by default, and should stay that way. Applied to ground-truth scar on the 30 scar-bearing training cases, the 5-SD rule retains only 12.7% of true scar pooled by volume and wipes out twelve cases entirely; the scar-in-myocardium constraint retains 51.7%. The fragment filter is close to inert — it alters 10 of 30 cases but retains 99.9% of true scar, so the frequent claim that it removes small patchy scar is not supported. Reproduce all of this with scripts/ablate_scar_refinement.py; see postprocessing.scar_refinement in the config.

Scoring conventions

Two conventions must be matched for a local score to be comparable to the challenge leaderboard, and both are easy to get wrong:

• Hausdorff distance is the maximum, not the 95th percentile. On this data the difference is large — mean maximum 8.05 mm against mean 95th percentile 2.24 mm over the 150 Cine structure-case measurements.
• Scar-mass relative absolute error averages |m̂ − m| / m over cases containing scar, excluding empty ones.

With those matched, scripts/score_local.py reproduces the platform for Dice throughout and for all Task 2 terms (Task 2 0.6458 locally against 0.65 reported). It does not reproduce Task 1. Pass --calibrate to print every RAE convention at once.

Install

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

Then install MedNeXt with this project's trainers applied — see mednext-patch/README.md for the exact commands. In short: clone MIC-DKFZ/MedNeXt at commit 0b78ed8, apply mednext-patch/upstream-compat.patch, copy in the three trainers from mednext-patch/new_trainers/, then pip install -e ..

The upstream fork is not vendored here. The patch is four lines across three files (two PyTorch 2.6 weights_only fixes and an env-var epoch count); the three new trainers are the 2D MedNeXt trainer plus the two long-axis comparison arms.

Configuration

Everything is driven by configs/default.yaml. No path is hardcoded in source, and paths are environment-expanded — ${VAR} requires the variable, ${VAR:-fallback} supplies a default — so a checkout runs on a new machine without editing YAML.

cp configs/env.example.sh configs/env.sh   # edit CMR_ROOT and CMR_DATA_ROOT
source configs/env.sh

nnU-Net writes tens of GB (unpacked training arrays, checkpoints) under CMR_ROOT, so point it at a volume with room — not necessarily the disk holding this checkout.

Workflow

source configs/env.sh

# 1. Inspect raw data (shapes, spacings, labels, mismatches)
python scripts/01_inspect_data.py

# 2. Convert to nnU-Net v1 raw format (+ stratified CV splits)
python scripts/02_convert_to_nnunet.py --make-splits

# 3. Train. Two workers can share the queue: give the second --reverse and it
#    walks the job list backwards, with lock directories preventing overlap.
python scripts/train_specialists.py --arm mednext --gpu 0
python scripts/train_specialists.py --arm mednext --gpu 1 --reverse

#    Plan without running anything:
python scripts/train_specialists.py --arm mednext --dry-run

# 4. Validation inference for one arm
python scripts/predict_specialists.py --arm mednext

# 5. Compare arms. Use matched fold counts: scoring a 5-fold ensemble against a
#    3-fold one understates the smaller arm.
python scripts/predict_specialists.py --arm nnunet --folds 0 1 2
python scripts/compare_backbones.py --arms mednext nnunet adamw

# 6. Confirm the comparison is controlled before trusting it
python scripts/audit_controlled.py

# 7. Score locally against the challenge formula
python scripts/score_local.py --arm mednext --calibrate

Reproducing the paper's analyses

python scripts/ablate_scar_refinement.py   # scar rules vs ground truth, per case
python scripts/analyse_hausdorff.py        # is the max driven by false positives or misses?
python scripts/compare_backbones.py        # three-arm backbone/optimiser comparison

Each writes CSVs under outputs/, and each is named in the corresponding section of the paper's supplementary material, so every reported number can be regenerated rather than taken on trust.

Project structure

src/
  config.py            # YAML-backed typed Config, environment expansion
  data/                # registry (single source of truth), parser, router, converter
  preprocessing/       # intensity normalisation, ROI cropping
  training/            # trainer wrapper, multi-job runner with locking, loss tracker
  inference/           # TTA ensemble, arm-aware predictor
  postprocessing/      # connected components, anatomical, temporal, scar refinement
  evaluation/          # metrics, clinical (LVEF/scar mass), challenge scoring
  utils/               # NIfTI io, logging
scripts/               # CLI entry points and analysis scripts
mednext-patch/         # MedNeXt changes: compat patch + the 2D/comparison trainers
tests/                 # unit tests

Design principles

• pathlib everywhere, type hints, Google-style docstrings, logging over print.
• MedNeXt / nnU-Net v1 is driven as a black box via its CLI (subprocess). The one exception is the added trainers, which necessarily live inside the package.
• Label maps live only in src/data/registry.py; filesystem roots and backbone choices live only in configs/default.yaml. Neither is duplicated in scripts.
• Failures surface early: the training runner checks that the results and preprocessed directories are writable before any GPU work, and distinguishes "another worker holds this job" from "the lock could not be created".

Data

The challenge dataset is not distributed here. It is available from the challenge organisers under their terms of participation. Point CMR_DATA_ROOT at your copy.

Licence

Apache License 2.0 — see LICENSE and NOTICE.

The files under mednext-patch/ are modifications of, or derivative works based on, MedNeXt and nnU-Net (Copyright DKFZ, Apache-2.0), and retain the original copyright headers. Those upstream projects are not redistributed here — they are obtained separately, as described in mednext-patch/README.md.

Citation

If you use this code, please cite the accompanying paper (MWM @ MICCAI 2026).