feat: optimize minimax support and add demo

.gitmodules

@@ -1,6 +1,6 @@
 [submodule "third_party/ggml"]
 	path = third_party/ggml
-	url = https://github.com/yiming-l21/ggml.git
+	url = https://github.com/Komorebi623/ggml.git
 	branch = edge-cudnn-sdpa
 [submodule "third_party/onednn"]
 	path = third_party/onednn

README.md

@@ -39,7 +39,7 @@ placement, and backend selection.
   - Shared **C API, CLI, HTTP server, and Python** interfaces across every family
 
 - **Fits large models into limited VRAM**
-  - **`--auto-fit`** — one flag auto-picks DiT **quantization** (`q8_0`→`q4_K`) *and* per-component placement to meet a hard VRAM budget
+  - **`--auto-fit`** — one flag auto-picks DiT **quantization** (`q8_0`→`q4_K`) and per-component placement; on single-device CUDA, pairing it with `--max-vram` also enforces an allocation ceiling
   - **Layered offload** — streams the diffusion transformer **one block at a time** (async double-buffered on CUDA), so 20 GB+ models run on a 24 GB or smaller card
   - **Per-component offload** (`--dit-offload` / `--text-encoder-offload` / `--vae-offload`) and **VAE tiling**
   - **`ed-convert`** — offline weight quantization to a **portable pre-quantized GGUF** (skips per-load CPU quantization), with per-tensor dtype rules and activation-calibrated **imatrix**
@@ -57,7 +57,7 @@ placement, and backend selection.
 - **2026-08-14:** 🚀 Added **MiniMax-H3 FL2VA and Ref2VA video+audio generation** with image, video, embedded/paired audio, and mixed references; full or pruned BF16 DiTs, persistent Q8_0 conversion, Q4_K_M weights, and automatic VRAM fitting are supported ([usage and H200 results](docs/minimax-h3.md)).
 - **2026-08-05:** 🚀 Completed the **RTX 4090 (24 GB) benchmark** — full cross-system speed / VRAM / image-quality across text-to-image, editing, and video ([results](docs/performance-4090.md)).
 - **2026-07-30:** 🚀 Added **per-component offload** (`--dit-offload` / `--text-encoder-offload` / `--vae-offload`), unifying all offload paths on one semantics.
-- **2026-07-29:** 🚀 Added **`--auto-fit`** — one flag picks DiT quantization *and* per-component placement to fit a hard VRAM budget.
+- **2026-07-29:** 🚀 Added **`--auto-fit`** — one flag picks DiT quantization and per-component placement; `--auto-fit --max-vram <GB>` enforces the requested single-device CUDA allocation ceiling.
 - **2026-07-27:** 🚀 Added **few-step distilled** auto-detection (Turbo/Lightning/schnell → 4–8 steps) and optional **SageAttention** for SD3/Wan.
 - **2026-07-23:** 🚀 Added **`ed-convert`** for offline weight quantization to portable pre-quantized GGUF (with activation-calibrated `--imatrix`).
 - **2026-07-11:** 🚀 **edge-dit.cpp v0.1.0-alpha** enters **public preview**.
@@ -162,8 +162,8 @@ Generation latency is the primary cross-runtime performance metric.
 
 MiniMax-H3 has a separate 124-frame H200 benchmark because it generates video
 and audio rather than one image. In the current resident-component BF16
-comparison, Edge is faster than Diffusers in all four FL2VA and all four Ref2VA
-generation paths while using less peak VRAM. FL2VA text-to-video takes
+comparison, edge-dit.cpp is faster than Diffusers in all four FL2VA and all four
+Ref2VA generation paths while using less peak VRAM. FL2VA text-to-video takes
 `51.396s` versus `53.986s`; Ref2VA image and mixed-reference generation reach
 `1.08x` and `1.06x` speedups. Full and pruned BF16 DiTs are supported directly,
 and either can be converted once to persistent Q8_0 GGUF. `--auto-fit` has also

docs/assets/minimax-h3-ref2va-h200-15s-metrics.json

@@ -0,0 +1,92 @@
+{
+  "date": "2026-08-16",
+  "device": "NVIDIA H200",
+  "sampling": {
+    "frames": 362,
+    "fps": 24,
+    "steps": 20,
+    "cfg": 1.0,
+    "seed": 157368968253448,
+    "sampler": "res_multistep",
+    "scheduler": "simple"
+  },
+  "tasks": {
+    "four_image_action": {
+      "width": 1280,
+      "height": 736,
+      "reference_images": 4,
+      "frameworks": {
+        "edge-dit.cpp": {
+          "end_to_end_seconds": 919.765,
+          "load_seconds": 17.773,
+          "conditioning_seconds": 2.641,
+          "dit_seconds": 839.890,
+          "video_vae_decode_seconds": 41.842,
+          "audio_vae_decode_seconds": 0.369,
+          "save_seconds": 5.519,
+          "peak_device_mib": 52667
+        },
+        "ComfyUI": {
+          "end_to_end_seconds": 1055.354,
+          "peak_device_mib": 49413,
+          "memory_sample_interval_seconds": 0.2,
+          "memory_sample_count": 3321,
+          "decoded_output_matches_original": true
+        },
+        "Diffusers": {
+          "end_to_end_seconds": 2343.937,
+          "load_seconds": 32.260,
+          "conditioning_seconds": 14.924,
+          "dit_seconds": 2263.720,
+          "video_vae_decode_seconds": 20.679,
+          "audio_vae_decode_seconds": 0.145,
+          "mux_seconds": 4.699,
+          "pytorch_max_allocated_mib": 109960,
+          "pytorch_max_reserved_mib": 110866
+        }
+      }
+    },
+    "two_character_portrait": {
+      "width": 736,
+      "height": 1280,
+      "reference_images": 2,
+      "frameworks": {
+        "edge-dit.cpp": {
+          "end_to_end_seconds": 888.636,
+          "load_seconds": 17.784,
+          "conditioning_seconds": 1.168,
+          "dit_seconds": 811.867,
+          "video_vae_decode_seconds": 41.871,
+          "audio_vae_decode_seconds": 0.355,
+          "save_seconds": 3.737,
+          "peak_device_mib": 52809
+        },
+        "ComfyUI": {
+          "end_to_end_seconds": 1025.864,
+          "conditioning_node_seconds": 33.140,
+          "sampler_node_seconds": 965.504,
+          "video_vae_decode_seconds": 19.947,
+          "audio_vae_decode_seconds": 0.501,
+          "save_seconds": 5.927,
+          "peak_device_mib": 49159
+        },
+        "Diffusers": {
+          "end_to_end_seconds": 1625.178,
+          "load_seconds": 32.838,
+          "conditioning_seconds": 7.176,
+          "dit_seconds": 1553.626,
+          "video_vae_decode_seconds": 20.567,
+          "audio_vae_decode_seconds": 0.144,
+          "mux_seconds": 3.481,
+          "pytorch_max_allocated_mib": 99189,
+          "pytorch_max_reserved_mib": 108646
+        }
+      }
+    }
+  },
+  "reference_resize": {
+    "edge-dit.cpp": "match; no upscaling",
+    "ComfyUI": "match; no upscaling",
+    "Diffusers": "official MiniMaxH3ImageReference; 2048px short edge"
+  }
+}

docs/cli.md

@@ -337,12 +337,21 @@ On-load weight type selection:
   --output flux-q4.png
 ```
 
+`--type` is a global loading policy: it applies to every eligible tensor in
+every loaded component (DiT, text encoder, video VAE, and audio VAE). Biases,
+norms, embeddings, protected tensors, and shapes incompatible with a block
+quant remain at their stored type.
+
 Supported `--type` values:
 
 ```text
-f32 f16 bf16 q4_0 q4_1 q5_0 q5_1 q8_0 q2_k q3_k q4_k q5_k q6_k
+preserve f32 f16 bf16 q4_0 q4_1 q5_0 q5_1 q8_0 q2_k q3_k q4_k q5_k q6_k
 ```
 
+`preserve` is the default and leaves every source tensor at its stored type.
+The older value `auto` remains accepted as a compatibility alias for
+`preserve`; it does not perform automatic quantization.
+
 > **Qwen-Image models and FP16:** the Qwen-Image family (`qwen-image`,
 > `qwen-image-edit`, and their distilled/lightning variants) is not supported in
 > FP16 — its DiT activations exceed FP16's dynamic range and silently saturate,
@@ -376,27 +385,36 @@ it off. The offload flags share one semantics — **weights kept on CPU and stag
 to the GPU per compute** (compute always runs on the GPU): `--offload-to-cpu`
 offloads the whole model, while `--dit-offload` / `--text-encoder-offload` /
 `--vae-offload` offload just that one component. `--auto-allocate` places each
-component (DiT, text encoder, VAE) under a hard VRAM cap of `min(--max-vram,
-free)`, keeping a component resident when it fits and offloading (staging) it
-otherwise.
+component (DiT, text encoder, VAE) against a planning budget of
+`min(--max-vram, free)`, keeping a component resident when it fits and
+offloading (staging) it otherwise. By itself this remains a placement-planning
+budget. On single-device CUDA, `--auto-fit --max-vram <GB>` additionally installs a guarded
+allocation ceiling below the requested value, reserving 1 GiB for external
+library workspaces. An allocation that cannot fit is rejected before the
+process crosses the requested ceiling.
 
 These options are workload dependent. Validate output quality and latency for
 the exact model and resolution you plan to run.
 
 ### Budget-driven placement (`--auto-allocate`) and full auto (`--auto-fit`)
 
-Two levels of automation size a run to a hard VRAM budget instead of tuning
+Two levels of automation size a run to a VRAM planning budget instead of tuning
 `--type` and offload flags by hand:
 
 - `--auto-allocate` — you pick the quantization (`--type`); the runtime decides,
   per component (DiT / text encoder / VAE), what stays resident on the GPU and
-  what streams from host, so the peak stays under `--max-vram`.
+  what streams from host. `--max-vram` is not a universal process-level peak clamp.
 - `--auto-fit` — fully automatic. It **implies `--auto-allocate`** and, in
   addition, chooses the quantization itself: the DiT is driven down the ladder
   `q8_0 → q4_k` to the highest level that stays resident within the budget, the
-  text encoder is set to `q8_0` (near-lossless, halves its footprint), and
-  placement is decided as above. `--auto-fit` **ignores `--type`** (it owns the
-  quantization) and logs that it is doing so.
+  text encoder is lowered to at most `q8_0` (an already smaller quant stays unchanged), and
+  placement is decided as above. For the text encoder and DiT, this automatic
+  decision supersedes the global `--type` without ever increasing an already
+  lower-precision source. The VAEs are not replanned: they still follow
+  `--type`, where `preserve` means keeping each source tensor's stored type.
+  With an explicit `--max-vram` on
+  single-device CUDA, it also enables the hard allocation guard; an intrinsically too-large
+  graph fails safely instead of exceeding the budget.
 
 `--auto-fit` measures each component's real compute-buffer footprint at the
 requested resolution (`-W`/`-H`, and `--frames` for video) to size the resident
@@ -416,7 +434,7 @@ the model's fixed 24 fps to the nearest legal `17k+5` frame count. Keep using
 mutually exclusive.
 
 ```bash
-# MiniMax-H3 Q8_0 under a 40 GiB hard placement budget.
+# MiniMax-H3 with automatic quantization/placement and a 40 GiB CUDA ceiling.
 ./build-cuda/bin/ed-cli --video \
   --diffusion-model /path/to/minimax_h3_fl2va-diffusers-Q8_0.gguf \
   --vae /path/to/minimax_h3_video_vae_fp16.safetensors \