[Bugfix][Core] Fix multimodal prompt building for rows without media

[natedemoss]

I was poking at --multimodal-keys with a dataset that mixes image rows and plain
text rows, and ended up finding three separate problems in _build_messages.

The prompt gets shattered when a row has no media

The placeholder split builds a regex alternation out of whatever media the row
actually carries. When the media column is None nothing goes into that set, the
pattern collapses to "()", and that matches the empty string at every position:

>>> from vime.utils.data import _build_messages
>>> _build_messages({"text": "Describe this image.", "images": None},
...                 "text", True, {"image": "images"})
[{'role': 'user', 'content': [{'type': 'text', 'text': 'D'},
                              {'type': 'text', 'text': 'e'},
                              {'type': 'text', 'text': 's'}, ...]}]

One content dict per character. These rows definitely reach here — filter_long_prompt
already has an explicit text-only branch for exactly this shape of dataset.

I want to be upfront that this is the least severe of the three. I checked
Qwen2-VL, Qwen3-VL and GLM-4.1V, and all three render the shattered content to
byte-identical text, so it's wasted work and a fragile representation rather than
corrupted training data. Still worth not doing.

A typo in --multimodal-keys silently drops every image

MultimodalTypes.get() returns None for a name it doesn't know, and the old code
just skipped those. So '{"images": "images"}' — plural, which is an easy thing to
type given the column is usually called images — quietly attaches no media at all
and you train text-only against a VLM dataset with nothing in the logs to tell you.

This one I'd call the real bug. I made it raise, and listed the supported types in the
--multimodal-keys help text since they weren't written down anywhere.

This is the only behaviour change in the PR that could break a currently-working run,
so shout if you'd rather it stayed a warning.

Rows already in list-of-dicts form abort the dataset load

If a message's content is already a list, the code deliberately passes it through
("no processing will be done"). But the leftover-media check afterwards doesn't know
that happened, so the row's media has no placeholder to be spent on and the whole
Dataset construction dies:

message['content'] is a list of dicts, no processing will be done.
AssertionError: Multimodal data count mismatch: 1 more image(s)than '<image>' placeholders in prompt

Now it only runs the check when placeholder expansion actually happened. (Also put the
missing space back in that message.)

Tests

tests/test_build_messages.py, wired into the synchronized upstream CPU tests job
next to test_filter_long_prompt.py. Nine cases: three fail on main, the other six
pin the substitution behaviour that should not move.

One asymmetry I left alone and pinned instead: images: None leaves the content as a
plain string, images: [] gives a single text dict. Collapsing [] into the no-media
path would be tidier but you'd lose the Not enough image data assert when a prompt
has <image> and an empty column, which seemed like the more useful behaviour to keep.

Verification

CPU jobs all pass locally (~740 tests across plugin contracts, upstream sync CPU,
utils and agent adapter). pre-commit clean.

Two things I couldn't run on my machine, neither touched by this change: the GPU
suites, and test_glm5_indexer_q_norm / test_rollout_metrics /
test_rollout_routing_replay_validation, which need megatron and vllm from the
vllm/vime:latest image.

test_cispo_loss and test_policy_loss fail for me on clean main too — Windows box
with no C++ toolchain, so torch.compile can't get through pick_vec_isa. Unrelated.

---

AI-assisted, per CONTRIBUTING: the diff and tests were written by Claude Opus 5
(Anthropic, via Claude Code) working from my direction, and it's attributed in the
commit trailer. I've reviewed every changed line and run the tests above.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[gemini-code-assist]

Code Review

This pull request introduces comprehensive unit tests for _build_messages and refactors the function to handle edge cases, such as raising a ValueError for unknown multimodal types, avoiding empty placeholder splitting, and ensuring leftover media validation only runs when substitution occurs. The review feedback suggests robustly handling single string or dictionary media entries by wrapping them in a list, adding a test for this behavior, and replacing the runtime assert statement with an explicit AssertionError to prevent it from being bypassed under Python optimization flags.

[LOGO127]

I found one remaining mixed-message case at 9574840e4479d328b536fa80b982dec7626a6b90: the inline-content pass-through works by itself, but adding a plain-text system or assistant turn makes it fail again.

from vime.utils.data import _build_messages

row = {
    "text": [
        {"role": "system", "content": "You are helpful."},
        {"role": "user", "content": [
            {"type": "image", "image": "a.png"},
            {"type": "text", "text": "Describe it."},
        ]},
    ],
    "images": ["a.png"],
}
_build_messages(row, "text", True, {"image": "images"})

This raises AssertionError: Multimodal data count mismatch: 1 more image(s) than '<image>' placeholders in prompt. Removing the system message passes. Adding a plain-string assistant turn after the inline user message also fails.

expanded_any becomes true for any string content, even if that string has no media placeholder, so the unrelated text turn re-enables the leftover check for inline media. This looks like an incomplete edge of the PR's stated pass-through fix, not a claim that this PR introduced the original mismatch bug.

Could the regression coverage include these mixed conversations and define how inline media relates to the row media column? Simply counting only matched placeholders would also need to preserve the existing all-string missing-placeholder validation.

Verified through the real vime.utils.data import on this head, Python 3.12.3 / CPU: three focused cases yielded 1 pass / 2 failures. No import stubs, downloads, GPU training or full-suite claim. This review/reproducer was AI-assisted. I'm leaving implementation with this PR rather than opening a duplicate.