Add Qwen 3.5 for Video Caption - #2344
Conversation
|
🌿 Preview your docs: https://nvidia-preview-bobchen-qwen-3-5.docs.buildwithfern.com/nemo/curator Here are the markdown pages you've updated: |
Greptile SummaryThis PR adds Qwen 3.5 support to video caption preparation, generation, enhancement, documentation, and tutorial configuration. It also introduces unrelated interleaved image-cache tests that currently target a nonexistent production API.
Confidence Score: 4/5The PR should not merge until the newly added interleaved cache tests are removed or the production cache API they exercise is implemented. The Qwen3.5 integration is coherently wired across preparation, generation, enhancement, documentation, and CLI selection, but the new interleaved tests deterministically invoke a method absent from every production stage class and will fail the test suite. Files Needing Attention: tests/stages/interleaved/test_stages.py and nemo_curator/stages/interleaved/stages.py Important Files Changed
Sequence DiagramsequenceDiagram
participant Clip as Video window
participant Prep as Caption preparation
participant VLM as Qwen3.5-9B
participant LM as Qwen3.5-27B
Prep->>Prep: Format video prompt with thinking disabled
Clip->>Prep: Sampled video frames
Prep->>VLM: Prompt and multimodal video data
VLM-->>Clip: Generated caption
Clip->>LM: Caption enhancement prompt
LM-->>Clip: Enhanced caption
Reviews (1): Last reviewed commit: "Remove unused files" | Re-trigger Greptile |
| df = task.to_pandas() | ||
| image_mask = df["modality"] == "image" | ||
|
|
||
| first = list(stage.iter_decoded_images(task=task, df=df, row_mask=image_mask)) |
There was a problem hiding this comment.
Missing decoded-image cache method
When the new interleaved tests run, they invoke iter_decoded_images, but neither the filter classes nor their base classes define that method, causing all three tests to fail with AttributeError.
Knowledge Base Used: Interleaved and synthetic data curation
Description
Add Qwen 3.5 for Video Caption
Usage
# Add snippet demonstrating usageChecklist