Skip to content

About

ZwZ-v2: RL-trained vision-language models (4B/9B/27B) that connect fine-grained visual evidence across image regions, trained with GRPO on 20.4K verified multi-region VQA examples.

Resources

Stars

17 stars

Watchers

0 watching

Forks

Latest commit

ย 

History

13 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Zooming without Zooming v2 (ZwZ2)


๐Ÿ“ƒ Paper | ๐Ÿ  Project | ๐Ÿค— ZwZ2-RL-VQA | ๐Ÿค— ZwZ2-4B | ๐Ÿค— ZwZ2-9B | ๐Ÿค— ZwZ2-27B

โœจ Introduction

Zooming without Zooming v2 connects fine-grained visual evidence across regions. We train ZwZ2-4B/9B/27B with reinforcement learning on 20.4K verified, multi-region VQA examples. Training uses full images; inference needs no zooming tools. The models transfer across perception, spatial reasoning, long-context understanding and multimodal agents.

ZwZ-v2 benchmark results

โš™๏ธ Method

EvidenceConnect turns evidence from 3โ€“7 image regions into verifiable supervision:

  1. Discover grounded regions and synthesize questions that connect their evidence.
  2. Verify answers through cross-model agreement and adaptive region-ablation tests.
  3. Train on the full-image view with GRPO, without supervised fine-tuning.

EvidenceConnect data construction pipeline

๐Ÿ“Š Performance

Results on 25 benchmarks across six domains (accuracy, %).

Full results table for 25 benchmarks

๐Ÿค— Models and Data

Model Base model
ZwZ2-4B Qwen3.5-4B
ZwZ2-9B Qwen3.5-9B
ZwZ2-27B Qwen3.6-27B

All three scales use the same 20.4K training examples, synthesized from SA-1B images with cross-model verification. Privileged crops are used during data construction only; the policy learns from the full-image view.

๐Ÿš€ Getting Started

Component Directory
Data synthesis data_synthesis/
Training verl/
Evaluation eval/

1. Data synthesis (data_synthesis/)

Setup

cd data_synthesis
cp configs/local_env.sh.example configs/local_env.sh   # then edit keys/model names
pip install openai httpx pillow pyarrow pytest

Set your API URL, keys and model names in configs/local_env.sh (gitignored).

Run

The only input is a folder of images:

# 1) Stage 01 โ€” scan the image folder, build a manifest, and drop
#    low-content images (resolution / entropy / edge density / std-dev):
python scripts/01_filter_manifest.py \
    --image_folder /path/to/images \
    --output_jsonl data/filtered_manifest.jsonl

# 2) Continuous synthesis over the filtered manifest:
bash run_pipeline_continuous.sh [TARGET_VALID] [BATCH_SIZE] [MAX_BATCHES] [MANIFEST]
#    MANIFEST defaults to data/filtered_manifest.jsonl (the stage-01 output)

Export training data

The synthesized records are exported to a verl-ready parquet for RL training:

python scripts/05_export_dataset.py \
    --validated_jsonl data/all_valid.jsonl \
    --output_parquet  mcq_train.parquet \
    --format mcq

--format mcq is the multiple-choice format consumed by the training launchers below. Any verl-format validation parquet can be used as the evaluation set.

Tests

pytest -q tests/

2. Training (verl/)

Train with GRPO using verl. Choose the launcher for your base model:

Model Script
Qwen3.5-4B verl/train_zwz2_qwen3_5_4b.sh
Qwen3.5-9B verl/train_zwz2_qwen3_5_9b.sh
Qwen3.6-27B verl/train_zwz2_qwen3_6_27b.sh

Setup

pip install -e ./verl   # install the vendored verl framework

Run

Set the model and dataset paths:

cd verl
MODEL_PATH=/path/to/Qwen3.5-9B \
TRAIN_FILE=/path/to/mcq_train.parquet \
TEST_FILE=/path/to/val.parquet \
bash train_zwz2_qwen3_5_9b.sh

Run the same way for the other sizes; checkpoints, logs and rollout dumps are written under outputs/<experiment-name>/ (override with EXPERIMENT_ROOT).


๐Ÿงช 3. Model Evaluation (eval/)

cd eval
python -m pip install -e '.[api,data]'

1. Configure

python -m zwz_eval init --directory local/my-eval --bench hr4k,hr8k

Edit local/my-eval/model.json with your model/judge paths, identities, GPU settings and output directory. Set downloaded annotation/image paths and input formats in local/my-eval/datasets.json. Replace all example values before running.

2. Prepare data

Download and extract the benchmarks, then prepare them:

bash scripts/prepare_benchmarks.sh local/my-eval

This writes local/my-eval/ready.json without modifying the original downloads.

3. Evaluate

bash scripts/evaluate.sh local/my-eval

The command evaluates the configured benchmarks serially, managing model startup, inference, required judging and service cleanup. To select a subset, append IDs such as hr4k,hr8k.

Benchmark data format

MMStar, BabyVision, VStar, HR-4K, HR-8K, MME-RealWorld-en/cn, ZoomBench, Perception-Bench, TreeBench, O3-Bench and CountQA require JSONL or a JSON array with images, query and response fields:

{
  "images": ["/path/to/images/image.png"],
  "query": "<image>Original question and answer choices",
  "response": "Reference answer"
}

Use format: normalized for these benchmarks; keep original questions, answers, image order and optional fields unchanged. Other benchmarks use their downloaded formats. See eval/README.md for the benchmark ID table, download instructions, data formats and optional API settings. AgentVista uses the same workflow and additionally requires Internet access for search/visit tools.


๐Ÿ™ Acknowledgments

This project builds upon:

  • Qwen3.5 and Qwen3.6 for the base models.
  • verl for the RL training framework.

๐Ÿ“ฎ Contact

For questions or collaborations, please contact:

If you encounter issues with dataset paths or evaluation setup, please share your feedback or contributions. Thank you for your support!

๐Ÿ“„ License

Apache License 2.0. See LICENSE.

๐Ÿ“„ Citation

@article{wei2026zoomingv2,
  title={Zooming without Zooming v2: From Seeing Details to Connecting Evidence for Generalizable Multimodal Perception},
  author={Wei, Lai and Li, Yatai and He, Liangbo and Lan, Jun and Gu, Renjie and Cao, Linhan and Lin, Junkai and Zhu, Huijia and Wang, Weiqiang and Kong, Linghe and Wang, Yue and Zhang, Zhuosheng and Huang, Weiran},
  journal={arXiv preprint arXiv:2610.37037},
  year={2026}
}

About

ZwZ-v2: RL-trained vision-language models (4B/9B/27B) that connect fine-grained visual evidence across image regions, trained with GRPO on 20.4K verified multi-region VQA examples.

Resources

Stars

17 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages