A comprehensive vqa benchmark specifically designed to evaluate the robustness of LVLMs against misleading visual inputs.
You can download MVI-Bench from kittyhy/MVI-Bench
For Qwen-VL series
Python: 3.10.18 torch: 2.6.0, transformers: 4.51.3
For InternVL3
Python: 3.9.23 torch: 2.8.0, transformers: 4.55.4
For SAIL-VL2
Python: 3.10.18 torch: 2.8.0, transformers: 4.51.0
For Molmo
Python: 3.9.23 torch: 2.8.0, transformers: 4.50.3
For LLaVA-OneVision
Python: 3.10.18 torch: 2.1.2, transformers: 4.45.0