pith. sign in

arxiv: 2605.14908 · v1 · pith:TDXIDFFTnew · submitted 2026-05-14 · 💻 cs.CV

SteerSeg: Attention Steering for Reasoning Video Segmentation

Ali Cheraghian , Hamidreza Dastmalchi , Abdelwahed Khamis , Morteza Saberi , Aijun An , Lars Petersson This is my paper

Pith reviewed 2026-06-30 20:55 UTC · model grok-4.3

classification 💻 cs.CV
keywords video reasoning segmentationattention steeringsoft promptschain-of-thoughtlarge vision-language modelsspatial groundingfrozen models
0
0 comments X

The pith

Input conditioning with soft prompts and chain-of-thought steers attention to improve video object segmentation in large vision-language models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that attention maps taken from large vision-language models for video reasoning segmentation are misaligned because they were optimized for text generation instead of spatial localization, producing diffuse signals. SteerSeg addresses this by adding learnable soft prompts and reasoning-guided chain-of-thought prompts at the input to reshape the attention distribution into more concentrated and instance-specific maps. These maps are turned into point prompts for a segmentation model, with tracklets selected by correlation scoring, while the underlying models stay frozen. A reader would care because the method keeps pretrained reasoning intact and trains only a small prompt set, yet still lifts performance on segmentation benchmarks after training on one dataset alone.

Core claim

Attention misalignment is the key bottleneck in attention-based grounding for video reasoning segmentation. SteerSeg steers attention at its source through input-level conditioning that combines learnable soft prompts with reasoning-guided Chain-of-Thought prompting. The soft prompts concentrate the attention distribution while the CoT attributes disambiguate among similar objects. The resulting maps supply point prompts to a segmentation model across keyframes, and candidate tracklets are ranked by correlation-based scoring. Only the soft prompts are trained; the LVLM and segmentation model remain frozen.

What carries the argument

Learnable soft prompts combined with Chain-of-Thought prompting that reshape the attention distribution inside the frozen LVLM.

If this is right

  • Attention maps shift from diffuse to spatially concentrated.
  • Ambiguity among visually similar objects is reduced by directing attention to the correct instance.
  • Pretrained reasoning capabilities remain available because the LVLM weights are untouched.
  • Performance rises on multiple video segmentation benchmarks after training only on Ref-YouTube-VOS.
  • The same frozen models generalize to new benchmarks without additional task-specific training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same input-conditioning pattern could be tested on static-image grounding or referring-expression tasks.
  • If soft prompts prove sufficient, full-model fine-tuning might be avoidable for other multimodal localization problems.
  • Extending the tracklet-ranking step to handle longer videos with frequent occlusions would test the method's temporal robustness.
  • Measuring whether the steered attention also improves downstream tasks such as action recognition could reveal broader utility.

Load-bearing premise

Attention misalignment is the main cause of weak grounding and can be fixed by input-level conditioning without creating new localization errors or damaging the model's original reasoning ability.

What would settle it

Applying the soft prompts and CoT prompts on held-out videos and measuring that the attention maps stay as diffuse or produce lower segmentation accuracy than the unconditioned baseline would show the steering does not work.

Figures

Figures reproduced from arXiv: 2605.14908 by Abdelwahed Khamis, Aijun An, Ali Cheraghian, Hamidreza Dastmalchi, Lars Petersson, Morteza Saberi.

Figure 1
Figure 1. Figure 1: (a) Diagnostic study on reasoning and grounding. While the LVLM often identifies the correct target object, the corresponding attention remains poorly localized, leading to inaccurate masks. (b) Effect of attention refinement on segmentation. Raw and contrast-based attention produce ambiguous localization, while soft prompts and CoT reasoning progressively improve attention quality and segmentation accurac… view at source ↗
Figure 2
Figure 2. Figure 2: Given a referring expression and video frames, the frozen LVLM first performs CoT [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Attention alignment and training loss evolution. Video Frames Frame Attention Text Tokens Visual Tokens High Attention Medium Attention Low Attention High Attention Low Attention 𝑓𝑡 𝑓𝑡+∆𝑡 Segmentation Mask Frame Attention Video Attention (a) (b) Expression: “What kind of vehicle has the most spacious interior for transporting bulky items?” v1 v2 v3 v4 v5 v6 [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative visualization of the proposed framework. (a) Video-level attention, frame-level [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effect of fusion weight α on ReVOS. Frame-only attention (α = 1) is better for referring queries, while video-only (α = 0) benefits reasoning tasks requiring temporal context. Performance peaks at α ≈ 0.3, showing the advantage of combining both. performance across all datasets and metrics, with the correlation trends remaining consistent with the segmentation results. Overall, the results suggest that CoT… view at source ↗
Figure 6
Figure 6. Figure 6: GUI interface used for human evaluation of the generated reasoning attributes on Reason [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
read the original abstract

Video reasoning segmentation requires localizing objects across video frames from natural language expressions, often involving spatial reasoning and implicit references. Recent approaches leverage frozen large vision-language models (LVLMs) by extracting attention maps and using them as spatial priors for segmentation, enabling training-free grounding. However, these attention maps are optimized for text generation rather than spatial localization, often resulting in diffuse and ambiguous grounding signals. In this work, we introduce SteerSeg, a lightweight framework that identifies attention misalignment as the key bottleneck in attention-based grounding and proposes to steer attention at its source through input-level conditioning. SteerSeg combines learnable soft prompts with reasoning-guided Chain-of-Thought (CoT) prompting. The soft prompts reshape the attention distribution to produce more spatially concentrated maps, while CoT-derived attributes resolve ambiguity among similar objects by guiding attention toward the correct instance. The resulting attention maps are converted into point prompts across keyframes to guide a segmentation model, while candidate tracklets are ranked and selected using correlation-based scoring. Our approach freezes the LVLM and segmentation model parameters and learns only a small set of soft prompts, preserving the model's pretrained reasoning capabilities while significantly improving grounding. Despite being trained only on Ref-YouTube-VOS, SteerSeg generalizes well across diverse benchmarks, significantly improving the spatial grounding capability of LVLMs. Project page: https://steerseg.github.io

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The paper proposes SteerSeg, a lightweight framework for reasoning video segmentation that identifies attention misalignment in frozen LVLMs as the central bottleneck. It steers attention via input-level conditioning using a small set of learnable soft prompts combined with reasoning-guided Chain-of-Thought (CoT) prompting, converts the resulting maps to point prompts for a segmentation model, and ranks tracklets via correlation scoring. The approach freezes the LVLM and segmentation model, trains only the soft prompts on Ref-YouTube-VOS, and claims more concentrated attention maps plus strong generalization across benchmarks while preserving pretrained reasoning.

Significance. If the empirical claims hold, the method would offer a parameter-efficient, training-free route to improve spatial grounding in attention-based LVLM pipelines without altering the core model weights. The explicit separation of soft-prompt steering from CoT attribute extraction and the correlation-based tracklet ranking are potentially reusable components, but the absence of any reported metrics prevents assessment of whether these elements deliver measurable gains over prior attention-extraction baselines.

major comments (3)
  1. [Abstract] Abstract: the central claim that attention misalignment is the dominant failure mode (rather than reasoning errors, object ambiguity, or downstream segmentation) is asserted without any supporting evidence, ablation, or isolation experiment; the joint presentation of soft prompts + CoT leaves open the possibility that gains, if any, arise from improved attribute extraction rather than attention reshaping.
  2. [Abstract] Abstract: the assertion that the method 'generalizes well across diverse benchmarks' after training exclusively on Ref-YouTube-VOS is unsupported by any cross-dataset numbers, protocol details, or analysis of whether the learned soft prompts overfit to that single training distribution.
  3. [Abstract] Abstract: no quantitative metrics, tables, figures, error analysis, or experimental protocol are supplied to substantiate the repeated claims of 'more spatially concentrated maps' or 'significantly improving the spatial grounding capability of LVLMs'.
minor comments (1)
  1. The project page URL is referenced but no additional implementation or reproducibility details are provided in the manuscript text.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and constructive feedback. We address each major comment below and commit to revisions that strengthen the manuscript's claims with additional evidence and analysis.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that attention misalignment is the dominant failure mode (rather than reasoning errors, object ambiguity, or downstream segmentation) is asserted without any supporting evidence, ablation, or isolation experiment; the joint presentation of soft prompts + CoT leaves open the possibility that gains, if any, arise from improved attribute extraction rather than attention reshaping.

    Authors: We agree that the abstract asserts attention misalignment as the central bottleneck without dedicated isolation experiments or ablations. The manuscript includes qualitative attention map visualizations contrasting baseline LVLM outputs with our steered results, but these do not fully isolate the factors. We will add an ablation study in the revised version that separately evaluates soft-prompt steering and CoT attribute extraction, reporting their individual effects on attention concentration and downstream segmentation accuracy to clarify the source of gains. revision: yes

  2. Referee: [Abstract] Abstract: the assertion that the method 'generalizes well across diverse benchmarks' after training exclusively on Ref-YouTube-VOS is unsupported by any cross-dataset numbers, protocol details, or analysis of whether the learned soft prompts overfit to that single training distribution.

    Authors: We acknowledge that the generalization claim in the abstract lacks quantitative cross-dataset support and protocol details. The manuscript currently presents qualitative results on additional benchmarks, but this is insufficient. In the revision we will include quantitative results on held-out datasets, specify the evaluation protocol, and analyze soft-prompt transfer to assess overfitting risks. revision: yes

  3. Referee: [Abstract] Abstract: no quantitative metrics, tables, figures, error analysis, or experimental protocol are supplied to substantiate the repeated claims of 'more spatially concentrated maps' or 'significantly improving the spatial grounding capability of LVLMs'.

    Authors: We agree that the abstract repeats claims of improved spatial concentration and grounding without quantitative backing, tables, or protocol details. While the manuscript contains illustrative figures of attention maps, it lacks numerical metrics. We will add quantitative measures (e.g., attention entropy, spatial IoU with ground-truth regions), performance tables with baselines, an error analysis, and a full experimental protocol section in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No significant circularity; standard empirical method with held-out evaluation

full rationale

The paper presents an empirical framework that learns a small set of soft prompts on Ref-YouTube-VOS to reshape LVLM attention maps, then evaluates the resulting point-prompted segmentation pipeline on other benchmarks. This is ordinary supervised training followed by cross-dataset testing rather than any derivation that reduces to its own inputs by construction. No self-definitional equations, fitted parameters renamed as independent predictions, load-bearing self-citations, or imported uniqueness theorems appear in the abstract or described approach. The identification of attention misalignment as a bottleneck is an empirical premise tested by the experiments, not a circular premise.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that attention misalignment can be corrected at the input level; the only explicit free parameter class is the set of learnable soft prompts whose values are fitted to data.

free parameters (1)
  • soft prompts
    Learnable parameters introduced to reshape the attention distribution of the frozen LVLM.
axioms (1)
  • domain assumption Attention misalignment between text-generation optimization and spatial localization is the key bottleneck in current attention-based grounding methods.
    Directly stated as the identified bottleneck that the framework is designed to address.

pith-pipeline@v0.9.1-grok · 5794 in / 1504 out tokens · 43685 ms · 2026-06-30T20:55:34.581416+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 7 canonical work pages · 6 internal anchors

  1. [1]

    Abnar and W

    S. Abnar and W. Zuidema. Quantifying attention flow in transformers. InAnnual Meeting of the Association for Computational Linguistics (ACL), 2020

    2020

  2. [2]

    Alayrac, J

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. Flamingo: A visual language model for few-shot learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  3. [3]

    Unleashing hierarchical reasoning: An LLM-driven framework for training-free referring video object segmentation

    Anonymous. Unleashing hierarchical reasoning: An LLM-driven framework for training-free referring video object segmentation. InAAAI Conference on Artificial Intelligence (AAAI), 2026

    2026

  4. [4]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2.5-VL technical report.arXiv preprint arXiv:2502.13923, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  5. [5]

    Z. Bai, T. He, H. Mei, P. Wang, Z. Gao, J. Chen, L. Liu, Z. Zhang, and M. Z. Shou. One token to seg them all: Language instructed reasoning segmentation in videos.Advances in Neural Information Processing Systems, 37:6833–6859, 2024

    2024

  6. [6]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, et al. Language models are few-shot learners. InAdvances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  7. [7]

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, B. Li, P. Luo, T. Lu, Y . Qiao, and J. Dai. InternVL: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  8. [8]

    Deitke, C

    M. Deitke, C. Clark, S. Lee, R. Tripathi, Y . Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, et al. Molmo and PixMo: Open weights and open data for state-of-the-art vision-language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  9. [9]

    H. Ding, C. Liu, S. He, X. Jiang, and C. C. Loy. Mevis: A large-scale benchmark for video segmentation with motion expressions. InProceedings of the IEEE/CVF international conference on computer vision, pages 2694–2703, 2023

    2023

  10. [10]

    The Llama 3 Herd of Models

    A. Dubey, A. Jauhri, A. Pandey, et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [11]

    S. Gong, L. Zhang, Y . Zhuge, X. Jia, P. Zhang, and H. Lu. Reinforcing video reasoning segmentation to think before it segments.arXiv preprint arXiv:2508.11538, 2025

    work page arXiv 2025

  12. [12]

    S. Gong, Y . Zhuge, L. Zhang, Z. Yang, P. Zhang, and H. Lu. The devil is in temporal token: High quality video reasoning segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  13. [13]

    S. H. Han, J. Hyun, P. Lee, M. Shim, D. Wee, and S. J. Kim. Decomposed attention fusion in MLLMs for training-free video reasoning segmentation. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  14. [14]

    E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022. URLhttps://openreview.net/forum?id=nZeVKeeFYf9. 10

    2022

  15. [15]

    P. Jin, R. Takanobu, W. Zhang, X. Cao, and L. Yuan. Chat-UniVi: Unified visual representation empowers large language models with image and video understanding. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  16. [16]

    S. Kang, J. Kim, J. Kim, and S. J. Hwang. Your large vision-language model only needs a few attention heads for visual grounding. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  17. [17]

    S. Kang, J. Kim, J. Kim, and S. J. Hwang. See what you are told: Visual attention sink in large multimodal models. InThe Thirteenth International Conference on Learning Representations,

  18. [18]

    URLhttps://openreview.net/forum?id=7uDI7w5RQA

  19. [19]

    Kao, Y .-W

    S.-h. Kao, Y .-W. Tai, and C.-K. Tang. CoT-RVS: Zero-shot chain-of-thought reasoning segmen- tation for videos. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  20. [20]

    Khoreva, A

    A. Khoreva, A. Rohrbach, and B. Schiele. Video object segmentation with language referring expressions. InAsian conference on computer vision, pages 123–141. Springer, 2018

    2018

  21. [21]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Dollár, and R. Girshick. Segment anything. InIEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  22. [22]

    X. Lai, Z. Tian, Y . Chen, Y . Li, Y . Yuan, S. Liu, and J. Jia. LISA: Reasoning segmentation via large language model. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  23. [23]

    B. Li, Y . Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y . Li, Z. Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  24. [24]

    J. Li, D. Li, S. Savarese, and S. Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational Conference on Machine Learning (ICML), 2023

    2023

  25. [25]

    Y . Li, C. Wang, and J. Jia. LLaMA-VID: An image is worth 2 tokens in large language models. InEuropean Conference on Computer Vision (ECCV), 2024

    2024

  26. [26]

    L. Lin, X. Yu, Z. Pang, and Y .-X. Wang. GLUS: Global-local reasoning unified into a single large language model for video segmentation. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  27. [27]

    Z. Lin, Y . Wang, and Z. Tang. Training-free open-ended object detection and segmentation via attention as prompts. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  28. [28]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  29. [29]

    H. Liu, C. Li, Y . Li, and Y . J. Lee. Improved baselines with visual instruction tuning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  30. [30]

    Z. Liu, Y . Dong, Z. Liu, W. Hu, J. Lu, and Y . Rao. Oryx MLLM: On-demand spatial-temporal understanding at arbitrary resolution. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  31. [31]

    S. Ma, Q. Fang, S. Guo, Y . Zhang, and Y . Feng. LLaV A-Mini: Efficient image and video large multimodal models with one vision token. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  32. [32]

    M. Maaz, H. Rasheed, S. Khan, and F. S. Khan. Video-ChatGPT: Towards detailed video understanding via large vision and language models. InAnnual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  33. [33]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning (ICML), 2021. 11

    2021

  34. [34]

    Rasheed, M

    H. Rasheed, M. Maaz, S. Shaji, A. Shaker, S. Khan, H. Cholakkal, R. M. Anwer, E. Xing, M.-H. Yang, and F. S. Khan. GLaMM: Pixel grounding large multimodal model. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  35. [35]

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Dollar, and C. Feichtenhofer. SAM 2: Segment anything in images and videos. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/fo...

    2025

  36. [36]

    Z. Ren, Z. Huang, Y . Wei, Y . Zhao, D. Fu, J. Feng, and X. Jin. PixelLM: Pixel reasoning with large multimodal model. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  37. [37]

    Seo, J.-Y

    S. Seo, J.-Y . Lee, and B. Han. Urvos: Unified referring video object segmentation network with a large-scale benchmark. InEuropean conference on computer vision, pages 208–223. Springer, 2020

    2020

  38. [38]

    X. Shen, Y . Xiong, C. Zhao, L. Wu, J. Chen, C. Zhu, Z. Liu, F. Xiao, B. Varadarajan, F. Bordes, Z. Liu, H. Xu, H. J. Kim, B. Soran, R. Krishnamoorthi, M. Elhoseiny, and V . Chandra. LongVU: Spatiotemporal adaptive compression for long video-language understanding. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  39. [39]

    E. Song, W. Chai, G. Wang, Y . Zhang, H. Zhou, F. Wu, H. Chi, X. Guo, T. Ye, Y . Zhang, Y . Lu, J.-N. Hwang, and G. Wang. MovieChat: From dense token to sparse memory for long video understanding. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  40. [40]

    S. Tong, E. Brown, P. Wu, S. Woo, M. Middepogu, S. C. Akula, J. Yang, S. Yang, A. Iyer, X. Pan, A. Wang, R. Fergus, Y . LeCun, and S. Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal LLMs. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  41. [41]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  42. [42]

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y . Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin. Qwen2-VL: En- hancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  43. [43]

    M. Wu, X. Cai, J. Ji, J. Li, O. Huang, G. Luo, H. Fei, G. Jiang, X. Sun, and R. Ji. ControlMLLM: Training-free visual prompt learning for multimodal large language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  44. [44]

    S. Wu, S. Jin, W. Zhang, L. Xu, W. Liu, W. Li, and C. C. Loy. F-LMM: Grounding frozen large multimodal models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  45. [45]

    C. Yan, H. Wang, S. Yan, X. Jiang, Y . Hu, G. Kang, W. Xie, and E. Gavves. Visa: Reasoning video object segmentation via large language models. InEuropean Conference on Computer Vision, pages 98–115. Springer, 2024

    2024

  46. [46]

    A. Yang, B. Yang, B. Hui, et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  47. [47]

    Zhang, M

    J. Zhang, M. Khayatkhoei, P. Chhikara, and F. Ilievski. MLLMs know where to look: Training- free perception of small visual details with multimodal LLMs. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  48. [48]

    Zhang, J

    Y . Zhang, J. Wu, W. Li, B. Li, Z. Ma, Z. Liu, and C. Li. Video instruction tuning with synthetic data. InInternational Conference on Learning Representations (ICLR), 2025. 12

    2025

  49. [49]

    J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, H. Tian, Y . Duan, W. Su, J. Shao, et al. InternVL3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  50. [50]

    Z. Zhu, J. Fan, Z. Liu, and F. Li. Training-free spatio-temporal decoupled reasoning video segmentation with adaptive object memory. InAAAI Conference on Artificial Intelligence (AAAI), 2026. 13 Appendix A Diagnostic Study and Annotation Interface To better understand the relationship between semantic reasoning and spatial grounding in LVLMs, we conducted...

    2026