Pith. sign in

REVIEW 3 major objections 6 minor 53 references

ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By assigning each high-resolution image token the position ID of its spatially overlapping thumbnail token, ID-Align restores the cross-resolution and image-text attention that RoPE's long-term decay erodes, improving benchmark performance.

desk verdict A simple, plausible fix for position-ID inflation in LLaVA-Next, with an honest but incomplete mechanism study; worth a serious referee. read the letter →

arxiv 2505.21465 v1 pith:I4BF7DSG submitted 2025-05-27 cs.CV cs.CL

Bozhou Li , Wentao Zhang This is my paper
classification cs.CVcs.CL
keywords vision-languagemodelsdynamichigh-resolutionRotaryPositionEmbeddingIDremappingcross-resolutionattentionRoPElong-termdecayID-Align
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ID-Align claims that dynamic high-resolution vision-language models are silently degraded by the choice of position IDs, and that a simple remap fixes it. These pipelines feed both a downscaled thumbnail and upscaled high-resolution crops to the model; the token count balloons, and with Rotary Position Embedding the inflated relative distances push attention away from the high-resolution tokens' corresponding thumbnail regions and away from the text. The paper first shows that RoPE's long-term decay is not mathematically guaranteed but does hold for the query and key distributions found in a real LLM. It then proposes ID-Align: each high-resolution token inherits the position ID of its spatially corresponding thumbnail token, collapsing positional distance while preserving spatial layout. On a standard dynamic high-resolution VLM, this yields consistent benchmark gains, including a 6.09-point improvement in MMBench relation reasoning.

What carries the argument

The load-bearing object is the pairing of Rotary Position Embedding with a position-ID grid. After the thumbnail and high-resolution crops are encoded, ID-Align constructs the high-resolution position IDs by taking the thumbnail's 2D ID grid, resizing it to the high-resolution grid, and rounding; each high-resolution token then carries the ID of the thumbnail region it overlaps. This keeps the RoPE relative distance between matching tokens near zero and stops positional indices from expanding with the large token increase caused by high-resolution encoding.

What would settle it

Compute RoPE attention scores versus relative distance for visual-to-visual and text-to-visual pairs inside the trained VLM; if no decay appears for visual tokens, the motivating mechanism collapses. Alternatively, ablate the correspondence rule by assigning each high-resolution token a random position ID from the same multiset; if benchmark gains remain, spatial alignment is not what carries the improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that RoPE's long-term decay, which the authors verify empirically on real LLM activations, actively degrades dynamic high-resolution VLMs by severing two necessary attention links: between a high-resolution token and the thumbnail token whose region it refines, and between image tokens and text. The proposed remedy, ID-Align, reorders position IDs so that high-resolution tokens take the position ID of their corresponding thumbnail token, keeping position IDs within the ranges seen in training and making relative distances encode spatial correspondence rather than raw token count. The paper shows attention maps shifting from unrelated regions to the relevant objects and text, with benchmark gains concentrated in global-information subtasks.

Load-bearing premise

The argument rests on the assumption that the rotary position embedding's long-term decay, measured on text-only activations, also governs visual tokens in real multimodal sequences and is the cause of the degraded cross-resolution and image-text attention.

Editorial extensions

If this is right

  • High-resolution image understanding can be improved without additional parameters, training data, or token compression, simply by reusing existing position-ID capacity.
  • Because relative distances are repurposed, position IDs stop encoding absolute token order for images and start encoding cross-resolution spatial correspondence.
  • ID-Align is compatible with other RoPE position-ID schemes that operate within a single image, so its gains can stack with those methods.
  • The gains concentrate in global-information subtasks such as relation reasoning and cross-instance fine-grained perception, matching the paper's attention-based explanation.
  • Models with a larger RoPE base show smaller gains, consistent with the claim that the effect is mediated by positional sensitivity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the decay mechanism transfers to other multimodal backbones, the same ID remapping should yield gains in any RoPE-based VLM that concatenates multiple image crops; this is directly testable outside the paper's base architecture.
  • A stronger test would swap the spatial correspondence rule for an arbitrary but equally sized remapping; if random ID sharing works just as well, spatial alignment is not the active ingredient.
  • The position-ID budget view suggests a design principle for very long multimodal contexts: keep semantic correspondences at small relative distances instead of letting raw token count inflate positional IDs, which could combine with token compression rather than compete with it.
  • Because the paper measures decay on text-only activations, replicating the measurement on visual keys and queries inside real multimodal sequences would pin down whether the mechanism is exactly the one proposed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ID-Align, a position-ID remapping scheme for LLaVA-Next style dynamic high-resolution VLMs. High-resolution image tokens are assigned the position IDs of their spatially corresponding thumbnail tokens, while the IDs of text and thumbnail tokens are unchanged. The motivation is that RoPE's long-term decay, which the paper argues is distribution-dependent, suppresses attention between high-resolution tokens and their thumbnail counterparts and between text and distant image tokens. The authors report benchmark gains with Vicuna-7B/CLIP and Qwen-2.5-7B/SigLIP, including a +6.09 point increase on MMBench Relation Reasoning, and provide attention visualizations and learning-curve plots. The paper also contains a mathematical and empirical analysis of when RoPE exhibits long-term decay (Section 3.1, Appendix A).

Significance. If the empirical claim holds, ID-Align is a simple, plug-in position-ID remapping that could improve dynamic high-resolution VLMs without architectural changes, and the released code would aid reproducibility. The Section 3.1 analysis of the distribution-dependence of RoPE decay (Eqs. 14-15) is a useful clarification of a subtle point and is a genuine contribution beyond the specific method. However, the central mechanism is currently supported only indirectly: the decay is measured on text-only activations, and the headline benchmark results come from single training runs. The skeptic's concern about transfer from text-only to visual q/k distributions is valid and load-bearing, because the paper itself argues the decay property is not universal. The stress-test concern lands.

major comments (3)
  1. [Section 3.1, Figure 3] The empirical demonstration of RoPE's long-term decay is obtained from text-only activations of Vicuna-7B on WikiText, whereas the method's mechanism concerns q/k pairs of visual tokens in a fine-tuned multimodal model. Because Eqs. (14)-(15) and Appendix A show the decay trend is distribution-dependent, this transfer is load-bearing and unverified. Please measure the average q-k inner product versus relative position for visual tokens, both thumbnail and high-resolution, in the fine-tuned LLaVA-Next model, or otherwise justify why the text-only measurement transfers. As written, the motivating mechanism could fail even if ID-Align still improves performance for other reasons.
  2. [Tables 1 and 2] The core empirical claim rests on single training runs with no error bars or multiple seeds. Several reported cells decline (e.g., Vicuna LR -1.69 in Table 2; Qwen MMStar -0.39, RealWorldQA -0.39, POPE -0.01 in Table 1), and the margins on other cells are small. Without variance estimates or paired significance tests, the statement that ID-Align 'significantly' improves performance is not supported. Please report means and standard deviations over at least three seeds, or a paired statistical comparison, and adjust the wording so that 'significant' is reserved for tested effects.
  3. [Section 4, first paragraph] The sentence 'increasing the difference between the position IDs of q and k can enhance their attention coefficient, while decreasing it can reduce it' is opposite to the paper's long-term decay premise and to the intended effect of ID-Align, which brings corresponding tokens closer in position-ID space. As written, this reverses the mechanism that motivates the method. Please correct the direction or explicitly restate that the effect is relative-distance dependent as shown in Section 3.1.
minor comments (6)
  1. [Abstract and Section 5.2] The abstract's '6.09% enhancement' should be '6.09 percentage points', since it is an absolute increase from 60.87 to 66.96 in MMBench Relation Reasoning.
  2. [Section 5.2] There is a typo: 'Vinca' should be 'Vicuna'.
  3. [Introduction, contributions] The text says the contributions are summarized into 'two points' but then lists three bullets; this should be corrected.
  4. [Appendix B] The word 'intenger' should be 'integer'.
  5. [Appendix D.2] The comparison with MRoPE and V2PE uses only the Qwen-2.5-0.5B model and, as stated, does not combine those methods with ID-Align. This is acceptable as a preliminary comparison, but the broader claim 'our method is superior' should be restricted to the reported setting.
  6. [Section 7] The limitation section does not mention that the RoPE-decay analysis was performed on text-only activations; adding this limitation would make the scope of the motivating evidence explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ID-Align is an empirically validated design on external benchmarks; the RoPE-decay analysis motivates the design but no fitted parameter is renamed as a prediction and no result reduces to an input by construction.

full rationale

The paper's derivation chain is: (1) Section 3.1 empirically measures RoPE long-term decay using text-only Vicuna-7B activations on WikiText; (2) Section 3.2 argues that this decay can suppress thumbnail-to-high-resolution and image-to-text attention; (3) Section 4 defines ID-Align by assigning high-resolution tokens the position IDs of their spatially corresponding thumbnail tokens; (4) Section 5 reports gains on external benchmarks. No equation in the paper converts a fitted parameter into a 'prediction': the decay curve in Figure 3 is descriptive, no constant from it is inserted into the method, and the benchmark results are genuine out-of-sample comparisons. ID-Align's construction directly encodes the desired correspondence, which is the designed mechanism rather than a hidden equivalence; the paper does not claim to derive the benchmark gains from the decay measurement. The only self-referential element is a non-load-bearing citation to a survey coauthored by one of the present authors (Bai et al., 2024) in the introduction's discussion of dataset curation. The paper's limitation section lists unexplored combinations (token compression, ViTs with dynamic resolution), which are scope restrictions, not circularity. The known weakness in the skeptic attack—that Section 3.1 measures text-only activations and the transfer to visual-token q/k distributions in fine-tuned VLMs is unverified—is an empirical assumption, not circular reasoning, because the external benchmarks would discriminate the method even if the motivating mechanism were incorrect. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The method introduces no new entities or fitted constants. The central claim rests on an empirical premise about RoPE behavior and a design assumption about repeated position IDs.

assumptions (3)
  • domain assumption RoPE exhibits long-term decay for q/k distributions in LLMs
    Empirically verified on text activations from Vicuna-7B (Section 3.1, Figure 3), assumed to extend to visual tokens.
  • domain assumption The LLaVA-Next training recipe (pretrain + finetune) is a valid testbed for the method
    Standard training pipeline; no modification to the training objective or loss.
  • ad hoc to paper Assigning identical position IDs to spatially corresponding tokens is a safe intervention that does not harm token distinguishability
    Core design choice of ID-Align; not proven, only empirically evaluated on two backbones.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models." pith.science (2026). https://pith.science/paper/I4BF7DSG

@misc{pith2026250521465,
  author       = {Pith},
  title        = {Pith review of: ID-Align: RoPE-Conscious Position Remapping for Dynamic High-Resolution Adaptation in Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I4BF7DSG}},
  note         = {Machine review of arXiv:2505.21465}
}
read the original abstract

Currently, a prevalent approach for enhancing Vision-Language Models (VLMs) performance is to encode both the high-resolution version and the thumbnail of an image simultaneously. While effective, this method generates a large number of image tokens. When combined with the widely used Rotary Position Embedding (RoPE), its long-term decay property hinders the interaction between high-resolution tokens and thumbnail tokens, as well as between text and image. To address these issues, we propose ID-Align, which alleviates these problems by reordering position IDs. In this method, high-resolution tokens inherit IDs from their corresponding thumbnail token while constraining the overexpansion of positional indices. Our experiments conducted within the LLaVA-Next framework demonstrate that ID-Align achieves significant improvements, including a 6.09% enhancement on MMBench's relation reasoning tasks and notable gains across multiple benchmarks. Our code is available at the following link: https://github.com/zooblastlbz/ID-Align.

Figures

Figures reproduced from arXiv: 2505.21465 by the authors.

Figure 1
Figure 1. Intuitive presentation of the original high-resolution method and ID-Align. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Flowchart of the Dynamic High-Resolution [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The Long-term Decay Property of RoPE. We randomly sampled 100 text data points from Wiki￾text and randomly selected 10 pairs of q-k from each layer of the Vicuna-7B model for computation. q ∼ N (µq, I), k ∼ N (µk, I), the following for￾mula holds: E[q ⊤Rmk] = µq T Rmµk ∀m ∈ Z (15) Furthermore, the trend of E[q ⊤Rmk] with respect to m is dependent on the value of µq, µk , and can be overall increasing or decreasing a… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Attention distributions from the red region in the high-resolution image and the red text towards thumbnail [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 1
Figure 1. Figure 1: (q_mean=1, k_mean=1) _base=10000 _base=10000000 0 200 400 600 800 1000 relative distance 16 14 12 10 8 6 4 2 average attention score [PITH_FULL_IMAGE:figures/full_fig_p013_1.png]
Figure 3
Figure 3. Figure 3: (q_mean=-1, k_mean=-1) _base=10000 _base=10000000 0 200 400 600 800 1000 relative distance 0.10 0.08 0.06 0.04 0.02 0.00 0.02 0.04 0.06 average attention score [PITH_FULL_IMAGE:figures/full_fig_p013_3.png]
Figure 5
Figure 5. Figure 5: Simulation of RoPE’s Long-term Properties under Different [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 9
Figure 9. Figure 9: Finetune Grad Norm D.1.2 Qwen [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Pretrain Loss [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Pretrain Grad Norm [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 8
Figure 8. Figure 8: Finetune Loss [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 13
Figure 13. Figure 13: Finetune Grad Norm D.2 Compare with Other Methods We also compared our method with MRoPE(Wang et al., 2024) and V2PE(Ge et al., 2024). Our method is not in competition with these methods; rather, it is compatible with them. The focus of these methods is on positional …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 11 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Tianyi Bai, Hao Liang, Binwang Wan, Yanran Xu, Xi Li, Shiyu Li, Ling Yang, Bozhou Li, Yifan Wang, Bin Cui, et al. 2024. A survey of multimodal large language model from a data-centric perspective. arXiv preprint arXiv:2405.16640

    arXiv 2024

  3. [3]

    Federico Barbero, Alex Vitvitskyi, Christos Perivolaropoulos, Razvan Pascanu, and Petar Veli c kovi \'c . 2024. Round and round we go! what makes rotary positional encodings useful? arXiv preprint arXiv:2410.06205

    arXiv 2024

  4. [4]

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. 2024. Internlm2 technical report. arXiv preprint arXiv:2403.17297

    arXiv 2024

  5. [5]

    Junbum Cha, Wooyoung Kang, Jonghwan Mun, and Byungseok Roh. 2024. Honeybee: Locality-enhanced projector for multimodal llm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13817--13827

    work page 2024

  6. [6]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024 a . An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, pages 19--35. Springer

    2024

  7. [7]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. 2024 b . Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330

    arXiv 2024

  8. [8]

    Lin Chen and Long Xing. 2024. https://doi.org/10.5281/zenodo.13935471 Open-llava-next: An open-source implementation of llava-next series for facilitating the large multi-modal model community. https://github.com/xiaoachen98/Open-LLaVA-NeXT

    work page doi:10.5281/zenodo.13935471 2024

Show all 53 references
  1. [9]

    Zhanpeng Chen, Mingxiao Li, Ziyang Chen, Nan Du, Xiaolong Li, and Yuexian Zou. 2025. Advancing general multimodal capability of vision-language models with pyramid-descent visual position encoding. arXiv preprint arXiv:2501.10967

    2025 arXiv

  2. [10]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. 2024 c . Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271

    2024 arXiv

  3. [11]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024 d . How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101

    2024

  4. [12]

    Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. Nvlm: Open frontier-class multimodal llms. arXiv preprint arXiv:2409.11402

    2024 arXiv

  5. [13]

    Zihang Dai. 2019. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860

    2019 arXiv

  6. [14]

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. 2024. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv preprint arXiv:2409.17146

    2024 arXiv

  7. [15]

    Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  8. [16]

    Junqi Ge, Ziyi Chen, Jintao Lin, Jinguo Zhu, Xihui Liu, Jifeng Dai, and Xizhou Zhu. 2024. https://api.semanticscholar.org/CorpusID:274656197 V2pe: Improving multimodal long-context capability of vision-language models with variable visual position encoding . ArXiv, abs/2412.09616

    2024 arXiv

  9. [17]

    Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In International conference on machine learning, pages 1243--1252. PMLR

    2017

  10. [18]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904--6913

    2017

  11. [19]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654

    2020 arXiv

  12. [20]

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. A diagram is worth a dozen images. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part IV 14, pages 2...

    2016

  13. [21]

    Bohao Li, Yuying Ge, Yi Chen, Yixiao Ge, Ruimao Zhang, and Ying Shan. 2024. Seed-bench-2-plus: Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint arXiv:2404.16790

    2024 arXiv

  14. [22]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023 a . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR

    2023

  15. [23]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023 b . Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 292--305

    2023

  16. [24]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024 a . Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

    2024 arXiv

  17. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 b . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  18. [26]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 c . https://llava-vl.github.io/blog/2024-01-30-llava-next/ Llava-next: Improved reasoning, ocr, and world knowledge

    2024

  19. [27]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024 d . Visual instruction tuning. Advances in neural information processing systems, 36

    2024

  20. [28]

    Xuanqing Liu, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh. 2020. Learning to encode position for transformer with continuous dynamical model. In International conference on machine learning, pages 6327--6335. PMLR

    2020

  21. [29]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. 2024 e . Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216--233. Springer

    2024

  22. [30]

    Zhijian Liu, Ligeng Zhu, Baifeng Shi, Zhuoyang Zhang, Yuming Lou, Shang Yang, Haocheng Xi, Shiyi Cao, Yuxian Gu, Dacheng Li, et al. 2024 f . Nvila: Efficient frontier visual language models. arXiv preprint arXiv:2412.04468

    2024 arXiv

  23. [31]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2...

    2022

  24. [32]

    Xin Men, Mingyu Xu, Bingning Wang, Qingyu Zhang, Hongyu Lin, Xianpei Han, and Weipeng Chen. 2024. Base of rope bounds context length. arXiv preprint arXiv:2405.14591

    2024 arXiv

  25. [33]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. https://arxiv.org/abs/1609.07843 Pointer sentinel mixture models . Preprint, arXiv:1609.07843

    2016 arXiv

  26. [34]

    Ofir Press, Noah A Smith, and Mike Lewis. 2021. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint arXiv:2108.12409

    2021 arXiv

  27. [35]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  28. [36]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  29. [37]

    Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri, and Anh Totti Nguyen. 2024. Vision language models are blind. In Proceedings of the Asian Conference on Computer Vision, pages 18--34

    2024

  30. [38]

    Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. 2018. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155

    2018 arXiv

  31. [39]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063

    2024

  32. [40]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  33. [41]

    Benyou Wang, Donghao Zhao, Christina Lioma, Qiuchi Li, Peng Zhang, and Jakob Grue Simonsen. 2019. Encoding word order in complex embeddings. arXiv preprint arXiv:1912.12333

    2019 arXiv

  34. [42]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  35. [43]

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. 2024. Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302

    2024 arXiv

  36. [44]

    Yun Xing, Yiheng Li, Ivan Laptev, and Shijian Lu. 2025. Mitigating object hallucination via concentric causal attention. Advances in Neural Information Processing Systems, 37:92012--92035

    2025

  37. [45]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  38. [46]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. arXiv preprint arXiv:2306.13549

    2023 arXiv

  39. [47]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975--11986

    2023

  40. [48]

    Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, and Dong Yu. 2024 a . Mm-llms: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601

    2024 arXiv

  41. [49]

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2024 b . https://arxiv.org/abs/2407.12772 Lmms-eval: Reality check on the evaluation of large multimodal models . Preprint, arX...

    2024 arXiv

  42. [50]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  43. [51]

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479

    2025 arXiv

  44. [52]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.