Pith. sign in

REVIEW 3 major objections 5 minor 39 references

AdaTP: Attention-Debiased Token Pruning for Video Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AdaTP claims that correcting two systematic attention biases in Video LLMs lets a training-free pruning pipeline cut FLOPs to 27.3% of vanilla on LLaVA-OneVision-7B without performance loss.

desk verdict Solid training-free token pruning with strong numbers, but the LLaVA-Video baseline row looks copied, Eq. 4 is ambiguous, and there's no code yet. read the letter →

arxiv 2505.20100 v1 pith:DZVJ2PMM submitted 2025-05-26 cs.CV cs.AI

Fengyuan Sun , Leqi Shen , Hui Chen , Sicheng Zhao , Jungong Han , Guiguang Ding This is my paper
classification cs.CVcs.AI
keywords videolargelanguagemodelstokenpruningattentionbiastraining-freeaccelerationvisualcompressionLLaVA-OneVisionMMEFLOPsreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video LLMs answer questions about long videos by feeding every frame's visual tokens through a language model, and that token flood makes inference expensive. Existing attempts to prune tokens by raw attention scores lose accuracy, and this paper argues the reason is that the scores are systematically biased. AdaTP corrects two biases, a global bias that over-weights the first and last frames and a local bias that over-retains the same spatial patches across similar frames, and then prunes using the debiased scores. On LLaVA-OneVision-7B it keeps, and on several benchmarks slightly beats, vanilla accuracy while using as little as 27.3% of the original FLOPs, with no training required.

What carries the argument

The pipeline is AdaTP (Attention-Debiased Token Pruning). Its load-bearing pieces are a segmentation step and two debiasing modules. The Global Debiasing Module uses the text encoder aligned with the visual encoder: for frame $i$ it computes the cosine similarity $\mathrm{sim}_i = v_{p i}^\top t_p / (\|v_{p i}\| \|t_p\|)$ between the frame's global visual token $v_{p i}$ and the global textual token $t_p$; each segment's average similarity is compared with threshold $\tau_t$, and segments above it get a boosted retention ratio $\alpha_\mathrm{boost}$ capped by $\gamma_\mathrm{cap}$. The Local Debiasing Module sorts each segment's high-attention tokens by score and greedily keeps at most one token per spatial position, removing cross-frame duplicates. Frames are first partitioned into segments by adjacent-frame cosine similarity with threshold $\tau_s$, and pruning is progressive from layer 2 to layer $N-12$.

What would settle it

Construct a set of video questions whose answers lie in the middle third of the video and whose question text has low cosine similarity to those frames (for example, asking 'what color is the vehicle?' while the text embedding is dominated by other words). If AdaTP, run with the paper's settings, prunes those frames and scores below the vanilla model on that set, the global debiasing premise is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that attention scores from the language model of a Video LLM are not a trustworthy token-importance signal until two biases are removed. Global attention bias makes high scores cluster at the beginning and end of the visual token sequence regardless of content, while local attention bias makes shallow layers concentrate scores on fixed spatial positions across all frames. AdaTP first splits frames into coherent segments by adjacent-frame cosine similarity, then uses a Global Debiasing Module that computes cosine similarity between each frame's global visual token and the text token from the aligned text encoder to identify text-relevant segments and give them larger retention budgets, and a Local Debiasing Module that greedily keeps at most one token per spatial position per segment. With raw attention replaced by this debiased guidance, progressive layer-by-layer pruning retains accuracy. In experiments, AdaTP sets state-of-the-art results on VideoMME, MLVU, and LongVideoBench across LLaVA-OneVision-0.5B/7B and LLaVA-Video-7B, and on LLaVA-OneVision-7B it matches or exceeds the vanilla model while using as little as 27.3% of its FLOPs.

Load-bearing premise

The load-bearing premise is that the cosine similarity between a frame's global visual embedding and the question's text embedding reliably marks which frames contain the information needed to answer; if that similarity is noisy or misaligned, the global debiasing module can down-weight the wrong segments and the no-degradation claim breaks.

Editorial extensions

If this is right

  • At the strongest tested compression on LLaVA-OneVision-7B, AdaTP uses 27.3% of vanilla FLOPs and scores 59.51 average across VideoMME, MLVU, and LongVideoBench versus 59.34 for the unpruned model, so the no-degradation claim holds across the three benchmarks.
  • Because AdaTP is training-free and plug-and-play, it can accelerate an already-deployed Video LLM without fine-tuning or new data; the paper applies it without modification to LLaVA-OneVision-0.5B/7B and LLaVA-Video-7B.
  • The ablation shows that each component contributes: adding segmentation, then global debiasing, then local debiasing raises the average score from 45.04 to 46.53 at similar FLOPs, so the bias corrections, not just the pruning schedule, are what preserve accuracy.
  • Dynamic videos are handled automatically: more scene changes mean more segments, which receive a larger fraction of the token budget, while static videos are compressed more aggressively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the global debiasing signal is only as good as the aligned text-encoder similarity, so questions whose answer is visually salient but semantically distant from the question text may have their key frames down-weighted; a per-question-type breakdown would reveal where the no-degradation claim starts to bend.
  • Editorial inference: the same two-bias diagnosis suggests that other token-reduction mechanisms, such as attention-based merging or pruning in image MLLMs with similar positional attention sinks, could benefit from the same debiasing treatment.
  • Editorial inference: the paper reports small gains over vanilla at high compression; one testable extension is whether pruned models are also more robust to irrelevant or adversarial frames, since redundant-token removal may act as a regularizer.
  • Editorial inference: the four tuning parameters (segment threshold, significance threshold, boost factor, and cap ratio) are fixed globally across all videos; making them adaptive per video or per segment could push the FLOPs/accuracy frontier further.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. AdaTP is a training-free, plug-and-play token pruning pipeline for Video LLMs. The paper argues that raw LLM attention scores used for token pruning suffer from two biases: a global bias that concentrates attention on the first and last frames, and a local bias that over-retains tokens from the same spatial positions across adjacent similar frames. AdaTP addresses these with two modules: a Global Debiasing Module that uses cosine similarity between CLIP/SigLIP global visual tokens and the text encoding to divide the video into significant and non-significant segments and allocates higher retention ratios to significant segments, and a Local Debiasing Module that removes spatially duplicated tokens within each segment. The method is evaluated on LLaVA-OneVision-0.5B/7B and LLaVA-Video-7B across VideoMME, MLVU, and LongVideoBench, reporting FLOPs reductions to roughly 26–44% of the vanilla model while maintaining or slightly improving accuracy. The central claim is that on LLaVA-OneVision-7B AdaTP uses only 27.3% of the vanilla FLOPs without performance degradation.

Significance. If the method is sound and reproducible, AdaTP would be a useful practical contribution to efficient Video LLM inference: it is training-free, model-agnostic, and targets a well-documented failure mode of existing attention-based pruning methods. The paper provides clear visual evidence of the two biases (Figs. 2–3), a reasonable ablation study (Tab. 2), and an extensive hyperparameter sweep (Tab. 4). However, the paper's central algorithmic equations contain a load-bearing inconsistency (Eqs. 4–5), a likely copy-paste error in a key baseline row (Table 1), and an unspecified FLOPs accounting methodology. These issues currently prevent the reader from reproducing the reported 27.3% FLOPs / no-degradation result, so the contribution cannot be fully assessed as written.

major comments (3)
  1. [Sec. 3.3, Eqs. (4)–(5)] The token allocation formula is ill-posed for a common operating regime. Equation (4) contains an undefined term 'nP' (the projector P is a function, not a scalar) and the min term does not enforce the stated cap; the correct bound for r1 should be γcap·n/Σ_{s∈S'}||s||, not nP/(Σ_{s∈S'}||s||·γcap). With Eqs. (4)–(5) as written, whenever the fraction f of frames in significant segments exceeds 1/αboost (≈45% for αboost=2.2), the numerator of Eq. (5) becomes negative, giving a negative retention ratio r2 for non-significant segments. The paper does not specify any clipping, renormalization, or fallback for this case. Since this allocation is the core of the Global Debiasing Module, the reported 27.3% FLOPs / no-degradation result is not reproducible from the equations as printed; the authors must correct the formula, specify the exact allocation procedure (including any clipping), and ideally report the empirical distribution of f across benchmarks to show that the negative-r2 regime does not occur.
  2. [Table 1, LLaVA-Video-7B vanilla row] The vanilla row for LLaVA-Video-7B is identical to the vanilla row for LLaVA-OneVision-7B (70.0, 56.7, 48.7, 58.44, 63.25, 56.32, 59.34). Since these are different models with different frame sampling (20 frames with newline tokens vs. 32 frames), this is almost certainly a copy-paste error. The corrected vanilla baseline is required to support the claim that AdaTP 'maintains performance' or 'surpasses' vanilla on LLaVA-Video-7B, and the comparison against FastV/VisionZip/Dycoke for this model should be re-evaluated with the correct baseline.
  3. [Sec. 4.1, FLOPs measurement] The paper states that FLOPs are collected via torch.profiler but does not specify whether the overhead of the Global Debiasing Module is included in the reported ratios. AdaTP runs an additional text-encoder forward pass and per-frame cosine-similarity computations; if these costs are excluded, the reported '27.3% FLOPs' is not an end-to-end comparison with the vanilla model. Please provide a breakdown of the token-pruning FLOPs versus the debiasing-overhead FLOPs, and confirm that the reported numbers are total end-to-end FLOPs for the question-answering pipeline.
minor comments (5)
  1. [Sec. 3.3, Eq. (3)] Please define exactly how the 'global visual token' vp_i and the 'global textual token' tp are obtained from the CLIP/SigLIP encoders (e.g., [CLS] token, mean pooling, or a specific hidden state). This is needed for reproducibility.
  2. [Sec. 3.3, Algorithm 1] Please clarify whether the local deduplication step reduces the token count below the ratio allocated by Eqs. (4)–(5). If so, the actual retained token fraction is lower than r, and the relationship between the scheduled pruning rate p and the measured FLOPs should be stated explicitly.
  3. [Sec. 4.2] All results are reported from a single run without error bars or significance tests. Given that the 'no degradation' claim on LLaVA-OneVision-7B rests on differences of ~0.2 points (e.g., 59.51 vs. 59.34), please report variance across at least a few seeds or state that the differences are within run-to-run noise.
  4. [Appendix A.1] There is a typo: 'couduct' should be 'conduct'.
  5. [Eq. (4)] The notation 'nP' is unclear; if it is meant to be 'n·p' where p is the compression schedule parameter, or simply 'n', please correct it. As written it is undefined.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central FLOPs/accuracy claims are empirical results on independent benchmarks, not consequences of the method's equations or of self-citations.

full rationale

The paper is an empirical token-pruning pipeline, not a derivation. Its central claims—maintaining performance while using as little as 27.3% of vanilla FLOPs—are measured on external benchmarks (VideoMME, MLVU, LongVideoBench) under a stated, fixed hyperparameter configuration, and no equation in the paper defines those benchmark scores in terms of its inputs. The global-debiasing relevance score (Eq. 3) is a cosine similarity computed from the frozen, aligned CLIP/SigLIP text and visual encoders, which is independent of the evaluation answers; the retention ratios in Eqs. 4–5 are construction constraints that enforce a token budget, not predictions that could be circular. Hyperparameters such as tau_s, tau_t, alpha_boost, and gamma_cap are explicitly hand-set and then ablated, and the limitation section acknowledges the hyperparameter count. The self-citations (Shen et al. 2024, 2025a, 2025b; Wang et al. 2024a) appear only in the related-work survey of existing token-compression methods and carry no load-bearing argument for AdaTP's design. The concern raised about Eq. 5 becoming negative for large significant-segment fractions is a potential internal consistency or reproducibility issue, not a circularity in which a claimed prediction reduces by construction to a fitted input or to a self-citation. Therefore, under the provided rules, no circular step is present, and the appropriate score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces four tunable hyperparameters plus the compression multiplier p. It relies on assumptions about attention-score informativeness, text-visual alignment, intra-segment similarity, and the completeness of the FLOPs measurement. No new physical or conceptual entities are postulated.

free parameters (5)
  • tau_t = 0.05
    Threshold for segment-text relevance; chosen by ablations on LLaVA-OneVision-0.5B.
  • alpha_boost = 2.2
    Upscale factor for retention ratio in significant segments; manually selected.
  • gamma_cap = 0.75
    Maximum fraction of tokens allowed in significant segments; manually selected.
  • tau_s = 0.95
    Frame similarity threshold for segment partitioning; set high to ensure near-duplicate frames within a segment.
  • p = 1.0, 1.5, 2.0
    Compression multiplier controlling overall retention; three settings tested.
assumptions (4)
  • domain assumption Attention scores from the language model are a valid proxy for token importance.
    The whole pipeline uses attention scores to select tokens; if scores are not informative even after debiasing, pruning would lose important content.
  • domain assumption The text encoder and visual encoder are aligned so that cosine similarity between frame-level visual tokens and text tokens reflects relevance to the question.
    Eq. (3) uses this similarity to identify significant segments. If the alignment is poor, the global debiasing would mis-allocate tokens.
  • domain assumption Frames within a segment are similar enough that keeping one token per spatial position preserves all needed information.
    Algorithm 1 deduplicates by spatial position; if there is meaningful motion or lighting change within a segment, this could discard useful tokens.
  • domain assumption torch.profiler FLOPs measurement includes all added computations of the proposed pipeline.
    The paper reports FLOPs as a fraction of vanilla but does not specify whether text-encoder and similarity costs are counted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AdaTP: Attention-Debiased Token Pruning for Video Large Language Models." pith.science (2026). https://pith.science/paper/DZVJ2PMM

@misc{pith2026250520100,
  author       = {Pith},
  title        = {Pith review of: AdaTP: Attention-Debiased Token Pruning for Video Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DZVJ2PMM}},
  note         = {Machine review of arXiv:2505.20100}
}
abstract

Video Large Language Models (Video LLMs) have achieved remarkable results in video understanding tasks. However, they often suffer from heavy computational overhead due to the large number of visual tokens generated from multiple video frames. Existing visual token compression methods often rely on attention scores from language models as guidance. However, these scores exhibit inherent biases: global bias reflects a tendency to focus on the two ends of the visual token sequence, while local bias leads to an over-concentration on the same spatial positions across different frames. To address the issue of attention bias, we propose $\textbf{A}$ttention-$\textbf{D}$ebi$\textbf{a}$sed $\textbf{T}$oken $\textbf{P}$runing for Video Large Language Models ($\textbf{AdaTP}$), a novel token pruning pipeline for Video LLMs. AdaTP integrates two dedicated debiasing modules into the pipeline, targeting global attention bias and local attention bias, respectively. Without the need for additional training, our method significantly reduces the computational overhead of Video LLMs while retaining the performance of vanilla models. Extensive evaluation shows that AdaTP achieves state-of-the-art performance in various commonly used video understanding benchmarks. In particular, on LLaVA-OneVision-7B, AdaTP maintains performance without degradation while using only up to $27.3\%$ FLOPs compared to the vanilla model. Our code will be released soon.

Figures

Figures reproduced from arXiv: 2505.20100 by the authors.

Figure 1
Figure 1. Visualization of attention bias. The visual tokens with top-10% attention scores are highlighted in red and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Visualization of local attention bias in Layer [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Illustration of our AdaTP pipeline. AdaTP mitigates attention bias by selectively retaining critical tokens [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Visualization example sampled from the VideoMME dataset. We adopt attention scores from layer 1 to [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Additional visualizations of local attention bias across different layers. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Additional visualizations of global attention bias across different layers. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 2 canonical work pages

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, and 1 others. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716--23736

    work page 2022

  2. [2]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024 a . An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, pages 19--35. Springer

    2024

  3. [3]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, and 1 others. 2024 b . Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271

    arXiv 2024

  4. [4]

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, and 1 others. 2024. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. arXiv preprint arXiv:2405.21075

    arXiv 2024

  5. [5]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. https://arxiv.org/abs/2408.03326 Llava-onevision: Easy visual task transfer . arXiv preprint arXiv:2408.03326

    arXiv 2024

  6. [6]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730--19742. PMLR

    2023

  7. [7]

    Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie. 2022. Not all patches are what you need: Expediting vision transformers via token reorganizations. arXiv preprint arXiv:2202.07800

    arXiv 2022

  8. [8]

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. 2023. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122

    arXiv 2023

Show all 39 references
  1. [9]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  2. [10]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 b . Llavanext: Improved reasoning, ocr, and world knowledge

    2024

  3. [11]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  4. [12]

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. 2023. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424

    2023 arXiv

  5. [13]

    Dmitrii Marin, Jen-Hao Rick Chang, Anurag Ranjan, Anish Prabhu, Mohammad Rastegari, and Oncel Tuzel. 2021. Token pooling in vision transformers. arXiv preprint arXiv:2110.03860

    2021 arXiv

  6. [14]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://api.semanticscholar.org/CorpusID:231591445 Learning transferable visual model...

    2021

  7. [15]

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. 2021. Dynamicvit: Efficient vision transformers with dynamic token sparsification. Advances in neural information processing systems, 34:13937--13949

    2021

  8. [16]

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. 2024. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. arXiv preprint arXiv:2403.15388

    2024

  9. [17]

    Leqi Shen, Guoqiang Gong, Tao He, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, and Guiguang Ding. 2025 a . Fastvid: Dynamic density pruning for fast video large language models. arXiv preprint arXiv:2503.11187

    2025

  10. [18]

    Leqi Shen, Tianxiang Hao, Tao He, Sicheng Zhao, Yifeng Zhang, Pengzhang Liu, Yongjun Bao, and Guiguang Ding. 2024. Tempme: Video temporal token merging for efficient text-video retrieval. arXiv preprint arXiv:2409.01156

    2024 arXiv

  11. [19]

    Leqi Shen, Tao He, Guoqiang Gong, Fan Yang, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, and Guiguang Ding. 2025 b . Llava-mlb: Mitigating and leveraging attention bias for training-free video llms. arXiv preprint arXiv:2503.11205

    2025 arXiv

  12. [20]

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. 2024. Dycoke: Dynamic compression of tokens for fast video large language models. arXiv preprint arXiv:2411.15024

    2024 arXiv

  13. [21]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    2023 arXiv

  14. [22]

    Ao Wang, Fengyuan Sun, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. 2024 a . [cls] token tells everything needed for training-free efficient mllms. arXiv preprint arXiv:2412.05819

    2024 arXiv

  15. [23]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024 b . Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  16. [24]

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. 2024. Longvideobench: A benchmark for long-context interleaved video-language understanding. Advances in Neural Information Processing Systems, 37:28828--28857

    2024

  17. [25]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2023. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453

    2023 arXiv

  18. [26]

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and 1 others. 2024. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. arXiv preprint arXiv:2410.17247

    2024 arXiv

  19. [27]

    Lin Xu, Yilin Zhao, Daquan Zhou, Zhijie Lin, See Kiong Ng, and Jiashi Feng. 2024. Pllava: Parameter-free llava extension from images to videos for video dense captioning. arXiv preprint arXiv:2404.16994

    2024 arXiv

  20. [28]

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. 2024. Visionzip: Longer is better but not necessary in vision language models. arXiv preprint arXiv:2412.04467

    2024

  21. [29]

    Hongxu Yin, Arash Vahdat, Jose M Alvarez, Arun Mallya, Jan Kautz, and Pavlo Molchanov. 2022. A-vit: Adaptive tokens for efficient vision transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10809--10818

    2022

  22. [30]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975--11986

    2023

  23. [31]

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and 1 others. 2024 a . Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407.12772

    2024 arXiv

  24. [32]

    Qizhe Zhang, Aosong Cheng, Ming Lu, Zhiyong Zhuo, Minqi Wang, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. 2024 b . [cls] attention is all you need for training-free visual token pruning: Make vlm inference faster. arXiv preprint arXiv:2412.01818

    2024 arXiv

  25. [33]

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. 2024 c . Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713

    2024 arXiv

  26. [34]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher R \'e , Clark Barrett, and 1 others. 2023. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information...

    2023

  27. [35]

    Yiwu Zhong, Zhuoming Liu, Yin Li, and Liwei Wang. 2024. Aim: Adaptive inference of multi-modal llms via token merging and pruning. arXiv preprint arXiv:2412.03248

    2024 arXiv

  28. [36]

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. 2024. Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264

    2024 arXiv

  29. [37]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

    2023 arXiv

  30. [38]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.