Pith. sign in

REVIEW 3 major objections 5 minor 36 references

Wavelet-based temporal and spatial token condensation preserves 99.6% of full-token video question-answering accuracy at 10x compression without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Wavelet-based temporal and spatial token condensation preserves 99.6% of full-token video question-answering accuracy at 10x compression without retraining.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A real training-free token condensation method with a solid empirical core, but the abstract overclaims and the scorer-alignment assumption deserves scrutiny. the 3 major comments →

arxiv 2607.23265 v2 pith:KAOWRQ6J submitted 2026-07-25 cs.CV

WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation

Yuhui Zeng , Wang Chen , Jinfa Huang , Tianyu Xie , Yongdong Luo , Jiayi Ji , Xiawu Zheng , jiebo Luo This is my paper
classification cs.CV
keywords waveziphigh-frequencyvideospatialcoefficientscomponentsefficientinference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video language models turn each frame into a grid of tokens and then run attention over all of them at once. Long videos mean tens of thousands of tokens, and attention grows quadratically, so inference becomes slow. Existing shortcuts either throw away tokens outright (hard pruning) or average nearby tokens uniformly (merging); both hurt fine details. WaveZip instead splits each video signal into low-frequency structure and high-frequency detail using wavelet transforms, because temporal redundancy lives in slow trends and the task-relevant visual details are concentrated in fast-changing high-frequency components.

For time, WaveZip takes frame-level relevance scores from a frozen image-text scorer, applies a one-dimensional wavelet transform, dampens high-frequency fluctuations in static segments, and reconstructs a stabilized score. That score decides how many tokens each frame gets. For space, a two-dimensional wavelet transform splits each frame's features into coarse and detail bands; the detail bands are reweighted by saliency from the scorer, then reconstructed. The final frame features are squeezed to the allocated token counts and fed into the unmodified language model.

On three long-video benchmarks, at 10% retained tokens, WaveZip reports 99.6% of full-token average accuracy, and it beats several pruning/merging baselines on average across ratios. Ablations show both temporal and spatial modules contribute. Caveats: no code is released, the frame-filtering threshold is never stated, and all numbers are single-run with no error bars, so the true margins are uncertain.

Core claim

Under an extreme 10x compression ratio (retained ratio ρ=0.1), WaveZip retains 99.6% of the full-token average performance (58.8 vs 59.0 on EgoSchema-500, LongVideoBench, and VideoMME with LLaVA-OneVision-7B), with no task-specific training, and the abstract asserts it 'consistently outperforming state-of-the-art methods.'

Load-bearing premise

WaveZip assumes the frozen BLIP-ITM cross-modal scorer's frame-level relevance r_t and spatial saliency M_t are faithful proxies for what the target LVLM actually needs, since these signals alone drive both the temporal token budget (Eq. 16) and the spatial modulation gate (Eq. 14). If the scorer's relevance ranking misaligns with the LVLM's attention, the 10x compression will cut evidence the model would have used. This enters at §3.2.1 (Cross-Modal Scorer) and is not verified against the LVLM's own attention.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. WaveZip is a training-free video token condensation method for LVLMs. It uses a frozen BLIP-ITM cross-modal scorer to derive frame-level relevance scores r_t and spatial saliency maps M_t; a Visual Change Estimator filters frames below a threshold τ; a wavelet temporal allocator rectifies r_t via 1D DWT with visual-change gating; a wavelet spatial modulator reweights high-frequency 2D DWT subbands by saliency; and a token assigner distributes a global budget B = ρN_v among retained frames. Experiments on EgoSchema-500, LongVideoBench, VideoMME, and LVBench across LLaVA-OneVision-7B, LLaVA-Video-7B, and Qwen2.5-VL report that WaveZip retains 99.6% of full-token average accuracy at ρ=0.1 and claims consistent average gains over state-of-the-art baselines. The paper includes controlled module ablations, wavelet-basis ablations, an efficiency analysis, and a query-type breakdown.

Significance. If the central empirical claim holds, the method is practically significant: it is training-free, plug-and-play, and shows consistent average gains across three backbones and several benchmarks, including an additional LVBench generalization test. The controlled ablations in Appendix B (Fig. 8) are a genuine strength, as are the honest query-type breakdown in Appendix E and the component-level runtime breakdown in Appendix F. However, the headline result is an average over a small benchmark suite, per-benchmark gains are not uniform, all numbers are single-run with no error bars, and the central cross-modal proxy assumption is not directly validated. These gaps currently under-support the strength of the claims.

major comments (3)
  1. [§4.2, Table 1] The headline 'retains 99.6%' and the claimed consistent improvement rest on single-run accuracy differences as small as 0.1–1.1 points. For a 500-sample subset, a 1-point difference is well within binomial sampling error; even on the full benchmarks, a 0.9-point average gap between WaveZip (58.8) and VFlowOpt (57.9) at ρ=0.1 is not interpretable without variance. Please report standard errors or paired bootstrap intervals over benchmark items (ideally with multiple seeds). Also, the abstract's 'consistently outperforming' is only true for the average: at ρ=0.1, WaveZip is below DyCoke on EgoSchema (62.3 vs 63.0) and below FastVID on LongVideoBench (54.7 vs 55.7); at ρ=0.1 on Qwen2.5-VL it is 3.6 points below VisionZip on EgoSchema (Table 2). The average claim and the per-benchmark claim should be reported separately and the wording adjusted.
  2. [§3.2.2, Eq. (8); §4.1 setup] The Frame Filter threshold τ is a free parameter that is never reported, ablated, or given a selection criterion. It determines the retained frame set M and directly changes both the temporal budget allocation (Eq. 16) and how the global budget B is spread across retained frames. Since λ is ablated and wavelet basis pairs are ablated, the omission of τ is conspicuous. If τ is nonzero, the effective number of frames can be much smaller than T and the actual per-frame token density changes; if τ=0, the VCE gate still affects WTA but the filter is vacuous. Please report the default τ, state whether it is fixed or tuned per benchmark, and provide a sensitivity sweep. Without this, the 10× compression claim is not reproducible.
  3. [§3.2.1, Table 4, Appendix E] The entire compression mechanism assumes the frozen BLIP-ITM cross-modal scores r_t and M_t are faithful proxies for what the target LVLM needs. Table 4 only replaces the scorer with another off-the-shelf scorer or with base-LVLM similarity; it does not measure agreement with the target model's own per-token attention, gradients, or an oracle relevance. The category breakdown in Fig. 10 is telling: at ρ=0.2 the largest losses are Action Recognition (−3.83), Object Reasoning (−2.86), Temporal Reasoning (−2.26), and OCR (−2.16), exactly the categories where a global image-text matching model should be most suspect. To support the claim that WaveZip preserves query-relevant evidence, I would expect at minimum a comparison of the frames/tokens selected by WaveZip against those selected by the target LVLM's own attention, or an oracle-ablation showing the ceiling. Without this, the central as
minor comments (5)
  1. [§3.1] Typo: 'ues' should be 'uses'.
  2. [§4.2] The text says 'ρ=0.2%' when referring to a retained ratio of 20%; this should be 'ρ=0.2' or '20%'.
  3. [Table 4] The retained ratio ρ is not stated in the table or its caption, making the cross-modal scorer comparison impossible to interpret. Please specify the ρ value and evaluation subset.
  4. [Abstract and §4.1] The abstract says '10× compression ratio' while §4.1 defines the retained ratio ρ. Please reconcile: does 10× mean ρ=0.1? Clarify the direction of the ratio to avoid ambiguity.
  5. [Figs. 3, 5, 8] These ablation figures report single-run accuracies with no error bars or significance information. Given the small observed differences, adding error bars or at least stating the number of samples would substantially improve interpretability.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical entities. Its epistemic load is carried by a frozen external scorer, an unreported threshold, and a few hand-selected hyperparameters; the central claim is empirical rather than derived.

free parameters (3)
  • λ (WSM modulation strength) = 1.5 (default)
    Controls the saliency gate amplitude in Eq. 13; ablated over {0.5,1.0,1.5,2.0} on the same evaluation benchmarks and fixed to 1.5. The paper shows insensitivity but the value is still hand-selected.
  • τ (VCE frame filter threshold) = not reported
    Frames with inter-frame cosine distance δ_t < τ are dropped (§3.2.2). The paper calls τ 'predefined' but never gives its value, so the number of retained frames and thus budget distribution under a fixed ρ is not reproducible.
  • Wavelet basis pairs (WTA/WSM) = Haar/db4 default
    Ablated in Fig. 3 across Haar, db4, bior4.4; default chosen without a clear winner. Basis choice is a hand-selected design decision.
axioms (4)
  • standard math Single-level 1D/2D DWT with Haar/db4 gives a localized, invertible multi-band decomposition supporting perfect reconstruction.
    Used in §3.2.3 Eqs (9)-(15); standard wavelet theory, but boundary effects in practice are ignored.
  • domain assumption Query-conditioned visual saliency correlates more strongly with high-frequency wavelet detail bands than with the low-frequency LL band.
    Motivates WSM (Eq. 14); supported by Fig. 7 Pearson correlations, but measurement details and sample sizes are not given.
  • domain assumption The frozen BLIP-ITM cross-modal scorer supplies relevance scores and saliency maps that faithfully rank the evidence the target LVLM needs.
    All budget allocation and spatial modulation flow from these scores (§3.2.1, Eqs. 14, 16); the paper tests three scorers with similar outcomes, but no analyzer-LVLM alignment analysis is provided.
  • domain assumption Averaged accuracy across EgoSchema (500-subset), LongVideoBench, and VideoMME is a meaningful proxy for 'full performance'.
    The 99.6% headline rests on a three-benchmark average; subset sizes and evaluation protocols vary, and no confidence intervals are reported.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation." pith.science (2026). https://pith.science/paper/KAOWRQ6J

@misc{pith2026260723265,
  author       = {Pith},
  title        = {Pith review of: WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KAOWRQ6J}},
  note         = {Machine review of arXiv:2607.23265}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Existing Large Vision-Language Models (LVLMs) struggle with long-form video understanding due to the quadratic computational cost of visual tokens. While recent efficient methods attempt to compress tokens via hard pruning or uniform merging, they operate strictly in the spatial feature domain, where robust structural context and discriminative semantic details are inherently entangled. In this work, we propose WaveZip, a joint signal-frequency-domain framework for efficient video inference. Driven by the insight that temporal redundancy resides in low-pass approximation scales while spatial saliency strongly correlates with high-frequency components, WaveZip leverages Discrete Wavelet Transforms (DWT) to disentangle these signals. Temporally, it employs 1D DWT to analyze query-frame relevance, and the resulting high-frequency coefficients are further gated by inter-frame differences, with both signals jointly driving the dynamic allocation of a precise frame-level token budget. Spatially, a 2D DWT decomposes features into low-frequency approximations and high-frequency detail components, where the high-frequency coefficients are modulated within query-salient regions to regulate spatial reconstruction. Importantly, WaveZip requires no task-specific training and can be seamlessly integrated into off-the-shelf LVLMs to boost inference efficiency. Extensive experiments on long video understanding benchmarks demonstrate that WaveZip retains 99.6% of the full performance under an extreme 10x compression ratio, consistently outperforming state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2607.23265 by Jiayi Ji, Jiebo Luo, Jinfa Huang, Tianyu Xie, Wang Chen, Xiawu Zheng, Yongdong Luo, Yuhui Zeng.

Figure 1
Figure 1. Figure 1: Comparison of token compression strategies. Un￾like hard selection or uniform merging, WaveZip performs query￾conditioned token condensation jointly across time and space, bal￾ancing temporal dynamics, spatial structure, and fine-grained de￾tails. Long videos produce thousands of visual tokens across space and time, causing the transformer attention to scale quadratically with token length. Consequently, e… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of WaveZip. The pipeline computes cross-modal relevance, rectifies temporal scores via WTA (Wavelet Temporal Allocator), preserves spatial saliency via WSM (Wavelet Spatial Modulator), and performs budget-guided compression. where v¯t denotes the spatially average-pooled feature of frame Xt. Applying min-max normalization over the temporal dimension yields the gating factor γt = Nmin-max(δt) ∈ [0,… view at source ↗
Figure 3
Figure 3. Figure 3: Ablation of wavelet bases in WTA and WSM on VideoMME with LLaVA-OV. Each group on the x-axis denotes a basis pair assigned to WTA and WSM, respectively (h: Haar, d: db4, b: bior4.4). Each dot represents the accuracy at a specific retained ratio (ρ ∈ {0.10, 0.15, 0.20, 0.25}), and the vertical seg￾ment spans the min–max range across all four ratios. Generalization across LVLM backbones [PITH_FULL_IMAGE:fig… view at source ↗
Figure 3
Figure 3. Figure 3: Ablation of wavelet bases in WTA and WSM on VideoMME with LLaVA-OV. Each group on the x-axis denotes a basis pair assigned to WTA and WSM, respectively (h: Haar, d: db4, b: bior4.4). Each dot represents the accuracy at a specific retained ratio (ρ ∈ {0.10, 0.15, 0.20, 0.25}), and the vertical seg￾ment spans the min–max range across all four ratios. 4.3. Ablation Study Wavelet Basis. We evaluate three wavel… view at source ↗
Figure 4
Figure 4. Figure 4: Ablation of core modules in WaveZip. Uniform denotes the uniform compression baseline, WTA adds only the Wavelet Temporal Allocator, and Full further adds the Wavelet Spatial Modulator. Results are reported at two retained ratios, ρ = 0.1 and ρ = 0.2. VideoMME LongVideoBench EgoSchema 30 40 50 60 70 Accuracy (%) Modulation Scale (λ) 0.5 1.0 1.5 2.0 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ablation of core modules in WaveZip. Uniform denotes the uniform compression baseline, WTA adds only the Wavelet Temporal Allocator, and Full further adds the Wavelet Spatial Modulator. Results are reported at two retained ratios, ρ = 0.1 and ρ = 0.2. VideoMME LongVideoBench EgoSchema 30 40 50 60 70 Accuracy (%) Modulation Scale (λ) 0.5 1.0 1.5 2.0 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation of the WSM modulation scale λ on LLaVA￾OV. Bars report accuracy for λ ∈ {0.5, 1.0, 1.5, 2.0}, showing that performance is relatively stable with only minor variations. uniform compression by introducing frequency-rectified temporal budget allocation. Adding WSM provides clear Method 16 Frames 32 Frames 96 Frames 128 Frames FastV 48.8 53.1 OOM OOM VisionZip 47.1 47.6 48.6 OOM FastVID 54.5 57.3 59.9… view at source ↗
Figure 5
Figure 5. Figure 5: Ablation of the WSM modulation scale λ on LLaVA￾OV. Bars report accuracy for λ ∈ {0.5, 1.0, 1.5, 2.0}, showing that performance is relatively stable with only minor variations. sensitivity to the specific wavelet family. This near-uniform pattern suggests that no single wavelet family offers a sys￾tematic advantage, and the performance gains of WaveZip arise primarily from the multi-band decomposition mech… view at source ↗
Figure 7
Figure 7. Figure 7: Frequency-band saliency analysis. Wavelet-band energy maps and Pearson correlation statistics show that query￾conditioned saliency aligns more strongly with high-frequency detail bands than with the low-frequency LL band, motivating saliency-guided high-frequency modulation in WSM. 4.4. Efficiency Analysis We evaluate the end-to-end inference efficiency of different token compression methods with LLaVA-Vid… view at source ↗
Figure 7
Figure 7. Figure 7: Frequency-band saliency analysis. Wavelet-band energy maps and Pearson correlation statistics show that query￾conditioned saliency aligns more strongly with high-frequency detail bands than with the low-frequency LL band, motivating saliency-guided high-frequency modulation in WSM. frequency approximation unchanged. 5. Conclusion In this paper, we present WaveZip, a training-free frame￾work that formulates… view at source ↗
Figure 6
Figure 6. Figure 6: End-to-end accuracy–efficiency trade-off across three benchmarks with LLaVA-Video-7B. Average latency and peak GPU memory are measured separately on a fixed 600-sample subset constructed by sampling 200 examples from each bench￾mark. Results are reported at ρ = 0.1 and ρ = 0.2, with marker size indicating peak GPU memory. Latency covers input prepro￾cessing, cross-modal scoring, token compression, LVLM pre… view at source ↗
Figure 8
Figure 8. Figure 8: Controlled mechanism ablation on VideoMMEwith LLaVA-Video-7B. We compare Raw Top-k vs. WTA for tem￾poral allocation and Saliency-Only vs. WSM for spatial modula￾tion under the same scorer, token budget, backbone, and evaluation subset. All configurations use 64 sampled frames and ρ = 0.1. -50 0 50 PC1 (29.8% var.) -40 -20 0 20 PC2 (11.1% var.) Frame Feature PCA Before/After Wavelet Before Wavelet After Wav… view at source ↗
Figure 8
Figure 8. Figure 8: Controlled mechanism ablation on VideoMMEwith LLaVA-Video-7B. We compare Raw Top-k vs. WTA for tem￾poral allocation and Saliency-Only vs. WSM for spatial modula￾tion under the same scorer, token budget, backbone, and evaluation subset. All configurations use 64 sampled frames and ρ = 0.1. -50 0 50 PC1 (29.8% var.) -40 -20 0 20 PC2 (11.1% var.) Frame Feature PCA Before/After Wavelet Before Wavelet After Wav… view at source ↗
Figure 9
Figure 9. Figure 9: PCA visualization of frame-level features before and after WSM. We project the original features and the WSM￾modulated features into the same PCA space. The first two princi￾pal components explain 29.8% and 11.1% of the variance, respec￾tively. with WSM. This indicates that the temporal gain does not come from naive relevance ranking, but from wavelet-based relevance rectification. Similarly, WSM improves … view at source ↗
Figure 9
Figure 9. Figure 9: PCA visualization of frame-level features before and after WSM. We project the original features and the WSM￾modulated features into the same PCA space. The first two princi￾pal components explain 29.8% and 11.1% of the variance, respec￾tively. with WSM. This indicates that the temporal gain does not come from naive relevance ranking, but from wavelet-based relevance rectification. Similarly, WSM improves … view at source ↗
Figure 10
Figure 10. Figure 10: Query-type analysis on VideoMME. We compare the full-input baseline and WaveZip at ρ = 0.2. The number shown for each category is the WaveZip accuracy, and the value in paren￾theses denotes the difference relative to the baseline. creasing accuracy from 59.3% to 60.3% with Raw Top-k and from 59.8% to 61.1% with WTA. This confirms that the spatial gain comes from frequency-aware high-frequency modulation r… view at source ↗
Figure 10
Figure 10. Figure 10: Query-type analysis on VideoMME. We compare the full-input baseline and WaveZip at ρ = 0.2. The number shown for each category is the WaveZip accuracy, and the value in paren￾theses denotes the difference relative to the baseline. creasing accuracy from 59.3% to 60.3% with Raw Top-k and from 59.8% to 61.1% with WTA. This confirms that the spatial gain comes from frequency-aware high-frequency modulation r… view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 16 linked inside Pith

  1. [1]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 1, 2, 5, 6

    Pith/arXiv arXiv 2025

  2. [2]

    To- ken merging: Your vit but faster.arXiv preprint arXiv:2210.09461, 2022

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. To- ken merging: Your vit but faster.arXiv preprint arXiv:2210.09461, 2022. 2

    Pith/arXiv arXiv 2022

  3. [3]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. InEuropean Conference on Computer Vision, pages 19–35. Springer, 2024. 2, 5, 6

    2024

  4. [4]

    Flashvid: Efficient video large lan- guage models via training-free tree-based spatiotemporal to- ken merging.arXiv preprint arXiv:2602.08024, 2026

    Ziyang Fan, Keyu Chen, Ruilong Xing, Yulin Li, Li Jiang, and Zhuotao Tian. Flashvid: Efficient video large lan- guage models via training-free tree-based spatiotemporal to- ken merging.arXiv preprint arXiv:2602.08024, 2026. 6

    arXiv 2026

  5. [5]

    Video-mme: The first-ever 8 comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever 8 comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 24108–24118, 2025. 5

    2025

  6. [6]

    Tosa: To- ken merging with spatial awareness.arXiv preprint arXiv:2506.20066, 2025

    Hsiang-Wei Huang, Wenhao Chai, Kuang-Ming Chen, Cheng-Yen Yang, and Jenq-Neng Hwang. Tosa: To- ken merging with spatial awareness.arXiv preprint arXiv:2506.20066, 2025. 2

    Pith/arXiv arXiv 2025

  7. [7]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 1, 2

    Pith/arXiv arXiv 2024

  8. [8]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 2, 5

    Pith/arXiv arXiv 2024

  9. [9]

    Blip: Bootstrapping language-image pre-training for uni- fied vision-language understanding and generation.arXiv preprint arXiv:2201.12086, 2022

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for uni- fied vision-language understanding and generation.arXiv preprint arXiv:2201.12086, 2022. 5

    Pith/arXiv arXiv 2022

  10. [10]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InIn- ternational conference on machine learning, pages 19730– 19742. PMLR, 2023. 2

    2023

  11. [11]

    Videochat: Chat-centric video understanding.arXiv preprint arXiv:2305.06355, 2023

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding.arXiv preprint arXiv:2305.06355, 2023. 1

    Pith/arXiv arXiv 2023

  12. [12]

    Llama-vid: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, pages 323–340. Springer, 2024

    2024

  13. [13]

    Video-llava: Learning united visual repre- sentation by alignment before projection

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual repre- sentation by alignment before projection. InProceedings of the 2024 Conference on Empirical Methods in Natural Lan- guage Processing, pages 5971–5984, 2024

    2024

  14. [14]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 2

    2023

  15. [15]

    Hiprune: Training-free visual token pruning via hierarchical attention in vision-language models.arXiv preprint arXiv:2508.00553, 2025

    Jizhihui Liu, Feiyi Du, Guangdao Zhu, Niu Lian, Jun Li, and Bin Chen. Hiprune: Training-free visual token pruning via hierarchical attention in vision-language models.arXiv preprint arXiv:2508.00553, 2025. 2

    Pith/arXiv arXiv 2025

  16. [16]

    St-llm: Large language models are effective tem- poral learners

    Ruyang Liu, Chen Li, Haoran Tang, Yixiao Ge, Ying Shan, and Ge Li. St-llm: Large language models are effective tem- poral learners. InEuropean Conference on Computer Vision, pages 1–18. Springer, 2024. 2

    2024

  17. [17]

    Quota: Query-oriented token assign- ment via cot query decouple for long video comprehension

    Yongdong Luo, Wang Chen, Weizhong Huang, Shukang Yin, Haojia Lin, Jinfa Huang, Chaoyou Fu, Jiayi Ji, Xiawu Zheng, and Jiebo Luo. Quota: Query-oriented token assign- ment via cot query decouple for long video comprehension. InProceedings of the AAAI Conference on Artificial Intelli- gence, pages 24160–24168, 2026. 2

    2026

  18. [18]

    Egoschema: A diagnostic benchmark for very long- form video language understanding.Advances in Neural In- formation Processing Systems, 36:46212–46244, 2023

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic benchmark for very long- form video language understanding.Advances in Neural In- formation Processing Systems, 36:46212–46244, 2023. 5

    2023

  19. [19]

    Llava-prumerge: Adaptive token reduction for efficient large multimodal models

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22857– 22867, 2025. 2, 5

    2025

  20. [20]

    Holitom: Holistic token merging for fast video large language models.arXiv preprint arXiv:2505.21334,

    Kele Shao, Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Holitom: Holistic token merging for fast video large language models.arXiv preprint arXiv:2505.21334,

    arXiv

  21. [21]

    Tempme: Video temporal token merging for efficient text- video retrieval.arXiv preprint arXiv:2409.01156, 2024

    Leqi Shen, Tianxiang Hao, Tao He, Sicheng Zhao, Yifeng Zhang, Pengzhang Liu, Yongjun Bao, and Guiguang Ding. Tempme: Video temporal token merging for efficient text- video retrieval.arXiv preprint arXiv:2409.01156, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Fastvid: Dynamic density pruning for fast video large language mod- els.arXiv preprint arXiv:2503.11187, 2025

    Leqi Shen, Guoqiang Gong, Tao He, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, and Guiguang Ding. Fastvid: Dynamic density pruning for fast video large language mod- els.arXiv preprint arXiv:2503.11187, 2025. 2, 5, 6

    arXiv 2025

  23. [23]

    Dycoke: Dynamic compression of tokens for fast video large language models

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Dycoke: Dynamic compression of tokens for fast video large language models. InProceedings of the Com- puter Vision and Pattern Recognition Conference, pages 18992–19001, 2025. 2, 5

    2025

  24. [24]

    LOOK-M: Look-once optimization in KV cache for efficient multi- modal long-context inference

    Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, and Li Yuan. LOOK-M: Look-once optimization in KV cache for efficient multi- modal long-context inference. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 4065– 4078, Miami, Florida, USA, 2024. Association for Compu- tational Linguistics. 2

    2024

  25. [25]

    Fourier-vlm: Compressing vision tokens in the frequency domain for large vision-language models.arXiv preprint arXiv:2508.06038, 2025

    Huanyu Wang, Jushi Kai, Haoli Bai, Lu Hou, Bo Jiang, Zi- wei He, and Zhouhan Lin. Fourier-vlm: Compressing vision tokens in the frequency domain for large vision-language models.arXiv preprint arXiv:2508.06038, 2025. 2

    Pith/arXiv arXiv 2025

  26. [26]

    Internvl3

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 1

    Pith/arXiv arXiv 2025

  27. [27]

    Lvbench: An extreme long video understanding benchmark

    Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiao- han Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yux- iao Dong, Ming Ding, and Jie Tang. Lvbench: An extreme long video understanding benchmark. InProceedings of the IEEE/CVF International Conference on Computer Vision,

  28. [28]

    Longvideobench: A benchmark for long-context interleaved video-language understanding.Advances in Neural Informa- tion Processing Systems, 37:28828–28857, 2024

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-language understanding.Advances in Neural Informa- tion Processing Systems, 37:28828–28857, 2024. 5

    2024

  29. [29]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy re- duction.arXiv preprint arXiv:2410.17247, 2024

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy re- duction.arXiv preprint arXiv:2410.17247, 2024. 2

    Pith/arXiv arXiv 2024

  30. [30]

    Visionzip: Longer 9 is better but not necessary in vision language models

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer 9 is better but not necessary in vision language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19792–19802, 2025. 2, 5, 6

    2025

  31. [31]

    Vflowopt: A token pruning frame- work for lmms with visual information flow-guided opti- mization

    Sihan Yang, Runsen Xu, Chenhang Cui, Tai Wang, Dahua Lin, and Jiangmiao Pang. Vflowopt: A token pruning frame- work for lmms with visual information flow-guided opti- mization. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 23924–23934, 2025. 2, 5

    2025

  32. [32]

    Wave-vit: Unifying wavelet and transformers for visual representation learning

    Ting Yao, Yingwei Pan, Yehao Li, Chong-Wah Ngo, and Tao Mei. Wave-vit: Unifying wavelet and transformers for visual representation learning. InEuropean conference on computer vision, pages 328–345. Springer, 2022. 2

    2022

  33. [33]

    A glimpse to compress: Dynamic visual token prun- ing for large vision-language models.arXiv preprint arXiv:2508.01548, 2025

    Quan-Sheng Zeng, Yunheng Li, Qilong Wang, Peng-Tao Jiang, Zuxuan Wu, Ming-Ming Cheng, and Qibin Hou. A glimpse to compress: Dynamic visual token prun- ing for large vision-language models.arXiv preprint arXiv:2508.01548, 2025. 2

    Pith/arXiv arXiv 2025

  34. [34]

    Video-llama: An instruction-tuned audio-visual language model for video un- derstanding.arXiv preprint arXiv:2306.02858, 2023

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video un- derstanding.arXiv preprint arXiv:2306.02858, 2023. 1, 2

    Pith/arXiv arXiv 2023

  35. [35]

    Beyond attention or similarity: Maximizing conditional diversity for token pruning in mllms.arXiv preprint arXiv:2506.10967,

    Qizhe Zhang, Mengzhen Liu, Lichen Li, Ming Lu, Yuan Zhang, Junwen Pan, Qi She, and Shanghang Zhang. Beyond attention or similarity: Maximizing conditional diversity for token pruning in mllms.arXiv preprint arXiv:2506.10967,

    Pith/arXiv arXiv

  36. [36]

    Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Zi- wei Liu, and Chunyuan Li. Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024. 1, 2, 5, 6 10 Appendix Appendix Contents A Supplemental Experimental Protocol B Controlled Wavelet Ablations C Additional Long-Video Generalization Results D Feature Distribution Analysis of ...

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.