Pith. sign in

REVIEW 3 major objections 5 minor 94 references

Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Long-video MLLMs can select their own query-relevant frames by reading their internal attention, cutting visual tokens by about half and gaining a 3.9x end-to-end speedup while improving accuracy over prior selectors.

desk verdict Genuinely novel evidence-driven selection mechanism, but the paper omits the one control that would separate token-budget savings from evidence quality. read the letter →

arxiv 2608.05780 v1 pith:Q4ZOU3RZ submitted 2026-08-06 cs.CV

Bo Zhang , Wenxin Wang , Feng Chen , Zhihao Zhang , Zixuan Wang , Changsheng Li , Yinjie Lei This is my paper · ORCID
classification cs.CV
keywords long-formvideounderstandingframeselectionvisualtokenreductionmultimodallargelanguagemodelsattentionmapssparseprefillingGRPOpolicyoptimizationdynamicspatiotemporalsampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long-video multimodal large language models (MLLMs) are expensive and limited by context, so many systems pre-select frames with an external model such as CLIP. This paper claims that external proxies are the wrong signal: the target MLLM's own attention maps, quickly approximated from a highly compressed sparse prefill, already encode where the query-relevant evidence lives. On that evidence the authors train a lightweight selector to decide, per timestamp, whether to keep a frame, how densely to sample its segment, and at what resolution, using GRPO with a reward that only pays for efficiency when at least one candidate in the group answers correctly. The result is a claimed 3.9x end-to-end speedup with roughly half the visual tokens, together with accuracy gains over prior selectors on LongVideoBench, MLVU, and Video-MME. A sympathetic reader would take away that internal attention can serve as a faithful, cheap, and directly trainable selection signal for long-video understanding.

What carries the argument

The load-bearing machinery is a sparse-evidence probing pass followed by a lightweight distribution-aware selector. The video timeline is partitioned into two-second segments; the midpoint frame of each segment is spatially compressed to 20 visual tokens, and block-sparse attention with top-p retention ($\tau_p=0.97$) approximates the target MLLM's layer-wise attention maps at low cost. Those maps are sliced into three cues without extra learned parameters: query-frame relevance $A^{qf}$, cross-frame temporal dependency $A^{ff}$, and intra-frame spatial saliency $A^{if}$. A shared MLP with three heads turns fused cues into per-timestamp categorical policies over retention $z_t$, local sampling rate $\rho_t\in\{1,2,4,8\}$, and spatial resolution $\lambda_t\in\{90\times160,360\times640,540\times960,720\times1280\}$. The policy is trained with GRPO under a reward that combines a binary correctness term with an efficiency term, the latter activated only when at least one candidate in the group is correct, so the policy cannot collapse to the cheapest input.

What would settle it

Build a probe set of long videos whose query-critical events last under two seconds and are positioned exactly midway between the two-second anchor midpoints. Run EviSelect's evidence-grounding pass and check whether its attention cues assign high relevance to the segment containing the event; if the model systematically misses such events and answers at chance, the claim that sparse prefill captures query-relevant evidence is falsified for short-duration events.

Watch

Extended reading notes

Core claim

The paper's central discovery is that evidence derived from the target MLLM's own attention can drive a fine-grained spatiotemporal sampling policy that beats both uniform sampling and external-proxy selectors at lower cost. Sparse prefilling over anchor frames—one 20-token frame every two seconds—produces attention maps that correlate at 0.87–0.89 with dense attention and recover over 98% of the attention mass, and these maps are decomposed into query-frame relevance, cross-frame dependency, and intra-frame spatial saliency. A 0.1B selector trained on this evidence retains, densifies, or up-resolves each temporal segment according to the video's local information density. On LLaVA-Video-7B the full method reaches 62.1, 69.1, and 64.0 on LongVideoBench, MLVU, and Video-MME with 1701 visual tokens, versus 61.4, 68.2, and 62.1 with 3360 tokens for TSPO, and it transfers, with some loss, to Qwen2.5-VL-7B.

Load-bearing premise

The load-bearing premise is that a uniformly spaced grid of two-second anchor frames, each compressed to 20 visual tokens, retains enough evidence for every query-relevant event; if a decisive event falls entirely between anchors, no selection policy can recover it.

Editorial extensions

If this is right

  • Frame selection for long-video MLLMs can be learned from the target model's own attention rather than from CLIP- or DINOv2-style external scores, removing the proxy-misalignment problem.
  • Adaptive local sampling rates and spatial resolutions let the token budget follow spatiotemporal information density, so fast actions and tiny text get more tokens while redundant scenes get fewer.
  • The same selector, with no retraining, transfers across open MLLMs (LLaVA-Video-7B and Qwen2.5-VL-7B) and still beats uniform sampling, though it loses some accuracy relative to a backbone-specific selector.
  • End-to-end latency drops to 2.55 seconds from TSPO's 9.98 seconds on the reported setup, a 3.9x speedup, making hour-scale video QA more practical.
  • Because internal attention is required, the approach does not apply to closed-source MLLMs whose attention maps are inaccessible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the uniform two-second anchor grid is the true ceiling; replacing it with a motion-aware or hierarchically adaptive first pass could recover short events the current failure analysis concedes are missed.
  • Beyond the paper: the counting failures the authors report suggest that selection alone cannot solve instance-aggregation errors; adding cross-frame instance association constraints to the reward could address queries where one object appears in many frames.
  • Beyond the paper: because the evidence prior is exactly the target MLLM's attention, any future improvement in the base MLLM's attention quality should automatically improve the selector, suggesting the framework is a compounding rather than one-off efficiency gain.
  • Beyond the paper: the evidence-source ablation, in which the internal prior outperforms both CLIP and random priors, implies a testable extension—use the same GRPO pipeline with a stronger internal signal to see whether accuracy and token savings scale together.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EviSelect, a framework for efficient long-video understanding that replaces external proxy scorers with evidence derived from the target MLLM's own attention maps. The method first runs a sparse prefilling pass over uniformly spaced, highly compressed anchor frames to approximate full attention, then decomposes these maps into query-frame relevance, cross-frame temporal dependency, and intra-frame spatial cues. A lightweight stochastic selector, trained with GRPO under a joint accuracy-efficiency reward, predicts per-timestamp retention decisions, local sampling rates, and spatial resolutions. The selected multi-scale frames are then fed to the target MLLM for answer generation. Experiments on LongVideoBench, MLVU, and Video-MME with LLaVA-Video-7B and Qwen2.5-VL-7B report consistent improvements over existing methods such as TSPO while using about half the answer-stage visual tokens and achieving a 3.9x end-to-end speedup. The paper also includes ablations, cross-backbone transfer experiments, sparse-dense attention alignment measurements, and a failure-case analysis.

Significance. If the reported results hold, EviSelect makes a genuinely useful contribution: it demonstrates that a target MLLM's own sparse prefilling attention can serve as a more faithful evidence prior than external CLIP or DINOv2 scorers, and that a small policy can translate this evidence into fine-grained, adaptive spatiotemporal sampling. The cross-backbone transfer study and the explicit failure-case analysis are commendable: they acknowledge the method's boundaries rather than overclaiming universality. The three-benchmark evaluation, component ablations, and efficiency measurements are coherent. However, the central claim of 'superior performance' is currently built on comparisons at different answer-stage token budgets, and the reported accuracy differences over TSPO are small (0.7 to 1.9 points) with no error bars. These issues are load-bearing for the paper's headline conclusion and must be addressed before the contribution is fully established.

major comments (3)
  1. [Table 3 and Table 8] The paper lacks a token-matched uniform-sampling control. All uniform-sampling baselines in Table 1 and Table 3 use 3360 answer-stage tokens for LLaVA-Video-7B, while EviSelect uses only 1701 tokens. Since the reward in Eq. (5) explicitly optimizes for lower token cost, the reported gains over TSPO (which also uses 3360 tokens) could be confounded with the lower operating point rather than with the evidence-driven allocation. The random-prior row in Table 8 is a trained policy with random evidence, not a uniform sample at the same token budget, and its token count is not stated. Please add a direct uniform-sampling baseline that matches EviSelect's answer-stage token count (e.g., uniformly sample frames so that the final input has exactly 1701 visual tokens for LLaVA-Video-7B and 1439 for Qwen2.5-VL-7B), and also report the token counts for the CLIP-prior and random-prior rows of Table 8. This experiment is cheap because it requires no retraining, yet it is the minimal control needed to verify that the evidence-driven mechanism, not merely the reduced token count, is responsible for the accuracy improvements.
  2. [Table 1 and Main Results] All accuracy numbers appear to be from single runs, and the differences over TSPO are small: 0.7 points on LongVideoBench, 0.9 points on MLVU, and 1.9 points on Video-MME for LLaVA-Video-7B, and comparable or smaller for Qwen2.5-VL-7B. Without repeated runs, error bars, or statistical significance testing, these differences may be within run-to-run noise. Please report mean and standard deviation over at least three seeds for the main comparisons, or provide bootstrap confidence intervals over the benchmark instances. This is necessary to support the claim that EviSelect achieves 'superior performance' rather than statistically indistinguishable accuracy at a lower token cost.
  3. [Figure 5 and Fig. 3] The paper's own failure analysis states that during evidence grounding, the uniformly spaced anchor frames can fail to capture brief events, and that evidence absent from the sampled input cannot be recovered by the MLLM. Because all sampling decisions (retention, rate, resolution) are made over this fixed anchor grid, the grid is a structural element of the method rather than a minor implementation detail. Figure 3 shows that a 2-second interval closely matches a 1-second interval on LongVideoBench, which partially mitigates the concern, but the failure case in Figure 5 indicates that the problem does occur. Please quantify how often this failure mode arises on the three benchmarks (for example, by comparing the 2-second default with a 1-second anchor grid and reporting the accuracy difference), and discuss whether the motion-aware or adaptive initial scan suggested in the paper would change the results. This would turn an acknowledged limitation into a measured one.
minor comments (5)
  1. [Figure 6] The label 'Frame Selcetion Time' contains a typo and should read 'Frame Selection Time'.
  2. [Table 7] The column headers 'Pear.', 'Spear.', 'JS', 'Top-10%', and 'Keep Mass-rec.' are not defined in the caption or the surrounding text; please spell out the full names and describe how each metric is computed.
  3. [Evidence Decomposition] The statement that Aif retains 'token-level causal attention within each anchor frame, retaining its spatial structure' needs a brief explanation of how the 2D spatial structure is recovered from the 1D visual-token sequence produced by the ViT, since frame patches are typically arranged in a raster order.
  4. [Equations (2) and (6)] The notation for the old policy appears inconsistently as 'πs,θold' in Eq. (2) and 'πs,θold' in Eq. (6); please use a single consistent subscript.
  5. [Conclusion] The limitations paragraph is appropriately honest about closed-source MLLMs and cross-backbone transfer, but please also state whether the code and trained selectors will be released for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: EviSelect's accuracy claims are judged on external benchmarks, and its sparse-prefill evidence prior is directly validated against dense attention.

full rationale

The central derivation chain is: (1) sparse prefilling over compressed anchors yields approximate target-MLLM attention maps; (2) a lightweight selector maps three attention-derived cues to retention, sampling-rate, and resolution decisions; (3) GRPO optimizes the selector with an accuracy-plus-efficiency reward; (4) the selected frames are fed to the target MLLM and scored on MLVU, LongVideoBench, and Video-MME. None of these steps is definitionally equivalent to the headline result. The token reduction is an explicit optimization target (Eq. 3), not a fitted parameter later relabeled as a prediction. The sparse-prefill approximation is not merely assumed from co-authored citations: Table 7 measures Pearson correlations of 0.87–0.89 and mass recovery above 0.985 against dense attention on 200 sampled videos, so the prior is empirically validated inside the paper. Co-authored references (He et al. 2024; Chen et al. 2026; Wang et al. 2026) appear as background or as modular components, and the key component is independently checked rather than taken on faith. Final accuracy is assessed on held-out public benchmarks, and cross-backbone transfer without retraining (Table 6) provides an additional check against self-referential overfitting. The acknowledged anchor-grid limitation (Fig. 5, left) and the absence of a token-matched uniform-sampling baseline are experimental confounds, but they do not make any reported equation or claimed prediction equivalent to its input by construction. No circular step satisfies the quoted-reduction test.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the selector and evidence cues are learned or constructed modules, not new objects requiring independent evidence. The central claim rests on a large set of hand-chosen hyperparameters and empirical assumptions about the reliability of sparse attention and anchor-frame coverage.

free parameters (7)
  • Per-frame visual token budget N_v = 20
    Chosen by hand; ablation in Fig. 6 (left) shows gains saturate and aggressive compression hurts accuracy.
  • Block size B for sparse attention = 20
    Chosen by hand; ablation in Fig. 6 (middle) trades finer evidence against frame-selection time.
  • Top-p retention threshold tau_p = 0.97
    Chosen by hand; ablation in Fig. 6 (right) shows performance saturates above 0.97.
  • Anchor interval = 2 seconds
    Chosen by hand; Fig. 3 shows 1s and 2s are close, while 4s and 8s degrade accuracy.
  • Sampling-rate candidate set R = {1, 2, 4, 8}
    Chosen by hand; ablation in Table 4 shows all four levels help.
  • Resolution candidate set L = {90x160, 360x640, 540x960, 720x1280} with LLaVA pooling equivalents
    Chosen by hand; ablation in Table 5 shows all four levels help.
  • GRPO hyperparameters = G=8, alpha=0.1, eta=0.5, epsilon=0.2, LR=5e-4, warmup schedule
    Chosen by hand in Implementation Details; no sensitivity analysis is reported.
assumptions (5)
  • domain assumption Block-sparse attention with top-p retention approximates dense attention maps of the target MLLM.
    Invoked in Evidence Grounding (Eq. 1), validated only on 200 sampled videos with Pearson correlations 0.87 to 0.89 (Table 7).
  • domain assumption Two-second uniform anchor frames capture all query-relevant events.
    Used to build the evidence grid; the failure analysis (Fig. 5) concedes that events between anchors are unrecoverable.
  • domain assumption The target MLLM's internal attention is a faithful indicator of query-relevant visual evidence.
    Core premise of evidence grounding; supported by Table 8 where internal evidence beats CLIP and random priors.
  • domain assumption Correctness of the final MLLM answer is a sufficient reward signal for frame selection.
    Defined in Eq. (5) with binary R_per; this assumes no reward hacking beyond the gated efficiency term.
  • ad hoc to paper Coarse token-level attention reconstructed from block affinities preserves frame alignment and spatial structure.
    Supplementary Eqs. (7) to (11) define the reconstruction; no independent validation beyond Table 7.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding." pith.science (2026). https://pith.science/paper/Q4ZOU3RZ

@misc{pith2026260805780,
  author       = {Pith},
  title        = {Pith review of: Evidence-Driven Dynamic Visual Selector for Efficient Long Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q4ZOU3RZ}},
  note         = {Machine review of arXiv:2608.05780}
}
read the original abstract

Recent advancements in MLLM-based long-form video understanding have mitigated inference-time computational cost and limited context lengths by selecting query-relevant frames. However, existing approaches predominantly rely on external proxy scorers and rigid heuristic rules, inevitably suffering from misalignment with the target MLLM's intrinsic evidence and failing to accommodate the non-uniform spatiotemporal information density. In this paper, we propose a fine-grained dynamic visual selection framework named EviSelect, grounded in the target MLLM internal attention evidence. Our method efficiently probes visual evidence via sparse prefilling as a structured prior to guide distribution-aware dynamic sampling. Specifically, we efficiently approximate attention maps of the target MLLM using highly compressed visual inputs and sparse attention, well-aligned to the full counterpart. Conditioned on three complementary attention components derived from this prior, we design a lightweight selector that not only precisely locates query-relevant timestamps but also adaptively adjusts the local sampling rate and spatial resolution. To enable evidence-conditioned spatiotemporal sampling, we formulate the selector as a stochastic policy and optimize it via GRPO under a joint accuracy--efficiency reward. By rewarding correct predictions under lower visual cost through group-relative comparisons, our method encourages the policy to allocate computation dynamically according to the information density of each video. Across three long video understanding benchmarks, EviSelect achieves superior performance compared to existing methods while reducing selected visual tokens by about 50\% and achieving a 3.9x end-to-end speedup.

Figures

Figures reproduced from arXiv: 2608.05780 by the authors.

Figure 1
Figure 1. Comparison of representative frame selection methods with EviSelect. Existing method (Liu et al. 2025a; Zhang et al. 2025a) typically rely on CLIP to compute query–frame similarity and apply rigid heuristic rules to select keyframes for MLLM. EviSelect rapidly approximates the target MLLM’s attention maps as evidence priors and designs a selector to predict three policy distributions for each timestamp: retention de… view at source ↗
Figure 2
Figure 2. Overview of our method. Our method efficiently probes visual evidence to enable distribution-aware dynamic sampling. We partition the video into uniform timestamps, construct a highly compressed visual input, and employ sparse attention to rapidly approximate the MLLM’s attention maps. A lightweight selector then predicts three policy distributions for each timestamp, including retention decision, sampling rate, and… view at source ↗
Figure 3
Figure 3. Effect of the anchor timestamp sampling interval. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between uniform sampling and our method. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Representative failure cases of EviSelect. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Influence of the per-frame token count Nv, block size B, and top-p threshold τp on LongVideoBench. increasing Nv improves accuracy but also increases frame￾selection time. The gains quickly saturate, while overly ag￾gressive compression causes a substantial performance…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 11 canonical work pages

  1. [1]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Seed-bench: Benchmarking multimodal large language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  2. [2]

    2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=

    Zipvl: Accelerating vision-language models through dynamic token sparsity , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization=

    2025

  3. [3]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    OmniSparse: Training-Aware Fine-Grained Sparse Attention for Long-Video MLLMs , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  4. [4]

    PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models

    PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models , author=. arXiv preprint arXiv:2606.22540 , year=

    work page Pith review arXiv

  5. [5]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Lion: Empowering multimodal large language model with dual-level visual knowledge , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  6. [6]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Vcoder: Versatile vision encoders for multimodal large language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  7. [7]

    arXiv preprint arXiv:2504.02438 , year=

    Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation , author=. arXiv preprint arXiv:2504.02438 , year=

    arXiv

  8. [8]

    arXiv preprint arXiv:2506.19225 , year=

    Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification , author=. arXiv preprint arXiv:2506.19225 , year=

    arXiv

Show all 94 references
  1. [9]

    arXiv preprint arXiv:2507.07966 , year=

    Scaling rl to long videos , author=. arXiv preprint arXiv:2507.07966 , year=

  2. [10]

    arXiv preprint arXiv:2501.12326 , year=

    Ui-tars: Pioneering automated gui interaction with native agents , author=. arXiv preprint arXiv:2501.12326 , year=

    arXiv

  3. [11]

    5-vl technical report , author=

    Qwen2. 5-vl technical report , author=. arXiv preprint arXiv:2502.13923 , year=

    arXiv

  4. [13]

    Journal of Visual Communication and Image Representation , volume=

    A review of video surveillance systems , author=. Journal of Visual Communication and Image Representation , volume=. 2021 , publisher=

    2021

  5. [14]

    arXiv preprint arXiv:2602.06733 , year=

    Pairwise is Not Enough: Hypergraph Neural Networks for Multi-Agent Pathfinding , author=. arXiv preprint arXiv:2602.06733 , year=

    arXiv

  6. [15]

    URL https://arxiv

    Cinepile: A long video question answering dataset and benchmark, 2024 , author=. URL https://arxiv. org/abs/2405.08813 , year=

    2024 arXiv

  7. [16]

    arXiv preprint arXiv:2503.12329 , year=

    Caparena: Benchmarking and analyzing detailed image captioning in the llm era , author=. arXiv preprint arXiv:2503.12329 , year=

    arXiv

  8. [17]

    arXiv preprint arXiv:2504.16072 , year=

    Describe anything: Detailed localized image and video captioning , author=. arXiv preprint arXiv:2504.16072 , year=

    arXiv

  9. [18]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Question aware vision transformer for multimodal reasoning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  10. [19]

    Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

    Dtrocr: Decoder-only transformer for optical character recognition , author=. Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

  11. [20]

    Advances in Neural Information Processing Systems , volume=

    Longvideobench: A benchmark for long-context interleaved video-language understanding , author=. Advances in Neural Information Processing Systems , volume=

  12. [21]

    arXiv e-prints , pages=

    Mlvu: A comprehensive benchmark for multi-task long video understanding , author=. arXiv e-prints , pages=

  13. [22]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  14. [23]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Lvbench: An extreme long video understanding benchmark , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  15. [24]

    arXiv preprint arXiv:2508.04369 , year=

    Tspo: Temporal sampling policy optimization for long-form video language understanding , author=. arXiv preprint arXiv:2508.04369 , year=

  16. [25]

    Bai, Jinze and Bai, Shuai and Yang, Shusheng and Wang, Shijie and Tan, Sinan and Wang, Peng and Lin, Junyang and Zhou, Chang and Zhou, Jingren , journal=

  17. [26]

    Hurst, Aaron and Lerer, Adam and Goucher, Adam P and Perelman, Adam and Ramesh, Aditya and Clark, Aidan and Ostrow, AJ and Welihinda, Akila and Hayes, Alan and Radford, Alec and others , journal=

  18. [27]

    arXiv preprint arXiv:2507.06261 , year=

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

    arXiv

  19. [28]

    Advances in neural information processing systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in neural information processing systems , volume=

  20. [29]

    International conference on machine learning , pages=

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  21. [30]

    arXiv preprint arXiv:2408.10188 , year=

    Longvila: Scaling long-context visual language models for long videos , author=. arXiv preprint arXiv:2408.10188 , year=

    arXiv

  22. [31]

    arXiv preprint arXiv:2406.16852 , year=

    Long context transfer from language to vision , author=. arXiv preprint arXiv:2406.16852 , year=

    arXiv

  23. [32]

    arXiv preprint arXiv:2502.05177 , year=

    Long-vita: Scaling large multi-modal models to 1 million tokens with leading short-context accuracy , author=. arXiv preprint arXiv:2502.05177 , year=

  24. [33]

    European Conference on Computer Vision , pages=

    Llama-vid: An image is worth 2 tokens in large language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  25. [34]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Video-xl: Extra-long vision language model for hour-scale video understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  26. [35]

    arXiv preprint arXiv:2506.21862 , year=

    LLaVA-Scissor: Token Compression with Semantic Connected Components for Video LLMs , author=. arXiv preprint arXiv:2506.21862 , year=

    arXiv

  27. [36]

    arXiv preprint arXiv:2304.07193 , year=

    Dinov2: Learning robust visual features without supervision , author=. arXiv preprint arXiv:2304.07193 , year=

    arXiv

  28. [37]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Adaptive keyframe sampling for long video understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  29. [38]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    BOLT: Boost Large Vision-Language Model Without Training for Long-form Video Understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  30. [39]

    arXiv preprint arXiv:2404.16994 , year=

    Pllava: Parameter-free llava extension from images to videos for video dense captioning , author=. arXiv preprint arXiv:2404.16994 , year=

    arXiv

  31. [40]

    arXiv preprint arXiv:2510.27280 , year=

    Focus: Efficient keyframe selection for long video understanding , author=. arXiv preprint arXiv:2510.27280 , year=

  32. [41]

    arXiv preprint arXiv:2506.22139 , year=

    Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs , author=. arXiv preprint arXiv:2506.22139 , year=

    arXiv

  33. [42]

    arXiv preprint arXiv:2506.00993 , year=

    FlexSelect: Flexible Token Selection for Efficient Long Video Understanding , author=. arXiv preprint arXiv:2506.00993 , year=

    arXiv

  34. [43]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Flexible Frame Selection for Efficient Video Reasoning , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  35. [44]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    M-llm based video frame selection for efficient video understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  36. [45]

    arXiv preprint arXiv:2510.13891 , year=

    K-frames: Scene-Driven Any-k Keyframe Selection for long video understanding , author=. arXiv preprint arXiv:2510.13891 , year=

  37. [46]

    arXiv preprint arXiv:2410.03226 , year=

    Frame-voyager: Learning to query frames for video large language models , author=. arXiv preprint arXiv:2410.03226 , year=

    arXiv

  38. [47]

    arXiv preprint arXiv:2404.01258 , year=

    Direct preference optimization of video large multimodal models from language model reward , author=. arXiv preprint arXiv:2404.01258 , year=

    arXiv

  39. [48]

    arXiv preprint arXiv:2503.14476 , year=

    Dapo: An open-source llm reinforcement learning system at scale , author=. arXiv preprint arXiv:2503.14476 , year=

    arXiv

  40. [49]

    arXiv preprint arXiv:2512.11534 , year=

    HFS: Holistic Query-Aware Frame Selection for Efficient Video Reasoning , author=. arXiv preprint arXiv:2512.11534 , year=

  41. [50]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  42. [51]

    Advances in neural information processing systems , volume=

    Direct preference optimization: Your language model is secretly a reward model , author=. Advances in neural information processing systems , volume=

  43. [52]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

  44. [53]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  45. [54]

    Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    Video-llava: Learning united visual representation by alignment before projection , author=. Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    2024

  46. [55]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Mvbench: A comprehensive multi-modal video understanding benchmark , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  47. [56]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Chat-univi: Unified visual representation empowers large language models with image and video understanding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  48. [57]

    arXiv preprint arXiv:2406.07476 , year=

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms , author=. arXiv preprint arXiv:2406.07476 , year=

    arXiv

  49. [58]

    Llavanext: Improved reasoning, ocr, and world knowledge , author=

  50. [59]

    arXiv preprint arXiv:2408.03326 , year=

    Llava-onevision: Easy visual task transfer , author=. arXiv preprint arXiv:2408.03326 , year=

    arXiv

  51. [60]

    arXiv preprint arXiv:2408.14023 , year=

    Video-ccam: Enhancing video-language understanding with causal cross-attention masks for short and long videos , author=. arXiv preprint arXiv:2408.14023 , year=

    arXiv

  52. [61]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Vila: On pre-training for visual language models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  53. [62]

    GPT-4V(ision) system card , url=

    OpenAI , year=. GPT-4V(ision) system card , url=

  54. [63]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    arXiv

  55. [64]

    arXiv preprint arXiv:2409.12961 , year=

    Oryx mllm: On-demand spatial-temporal understanding at arbitrary resolution , author=. arXiv preprint arXiv:2409.12961 , year=

    arXiv

  56. [65]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Nvila: Efficient frontier visual language models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  57. [66]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Apollo: An exploration of video understanding in large multimodal models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  58. [67]

    arXiv preprint arXiv:2408.04840 , year=

    mplug-owl3: Towards long image-sequence understanding in multi-modal large language models , author=. arXiv preprint arXiv:2408.04840 , year=

    arXiv

  59. [68]

    arXiv preprint arXiv:2412.05271 , year=

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling , author=. arXiv preprint arXiv:2412.05271 , year=

    arXiv

  60. [69]

    arXiv preprint arXiv:2410.17434 , year=

    Longvu: Spatiotemporal adaptive compression for long video-language understanding , author=. arXiv preprint arXiv:2410.17434 , year=

    arXiv

  61. [70]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    arXiv

  62. [71]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Lmms-eval: Reality check on the evaluation of large multimodal models , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  63. [72]

    arXiv preprint arXiv:2510.20622 , year=

    SeViCES: Unifying Semantic-Visual Evidence Consensus for Long Video Understanding , author=. arXiv preprint arXiv:2510.20622 , year=

  64. [73]

    arXiv preprint arXiv:2407.03104 , year=

    Keyvideollm: Towards large-scale video keyframe selection , author=. arXiv preprint arXiv:2407.03104 , year=

    arXiv

  65. [74]

    arXiv preprint arXiv:2502.13189 , year=

    Moba: Mixture of block attention for long-context llms , author=. arXiv preprint arXiv:2502.13189 , year=

    arXiv

  66. [75]

    arXiv preprint arXiv:2410.08584 , year=

    Zipvl: Efficient large vision-language models with dynamic token sparsification , author=. arXiv preprint arXiv:2410.08584 , year=

    arXiv

  67. [76]

    European Conference on Computer Vision , pages=

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  68. [77]

    arXiv preprint arXiv:2512.04000 , year=

    Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding , author=. arXiv preprint arXiv:2512.04000 , year=

  69. [78]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Llava-prumerge: Adaptive token reduction for efficient large multimodal models , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  70. [79]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Tempcompass: Do video llms really understand videos? , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  71. [80]

    arXiv preprint arXiv:2512.06866 , year=

    Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior , author=. arXiv preprint arXiv:2512.06866 , year=

  72. [81]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  73. [82]

    arXiv preprint arXiv:2510.18234 , year=

    Deepseek-ocr: Contexts optical compression , author=. arXiv preprint arXiv:2510.18234 , year=

    arXiv

  74. [83]

    arXiv preprint arXiv:2502.02770 , year=

    Twilight: Adaptive Attention Sparsity with Hierarchical Top- p Pruning , author=. arXiv preprint arXiv:2502.02770 , year=

  75. [84]

    arXiv preprint arXiv:2501.00574 , year=

    Videochat-flash: Hierarchical compression for long-context video modeling , author=. arXiv preprint arXiv:2501.00574 , year=

    arXiv

  76. [85]

    arXiv preprint arXiv:2601.22582 , year=

    MC-GRPO: Median-Centered Group Relative Policy Optimization for Small-Rollout Reinforcement Learning , author=. arXiv preprint arXiv:2601.22582 , year=

  77. [86]

    arXiv preprint arXiv:2502.06428 , year=

    Cos: Chain-of-shot prompting for long video understanding , author=. arXiv preprint arXiv:2502.06428 , year=

    arXiv

  78. [87]

    arXiv preprint arXiv:2410.02713 , year=

    Llava-video: Video instruction tuning with synthetic data , author=. arXiv preprint arXiv:2410.02713 , year=

    arXiv

  79. [88]

    arXiv preprint arXiv:2505.23747 , year=

    Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence , author=. arXiv preprint arXiv:2505.23747 , year=

    arXiv

  80. [89]

    arXiv preprint arXiv:2505.10978 , year=

    Group-in-group policy optimization for llm agent training , author=. arXiv preprint arXiv:2505.10978 , year=

    arXiv

  81. [90]

    arXiv preprint arXiv:2502.01951 , year=

    On the emergence of position bias in transformers , author=. arXiv preprint arXiv:2502.01951 , year=

    arXiv

  82. [91]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Identifying and mitigating position bias of multi-image vision-language models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  83. [92]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    The devil in linear transformer , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  84. [93]

    arXiv preprint arXiv:2504.18579 , year=

    Sparsity Forcing: Reinforcing Token Sparsity of MLLMs , author=. arXiv preprint arXiv:2504.18579 , year=

  85. [94]

    arXiv preprint arXiv:2603.06199 , year=

    Flashprefill: Instantaneous pattern discovery and thresholding for ultra-fast long-context prefilling , author=. arXiv preprint arXiv:2603.06199 , year=

  86. [95]

    arXiv preprint arXiv:2502.20766 , year=

    Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference , author=. arXiv preprint arXiv:2502.20766 , year=

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.