Pith. sign in

REVIEW 3 major objections 6 minor 38 references

One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Long-video frame selection can be reduced to a single Matryoshka ranking: one query-conditioned priority sequence whose prefixes serve any frame budget.

desk verdict MEC is a genuinely useful one-pass multi-budget frame selector; the ranking idea holds, but the sparse-discovery blind spot it documents needs a recall measurement before 'any budget' is credible. read the letter →

arxiv 2608.05707 v1 pith:BIWC4Q3P submitted 2026-08-06 cs.CV

Wang Chen , Yu Chen , Xiang Wang , Shuai Li , Jinfa Huang , Xiawu Zheng This is my paper · ORCID
classification cs.CV
keywords long-videounderstandingframeselectionMatryoshkarankinglargemultimodalmodelstraining-freebudget-independentvideoquestionansweringtemporalcoverage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that frame selection for long-video understanding does not need to be rerun for every frame budget. It claims that a single query-conditioned priority ranking, built once, can serve any budget simply by truncating it to the prefix of that length, provided the ranking is position-adaptive: early positions concentrate answer-critical evidence, later positions add temporal coverage and visual diversity. To support this, the paper introduces MEC, a training-free pipeline that builds a reusable sparse index of the video, discovers candidate frames by sparse probing and local zooming around high-evidence anchors, and greedily constructs the ranking with a position-dependent score. On four long-video benchmarks and six budgets, the paper reports that this one ranking improves accuracy over uniform sampling by an average of 3.77 percentage points, stays competitive with strong training-free selectors, and reduces end-to-end selection latency by 47.37–51.19%.

What carries the argument

The central object is the Matryoshka ranking itself: a sequence of distinct original-frame indices, built once per query, whose every prefix is a valid selection for the corresponding budget. The construction is carried by a greedy algorithm that, at position $k$, scores each remaining candidate $n$ by $u_k(n)=w_e(k)E_n(q)+w_c(k)H_n(k)+w_dD_n(k)$, where $E_n(q)$ is the evidence score (a weighted combination of frame–query matching, visual change, and observability), $H_n(k)$ is temporal coverage relative to the already chosen prefix, $D_n(k)$ is visual diversity, and the weights move from evidence-dominated ($w_e=0.6$) to coverage-dominated ($w_e=0.2$) as $k$ grows. Because the greedy step never replaces a selected frame, every shorter prefix is literally the beginning of every longer one, which guarantees nesting by construction. Efficiency comes from a reusable sparse index: duration-adaptive probes, low-resolution grayscale appearance encodings, local change scores, and observability priors are computed once and reused across questions.

What would settle it

The paper itself shows a 48-minute planter-comparison video where the final completed state is missed by sparse probing: MEC answers incorrectly at all six budgets, while uniform sampling succeeds from $K=32$ onward. Checking a set of such “brief endpoint” questions — where the answer depends on a single short state at a time boundary — and measuring how often MEC fails at all budgets while uniform succeeds would settle whether the central claim is conditional on discovery recall.

Watch

Extended reading notes

Core claim

The paper's central claim is that the right object to compute is not a budget-specific subset but a budget-independent Matryoshka ranking $R=\Psi(V,q,M)=(r_1,\dots,r_M)$, where the selected set for any budget $K\le M$ is the prefix $F_K=\{r_1,\dots,r_K\}$. The distinctive thesis is that rank positions carry different roles: the first ranks should be maximally evidence-bearing for the query, while later ranks should progressively fill the timeline and diversify appearance, so that longer prefixes preserve the evidence of shorter ones while adding context. MEC realizes this by scoring candidates with a mixture of query–frame relevance, local visual change, and observability, activating the highest-scoring temporal segments, zooming around anchors to add fine-grained candidates, and then greedily extending the ranking with a position-adaptive weight schedule. The reported outcome is that this single ranking, with one fixed configuration across models and benchmarks, improves accuracy over uniform sampling by 3.77 percentage points on average and matches the accuracy of strong selectors while cutting selection latency by roughly half.

Load-bearing premise

If a question's decisive evidence never enters the sparse candidate pool — because it falls between probes, in an inactive segment, or outside the zoomed anchor intervals — then no frame budget, however large, can recover it, and the ranking claim collapses for that question.

Editorial extensions

If this is right

  • Deployments can choose the frame budget at inference time: the same cached ranking serves $K=8$ and $K=256$ without rerunning the selector, so latency and accuracy can be traded on demand.
  • Tight budgets benefit most; the paper reports gains over uniform sampling that reach 8.30 points on one benchmark and 12.06 points on another at $K=8$, narrowing as the budget grows.
  • The one configuration transfers across downstream model families and scales, so a single selector can ship with a system without per-model calibration.
  • Selection cost drops sharply: candidates evaluated shrink by about 74%, and end-to-end latency falls by 47.37–51.19% relative to the compared selectors on the long-video benchmark used for the timing study.
  • Evidence selected under a tight budget is never replaced under a larger budget, addressing the instability of budget-conditioned selection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the nesting property holds in practice, the ranking can be computed once per video-query pair and then cached; any later change in the deployment budget, downstream model context window, or latency requirement costs nothing beyond truncation.
  • Because the failure mode is confined to the discovery stage, a natural extension is to add uncertainty-triggered refinement: if neighboring probes disagree sharply or the query includes temporal comparatives like “first and last,” increase local probing around inactive segment boundaries before ranking.
  • The framework separates two testable claims: sparse discovery recall and ranking quality. Datasets biased toward brief, isolated state changes can benchmark the first; datasets with many visible but redundant events can benchmark the second.
  • One could test near-optimality of the greedy ranking by comparing its prefixes against a beam-search or local-swap variant on a small subset; if large gaps appear, the position-adaptive surrogate is not the bottleneck.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes MEC (Matryoshka Evidence-to-Context) Frame Selection, a training-free method for long-video frame selection that constructs a single budget-independent ranked list of frames. The ranking is built by (i) creating a reusable sparse index of low-resolution appearance, local visual change, and observability; (ii) discovering query-conditioned evidence via duration-adaptive sparse probing, segment activation, and anchor-centered local zooming; and (iii) greedily selecting frames with a position-adaptive score that emphasizes evidence in early ranks and temporal coverage plus visual diversity in later ranks. Any budget K is served by truncating the ranking to its K-th prefix and restoring chronological order. Experiments on four long-video benchmarks (Video-MME, Video-MME-v2, LongVideoBench, MLVU) with three LMMs and six frame budgets report an average +3.77 pp accuracy gain over uniform sampling, competitiveness with AKS and WFS-SB, and lower cached end-to-end selection latency. The paper includes ablations on the ranking schedule and components, an analysis of the ranking horizon M, and an explicitly documented failure case in which sparse discovery misses a decisive frame.

Significance. The problem is practically important and the proposed formulation is clean: serving every budget from one truncated ranking is attractive for deployment, and the paper makes a falsifiable claim that can be independently checked. Strengths include a shared configuration across benchmarks, LMMs, and budgets; three downstream LMMs; six to eleven budget points; component and schedule ablations; and an unusually honest limitations section that includes a concrete failure case (Figure 7). The method is training-free and does not require accessing the LMM's internal tokens. However, the central 'any budget' claim is only as strong as the candidate-generation stage, and the main accuracy and latency claims lack statistical and amortization detail. These gaps are fixable and should be addressed before publication.

major comments (3)
  1. [Section D / Figure 7 (Appendix E.3)] The 'any budget' claim is load-bearing on sparse-discovery recall. Because every prefix FK is drawn from the candidate pool C(q), a decisive frame that never enters C(q) cannot be recovered by coverage or diversity at any larger budget. The paper itself documents this in Figure 7: for the planter comparison question, MEC fails at all six budgets while uniform sampling succeeds from K=32 onward. Section D acknowledges this 'sparse-discovery blind spot,' but the manuscript reports no statistic on how frequently such omissions occur. I request a candidate-pool recall analysis: for example, measure the fraction of questions for which the pool contains the frames selected by a dense oracle (or the answer-critical frames identified by a stronger reference), and report per-benchmark recall. Without this, the central claim that one ranking serves 'any budget' is conditional on an unvalidated premise.
  2. [Table 1 / Section 4.2] The main accuracy claim lacks statistical support. The reported differences are often small (e.g., Video-MME Overall at K=8: MEC 63.85 vs WFS-SB 63.52; at K=128: 73.74 vs 73.37), and the 3.77 pp average improvement over uniform is a point estimate with no confidence intervals or significance tests. Across 24 benchmark-by-budget cells, many gains may be within sampling noise. Please report bootstrap confidence intervals or paired statistical tests over benchmark questions/videos for the headline comparisons, at least for the full MEC vs. uniform and MEC vs. strongest baseline.
  3. [Table 4 / Section 4.3] The end-to-end latency reduction of 47.37–51.19% is measured with 'reusable caches available,' but the amortization model is unspecified. It is unclear whether the 5.48 s of MEC preprocessing per query includes building the sparse index once per video (amortized over the three questions in Video-MME) or rebuilding it per query, and whether the same cache reuse applies to AKS and WFS-SB. Since the latency claim is in the abstract, the paper should state the cache-reuse assumption explicitly and, ideally, report both cold and amortized numbers.
minor comments (6)
  1. [Throughout] There are several typographical and ligature artifacts, e.g., 'Efficiently', 'official', 'difficult', and 'Y et scaling' in the Introduction; these should be cleaned up.
  2. [Table 1] The header uses 'L VB' and 'ML VU' without expanding them in the caption or a table note. Please use full names or define the abbreviations directly below the table.
  3. [Section 4.2] The 3.77 pp average treats the four benchmarks equally, despite their very different sizes and task mixtures. Please also report per-benchmark averages over budgets, since the gains are much larger on LongVideoBench and MLVU than on Video-MME.
  4. [Figure 4 (left)] The 11-budget comparison is dense; adding numerical labels or a small supplementary table would make the per-budget gains easier to read and verify.
  5. [Table 1 / Baselines] The comparison is limited to uniform sampling, AKS, and WFS-SB. Please justify the omission of other recent selectors mentioned in the related work (e.g., BOLT, Nar-KFC), or include at least one at a subset of budgets.
  6. [Algorithm 1] The loop 'for k = 2, . . . , M' assumes M ≤ |C(q)|; this restriction is stated in Section 3.3 but adding an explicit assertion in the algorithm pseudocode would prevent misuse.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning found: MEC's ranking is built from query- and video-derived surrogates, and all accuracy numbers are externally measured outcomes.

full rationale

The claimed derivation chain is self-contained and non-circular. MEC constructs a single budget-independent ranking R = (r1,...,rM) by scoring candidate frames with query–frame relevance (BLIP-2 ITM), local visual change, observability, temporal coverage, and visual diversity, then greedily selecting positions under a hand-set position-adaptive schedule (Equations 5–12, Algorithm 1). None of these inputs are defined in terms of the benchmark labels or the reported accuracies; the 3.77 pp average improvement over uniform sampling is a measured outcome of feeding truncated prefixes to downstream LMMs, not an output of the construction. The weights (λr, λΔ, λo, w_e^+, w_e^-, wd, γ) are fixed modeling choices, not fitted parameters, and the paper reports the same configuration across benchmarks, LMMs, and budgets. Exact prefix nesting is a definitional property of the method's output format (Equation 2), not a circular prediction of performance. The authors' self-citations [5, 6] appear only as baseline selectors (WFS-SB and event-anchored selection) rather than as evidence for the central claim, and no uniqueness theorem or load-bearing prior result by the same authors is invoked. The acknowledged sparse-discovery limitation (Section D, Figure 7) weakens the unconditional reading of 'any budget' and is a correctness/recall risk, but it is an honest boundary condition on the method, not circularity. The ablations and comparisons against AKS and WFS-SB give the framework independent empirical content, so the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method relies on several hand-set hyperparameters and domain assumptions about evidence discovery. The candidate pool is bounded by sparse probing and segment activation, and the ranking uses a heuristic surrogate for prefix utility. The paper explicitly acknowledges the discovery blind spot in Section D and Figure 7. No new physical or conceptual entities with independent falsifiable handles are introduced.

free parameters (6)
  • Evidence mixture weights = λr=0.7, λΔ=0.2, λo=0.1
    Hand-set combination of relevance, visual change, and observability in Eq. (7). Not fitted to data, but the central evidence score depends on these values.
  • Ranking weight schedule = w_e+=0.6, w_e-=0.2, γ=1, wd=0.2
    Position-adaptive weights in Eq. (11) and Eq. (12) that control how evidence, temporal coverage, and diversity are traded across ranks.
  • Sparse probe parameters = P0=P1=256, τp=1800 s
    Duration-adaptive probe count from Eq. (4) bounds the sparse index size; chosen by hand.
  • Segment activation ratio = η=0.25
    Fraction of highest-scoring segments retained in Section 3.3. Directly limits which parts of the video are refined.
  • Local zoom density = Lz=2
    Number of interior frames sampled on each side of an anchor in Eq. (8). Affects recall around active segments.
  • Ranking horizon = M=256
    Maximum ranking length, used in Eq. (11) and Algorithm 1; the paper shows M affects prefix quality in Table 3.
assumptions (4)
  • domain assumption BLIP-2 ITM score ρn(q) is a valid proxy for frame-level evidence relevance to the question.
    Used in Eq. (7) as the dominant evidence term (λr=0.7); the paper does not validate ITM against downstream utility and acknowledges it may miss pair or temporal dependencies (Section D).
  • domain assumption Sparse probing plus segment activation and local zoom recovers answer-critical frames.
    Eq. (4) and Eq. (8) bound the candidate pool; Figure 7 shows this fails for a decisive endpoint, so the method assumes most evidence is near visually distinctive probes.
  • ad hoc to paper The greedy surrogate score uk(n) approximates the prefix utility QK in Eq. (3).
    The position-adaptive weights and coverage/diversity terms are hand-designed, not derived; Section D admits the greedy construction does not guarantee global optimality.
  • domain assumption Observability prior On captures usefulness for LMMs.
    Sharpness and exposure (Eqs. 16-20) are generic image quality cues, not validated against model perception.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding." pith.science (2026). https://pith.science/paper/BIWC4Q3P

@misc{pith2026260805707,
  author       = {Pith},
  title        = {Pith review of: One Ranking, Any Budget: Matryoshka Evidence-to-Context Frame Selection for Long-Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BIWC4Q3P}},
  note         = {Machine review of arXiv:2608.05707}
}
read the original abstract

Frame selection is essential for applying Large Multimodal Models (LMMs) to long videos due to severe frame redundancy and limited context windows. Since the appropriate frame budget varies with the downstream LMM, reasoning demands, and latency constraints, a practical selector should serve multiple budgets. However, existing methods typically optimize an isolated frame subset for each predefined budget: when the budget changes, previously selected evidence may be replaced rather than progressively augmented. Ranking frames by a fixed score would allow prefix reuse across budgets, but it ignores the distinct roles of different ranking positions. In this paper, we formulate long-video frame selection as a Matryoshka ranking problem: constructing a single priority sequence whose small prefixes concentrate query-conditioned evidence, while progressively larger prefixes preserve this evidence and add broader temporal context. Efficiently constructing such a ranking is itself challenging, as densely sampling long videos and evaluating frame-query relevance incurs substantial overhead. We therefore introduce Matryoshka Evidence-to-Context (MEC) Frame Selection, a training-free framework that builds a reusable sparse video index, discovers candidates through sparse probing and local zooming, and greedily constructs a position-adaptive ranking: early positions emphasize evidence; later positions progressively favor temporal coverage while preserving visual diversity. A single ranking can thus be truncated to any target budget without rerunning the selector. Across four benchmarks and six frame budgets, MEC improves average accuracy over uniform sampling by 3.77 percentage points, matches strong state-of-the-art selectors, and reduces end-to-end selection latency by 47.37-51.19%.

Figures

Figures reproduced from arXiv: 2608.05707 by the authors.

Figure 1
Figure 1. Motivation and empirical trade-off of MEC. (a) Budget-conditioned selection independently optimizes each subset and may replace previously selected evidence; fixed-score ranking reuses prefixes but struggles to balance performance across budgets. MEC instead constructs one Matryoshka ranking whose prefixes preserve early evidence and progressively extend temporal context. (b) On Video-MME Long, each marker is annota… view at source ↗
Figure 2
Figure 2. Overview of MEC. A reusable sparse video index caches low-resolution appearance encodings, local visual-change estimates, and observability priors. For each query, MEC scores multi-cue evidence, activates promising segments, and performs local zooming before constructing a position-adaptive Matryoshka ranking whose nested prefixes support any frame budget. 3.2 Reusable Sparse Video Index Exhaustively analyzing all N… view at source ↗
Figure 3
Figure 3. Generalization across downstream LMMs and input budgets K ∈ {8, 16, 32, 64, 128}. (a) Video-MME Overall with LLaVA-OneVision-2-8B-Instruct, comparing uniform frames, native codec-stream input, and MEC. (b) Uniform accuracy and the corresponding MEC gain on Video-MME and LongVideoBench (LVB) with InternVL3.5-8B-Instruct. 800 videos and 3,200 questions organized around information aggregation, temporal dynamics, and c… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Fine-grained budget behavior and thinking-mode evaluation on Video-MME Overall with Qwen3.5-9B. The left panel evaluates Uniform and MEC over 11 input budgets; the shaded region marks the advantage of MEC. The right panel compares the two selectors at five budgets with…
Figure 5
Figure 5. Figure 5: Evidence-to-context expansion from a single MEC ranking for a ceramic-making question. The upper timeline summarizes raw-clay preparation, shaping, firing, painting, and the finished products. Colors mark frames introduced at ranks 1–8, 9–16, and 17–32. The K = 8 prefi…
Figure 6
Figure 6. Figure 6: Successful example of procedural ordering. The query asks for the order in which four cardboard devices are introduced: MacBook (04:30), iPad (10:42), Apple Watch (21:41), and iPhone 13 (25:25). Red, orange, and blue mark frames entering the shared ranking at positions…
Figure 7
Figure 7. Figure 7: Failure on a comparison between the first and last objects in a 48.3-minute video. The answer requires comparing the first completed planter at 164.6 s with the last at 2871.7 s; both have black-and-brown bodies. Although the selected prefixes contain the first planter…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 16 canonical work pages

  1. [2]

    LLaV A-OneVision-2: Towards next-generation perceptual intelligence, 2026

    Xiang An, Yin Xie, Feilong Tang, Yunyao Y an, Huajie Tan, Didi Zhu, Changrui Chen, Xiuwei Zhao, Bin Qin, Kaicheng Y ang, Yifei Shen, Yuanhan Zhang, Kaichen Zhang, Wenkang Zhang, Zheng Cheng, Nansen Zhang, Chunsheng Wu, Chunjiang Ge, Zimin Ran, Dehua Song, Chunyuan Li, Shikun Feng, Ming Hu, Zhangquan Chen, Junbo Niu, Bo Li, Ziyong Feng, Ziwei Liu, Zongyuan...

    arXiv 2026

  2. [3]

    Qwen3-VL technical report, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Y ang Liu, Dayiheng Liu, Shixuan...

    arXiv 2025

  3. [4]

    Matryoshka multimodal models

    Mu Cai, Jianwei Y ang, Jianfeng Gao, and Y ong Jae Lee. Matryoshka multimodal models. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2405.17430

    arXiv 2025

  4. [5]

    Event-anchored frame selection for effective long-video understanding, 2026

    Wang Chen, Y ongdong Luo, Yuhui Zeng, Luojun Lin, Tianyu Xie, Fei Chao, Rongrong Ji, and Xiawu Zheng. Event-anchored frame selection for effective long-video understanding, 2026. URL https://arxiv.org/abs/2603.00983

    work page arXiv 2026

  5. [6]

    Wavelet-based frame selection by detecting semantic boundary for long video understanding, 2026

    Wang Chen, Yuhui Zeng, Y ongdong Luo, Tianyu Xie, Luojun Lin, Jiayi Ji, Y an Zhang, and Xiawu Zheng. Wavelet-based frame selection by detecting semantic boundary for long video understanding, 2026. URL https://arxiv.org/abs/2603.00512

    arXiv 2026

  6. [7]

    Molmo2: Open weights and data for vision-language models with video understanding and grounding, 2026

    Christopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park, Mohammadreza Salehi, Rohun Tripathi, Sangho Lee, Zhongzheng Ren, Chris Dongjoo Kim, Yinuo Y ang, Vincent Shao, Yue Y ang, Weikai Huang, Ziqi Gao, Taira Anderson, Jianrui Zhang, Jitesh Jain, George Stoica, Winson Han, Ali Farhadi, and Ranjay Krishna. Molmo2: Open weights and data for vision-language...

    arXiv 2026

  7. [8]

    The blur effect: Perception and estimation with a new no-reference perceptual blur metric

    Frédérique Crété-Roffet, Thierry Dolmière, Patricia Ladret, and Marina Nicolas. The blur effect: Perception and estimation with a new no-reference perceptual blur metric. In Human Vision and Electronic Imaging XII , volume 6492, page 64920I. SPIE, 2007. doi: 10.1117/12.702790

    doi:10.1117/12.702790 2007

  8. [9]

    MatFormer: Nested transformer for elastic inference, 2023

    Devvrit, Sneha Kudugunta, Aditya Kusupati, Tim Dettmers, Kaifeng Chen, Inderjit Dhillon, Yulia Tsvetkov, Hannaneh Ha- jishirzi, Sham Kakade, Ali Farhadi, and Prateek Jain. MatFormer: Nested transformer for elastic inference, 2023. URL https://arxiv.org/abs/2310.07707

    arXiv 2023

Show all 38 references
  1. [10]

    Agentic keyframe search for video question answering, 2025

    Sunqi Fan, Meng-Hao Guo, and Shuojin Y ang. Agentic keyframe search for video question answering, 2025. URL https: //arxiv.org/abs/2503.16032

    2025 arXiv

  2. [11]

    Bo Fang, Wenhao Wu, Qiangqiang Wu, Yuxin Song, and Antoni B. Chan. Threading keyframe with narratives: MLLMs as strong long video comprehenders, 2025. URL https://arxiv.org/abs/2505.24158

    2025 arXiv

  3. [12]

    CaptionFormer: Unified segmentation, tracking, and captioning for spatio-temporal objects

    Gabriel Fiastre, Antoine Y ang, and Cordelia Schmid. CaptionFormer: Unified segmentation, tracking, and captioning for spatio-temporal objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 39518–39528, 2026. URL https://openacces...

    2026

  4. [13]

    Video-MME: The first-ever comprehensive evaluation benchmark of multi-modal LLMs in video analysis

    Chaoyou Fu, Yuhan Dai, Y ongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Y anwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xiawu Zheng, Enhong Chen, Caifeng Shan, Ran He, and Xing Sun. Video-MME: The first-...

    2025

  5. [14]

    Video-MME-v2: Towards the next stage in benchmarks for comprehensive video understanding, 2026

    Chaoyou Fu, Haozhi Yuan, Yuhao Dong, Yi-Fan Zhang, Yunhang Shen, Xiaoxing Hu, Xueying Li, Jinsen Su, Chengwu Long, Xiaoyao Xie, Y ongkang Xie, Xiawu Zheng, Xue Y ang, Haoyu Cao, Yunsheng Wu, Ziwei Liu, Xing Sun, Caifeng Shan, and Ran He. Video-MME-v2: Towards the next stage in...

    2026 arXiv

  6. [16]

    Matryoshka query transformer for large vision-language models, 2024

    Wenbo Hu, Zi- Yi Dou, Liunian Harold Li, Amita Kamath, Nanyun Peng, and Kai-Wei Chang. Matryoshka query transformer for large vision-language models, 2024. URL https://arxiv.org/abs/2405.19315

    2024 arXiv

  7. [17]

    Matryoshka representation learning

    Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard- Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. Matryoshka representation learning. In Advances in Neural Information Processing Systems, volume ...

    2022

  8. [18]

    BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023. URL https://arxiv.org/abs/2301.12597

    2023 arXiv

  9. [19]

    VideoChat-Flash: Hierarchical compression for long-context video modeling,

    Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chenting Wang, Yu Qiao, Y ali Wang, and Limin Wang. VideoChat-Flash: Hierarchical compression for long-context video modeling,

  10. [20]

    BOLT: Boost large vision-language model without training for long-form video understanding

    Shuming Liu, Chen Zhao, Tianqi Xu, and Bernard Ghanem. BOLT: Boost large vision-language model without training for long-form video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3318–3327, 2025. URL https://openacce...

    2025

  11. [21]

    Keyframe-oriented vision token pruning: Enhancing efficiency of large vision language models on long-form video pro- cessing

    Yudong Liu, Jingwei Sun, Yueqian Lin, Jianyi Zhang, Jingyang Zhang, Ming Yin, Qinsi Wang, Hai Li, and Yiran Chen. Keyframe-oriented vision token pruning: Enhancing efficiency of large vision language models on long-form video pro- cessing. In Proceedings of the IEEE/CVF Interna...

    2025

  12. [22]

    QuoTA: Query-oriented token assignment via CoT query decouple for long video comprehension,

    Y ongdong Luo, Wang Chen, Xiawu Zheng, Weizhong Huang, Shukang Yin, Haojia Lin, Chaoyou Fu, Jinfa Huang, Jiayi Ji, Jiebo Luo, and Rongrong Ji. QuoTA: Query-oriented token assignment via CoT query decouple for long video comprehension,

  13. [23]

    Exposure fusion

    Tom Mertens, Jan Kautz, and Frank Van Reeth. Exposure fusion. In 15th Pacific Conference on Computer Graphics and Applications, pages 382–390. IEEE, 2007. doi: 10.1109/PG.2007.17

    2007 doi

  14. [24]

    URL https://arxiv.org/abs/2503.08689

    arXiv

  15. [25]

    MovieRecapsQA: A multimodal open-ended video question-answering benchmark

    Shaden Shaar, Bradon Thymes, Sirawut Chaixanien, Claire Cardie, and Bharath Hariharan. MovieRecapsQA: A multimodal open-ended video question-answering benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4537–4546, 2026. URL ht...

    2026

  16. [26]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URL https://qwen.ai/blog?id=qwen3.5

    2026

  17. [27]

    InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency, 2025

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Y e, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Y ang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian,...

    2025 arXiv

  18. [28]

    Adaptive keyframe sampling for long video un- derstanding

    Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Y e. Adaptive keyframe sampling for long video un- derstanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 29118–29128,

  19. [29]

    URL https://arxiv.org/abs/2502.21271

    doi: 10.1109/CVPR52734.2025.02711. URL https://arxiv.org/abs/2502.21271

    2025

  20. [30]

    TimeLens: Rethinking video temporal grounding with multimodal LLMs

    Jun Zhang, Teng Wang, Yuying Ge, Yixiao Ge, Xinhao Li, and Limin Wang. TimeLens: Rethinking video temporal grounding with multimodal LLMs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10419–10429, 2026. URL https://openaccess.the...

    2026

  21. [31]

    LongVideoBench: A benchmark for long-context interleaved video- language understanding, 2024

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. LongVideoBench: A benchmark for long-context interleaved video- language understanding, 2024. URL https://arxiv.org/abs/2407.15754

    2024 arXiv

  22. [32]

    WaveZip: Wavelet- driven space-time decoupling for video token condensation, 2026

    Yuhui Zeng, Wang Chen, Jinfa Huang, Tianyu Xie, Y ongdong Luo, Jiayi Ji, Xiawu Zheng, and Jiebo Luo. WaveZip: Wavelet- driven space-time decoupling for video token condensation, 2026. URL https://arxiv.org/abs/2607.23265. 12 MAC-AutoML

    2026 arXiv

  23. [33]

    Deep video discovery: Agentic search with tool use for long-form video understanding

    Xiaoyi Zhang, Zhaoyang Jia, Zongyu Guo, Jiahao Li, Bin Li, Houqiang Li, and Y an Lu. Deep video discovery: Agentic search with tool use for long-form video understanding. In Advances in Neural Information Processing Systems , 2025. URL https://arxiv.org/abs/2505.18079

    2025

  24. [34]

    LMMs-Eval: Reality check on the evaluation of large multimodal models, 2024

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Y ang, Chunyuan Li, and Ziwei Liu. LMMs-Eval: Reality check on the evaluation of large multimodal models, 2024. URL https://arxiv.org/abs/2407.12772

    2024 arXiv

  25. [35]

    Q-Frame: Query-aware frame selection and multi- resolution adaptation for video-llms

    Shaojie Zhang, Jiahui Y ang, Jianqin Yin, Zhenbo Luo, and Jian Luan. Q-Frame: Query-aware frame selection and multi- resolution adaptation for video-llms. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22056–22065, 2025. doi: 10.1109/ICCV517...

    2025

  26. [36]

    A.I.R.: Enabling adaptive, iterative, and reasoning-based frame selection for video question answering

    Yuanhao Zou, Shengji Jin, Andong Deng, Y oupeng Zhao, Jun Wang, and Chen Chen. A.I.R.: Enabling adaptive, iterative, and reasoning-based frame selection for video question answering. In International Conference on Learning Representations ,

  27. [37]

    Shot-aware frame sampling for video understanding, 2026

    Mengyu Zhao, Di Fu, Y ongyu Xie, Jiaxing Zhang, Zhigang Yuan, Shirin Jalali, and Y ong Cao. Shot-aware frame sampling for video understanding, 2026. URL https://arxiv.org/abs/2603.17374

    2026

  28. [38]

    ML VU: Benchmarking multi-task long video understanding, 2024

    Junjie Zhou, Y an Shu, Bo Zhao, Boya Wu, Zhengyang Liang, Shitao Xiao, Minghao Qin, Xi Y ang, Y ongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. ML VU: Benchmarking multi-task long video understanding, 2024. URL https://arxiv.org/abs/ 2406.04264

    2024 arXiv

  29. [2025]

    URL https://arxiv.org/abs/2501.00574

    arXiv

  30. [2026]

    13 MAC-AutoML Appendix overview

    URL https://arxiv.org/abs/2510.04428. 13 MAC-AutoML Appendix overview. The supplementary material is organized as follows: • Section A reviews additional literature on Matryoshka representations and visual-token compression. • Section B details the four benchmarks, three downs...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.