Pith. sign in

REVIEW 4 major objections 6 minor 17 references

CREST: Curvature-Regulated Event-Centric Sampling for Efficient Long-Video Understanding

T0 review · 4 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Pith's one-line read CREST claims that the local curvature of query–frame relevance scores, not their magnitude alone, determines where informative frames lie, enabling a training-free selection rule that rivals costlier retrieval pipelines.

desk verdict Simple, clean frame-selection heuristic with a plausible but small win over AKS under matched budgets; the flashier MIRA-retention claim is confounded by unequal frame budgets and the abstract overstates it. read the letter →

arxiv 2605.09223 v3 pith:ZE5KO7FX submitted 2026-05-09 cs.CV

classification cs.CV
keywords frameselectionlong-videounderstandingcurvaturequery-conditionedrelevancenon-maximumsuppressiontraining-freetemporalgeometryvideoquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CREST is a training-free frame selection method for long-video understanding that treats query–frame relevance as a temporal signal and uses its local curvature to guide a greedy non-maximum-suppression loop. Sharp curvature marks brief decisive events, so the suppression radius contracts there to keep dense evidence; flat regions get broader suppression to avoid redundancy. Under a shared backbone and frame budget, the paper reports higher accuracy than the AKS relevance–coverage baseline on LongVideoBench and VideoMME, and 93–95% of the accuracy of the stronger MIRA pipeline at only 3–4% of its preprocessing cost. If these results hold, a simple geometric rule can replace expensive multi-stage retrieval for video question answering.

What carries the argument

The central mechanism is curvature-regulated non-maximum suppression. Relevance scores are normalized and a discrete second-difference κ_t = |s_{t+1} − 2s_t + s_{t−1}| is computed. The baseline suppression radius R_base = T/M is divided by (1+κ_i) for each selected peak, so high-curvature event peaks keep a narrow suppression window and flat regions a wide one. An exponential decay with rate λ = ln 2/(ρM) progressively restores suppressed scores, letting the budget spread across the whole video.

What would settle it

Replace the CLIP relevance scores on a fixed video set with smoothed or high-frequency-noisy versions (keeping frames and queries identical) and compare CREST vs AKS accuracy: if CREST's advantage disappears or reverses when curvature no longer tracks genuine events, the mechanism's dependence on that premise is confirmed. A simpler check: on a video where the decisive information is a slow, low-curvature transition, CREST should miss it relative to top-K selection.

Watch

Extended reading notes

Core claim

The paper's central claim is that local temporal geometry of query-conditioned relevance scores, not just score magnitude, determines where informative frames lie. CREST computes the discrete second derivative κ_t = |s_{t+1} − 2s_t + s_{t−1}| on normalized CLIP relevance scores, then performs greedy selection with suppression radius R_i = R_base/(1+κ_i), so high-curvature event peaks retain dense neighboring evidence and flat regions are suppressed more broadly. An exponential decay on past suppression radii lets suppressed regions re-enter the candidate pool, spreading selections across the full video. The reported result is that this simple rule outperforms the AKS relevance–coverage basel

Load-bearing premise

The load-bearing premise is that the discrete second difference of the query–frame relevance score is a reliable indicator of where informative events occur, and that the specific mapping R_i = R_base/(1+κ_i) is the right way to convert that local shape into a suppression radius.

Editorial extensions

If this is right

  • If CREST's claim holds, a training-free geometric rule can substitute for multi-stage retrieval pipelines, cutting preprocessing time by roughly 26–31× and peak memory by about 10.7×.
  • At a fixed frame budget, CREST extracts more usable evidence per frame than AKS, so the limiting factor in long-video QA shifts from frame count to selection policy.
  • The benefit is largest when relevant evidence is spread over long temporal spans (the paper's 'extended relevance' regime), suggesting the method suits videos with slowly developing narratives.
  • CREST's scoring mechanism is agnostic to the relevance model: gains persist across CLIP, BLIP, and Sevila scorers, so it can be layered on different backbones.
  • The query-conditioned design means the method applies directly to video question answering but not to query-agnostic summarization, as the paper itself notes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The curvature heuristic may transfer beyond CLIP relevance scores to any temporal proxy—attention weights, object-detection confidence, optical-flow magnitude—where sharp transitions often mark state changes.
  • Because the paper compares CREST at 32 frames against MIRA at 64 frames, the reported accuracy gap is confounded by budget; a head-to-head at M=64 would likely close part of the gap and could also expose whether CREST's cost advantage grows with budget.
  • The decay parameter ρ is fixed at 0.5 without tuning; adapting it to video length or event density is a natural testable extension that could improve robustness on unusual temporal distributions.
  • The TempRel diagnostic—two controlled regimes of extended and hierarchical relevance—could serve as a reusable microbenchmark for frame-selection methods, isolating where different selection policies fail.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CREST, a training-free frame selection method for long-video question answering. CREST computes CLIP-based query–frame relevance scores, estimates local temporal curvature via second-order finite differences, and performs greedy selection with curvature-adaptive non-maximum suppression: high-curvature peaks receive narrow suppression radii to preserve dense evidence, while flat regions receive broader suppression. An exponential decay of suppression radii over selection steps allows previously suppressed regions to re-enter the candidate pool. Experiments on LongVideoBench, VideoMME, and a newly introduced TempRel diagnostic report that CREST outperforms AKS under a fixed backbone and frame budget, and retains 93–95% of MIRA's accuracy at 3–4% of MIRA's preprocessing cost. Pairwise LLM-as-a-judge evaluations also report higher win rates for CREST-selected frames.

Significance. If the central claims hold, the paper offers a simple, parameter-light geometric heuristic that improves over a relevance–coverage baseline at no training cost and with deterministic behavior. The method's reliance on an external CLIP scorer and a fixed decay parameter (ρ=0.5) is honest, and the authors make code and a diagnostic benchmark available — strengths worth noting. However, the headline MIRA retention claim is currently confounded by unequal frame budgets, and the ablation table contains an internal inconsistency that undermines the claimed complementary contributions of curvature and decay. The AKS advantage, while consistent across budgets and scorers, is numerically small and reported without significance measures. The paper is a plausible incremental contribution pending these corrections.

major comments (4)
  1. [Abstract; Table 1; Limitations] The headline claim that CREST 'retains 93–95% of the accuracy of MIRA' is not a controlled comparison. In Table 1, CREST uses M=32 frames while MIRA uses M=64, a difference the authors acknowledge in the Limitations. The abstract's phrase 'Under a fixed backbone and frame budget' does not apply to this comparison. Moreover, the stated range is arithmetically wrong: LVB retention is 60.21/64.5 = 93.3% and VideoMME retention is 65.04/66.2 = 98.2%, so the honest range is 93–98%. This is load-bearing because the accuracy/efficiency trade-off is the paper's key contribution. Please correct the abstract and Figure 1, and either run CREST at M=64 or explicitly qualify the retention as computed under unequal budgets.
  2. [Table 5; §3.3] In the ablation study, the rows 'w/o curvature' and 'w/o both' are identical across all scorers and both datasets (e.g., 59.31, 58.12, 58.79, 63.67, 64.04, 63.89). This is inconsistent with the text's claim that 'curvature modulation and temporal decay contribute independently and complementarily.' If decay had an independent effect, removing both components should degrade performance relative to removing curvature only. The identical numbers suggest either an implementation bug, a copy-paste error in the table, or that temporal decay has no effect when curvature is removed. This directly affects the interpretation of the ablation and must be investigated and corrected.
  3. [§3.2, Table 1] The advantage of CREST over AKS is +0.45% on LongVideoBench and +0.56% on VideoMME. The paper states 'Results are deterministic across multiple runs,' which rules out run-to-run variance, but benchmark accuracy is still a sample of questions. These small margins could be driven by a few items. A per-category breakdown or a paired bootstrap over questions would demonstrate that the improvement is consistent rather than concentrated in a small subset. As the AKS comparison is the paper's only fully budget-controlled claim, the missing variability analysis is a substantive gap.
  4. [§3.5, Appendix B.2] The LLM-as-a-judge protocol is biased toward verbosity and detail enumeration. The judge prompt explicitly instructs the judge to score higher for 'more exhaustive listing' and 'better readiness for follow-up questions,' and to 'assume every statement in both descriptions is already strictly limited to visible frames.' CREST's event-centric sampling may produce longer, more detailed descriptions by construction, so the win rates (60.58% and 54.50%) may reflect detail richness rather than factual grounding. The paper should validate the judge with human agreement or an analysis of whether the judged preferences correlate with factual accuracy (e.g., precision of statements) rather than only coverage.
minor comments (6)
  1. [Algorithm 1, line 13] The instruction 'restores t for t newly outside N(j, R_j^(n))' is ambiguous. It should specify that the restored score is the original normalized relevance value, and clarify the interaction with the zeroing operation in line 9, especially when a restored score has been re-zeroed by a later selected peak.
  2. [Table 2 caption] The caption states both methods are evaluated 'under the same frame budget,' but Table 1 shows MIRA at M=64 and CREST at M=32. If Table 2 indeed used the same budget, clarify which budget and why; otherwise, correct the caption to avoid confusion.
  3. [§3.4, TempRel] TempRel is introduced as a new diagnostic benchmark but the main text gives no details on its construction (number of videos/questions, source, annotation procedure, or the exact definition of the ER and HR regimes). Please add a description or point clearly to the supplementary material, since the TempRel results are part of the paper's contribution.
  4. [References] Several reference entries are incomplete, e.g., 'Cheng and 1 others', 'Liu and 1 others', 'Islam and 1 others'. These appear to be truncated author lists and should be completed for camera-ready.
  5. [Limitations / ρ sensitivity] The decay parameter ρ is fixed at 0.5 without tuning. The paper acknowledges this, but a small sensitivity table (e.g., ρ ∈ {0.3, 0.5, 0.7}) would make the 'no tuning' claim more credible and characterize the method's robustness, as the Limitations already call for.
  6. [Figure 1 and Abstract] The '93–95%' figure appears in both the abstract and Figure 1. As noted in Major Comment 1, the actual retention is 93.3% (LVB) and 98.2% (VideoMME). Please update all instances to reflect the correct range.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: CREST is a parameter-free heuristic evaluated on external benchmarks; the MIRA comparison is budget-confounded but not circular.

full rationale

CREST's selection rule is not fitted to the benchmarks. The only inputs are external CLIP relevance scores; the curvature is defined as a discrete second difference (κ_t = |s_{t+1} − 2s_t + s_{t−1}|); R_base = T/M is the average uniform spacing; the suppression radius is R_i = R_base/(1 + κ_i); and the decay rate λ = ln2/(ρM) uses ρ = 0.5 fixed without tuning. None of these parameters are learned from or optimized against LongVideoBench, VideoMME, or TempRel outcomes, and the paper does not claim to derive accuracy from curvature—it reports an empirical comparison. The central comparison against AKS uses the same backbone and the same frame budget (M = 32). The headline MIRA retention figure is not a controlled comparison, as the paper itself states in the Limitations: 'The accuracy comparison in Table 1 is conducted under different frame budgets: CREST passes M=32 frames to the downstream MLLM, while MIRA uses M=64... The reported accuracy gap therefore reflects a combination of selection quality and budget difference.' This is a real interpretive caveat, and the abstract's '93–95%' retention wording is also imprecise (60.21/64.5 ≈ 93.3% on LVB, 65.04/66.2 ≈ 98.3% on VideoMME), but this is a confounded comparison rather than a circular derivation: the retention ratio is arithmetic on independently reported accuracies, not a fitted parameter renamed as a prediction. The TempRel diagnostic and LLM-as-a-judge prompts are author-designed and may favor event-dense, verbose frame sets, but no equation defines success in terms of CREST's own curvature values, so this is an evaluation-design risk, not self-justification. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The derivation chain is self-contained and empirically checked against external benchmarks; no significant circularity is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The method rests on the assumption that the second difference of CLIP relevance scores is a usable proxy for event salience and that the modulation formulas are reasonable. No free parameters are fitted to benchmark data except the fixed decay coefficient ρ=0.5. The TempRel benchmark is an author-built entity without independent validation.

free parameters (1)
  • decay rate ρ = 0.5
    Fixed across all experiments without tuning; controls exponential decay of suppression radii. Sensitivity to ρ is uncharacterized, and the ablation shows removing decay sometimes improves accuracy (Table 5), so the choice is not innocuous.
assumptions (4)
  • domain assumption CLIP relevance scores s_t = s(Q, F_t) are a valid proxy for the utility of frame F_t for answering query Q.
    Section 2.1 states that no frame-level supervision is available and the objective is approximated using query–frame relevance as a proxy. The entire method operates on this proxy.
  • ad hoc to paper Local curvature κ_t = |s_{t+1} − 2s_t + s_{t−1}| captures the salience of temporal events.
    Section 2.3 asserts that high curvature corresponds to brief decisive events and low curvature to redundant regions, with no theoretical justification or independent validation beyond qualitative examples.
  • ad hoc to paper Suppression radius R_i = R_base/(1+κ_i) with R_base = T/M is the correct way to trade density against coverage.
    Section 2.3 defines this functional form without considering alternatives; the result depends on this specific modulation.
  • ad hoc to paper Temporal decay R_j exp(−λΔs) with λ = ln2/(ρM) enables coverage without oversuppression.
    Section 2.3 and Algorithm 1 introduce this mechanism; ablation shows it is not consistently beneficial, undermining the assumption that it is a necessary component.
invented entities (1)
  • TempRel benchmark
    purpose: Diagnostic dataset with Extended-Relevance and Hierarchical-Relevance regimes to evaluate frame selection under controlled temporal structures.
    Constructed by the authors, not released, and has no external validation. It is used as evidence for CREST's generalization, creating a risk of self-confirmation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CREST: Curvature-Regulated Event-Centric Sampling for Efficient Long-Video Understanding." pith.science (2026). https://pith.science/paper/ZE5KO7FX

@misc{pith2026260509223,
  author       = {Pith},
  title        = {Pith review of: CREST: Curvature-Regulated Event-Centric Sampling for Efficient Long-Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZE5KO7FX}},
  note         = {Machine review of arXiv:2605.09223}
}
read the original abstract

Selecting informative frames from long videos is a combinatorial problem that existing methods address either through efficient heuristics without explicit modeling of query-conditioned temporal structure, or through multi stage retrieval pipelines with substantial preprocessing cost. We propose \textbf{CREST}, a training-free frame selection method grounded in the temporal geometry of query--frame relevance. CREST is based on the observation that relevance over time exhibits structured local variation: sharp curvature around salient events and flatter regions in redundant segments. By using local curvature to guide selection, CREST allocates a fixed frame budget more effectively across brief decisive events and slowly evolving evidence. Under a fixed backbone and frame budget, CREST achieves higher accuracy than AKS, a lightweight relevance--coverage baseline, on LongVideoBench and VideoMME, while retaining 93--95\% of the accuracy of MIRA, a stronger multi-stage retrieval pipeline, at only 3--4\% of its preprocessing cost.\footnote{Code and implementation details are included in the supplementary material and will be released publicly upon acceptance.} On TempRel, our diagnostic benchmark for temporal frame selection, CREST achieves a 6.88\% relative improvement over AKS. Pairwise LLM-as-a-judge evaluation further shows that CREST-selected frames yield more coherent frame-conditioned descriptions, with win rates of 60.58\% and 54.50\% on the two benchmarks. These results show that local temporal geometry provides a simple and efficient basis for long-video frame selection.

Figures

Figures reproduced from arXiv: 2605.09223 by the authors.

Figure 1
Figure 1. Efficiency–performance trade-off of CATS compared to MIRA. CATS reduces preprocessing time by up to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 1
Figure 1. Efficiency–performance trade-off of CREST com [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison of keyframe selection strategies. Given the same query, AKS selects frames based on a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: Overview of the proposed CATS framework. Given a user query and a video, frame-level relevance scores [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 3
Figure 3. Figure 3: Illustration of the CREST selection process. At each iteration, the highest-scoring frame is selected and a curvature [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 4
Figure 4. Figure 4: Illustration of the CATS selection process. Given a query and a video, frame-level relevance scores form [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: This is an example of verifiability beyond answer correctness. Both methods predict the correct answer, but [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 5
Figure 5. Figure 5: Insufficient grounding leads to hallucinated rationale and failure on follow-up queries. CREST preserves critical [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: From rationale fidelity to future-query reasoning. Although both methods produce the correct answer, [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 13 linked inside Pith

  1. [2]

    23-04-2018 11:06 AM

    Evaluate how additional visible elements from the same scene/domain are in- cluded that could support future ques- tions on the same topic. Score higher for: • More exhaustive coverage of other objects, steps, contextual elements, variations, or supporting details in- side the exact same scene (even if not needed for the current question). • Better readin...

  2. [3]

    Cheng and 1 others

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks.arXiv preprint arXiv:2312.14238. Cheng and 1 others

  3. [4]

    Videomme: A com- prehensive evaluation benchmark for multi-modal large language models in video understanding.arXiv preprint arXiv:2405.21075. X. Gao and 1 others

  4. [6]

    Liu and 1 others

    Video-llava: Learning united visual representation by alignment before projection.arXiv preprint arXiv:2311.10122. Liu and 1 others

  5. [7]

    Visual instruction tuning.arXiv preprint arXiv:2304.08485. X. Liu, Y . Chen, and 1 others. 2025b. Nvila: Ef- ficient frontier visual language models.Preprint, arXiv:2501.06175. OpenAI

  6. [8]

    Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye

    Longvu: Spatiotemporal adaptive compression for long video-language under- standing.arXiv preprint arXiv:2410.17434. Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. 2025a. Adaptive keyframe sam- pling for long video understanding.arXiv preprint arXiv:2502.21271. Z. Tang and 1 others. 2025b. Tspo: Temporal agent training via grou...

  7. [9]

    Longvideobench: A benchmark for long-form video understanding.arXiv preprint arXiv:2406.14514. J. Xu and 1 others. 2024a. Sf-llava: Semantic-free lunch for video large language models. Preprint. Minghao Xu and 1 others. 2024b. Pllava: Parameter- efficient large language and vision assistant for video understanding.arXiv preprint arXiv:2404.02123. Z. Xu an...

  8. [10]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.xxxxx. Jinhui Ye, Zihan Wang, Haosen Sun, Keshigeyan Chan- drasegaran, Zane Durante, Cristobal Eyzaguirre, Yonatan Bisk, Juan Carlos Niebles, Ehsan Adeli, Li Fei-Fei, Jiajun Wu, and Manling Li. 2025a. T*: Re-thinking temporal search for long-form video un- derstanding.arXiv preprint ar...

Show all 17 references
  1. [11]

    Frame-voyager: Learning to query frames for video large language models.arXiv preprint arXiv:2410.03226. Yuan

  2. [12]

    InFindings of the Association for Computational Linguistics: NAACL 2025, pages 881–916

    Lmms-eval: Reality check on the evalua- tion of large multimodal models. InFindings of the Association for Computational Linguistics: NAACL 2025, pages 881–916. Yifan Zhang, Chunyuan Li, Haotian Liu, and 1 oth- ers. 2024a. Llava-video: Large language-and-vision assistant for v...

  3. [13]

    Minigpt-4: Enhancing vision-language understand- ing with advanced large language models.arXiv preprint arXiv:2304.10592. O. Zohar and 1 others

  4. [14]

    Preprint

    Apollo: Training and inference guidelines for long video comprehension. Preprint. A Related Works Early MLLMs such as LLaV A (Liu et al., 2023), Qwen-VL (Bai et al., 2023), MiniGPT-4 (Zhu et al., 2023), InternVL (Chen et al., 2023), GPT-4o (Ope- nAI, 2024), and NVILA (Liu et a...

  5. [15]

    PLLaV A (Xu et al., 2024b) improves parameter efficiency, yet its performance is directly bounded by the quality of the frames it receives

    employs sparse uniform sampling, which misses temporally localized events, while LLaV A-Video (Zhang et al., 2024b) increases den- sity but remains constrained by context length. PLLaV A (Xu et al., 2024b) improves parameter efficiency, yet its performance is directly bounded ...

  6. [16]

    These meth- ods improve accuracy but require additional data, training, and careful tuning, limiting their general- ity across deployment settings

    distills effective practices for long-range temporal understanding, and TSPO (Tang et al., 2025b) ex- plores lightweight policy adaptation. These meth- ods improve accuracy but require additional data, training, and careful tuning, limiting their general- ity across deployment...

  7. [2023]

    Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Hao- tian Tang, Shang Yang, Zhijian Liu, and 1 oth- ers

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966. Yukang Chen, Fuzhao Xue, Dacheng Li, Qinghao Hu, Ligeng Zhu, Xiuyu Li, Yunhao Fang, Hao- tian Tang, Shang Yang, Zhijian Liu, and 1 oth- ers

  8. [2024]

    Zhe Chen, Yizhou Wang, and 1 others

    Longvila: Scaling long-context visual language models for long videos.arXiv preprint arXiv:2408.10188. Zhe Chen, Yizhou Wang, and 1 others

  9. [2025]

    Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy

    Bimba: State space models for long video understanding.arXiv preprint. Rui Li, Xiaohan Wang, Yuhui Zhang, Zeyu Wang, and Serena Yeung-Levy. 2025a. Temporal preference op- timization for long-form video understanding.arXiv preprint arXiv:2501.13919. Z. Li and 1 others. 2025b. P...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.