Pith. sign in

REVIEW 3 major objections 4 minor 4 cited by

A model's own output uncertainty, measured as it looks at each candidate crop or frame, can serve as the guidance signal that selects the right visual evidence — allowing off-the-shelf multimodal LLMs to beat specialized fine-tuned systems

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 13:23 UTC pith:5DRIVFIW

load-bearing objection A genuinely new training-free idea with strong UG-Ground results, but internal numerical conflicts in the Visual Search tables make the headline evidence unreliable until fixed. the 3 major comments →

arxiv 2510.00705 v3 pith:5DRIVFIW submitted 2025-10-01 cs.CV

Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs

Sanghwan Kim , Rui Xiao , Stephan Alaniz , Yongqin Xian , Zeynep Akata This is my paper
classification cs.CV
keywords uncertainty guidancemultimodal large language modelsentropy minimizationvisual searchlong video understandingtemporal groundingtraining-freetoken entropy
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that a multimodal LLM's intrinsic uncertainty is not just a byproduct of confusion but a usable navigation signal: when shown the visual evidence relevant to a query, the model's output entropy drops, and this drop can be measured before committing to an answer. The authors turn three hard perception tasks — finding small objects in high-resolution images, picking key moments in long videos, and pinpointing an event's start and end — into a single search problem: among candidate crops, frames, or windows, choose the one with the lowest uncertainty, then answer from it. Because the selection uses only the model's own token probabilities, the method is training-free and model-agnostic. The central claim is that this simple 'score-then-answer' mechanism lets standard, off-the-shelf MLLMs reach accuracy competitive with — and in temporal grounding beyond — systems explicitly fine-tuned for those tasks.

Core claim

The central discovery is a strong inverse correlation between an MLLM's output entropy and the correctness of its answer to a visual query, and the operational claim that minimizing this entropy at inference time is a risk-minimization strategy. The paper demonstrates this on a high-resolution visual-search benchmark by zooming into the target object and observing that accuracy rises as entropy falls, then validates it across three tasks and several open MLLM families. The intended contribution is a unified, training-free protocol: score candidate visual inputs by average token entropy (or, for yes/no probes, by the difference in probability between the 'yes' and 'no' tokens), select the inp

What carries the argument

Token Entropy: the mean Shannon entropy of the model's per-token output distributions over the generated answer sequence, used to score image crops and video frames; lower entropy signals higher confidence and, by the paper's claim, higher relevance. Binary Response Confidence (BRC): p1('yes') − p1('no') from the first decoded token, used for yes/no event probes. The framework wraps these scores in a sliding-window search; for temporal grounding it reduces the resulting score sequence to a maximum-subarray problem solved with Kadane's algorithm, so the predicted interval is the contiguous segment of highest summed confidence.

Load-bearing premise

The load-bearing premise is that the visual input minimizing the model's output uncertainty is the one containing the evidence needed for the correct answer — an empirical correlation that the paper explicitly states is not a formal guarantee.

What would settle it

For a fixed MLLM and query set, compare the accuracy of answers generated from the minimum-entropy candidate against answers from a uniform random candidate and from the full image or full frame set. If, on a query set with intentionally plausible but wrong distractors, the minimum-entropy pick is not systematically more accurate, the central claim fails. A simpler check: whether the reported strong negative correlation between entropy and accuracy (Pearson coefficient below −0.9 on the paper's benchmarks) reproduces on a new benchmark with a different model.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Off-the-shelf MLLMs, without any fine-tuning, can perform fine-grained visual localization at a level competitive with specialized systems, lowering the cost of deploying such capabilities.
  • The same uncertainty-scoring mechanism transfers across tasks and models, suggesting that the principle generalizes to other sparse-evidence visual problems.
  • Since the uncertainty signal is the only guide, models with better calibration should benefit more, implying the approach scales with improvements in base MLLMs.
  • A scorer model larger than the answering model further improves accuracy, indicating the uncertainty signal itself can be strengthened independently of the final generator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same minimum-uncertainty principle could be applied to other generate-then-select loops, such as choosing context windows in retrieval-augmented generation or picking among candidate rationales before producing a final answer.
  • Editorial inference: the paper's failure cases — when the selected crop captures one of two small objects and misses the other — suggest a testable extension: after selecting a low-entropy input, check whether the model's entropy changes when neighboring crops are added, to catch cases where evidence is split across multiple regions.
  • Editorial inference: if a confidently-wrong distractor yields low entropy, the method would fail; a stress-test that deliberately constructs such distractors could delimit the regime where the assumption holds.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a training-free Uncertainty-Guided (UG) framework for fine-grained MLLM perception. The central insight is that an MLLM's output uncertainty, measured by average token entropy or by a binary 'yes/no' confidence score, decreases when the presented visual input is informative for the query. The framework operationalizes this insight in a score-then-answer procedure: generate candidate crops, frames, or temporal windows; score each candidate by uncertainty; and answer from the minimum-uncertainty input(s). Three instantiations are presented: UG-Search for visual search in high-resolution images, UG-Sample for long-video frame selection, and UG-Ground for temporal grounding using a maximum-subarray over BRC scores. Experiments on V* Bench/HR-Bench, Video-MME/MLVU/LongVideoBench, and Charades-STA/ActivityNet report large gains over base MLLMs and competitiveness with specialized, fine-tuned systems. The authors explicitly note in Appendix C that the entropy-correctness relationship is a heuristic, not a formal guarantee.

Significance. If the reported numbers are correct, the contribution is significant: a single, model-agnostic, training-free mechanism improves three challenging localization tasks, and on temporal grounding it turns untrained base MLLMs into competitive zero-shot localizers. The paper has concrete strengths: it evaluates a broad set of open MLLMs (LLaVA-OneVision, LLaVA-Video, Qwen2.5-VL, InternVL2.5, InternVideo2.5), reports ablations for crop/window size, top-k selection, stride, and scorer-generator decoupling (Appendix F.3), and discloses failure cases (Appendix G). It also explicitly disclaims the heuristic status of the framework in Appendix C. However, the main evidence is currently not internally consistent: the same configuration in Table 1 and Appendix E.1 yields different V* Bench accuracies, and Table 2's Qwen baseline/UG-Search rows do not match Table 1. These contradictions prevent the reported headline conclusions from being reproduced from the manuscript as written.

major comments (3)
  1. [Table 1 vs Appendix E.1 (Table 6)] The headline result for InternVL2.5-8B on V* Bench is not self-consistent. Table 1 reports the UG-Search row with V* Bench Overall 91.1 (baseline 71.7). Appendix E.1, Table 6 reports the same baseline 71.7 and, for the configuration explicitly adopted for all main experiments ('top-1 selection with resizing', Section E.1), a UG-Search accuracy of 83.3; even the best top-4 configuration is 84.8. The 7.8-point gap between 83.3 and 91.1 is not explained by any procedural difference in the manuscript. Since Table 1 is the principal evidence that UG-Search is competitive with fine-tuned systems on V* Bench, this inconsistency must be resolved.
  2. [Table 2 vs Table 1, Qwen2.5-VL-7B rows] The SOTA comparison in Table 2 is internally inconsistent with Table 1. Table 1 gives Qwen2.5-VL-7B baseline overall scores 76.4 (V*), 71.0 (HR4K), 65.4 (HR8K) and UG-Search scores 85.3, 74.3, 69.3, respectively. Table 2 lists the same model's baseline as 64.9, 60.1, 53.1 and its UG-Search row as 81.7, 74.9, 66.3. If these rows refer to the same base model/checkpoint, the comparison is invalid; if they do not, the table is mislabeled. The LLaVA row is consistent between the two tables, making this an apparent data error rather than a different evaluation protocol.
  3. [Section 4.1, Table 1 (FCP splits) and Section 4.4] The text claims 'substantial and consistent performance gains across all base models', but several FCP deltas are 0.0 (LLaVA-OV HR4K FCP), -0.3 (LLaVA-OV HR8K FCP), -0.8 (Qwen HR4K FCP), and -0.5 (Qwen HR8K FCP). With no error bars or repeated-seed statistics, these near-zero and negative deltas are not discriminable from noise, and the claim of consistency across splits is not supported. Please report standard deviations or bootstrap intervals, at least for the main results, and qualify the 'consistent' claim accordingly.
minor comments (4)
  1. [Equation (2) vs Appendix D.3] Equation (2) defines BRC as p_1('yes') - p_1('no'), but the prompt in Appendix D.3 asks the model to answer with option letters 'A' or 'B', and D.3 says the score is computed from tokens corresponding to 'A' and 'B'. Please align the notation: define p_1('A') - p_1('B') or clarify that 'yes'/'no' are labels for the options.
  2. [Figure 1 and Figure 5] The figures report strong inverse correlations (-0.90 and -0.95) but do not show confidence intervals or per-point sample sizes. For sub-task accuracies based on a small number of questions, error bars or at least counts would help assess the strength of the correlation.
  3. [Section 4.4 / Figure 3(a)] The text says performance 'peaks at a crop size of 1/8 and begins to saturate from 1/6', but the chosen configuration is 1/6. This is presumably an efficiency-accuracy trade-off, but the sentence should state that explicitly.
  4. [Table 10] Several gains in Table 10 are very small, e.g., +0.4 on NextQA for LLaVA-OV-7B and +0.2 for InternVideo2.5-8B. The text describes these as consistent improvements; a one-sentence caveat about noise would be appropriate given the absence of error bars.

Circularity Check

0 steps flagged

No significant circularity found; the UG framework is an empirically validated heuristic with predictions measured against external benchmarks.

full rationale

I walked the paper's derivation chain and checked each of the seven circularity patterns. The central principle is the empirical hypothesis that an MLLM's output entropy decreases when it is given visually relevant input; this is explicitly validated against ground-truth zoom-ins in Figure 1 and is not derived from the UG equations. The scoring metrics (Eq. 1 and Eq. 2) are well-defined functions of the model's output distribution, while task performance is measured independently on external benchmarks with ground-truth labels. No parameter fitted on the motivating correlation is renamed as a prediction; the selection rule is a fixed, parameter-free minimization of entropy or maximization of BRC. There are no self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The paper itself disclaims a formal guarantee in Appendix C ('this relationship does not constitute a formal guarantee'), which is consistent with the conclusion that the method is an empirically tested heuristic rather than a circular derivation. The numerical inconsistencies between Tables 1/2 and Appendix E.1/Table 6 raised by the skeptical review are correctness/reproducibility concerns, not constructional circularity: they do not show that any reported prediction is equivalent by definition to a fitted input. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

9 free parameters · 5 axioms · 0 invented entities

The framework introduces no new physical or architectural entities. Its load-bearing additions are hyperparameters (crop sizes, window sizes, strides, frame counts) chosen by ablation, and a domain assumption about MLLM calibration. The central empirical claim rests on these choices; a different hyperparameter regime could reduce the reported gains.

free parameters (9)
  • UG-Search crop size fraction = 1/6 of smaller image dimension
    Chosen via ablation on V* Bench (Section 4.4); performance peaks near 1/8 but 1/6 adopted as a balance between detail and context.
  • UG-Search stride = half the crop size
    Set by implementation; not systematically ablated but affects candidate coverage.
  • UG-Search top-k crops = 1 (main), 4 (best in ablation)
    Ablation E.1 shows top-4 gives 84.8 vs 83.3 for top-1 on V* Bench; top-1 used for simplicity and generality.
  • UG-Search crop resizing = True
    Ablation E.1: resizing crop to original resolution gives +1.1% vs no resizing.
  • UG-Sample number of selected frames = 8 from a pool of 256
    Follows prior protocols (Liu et al. 2025; Huang et al. 2025); not independently justified.
  • UG-Sample window size = 1 frame
    Ablation Figure 3(a): single-frame windows are optimal for Video-MME.
  • UG-Ground window size = 15 frames
    Ablation Table 7 on Charades-STA: 15-frame window gives best mIoU 51.0.
  • UG-Ground stride = 1 frame (main), up to 3 without loss
    Ablation Table 7: stride 1 and 3 both give 51.0; main experiments use stride 1.
  • Sampling FPS for UG-Ground = 3 FPS (Charades-STA), 1 FPS (ActivityNet Captions)
    Set to manage computational cost; not systematically ablated.
axioms (5)
  • domain assumption An MLLM's output entropy and BRC score are reliable proxies for prediction correctness on the evaluated tasks.
    Stated as the core hypothesis in Section 3.1; Appendix C explicitly says no formal guarantee and notes dependence on calibration.
  • domain assumption The target object or event is contained in at least one candidate crop, frame, or sliding window.
    Required for the search to succeed; Appendix G documents failure cases where the selected crop misses one of two distant objects.
  • standard math Kadane's algorithm finds the maximum-sum subarray.
    Used to convert the BRC score sequence into start/end times in Section 3.3 and Appendix D.3.
  • domain assumption The model's per-token output distribution is accessible and can be averaged to give a meaningful entropy.
    All scoring in Eq. 1 and Eq. 2 depends on vocabulary-head probabilities; assumed by the framework.
  • domain assumption Evaluation through LMMs-Eval with the library's default decoding is comparable across methods.
    Stated in Section 4 and Appendix D; batch size is forced to 1 to avoid known MLLM batch artifacts.

pith-pipeline@v1.3.0-alltime-deepseek · 21969 in / 14080 out tokens · 413680 ms · 2026-08-04T13:23:26.863151+00:00 · methodology

0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) often struggle with fine-grained perception, such as identifying small objects in high-resolution images or detecting key moments in long videos. Existing methods typically rely on complex, task-specific fine-tuning, which reduces generalizability and increases system complexity. In this work, we propose an effective, training-free framework that uses an MLLM's intrinsic uncertainty as proactive guidance. Our core insight is that a model's uncertainty decreases when provided with relevant visual information. We introduce a unified mechanism that scores candidate visual inputs by response uncertainty, enabling the model to autonomously focus on the most informative data. We apply this simple principle to three challenging visual tasks: Visual Search, Long Video Understanding, and Temporal Grounding, allowing off-the-shelf MLLMs to achieve performance competitive with specialized, fine-tuned systems. Our results demonstrate that leveraging intrinsic uncertainty is a powerful strategy for improving fine-grained multimodal performance.

Figures

Figures reproduced from arXiv: 2510.00705 by Rui Xiao, Sanghwan Kim, Stephan Alaniz, Yongqin Xian, Zeynep Akata.

Figure 1
Figure 1. Figure 1: Empirical validation of our core hypothesis on V ∗ Bench. As the visual input is increas￾ingly focused on the target object (higher zoom-in ratio), the MLLM’s output entropy (red line, right axis) decreases while task accuracy (blue line, left axis) consistently increases. This demonstrates a strong inverse correlation, motivating entropy minimization as a guidance signal. entropy for post-hoc error detect… view at source ↗
Figure 2
Figure 2. Figure 2: An overview of our Uncertainty-Guided (UG) Framework. The framework follows a two-stage process: (1) Scoring Stage: Candidate visual inputs (image crops or video frames) are scored using the MLLM’s intrinsic uncertainty, measured by either Token Entropy or Binary Response Confidence (BRC) score. (2) Answering Stage: The input with the lowest uncertainty are used for a final inference to generate the defini… view at source ↗
Figure 3
Figure 3. Figure 3: (a) Ablation on Input Granularity: Performance as a function of visual crop size (for UG-Search) and frame window size (for UG-Sample and UG-Ground). (b) Scaling Properties: Performance of the baseline and our UG-enhanced models across the InternVL-2.5 family. image’s smaller dimension. Performance peaks at a crop size of 1/8 and begins to saturate from 1/6. This suggests an optimal balance: the crop must … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Results. (a) UG-Search localizes small target objects by selecting the lowest￾entropy crop (red box). (b) UG-Sample identifies key semantic frames (red box) with the lowest entropy from a long video. (c) UG-Ground pinpoints the correct event timeline by finding the peak in its BRC score sequence. event correctly. Our method, however, filters the timeline to provide the necessary context relevan… view at source ↗
Figure 5
Figure 5. Figure 5: (a) Correlation between sub-task accuracy and entropy. (b) Entropy distribution for correct [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative Results of UG-Search. We compare qualitative results from UG-Search (ours) and InternV2.5-8B (baseline) on V ∗ Bench. Red rectangle expresses the visual crop with the lowest entropy that is selected by UG-Search. UG-Search successfully capture the relevant object in examples of the first two rows while the last row exemplifies the failure cases where UG-Search capture one object and miss the an… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative Results of UG-Sample. We compare qualitative results from UG-Sample (ours) and InternV2.5-8B (baseline) on Video-MME. Red rectangle expresses the frames selected by UG-Sample. We provide a zoom-in version of the selected frames for better visualization. Selected frames are combined into a single context, and used for final inference to answer the query. 22 [PITH_FULL_IMAGE:figures/full_fig_p02… view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative Results of UG-Ground. We compare qualitative results from UG-Ground (ours) and InternV2.5-8B (baseline) on ActivityNet Captions. The orange, blue and green bars show the grounding results of baseline, UG-Ground, and ground truth respectively. Our UG-Ground method transforms the video into a sequence of BRC scores and then find the subarray with the maximum sum. 23 [PITH_FULL_IMAGE:figures/full… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes

    cs.AI 2026-05 conditional novelty 6.0

    DenoiseRL injects wrong reasoning prefixes from weak models into RL rollouts and trains the policy to recover, improving GRPO/DAPO mathematical reasoning on named benchmarks.

  2. Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines

    cs.CV 2026-04 unverdicted novelty 6.0

    Zoom consistency provides a geometric, cross-model confidence signal in zoom-in grounding pipelines that correlates with prediction correctness and enables modest gains in specialist-generalist routing.

  3. LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models

    cs.CV 2026-02 conditional novelty 6.0

    A confidence-based when-to-look gate plus semantic-guided attention where-to-look module improves training-free fine-grained visual reasoning accuracy and reduces inference cost versus search-based baselines.

  4. DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes

    cs.AI 2026-05 unverdicted novelty 4.0

    DenoiseRL optimizes recovery from noisy prefixes in weak-model reasoning failures to improve performance and self-correction on math and general reasoning benchmarks without external supervision.

Reference graph

Works this paper leans on

27 extracted references · 21 linked inside Pith · cited by 3 Pith papers

  1. [1]

    (2017) were sampled at 3 FPS, while videos from ActivityNet Captions Krishna et al

    To manage computa- tional load, videos from Charades-STA Gao et al. (2017) were sampled at 3 FPS, while videos from ActivityNet Captions Krishna et al. (2017) were sampled at 1 FPS. For baseline comparisons, we restricted the input for the base MLLMs to 64 frames considering their context length limitations. E EXTRAABLATIONSTUDIES This section provides fu...

    2017

  2. [3]

    Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923,

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report.arXiv preprint arXiv:2502.13923,

    Pith/arXiv arXiv

  3. [8]

    Language mod- els (mostly) know what they know.arXiv preprint arXiv:2207.05221,

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language mod- els (mostly) know what they know.arXiv preprint arXiv:2207.05221,

    Pith/arXiv arXiv

  4. [9]

    Tag: A simple yet effective temporal-aware approach for zero-shot video temporal grounding.arXiv preprint arXiv:2508.07925,

    Jin-Seop Lee, SungJoon Lee, Jaehan Ahn, YunSeok Choi, and Jee-Hyong Lee. Tag: A simple yet effective temporal-aware approach for zero-shot video temporal grounding.arXiv preprint arXiv:2508.07925,

    Pith/arXiv arXiv

  5. [10]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326,

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326,

    Pith/arXiv arXiv

  6. [11]

    Textcot: Zoom in for enhanced multimodal text-rich image understanding.arXiv preprint arXiv:2404.09797,

    Bozhi Luan, Hao Feng, Hong Chen, Yonghui Wang, Wengang Zhou, and Houqiang Li. Textcot: Zoom in for enhanced multimodal text-rich image understanding.arXiv preprint arXiv:2404.09797,

    Pith/arXiv arXiv

  7. [12]

    Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context.arXiv preprint arXiv:2403.05530,

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal under- standing across millions of tokens of context.arXiv preprint arXiv:2403.05530,

    Pith/arXiv arXiv

  8. [13]

    The information bottleneck method.arXiv preprint physics/0004057,

    Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method.arXiv preprint physics/0004057,

    Pith/arXiv arXiv

  9. [15]

    Valid: Mitigating the hallucination of large vision language models by visual layer fusion contrastive decoding.arXiv preprint arXiv:2411.15839,

    Jiaqi Wang, Yifei Gao, and Jitao Sang. Valid: Mitigating the hallucination of large vision language models by visual layer fusion contrastive decoding.arXiv preprint arXiv:2411.15839,

    Pith/arXiv arXiv

  10. [16]

    Divide, conquer and combine: A training-free framework for high-resolution image percep- tion in multimodal large language models

    Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image percep- tion in multimodal large language models. InAAAI, 2025a. Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et ...

    Pith/arXiv arXiv

  11. [18]

    A survey on video temporal grounding with multimodal large language model.arXiv preprint arXiv:2508.10922, 2025a

    12 Preprint Jianlong Wu, Wei Liu, Ye Liu, Meng Liu, Liqiang Nie, Zhouchen Lin, and Chang Wen Chen. A survey on video temporal grounding with multimodal large language model.arXiv preprint arXiv:2508.10922, 2025a. Penghao Wu and Saining Xie. V?: Guided visual search as a core mechanism in multimodal llms. InCVPR,

    Pith/arXiv arXiv

  12. [19]

    Generate, but verify: Reducing hallucination in vision-language models with retrospective resam- pling.arXiv preprint arXiv:2504.13169, 2025b

    Tsung-Han Wu, Heekyung Lee, Jiaxin Ge, Joseph E Gonzalez, Trevor Darrell, and David M Chan. Generate, but verify: Reducing hallucination in vision-language models with retrospective resam- pling.arXiv preprint arXiv:2504.13169, 2025b. Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. Next-qa: Next phase of question- answering to explaining temporal...

    arXiv

  13. [20]

    Mllms know where to look: Training-free perception of small visual details with multimodal llms.ICLR, 2025a

    Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. Mllms know where to look: Training-free perception of small visual details with multimodal llms.ICLR, 2025a. Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Reality check on the ev...

    Pith/arXiv arXiv

  14. [21]

    Deepeyes: Incentivizing” thinking with images” via reinforcement learning.arXiv preprint arXiv:2505.14362,

    Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing” thinking with images” via reinforcement learning.arXiv preprint arXiv:2505.14362,

    Pith/arXiv arXiv

  15. [22]

    From seconds to hours: Reviewing mul- timodal large language models on comprehensive long video understanding.arXiv preprint arXiv:2409.18938,

    Heqing Zou, Tianze Luo, Guiyang Xie, Fengmao Lv, Guangcong Wang, Junyang Chen, Zhuochen Wang, Hansheng Zhang, Huaijian Zhang, et al. From seconds to hours: Reviewing mul- timodal large language models on comprehensive long video understanding.arXiv preprint arXiv:2409.18938,

    Pith/arXiv arXiv

  16. [23]

    Entropy, a foundational concept from information theory (Shannon, 1948), provides a formal measure of the uncertainty inherent in a probability distribution

    13 Preprint A INTERPRETATION OFUNCERTAINTY INMLLMS Quantifying uncertainty is a cornerstone of building reliable and interpretable machine learning systems (Ghahramani, 2015; Gal & Ghahramani, 2016; Lakshminarayanan et al., 2017). Entropy, a foundational concept from information theory (Shannon, 1948), provides a formal measure of the uncertainty inherent...

    2015

  17. [24]

    Pretrained on massive text corpora, LLMs learn to generate reliable probability distributions over a predefined vocabulary

    has renewed interest in entropy as a vital tool for understanding and refining model performance (Shannon, 1951; Kadavath et al., 2022). Pretrained on massive text corpora, LLMs learn to generate reliable probability distributions over a predefined vocabulary. This capability allows for the direct application of entropy-based methods to address critical c...

    1951

  18. [26]

    A” and “B

    and HR-Bench (Wang et al., 2025a), feature multiple-choice questions. The model’s output typically consists of a single answer token (e.g., “A”) followed by an end-of-sequence token (⟨EOS⟩). We calculate the average token entropy across all generated tokens, including both the answer and the⟨EOS⟩token. D.2 UG-SAMPLE In UG-Sample, we treat each video frame...

    2025

  19. [2000]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971,

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth ´ee Lacroix, Baptiste Rozi `ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971,

    Pith/arXiv arXiv

  20. [2008]

    This finding aligns with principles from curriculum learning, where task difficulty can be measured by model uncertainty (Bengio et al., 2009; Kumar et al., 2010)

    between entropy and accuracy, achieving -0.90 and -0.95 for MME-RealWorld and Video-MME respectively, which represents strong negative correlation. This finding aligns with principles from curriculum learning, where task difficulty can be measured by model uncertainty (Bengio et al., 2009; Kumar et al., 2010). Furthermore, Figure 1 (b) depicts the entropy...

    2009

  21. [2017]

    The internal state of an llm knows when it’s lying.arXiv preprint arXiv:2304.13734,

    Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying.arXiv preprint arXiv:2304.13734,

    Pith/arXiv arXiv

  22. [2019]

    Threading keyframe with narratives: Mllms as strong long video comprehenders.arXiv preprint arXiv:2505.24158,

    Bo Fang, Wenhao Wu, Qiangqiang Wu, Yuxin Song, and Antoni B Chan. Threading keyframe with narratives: Mllms as strong long video comprehenders.arXiv preprint arXiv:2505.24158,

    Pith/arXiv arXiv

  23. [2020]

    Timemarker: A versatile video-llm for long and short video understanding with superior temporal localization ability.arXiv preprint arXiv:2411.18211, 2024a

    Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, and Lin Ma. Timemarker: A versatile video-llm for long and short video understanding with superior temporal localization ability.arXiv preprint arXiv:2411.18211, 2024a. Shiqi Chen, Miao Xiong, Junteng Liu, Zhengxuan Wu, Teng Xiao, Siyang Gao, and Junxian He. In-context sharpness as alerts: An inner represe...

    Pith/arXiv arXiv

  24. [2022]

    Internvideo2.5: Empowering video mllms with long and rich context modeling.arXiv preprint arXiv:2501.12386, 2025b

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, et al. Internvideo2.5: Empowering video mllms with long and rich context modeling.arXiv preprint arXiv:2501.12386, 2025b. Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long-context interleaved video-la...

    Pith/arXiv arXiv

  25. [2023]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966,

    Pith/arXiv arXiv

  26. [2024]

    Frag: Frame selec- tion augmented generation for long video and long document understanding.arXiv preprint arXiv:2504.17447,

    De-An Huang, Subhashree Radhakrishnan, Zhiding Yu, and Jan Kautz. Frag: Frame selec- tion augmented generation for long video and long document understanding.arXiv preprint arXiv:2504.17447,

    Pith/arXiv arXiv

  27. [2025]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    Pith/arXiv arXiv