Pith. sign in

REVIEW 5 major objections 5 minor 95 references

LookBack: Where and How to Score LVLM Responses via Visual Reference Usage

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LookBack, a training-free scorer that combines token likelihood with a visual lookback attention signal, consistently improves Best-of-N response selection for vision-language models across four benchmarks and three models.

desk verdict A cheap and honest scoring trick that works on average, but the grounding claim is under-tested and the evaluation has fixable holes. read the letter →

arxiv 2608.11847 v1 pith:53DB3CTI submitted 2026-08-12 cs.CV cs.AIcs.CL

Beomsik Cho , Jinhyeong Kim , Dongseok Lee , Jaehyung Kim This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords Best-of-Nselectionvisualhallucinationlargevision-languagemodelsresponsescoringattentionweightsgroundingtraining-freeself-certainty
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that choosing among sampled responses of a vision-language model fails when the scorer only measures text confidence, because that confidence barely changes when the image is removed. To fix this, it proposes LookBack, a training-free scorer that combines each token's likelihood with a visual lookback score, the fraction of attention that token pays to image tokens, and weights tokens by how visually relevant they are. Across four image-grounded benchmarks and three different vision-language models, LookBack selects better Best-of-N responses than existing linguistic and vision-side baselines, at negligible extra cost. A sympathetic reader would care because it suggests model-internal attention is a usable, low-cost calibration signal for visual grounding, without reward models or extra inference.

What carries the argument

The load-bearing object is the token-level visual lookback score $A_t$, defined in Eq. (3) as the mean over layers and heads of the fraction of attention weight from the prediction step that falls on vision-token positions $P_v\subset C_t$. It feeds two places: the lookback-calibrated token score $u_t = \log(p_t) + \alpha\log(A_t)$, which lowers the score of high-likelihood tokens that do not attend to the image; and the visual relevance distribution $q_\lambda(t) = A_t^\lambda/\sum_j A_j^\lambda$, the closed-form solution of an entropy-regularized relevance maximization (Proposition 1) that concentrates response-level weight on visually attended positions. The final score $S(y|x,v)=\sum_t q_\lambda(t)u_t$ is equivalent to maximizing a visual-relevance-weighted geometric mean of $p_t A_t^\alpha$ (Corollary A.0.1), which makes it length-normalized. All quantities come from the generation forward pass, so no extra inference or external model is needed.

What would settle it

Zero out or randomize the attention paid to vision tokens while keeping the token probability distributions unchanged, then rerun LookBack Best-of-N selection on the same four benchmarks; if selection accuracy stays the same as with intact attention, then visual lookback is not the mechanism behind the reported gains.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that output-space confidence in LVLMs is largely image-insensitive: the distribution of Self-Certainty scores barely shifts when the image is removed, and the same top candidate is selected 0.36–0.64 of the time, far above the 1/25 random rate. Visual lookback score, defined as the mean over layers and heads of the fraction of attention each generation step directs to vision tokens, is higher for visually referential words (nouns, adjectives, numerals) and lower for function words, exactly opposite to confidence. LookBack combines the two by scoring each token as $\log(p_t) + \alpha\log(A_t)$ and aggregating under the distribution $q_\lambda(t)\propto A_t^\lambda$. The result is that Best-of-N selection improves consistently across VQAv2, CHAIR, AMBER, and HallusionBench and over the three tested LVLMs, with an average gain from 65.37% to 68.62% over random selection, requiring no auxiliary verifier, training, or extra inference pass.

Load-bearing premise

The whole construction rests on the assumption that the fraction of attention a token pays to image tokens faithfully measures how much that token actually relies on the image; if attention is miscalibrated or visual information flows through other pathways, the lookback calibration stops working.

Editorial extensions

If this is right

  • Best-of-N selection for vision-language models can be improved without reward models or extra inference, by reusing attention weights computed during sampling.
  • The diagnostic itself, removing the input image and measuring top-1 agreement of confidence scores, is a reusable sanity test for whether any LVLM response scorer is actually sensitive to the image.
  • Because the scorer needs internal attention weights, the method is usable on open-architecture LVLMs but not on black-box API-only models.
  • The same source-aware scoring principle should transfer to any generation task where high-confidence tokens must refer back to a designated part of the context, such as retrieved documents, tool outputs, or instruction tokens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to turn the calibrated token score into a training signal, for example by training a model to increase $A_t$ for high-likelihood tokens, potentially reducing hallucination without a reward model; the paper only uses the score for selection.
  • The paper averages $A_t$ over all layers and heads, an easy extension would be to select or learn layer and head subsets, since attention calibration varies across architectures and the authors' own limitations note this dependence.
  • The entropy-regularized weighting suggests an adaptive $\lambda$ per response: responses with attention concentrated on a few tokens might need sharper weighting, while diffuse attention might need smoother weighting to avoid over-penalizing valid low-attention tokens.
  • The per-model hyperparameters differ widely ($\alpha=7$ for LLaVA versus $0.25$ for InternVL3), so cross-model transfer of hyperparameters without tuning is untested; a further study could examine whether $\alpha$ tracks attention magnitude.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper studies Best-of-N response selection for LVLMs. It first presents a diagnostic showing that an output-space confidence scorer (Self-Certainty) is largely insensitive to the input image: score distributions barely shift when the image is removed, and top-1 agreement between image-present and image-absent scoring is 0.36–0.64. It then proposes LOOKBACK, a training-free scorer that adds a token-level visual lookback score A_t (the fraction of attention from each output token to vision tokens, averaged over layers and heads) to the log-likelihood, and aggregates tokens with a visual-relevance distribution q_lambda derived from A_t. The final score is a weighted average of log p_t + alpha log A_t under q_lambda. Experiments on VQAv2, CHAIR, AMBER, and HallusionBench with LLaVA-1.5-7B, Qwen2.5-VL-7B, and InternVL3-8B report consistent improvements in Best-of-N selection over SC, USC, CLIPScore, and VAUQ, with small scoring overhead.

Significance. The paper's motivation is clearly demonstrated: the image-removal diagnostic for Self-Certainty is simple and visually convincing. The proposed score is elegant, parameter-light in form (two hyperparameters), does not require external models, and the paper ships code and a reproducible experimental protocol. If the reported gains hold out-of-sample, LOOKBACK would be a useful, lightweight addition to the LVLM response-selection toolbox, and the attention-based calibration idea is worth publishing. However, the significance is currently limited by the evaluation protocol: the two free hyperparameters are tuned per model on the same benchmark suite that is used for the headline numbers, no variance or significance information is reported, and the central interpretation of A_t as visual reference usage is only partially validated (the authors themselves note in the Limitations that lookback is only a proxy and can reward incorrect responses).

major comments (5)
  1. [§5.2, Table 1] The headline claim of consistent improvement is not supported across all cells. For InternVL3-8B on HallusionBench at N=25, LookBack (56.85) is below Random (57.19) and only ties SC (56.85). This is a clear counterexample to the abstract's 'consistently improves' and should be discussed explicitly, along with a tempered claim and per-cell analysis.
  2. [§5.1, Table 1] The per-model hyperparameters (alpha, lambda) are set on the same benchmarks on which the main results are reported, with no validation split and no sensitivity analysis in the main paper. This makes the reported 4.97% relative gain over random partly a result of tuning rather than a parameter-free prediction. Please provide results with fixed hyperparameters shared across models, a validation/test split, or per-benchmark sensitivity curves for all three models.
  3. [§5.2, Table 1] No variance or statistical significance is reported anywhere. With 1,000 sampled instances, the standard error of a percentage near 60–80 is roughly 1.2–1.5 points, so many of the reported differences (e.g., LLaVA CHAIR N=25: 74.43 vs 73.53; Qwen VQAv2 N=5: 68.23 vs 66.88) are within plausible noise. Please report standard errors or confidence intervals, and state which cells are statistically distinguishable from the best baseline.
  4. [§4.1, Eq. (3)] The central claim that A_t measures 'visual reference usage' is not established. A_t averages attention over all vision tokens and all layers, so it rewards attention to any image token, including global/CLS tokens, background patches, or irrelevant distractors. The qualitative examples in Figs. 9–10 actually demonstrate this: lookback alone highlights cars/house/truck and child/toothbrush, which are not the question-relevant evidence. The POS analysis (Fig. 4) only shows that content words attend to vision tokens more than function words on average; it never compares A_t between correct and hallucinated tokens. Please add (i) a mismatched-image control (e.g., the same question paired with a different image) and (ii) a token-level analysis of A_t on hallucinated vs. correct claims, to test whether A_t reflects actual grounding rather than any image attention.
  5. [§3.1, Figs. 2–3] The image-insensitivity diagnostic is run only for Self-Certainty and only for LLaVA-1.5-7B and Qwen2.5-VL-7B. The interpretation of the top-1 agreement (0.36–0.64) as 'barely changes selection' lacks a reference point: a vision-sensitive scorer under the same image-removal perturbation might also agree substantially because most candidates remain similar in quality. Please add a reference comparison, or alternatively show that the agreement of LookBack (or another vision-sensitive signal) under image removal is substantially lower.
minor comments (5)
  1. [§5.2 and Table 1] The main text states 'improves the average score from 65.37% to 68.62%', but Table 1's Average column does not contain these numbers; please clarify the aggregation across model–benchmark–N cells.
  2. [§D, Figs. 9–10] The color scale for token-level highlighting is not described; please state what the darkness indicates (e.g., z-scored score, normalized within response).
  3. [§6, Limitations] The Limitations section correctly acknowledges that visual lookback is only a proxy and that attention to image tokens does not guarantee correctness; please move this caveat earlier (e.g., at the end of Sec. 3.2) so it qualifies the motivation before the method is introduced.
  4. [§B.2] For USC, the text reports averaging over three runs but does not report the variance; since USC is stochastic, please give the standard deviation or justify that it is negligible.
  5. [§5.1, Implementation details] Please report the number of instances used per benchmark for the main experiments; the motivation analysis uses 1,000, but the main tables do not state N.

Circularity Check

1 steps flagged · score 4.0 of 10

LookBack's formal equations are self-contained, but its central empirical claim is partially in-sample because the two hyperparameters (α, λ) are set per model on the evaluation suite without a described validation split.

  1. fitted input called prediction [Section 5.1 (Implementation details), Eq. 8, Table 1]
    "We set these hyperparameters on a per-model basis. Specifically, we use (α,λ) = (7.0,1.5) for LLaVA-1.5-7B, (0.5,1.25) for Qwen2.5-VL-7B, and (0.25,1.25) for InternVL3-8B. See Appendix B.2 for details."

    Eq. 8 defines the final score S(y|x,v) as a weighted combination of log p_t and log A_t with exponents α and λ, so these two parameters directly determine the response ranking and hence the reported Best-of-N accuracy. The paper then sets (α, λ) per model with no held-out validation split described, and reports improvements on the same four benchmarks that were used to select the values. The tuning signal and the reported evaluation metric are therefore the same; the 'consistently improves' result is a fitted optimum of the reported numbers rather than an independent, parameter-free prediction of the benefit of visual lookback. This is partial circularity: the structural hypothesis still has independent content, but the headline gain is not fully out-of-sample.

full rationale

The formal derivation chain (Eq. 3 through Eq. 8) is self-contained: A_t is defined as a normalized attention fraction over vision tokens, u_t is a log-linear combination of token likelihood and lookback, and q_λ is the closed-form solution of a stated entropy-regularized optimization (Proposition 1, Appendix A.2). No step derives the result from the conclusion, and the POS analysis in Sec. 3.2 gives an independent, though limited, check that content words receive higher A_t than function words. The Limitations section explicitly concedes that 'visual lookback is only a proxy for visual reference usage, not a guarantee of factual correctness,' so the paper does not hide the gap between the attention statistic and true grounding. The main circularity concern is experimental rather than derivational: Section 5.1 states that α and λ are set on a per-model basis, and no validation split is described, while Table 1 reports gains on the same benchmarks. Because the reported score in Eq. 8 is a direct function of these tuned parameters, the 'consistently improves' claim is partly an in-sample fitting result. Still, this is not total circularity: the central idea that attention to vision tokens carries selection signal has independent empirical content, and the self-citations in Related Works are background rather than load-bearing.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method depends on attention as a proxy for grounding, on the validity of the image-removal probe, and on free hyperparameters alpha and lambda tuned per model. No invented entities are introduced.

free parameters (2)
  • alpha (token-level lookback calibration strength) = 7.0 (LLaVA-1.5-7B), 0.5 (Qwen2.5-VL-7B), 0.25 (InternVL3-8B)
    Controls how much the visual lookback score modifies token likelihood in Eq. 4; set per model with no validation split described, so it is fitted to the evaluation setup.
  • lambda (visual relevance weighting strength) = 1.5 (LLaVA-1.5-7B), 1.25 (Qwen2.5-VL-7B), 1.25 (InternVL3-8B)
    Controls sharpness of the visual relevance distribution q_lambda in Eqs. 6 and 7; set per model with no validation split described.
assumptions (3)
  • domain assumption Attention weights are a causally meaningful measure of how much each generated token references vision tokens.
    The visual lookback score A_t in Eq. 3 averages attention fractions to vision tokens across layers and heads; the method assumes this reflects reference usage. The authors flag this as a proxy in Limitations.
  • domain assumption Output-space confidence computed without the image is a valid probe for measuring the image sensitivity of confidence-based selection.
    The motivation in Section 3 infers that confidence is image-insensitive because SC with and without the image rank candidates similarly; this assumes the image-removal probe is a valid measurement of sensitivity.
  • domain assumption The sampled candidate pools contain enough quality variation for Best-of-N selection to be meaningful.
    The paper explicitly uses oracle gaps (Table 2) to justify headroom; the scoring evaluation depends on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LookBack: Where and How to Score LVLM Responses via Visual Reference Usage." pith.science (2026). https://pith.science/paper/53DB3CTI

@misc{pith2026260811847,
  author       = {Pith},
  title        = {Pith review of: LookBack: Where and How to Score LVLM Responses via Visual Reference Usage},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/53DB3CTI}},
  note         = {Machine review of arXiv:2608.11847}
}
abstract

Large Vision-Language Models (LVLMs) integrate visual perception with language generation, enabling responses that span image understanding and complex reasoning. However, LVLMs do not just inherit the text-level hallucinations; they also hallucinate against the image, producing fluent responses ungrounded in what they see. This makes LVLM response scoring inherently harder, and our diagnostics show that existing confidence-based metrics adopted from LLMs are insufficient for LVLMs. Specifically, removing the input image barely changes confidence-based selection, suggesting that output-space confidence primarily captures textual plausibility rather than agreement with the image. To address this gap, we propose LookBack, a training-free LVLM response scoring method that augments token likelihood with visual lookback score, a lightweight measure of how strongly each response token refers to image tokens. Across four benchmarks and three models, LookBack consistently improves Best-of-$N$ selection over existing baselines with negligible additional overhead.

Figures

Figures reproduced from arXiv: 2608.11847 by the authors.

Figure 1
Figure 1. Overview of LOOKBACK: combining output-space confidence with visual reference usage. Given an image and a question, an LVLM generates multiple candidate responses and our goal is to select the proper one. Output-space confidence can assign high scores to fluent but visually unsupported tokens, while visual reference usage highlights tokens that directly look back to the image but does not by itself ensure response c… view at source ↗
Figure 2
Figure 2. SC score distributions remain similar with and without the image. SC score distributions under image-present and image-removed conditions highly overlap, with closely aligned means across setups. 3.1 Is Output-Space Confidence Sensitive to Visual Input? We first examine whether SC is affected by the visual input by comparing its score distributions under image-present and image-absent conditions. For each generated … view at source ↗
Figure 3
Figure 3. Top-1 agreement with image-conditioned SC: image-removal consistency and visual-lookback complementarity. We report the fraction of instances in which two scorers select the same top-ranked response from N = 25 candidates. largely unchanged even without the image. Thus, output-space confidence alone cannot reliably indi￾cate whether a response is grounded in the image. 3.2 Visual Lookback Complements Token-Level Con… view at source ↗
Figures from the paper (5 more)
Figure 7
Figure 7. Figure 7: Scoring overhead. We compare the aver￾age scoring time per response across three LVLMs on CHAIR. Error bars indicate standard deviation. heavily. In the bottom panel, increasing λ improves performance when α = 0, but with α = 7, perfor￾mance peaks at a moderate λ and d…
Figure 6
Figure 6. Figure 6: Ablation study. We ablate token-level calibra￾tion strength α and the response-level visual weighting strength λ on AMBER with LLaVA-1.5-7B. Top: per￾formance when varying α with fixed λ. Bottom: perfor￾mance when varying λ with fixed α. Dashed lines show the correspon…
Figure 8
Figure 8. Figure 8: Visual lookback and SC emphasize different POS groups. We report mean z-scored Visual lookback and Self-Certainty (SC) scores for each POS category. Across models and benchmarks, visual POS categories tend to receive higher Visual lookback scores, whereas textual categ…
Figure 9
Figure 9. Figure 9: Token-level comparison of response scorers. SC favors a fluent but incorrect response, and Visual Lookback Score highlights visually referential tokens but remains insufficient on its own. LOOKBACK combines confidence and visual lookback to select the visually grounded…
Figure 10
Figure 10. Figure 10: Token-level behavior on a visually complex example. Visual lookback Score broadly highlights image-referential tokens, whereas SC places large scores on a few confident tokens. LOOKBACK instead emphasizes question-relevant visual evidence, selecting the correct respon…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

95 extracted references · 43 canonical work pages

  1. [1]

    arXiv preprint arXiv:2412.10302 , year=

    Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding , author=. arXiv preprint arXiv:2412.10302 , year=

    arXiv

  2. [2]

    arXiv preprint arXiv:2302.13971 , year=

    Llama: Open and efficient foundation language models , author=. arXiv preprint arXiv:2302.13971 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  4. [4]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    arXiv

  5. [5]

    arXiv preprint arXiv:2308.12966 , year=

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond , author=. arXiv preprint arXiv:2308.12966 , year=

    arXiv

  6. [6]

    arXiv preprint arXiv:2304.14178 , year=

    mplug-owl: Modularization empowers large language models with multimodality , author=. arXiv preprint arXiv:2304.14178 , year=

    arXiv

  7. [7]

    Visual instruction tuning , author=

  8. [8]

    InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning , author=

Show all 95 references
  1. [9]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks , author=

  2. [10]

    arXiv preprint arXiv:2306.15195 , year=

    Shikra: Unleashing multimodal llm's referential dialogue magic , author=. arXiv preprint arXiv:2306.15195 , year=

    arXiv

  3. [11]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models , author=

  4. [12]

    Improved baselines with visual instruction tuning , author=

  5. [13]

    5-vl technical report , author=

    Qwen2. 5-vl technical report , author=. arXiv preprint arXiv:2502.13923 , year=

    arXiv

  6. [14]

    VQA: Visual Question Answering , author =

  7. [15]

    Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization , author =

  8. [16]

    Li, Junnan and Li, Dongxu and Savarese, Silvio and Hoi, Steven , booktitle = icml, year =

  9. [17]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi , author=

  10. [18]

    You, Quanzeng and Jin, Hailin and Wang, Zhaowen and Fang, Chen and Luo, Jiebo , title =

  11. [19]

    Show and tell: A neural image caption generator , author=

  12. [20]

    Show, attend and tell: Neural image caption generation with visual attention , author=

  13. [21]

    From recognition to cognition: Visual commonsense reasoning , author=

  14. [22]

    arXiv preprint arXiv:2205.06907 , year=

    Multimodal conversational AI: A survey of datasets and approaches , author=. arXiv preprint arXiv:2205.06907 , year=

    arXiv

  15. [23]

    arXiv preprint arXiv:2409.11402 , year=

    Nvlm: Open frontier-class multimodal llms , author=. arXiv preprint arXiv:2409.11402 , year=

    arXiv

  16. [24]

    Ferret: Refer and ground anything anywhere at any granularity , author=

  17. [25]

    arXiv preprint arXiv:2405.20797 , year=

    Ovis: Structural embedding alignment for multimodal large language model , author=. arXiv preprint arXiv:2405.20797 , year=

    arXiv

  18. [26]

    Self-introspective decoding: Alleviating hallucinations for large vision-language models , author=

  19. [27]

    See what you are told: Visual attention sink in large multimodal models , author=

  20. [28]

    Damro: Dive into the attention mechanism of lvlm to reduce object hallucination , author=

  21. [29]

    Mitigating object hallucinations in large vision-language models through visual contrastive decoding , author=

  22. [30]

    Multi-modal hallucination control by visual information grounding , author=

  23. [31]

    Science China Information Sciences (SCIS) , year=

    Woodpecker: Hallucination correction for multimodal large language models , author=. Science China Information Sciences (SCIS) , year=

  24. [32]

    arXiv preprint arXiv:2310.00754 , year=

    Analyzing and mitigating object hallucination in large vision-language models , author=. arXiv preprint arXiv:2310.00754 , year=

    arXiv

  25. [33]

    Contrastive decoding: Open-ended text generation as optimization , author=

  26. [34]

    interpreting GPT: the logit lens , author=

  27. [35]

    International Conference on Pattern Recognition (ICPR) , year=

    Branchynet: Fast inference via early exiting from deep neural networks , author=. International Conference on Pattern Recognition (ICPR) , year=

  28. [36]

    Depth-adaptive transformer , author=

  29. [37]

    Confident adaptive language modeling , author=

  30. [38]

    Dola: Decoding by contrasting layers improves factuality in large language models , author=

  31. [39]

    Mllm can see? dynamic correction decoding for hallucination mitigation , author=

  32. [40]

    Object hallucination in image captioning , author=

  33. [41]

    Evaluating object hallucination in large vision-language models , author=

  34. [42]

    arXiv preprint arXiv:2311.07397 , year=

    Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation , author=. arXiv preprint arXiv:2311.07397 , year=

    arXiv

  35. [43]

    Microsoft coco: Common objects in context , author=

  36. [44]

    A-okvqa: A benchmark for visual question answering using world knowledge , author=

  37. [45]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering , author=

  38. [46]

    arXiv preprint arXiv:2403.18715 , year=

    Mitigating hallucinations in large vision-language models with instruction contrastive decoding , author=. arXiv preprint arXiv:2403.18715 , year=

    arXiv

  39. [47]

    Paying more attention to image: A training-free method for alleviating hallucination in lvlms , author=

  40. [48]

    arXiv preprint arXiv:2405.17820 , year=

    Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models , author=. arXiv preprint arXiv:2405.17820 , year=

    arXiv

  41. [49]

    Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation , author=

  42. [50]

    Halc: Object hallucination reduction via adaptive focal-contrast decoding , author=

  43. [51]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  44. [52]

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models , author=

  45. [53]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering , author=

  46. [54]

    Code: Contrasting self-generated description to combat hallucination in large multi-modal models , author=

  47. [55]

    Interpreting and editing vision-language representations to mitigate hallucinations , author=

  48. [56]

    Devils in middle layers of large vision-language models: Interpreting, detecting and mitigating object hallucinations via attention lens , author=

  49. [57]

    arXiv preprint arXiv:2312.10997 , volume=

    Retrieval-augmented generation for large language models: A survey , author=. arXiv preprint arXiv:2312.10997 , volume=

    arXiv

  50. [58]

    Retrieval-augmented generation for knowledge-intensive nlp tasks , author=

  51. [59]

    Biometrics bulletin , year=

    Individual comparisons by ranking methods , author=. Biometrics bulletin , year=

  52. [60]

    arXiv preprint arXiv:2412.05271 , year=

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling , author=. arXiv preprint arXiv:2412.05271 , year=

    arXiv

  53. [61]

    arXiv preprint arXiv:2504.10479 , year=

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models , author=. arXiv preprint arXiv:2504.10479 , year=

    arXiv

  54. [62]

    arXiv preprint arXiv:2112.00861 , year=

    A general language assistant as a laboratory for alignment , author=. arXiv preprint arXiv:2112.00861 , year=

    arXiv

  55. [63]

    Transformer Circuits Thread , year=

    A mathematical framework for transformer circuits , author=. Transformer Circuits Thread , year=

  56. [64]

    arXiv preprint arXiv:2212.08073 , year=

    Constitutional ai: Harmlessness from ai feedback , author=. arXiv preprint arXiv:2212.08073 , year=

    arXiv

  57. [65]

    International Conference on Acoustics, Speech and Signal Processing (ICASSP) , year=

    Fuzzllm: A novel and universal fuzzing framework for proactively discovering jailbreak vulnerabilities in large language models , author=. International Conference on Acoustics, Speech and Signal Processing (ICASSP) , year=

  58. [66]

    Proceedings of the 5th Workshop on Machine Learning and Systems , year=

    Understanding oversubscribed memory management for deep learning training , author=. Proceedings of the 5th Workshop on Machine Learning and Systems , year=

  59. [67]

    Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , year=

    On the dangers of stochastic parrots: Can language models be too big? , author=. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , year=

    2021

  60. [68]

    Advances in Neural Information Processing Systems , year=

    Many-shot jailbreaking , author=. Advances in Neural Information Processing Systems , year=

  61. [69]

    URL https://arxiv

    Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023 , author=. URL https://arxiv. org/abs/2312.06674 , year=

    2023 arXiv

  62. [70]

    Neural computation , volume=

    Training products of experts by minimizing contrastive divergence , author=. Neural computation , volume=. 2002 , publisher=

    2002

  63. [71]

    Bonbon alignment for large language models and the sweetness of best-of-n sampling , author=

  64. [72]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters , author=

  65. [73]

    Self-consistency improves chain of thought reasoning in language models , author=

  66. [74]

    Chain-of-thought reasoning without prompting , author=

  67. [75]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  68. [76]

    arXiv preprint arXiv:2211.14275 , year=

    Solving math word problems with process-and outcome-based feedback , author=. arXiv preprint arXiv:2211.14275 , year=

    arXiv

  69. [77]

    Let's verify step by step , author=

  70. [78]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations , author=

  71. [79]

    arXiv preprint arXiv:2503.10291 , year=

    Visualprm: An effective process reward model for multimodal reasoning , author=. arXiv preprint arXiv:2503.10291 , year=

    arXiv

  72. [80]

    arXiv preprint arXiv:2508.03556 , year=

    Vrprm: Process reward modeling via visual reasoning , author=. arXiv preprint arXiv:2508.03556 , year=

    arXiv

  73. [81]

    Scalable best-of-n selection for large language models via self-certainty , author=

  74. [82]

    arXiv preprint arXiv:2506.17251 , year=

    Training-free LLM Verification via Recycling Few-shot Examples , author=. arXiv preprint arXiv:2506.17251 , year=

  75. [83]

    arXiv preprint arXiv:2207.05221 , year=

    Language models (mostly) know what they know , author=. arXiv preprint arXiv:2207.05221 , year=

    arXiv

  76. [84]

    Semantic density: Uncertainty quantification for large language models through confidence measurement in semantic space , author=

  77. [85]

    Learning to reason without external rewards , author=

  78. [86]

    INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling , author=

  79. [87]

    Findings of the Association for Computational Linguistics: NAACL 2025 , year=

    Mitigating hallucinations in large vision-language models via summary-guided decoding , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , year=

    2025

  80. [88]

    Findings of the Association for Computational Linguistics: ACL 2024 , year=

    Prometheus-vision: Vision-language model as a judge for fine-grained evaluation , author=. Findings of the Association for Computational Linguistics: ACL 2024 , year=

    2024

  81. [89]

    Llava-critic: Learning to evaluate multimodal models , author=

  82. [90]

    Clipscore: A reference-free evaluation metric for image captioning , author=

  83. [91]

    Calibrated self-rewarding vision language models , author=

  84. [92]

    arXiv preprint arXiv:2602.21054 , year=

    VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation , author=. arXiv preprint arXiv:2602.21054 , year=

    arXiv

  85. [93]

    arXiv preprint arXiv:2311.17311 , year=

    Universal self-consistency for large language model generation , author=. arXiv preprint arXiv:2311.17311 , year=

    arXiv

  86. [94]

    arXiv preprint arXiv:2510.06953 , year=

    Revisiting the Uniform Information Density Hypothesis in LLM Reasoning Traces , author=. arXiv preprint arXiv:2510.06953 , year=

    arXiv

  87. [95]

    arXiv preprint arXiv:1904.09751 , year=

    The curious case of neural text degeneration , author=. arXiv preprint arXiv:1904.09751 , year=

    1904 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.