{"id":"fecf0dd2-be58-4fc5-b4a3-6fdccf88301f","arxiv_id":"2608.11474","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"At each decoding step, TTH blends the LVLM's logits for candidate object tokens with CLIP image-text similarity scores, weighted by the model's uncertainty, to suppress hallucinated objects.","lead":"This paper introduces TTH, a training-free method that reduces object hallucinations in vision-language models by checking candidate object words against the image with CLIP during decoding. It reports small but consistent gains over prior methods on three benchmarks across three model families.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No head-to-head comparison with the closest prior CLIP-guided decoding method (SiB, ref [13]); TTH's claimed novelty and margins over prior baselines are unsubstantiated.","rationale":"The reader's overall CONDITIONAL verdict is sound, but the reader's nominated weakest assumption (CLIP similarity reliability and rescaling in Eq. 7) is not the most load-bearing issue. The paper's own experiments provide empirical evidence that CLIP-based validation improves CHAIR, OPOPE, and LLaVA-Bench, so the CLIP-reliability concern is a robustness caveat rather than a direct falsifier of the central claim. The deeper problem is that the method is closely related to SiB (ref [13]), the closest prior CLIP-guided decoding work, yet SiB is neither correctly characterized nor benchmarked. If SiB is a single-pass, logit-level, training-free method, then TTH's stated motivation and its claim to 'surpass previous baselines' are unsupported because the decisive prior baseline was omitted. This matches the reader's rationale identifying SiB as the main blocker, but differs from the reader's weakest_assumption field. I also credit the paper for releasing code and reporting consistency across three LVLMs, which is why the verdict stays CONDITIONAL rather than REJECT: the empirical evidence is real but the positioning and comparison set are incomplete. The concrete check (running SiB under the same protocol) would settle whether the concern actually lands: if TTH beats SiB, the conditional concern dissolves; if it does not, the paper must be reframed as an incremental variant or rejected as a novel contribution.","tokens_in":17135,"tokens_out":9561,"duration_ms":90661,"concrete_test":"Run SiB (the released code for arXiv:2402.15300) under the exact protocol of Section 4.1: same three LVLMs (LLaVA-1.5, MiniGPT-4, mPLUG-Owl2), same CHAIR/OPOPE/LLaVA-Bench evaluation, same three seeds, and same beam settings. Record CHAIRS, CHAIRI, OPOPE accuracy/precision/F1, and throughput. If TTH does not outperform SiB by a clear margin (e.g., >1 point CHAIRS or comparable with lower cost), or if SiB is confirmed to be single-pass and non-invasive, then the paper's novelty and superiority claims fail and the contribution is an incremental variant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of TTH is that it is a 'novel training-free method' that 'surpasses previous baselines' while avoiding multiple decoding rounds and internal-state editing. The paper does cite SiB (Deng et al., arXiv:2402.15300) as ref [13], but only in a passing sentence lumping it with methods that rely on 'multiple generations from the original prompt,' and it is never included in any comparison table. If SiB is in fact a single-pass, training-free, logit-level CLIP-guided decoding method, then TTH's core mechanism (top-k object candidates, CLIP similarity 'a photo of {w_i}', rescaling to the LVLM logit range, entropy-weighted fusion) is a minor variant of SiB, not a new approach. The reported gains over Nullu, OPERA, HALC, and VCD do not establish an advance over the closest prior work: omitting SiB from Tables 1 and 2 means a reader cannot tell whether TTH outperforms SiB at all. This directly threatens the abstract's novelty claim and the claim of addressing limitations that SiB may not have. The paper's own Section 2 text about ref [13] is also internally questionable: it describes SiB as requiring multiple generations, which contradicts the single-pass design suggested by its title and methodology. A load-bearing part of the paper's argument is therefore unsupported: either TTH must be shown to differ meaningfully from and outperform SiB, or the contribution must be reframed as incremental.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Test-Time Hallucination Mitigation (TTH), a training-free decoding-time method for reducing object hallucinations in large vision-language models. At each decoding step, TTH takes the top-k tokens from the LVLM's logits, filters them to object candidates using WordNet, verifies each candidate by computing CLIP similarity between the image and the prompt 'a photo of {w}', rescales these similarities into the LVLM logit range, and fuses the two logit sets with an entropy-based weighting that upweights CLIP when the LVLM is uncertain. The method is evaluated on CHAIR, OPOPE, and LLaVA-Bench across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, reporting consistent improvements over Nullu, OPERA, HALC, VCD, and greedy/beam baselines, with a reported modest throughput cost. The paper also provides ablations over the number of candidates and the fusion weighting scheme.","tokens_in":17424,"tokens_out":8826,"duration_ms":66351,"significance":"If the results hold, TTH is a simple, practical plug-in that reduces object hallucinations without retraining or internal-state modification, and the authors release code and report three-seed averages with fixed random subsets, which aids reproducibility. The consistency of the gains across three LVLMs and three benchmarks, including the OPOPE accuracy improvements over strong baselines, is a useful contribution, and the efficiency analysis in Figure 4 quantifies the practical cost. The principal caveat is that the closest prior work, SiB (CLIP-guided decoding), is not benchmarked, so the claimed novelty and advantage over existing single-pass CLIP-guided methods remain unverified.","major_comments":[{"comment":"The manuscript does not include a comparison with SiB (ref. [13]), the closest prior work in CLIP-guided decoding, and the related-work description of [13] as relying on 'multiple generations from the original prompt' appears inconsistent with SiB's stated single-pass CLIP-guided decoding design. Since the paper's central novelty claim is that TTH is the first training-free single-pass method that avoids both multiple decoding rounds and internal-state editing, the absence of SiB from Tables 1 and 2 leaves that claim unsubstantiated. The authors should add SiB as a baseline, report its results on CHAIR and OPOPE for the same LVLMs, and either demonstrate a clear margin or explicitly characterize the technical differences (e.g., candidate selection, rescaling, fusion) that justify the claimed advance.","section":"§2, Related Work; §4.1, Baselines; Tables 1–2"},{"comment":"The min-max rescaling of CLIP similarities into the range of the LVLM candidate logits (Eq. 7) can become ill-conditioned when the candidate logit range is very narrow, amplifying small differences in CLIP scores; conversely, the entropy-weighted fusion (Eq. 9) can allow noisy CLIP scores to override a confident LVLM prediction. The paper provides no analysis of this sensitivity, such as the distribution of candidate logit ranges, the calibration of CLIP similarity scores across images, or failure cases where TTH changes a correct object into an incorrect one. A concrete test would be to compare the proposed rescaling against alternatives (e.g., softmax temperature matching or score standardization) and to report the frequency and severity of such flips.","section":"§3.2, Eq. (7) and Eq. (9)"},{"comment":"The ablation over the number of candidates shows a sharp optimum at k=10, and all main results use this value, but the paper does not state how k was selected or whether it was chosen on the same MSCOCO subset used for evaluation. If k was tuned on the evaluation set, the reported improvements may reflect some degree of overfitting. The authors should clarify the selection procedure, report whether k was chosen on a validation split, and provide results for neighboring values of k (e.g., k=5, 15, 20) in the main tables to demonstrate robustness.","section":"§4.3, Figure 5; §4.2, Tables 1–2"}],"minor_comments":[{"comment":"The method name is inconsistent across the paper: the title uses 'Test-Time Hallucination Control', the abstract says 'Test-Time Hallucination Mitigation (TTH)', and Section 6 calls it 'Test-Time Hallucination (TTH)'; the authors should standardize the name.","section":"Title; Abstract; §6"},{"comment":"The image and the query are both denoted by the symbol I in Equation (1) and the surrounding text, which is confusing; please use distinct symbols, for example I_img for the image and q for the prompt.","section":"§3.1, Eq. (1)"},{"comment":"The sentence 'The first category of hallucination mitigation approaches relies on extensive fine-tuning [6,13,30,44,51,74]' misplaces several decoding-based methods (HALC [6], SiB [13], and VCD [30]) in the fine-tuning category; these are inference-time or decoding methods and should be cited in the appropriate category.","section":"§1, Introduction, page 2"},{"comment":"The LLaVA-Bench evaluation does not report standard deviations, the number of independent GPT-4V evaluations, or the exact prompt used for scoring; please provide these details to allow reproducibility and to assess the stability of the reported improvements.","section":"§4.2, GPT-4V Aided Evaluation on LLaVA-Bench; Table 3"},{"comment":"The annotations '3% 6xFaster' and '5% 1.5xFaster' in Figure 4 are ambiguous; please clarify what the percentages refer to and how the speedup factors are computed.","section":"Figure 4"},{"comment":"References [67] and [68] appear to be duplicates of the same preprint; please merge or remove the redundant entry.","section":"References"},{"comment":"The limitations section does not mention that only object candidates that appear in the LVLM's top-k set can be validated; if the correct object is outside the top-k, TTH cannot introduce it. This is an inherent design limitation and should be stated explicitly.","section":"§5, Limitations"}],"recommendation":"major_revision","confidential_remarks":"The missing SiB comparison is the most serious issue, because if SiB is indeed a single-pass, logit-level CLIP-guided decoding method, the paper's central novelty claim is not supported by the current experiments. I would encourage the editor to request a direct comparison and a corrected related-work description of SiB before further consideration. The paper also contains a number of presentation issues (naming inconsistency, mis-categorized citations, duplicate references) that should be cleaned up."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nShort version: this is a plausible, well-engineered method for reducing object hallucination in LVLMs, but the paper oversells its novelty by failing to benchmark or accurately describe the closest prior work, which is one of its own references. That gap needs fixing before I'd trust the claims.\n\nWhat is actually good: TTH is simple, training-free, and runs in a single pass at the logit level. The experiments cover three LVLM families with CHAIR, OPOPE, and LLaVA-Bench, and the results are consistently positive, if modest over the strongest baseline (Nullu). The efficiency analysis is honest: TTH is much faster than OPERA or HALC, though slower than greedy decoding. The entropy-adaptive fusion is a reasonable twist, and the ablation supports it over fixed weighting. Code is released.\n\nThe soft spots are load-bearing. The mechanism — top-k object candidates, CLIP scoring with \"a photo of {w}\" prompts, rescaling into the LVLM logit range, and fusing during decoding — is essentially what Deng et al.'s Seeing is Believing (SiB, arXiv:2402.15300, cited as [13]) does. The paper's related work describes SiB as requiring \"multiple generations from the original prompt,\" which is inaccurate if SiB is a single-pass logit-level method, as its title and methodology suggest. SiB never appears in any comparison table, so the reader cannot tell whether TTH outperforms it or is simply a re-implementation with a different weighting schedule. The adaptive weighting, borrowed from the authors' own prior work, may be a real improvement, but that is an incremental contribution, not a new method.\n\nThere is also a tunability concern: the top-k candidate pool is tuned on the same benchmarks used for the main results, and the ablation shows performance depends on it. Not fatal, but it weakens the generality claim.\n\nBottom line: this paper deserves serious peer review, but only if the authors add SiB as a baseline and reframe the contribution honestly. The experimental work is real and the method is useful; the novelty argument as it stands is not.\n\nMy recommendation: engage with it, but tell the authors the comparison and framing must be fixed before acceptance.\n\nBest,","headline":"A useful, well-tested decoding-time hallucination reducer, but the novelty claim collapses without a head-to-head against Seeing is Believing, which the paper mischaracterizes and omits.","tokens_in":17992,"tokens_out":3898,"would_cite":false,"duration_ms":32697,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding a zero-shot image-text validator at decoding time suppresses object hallucinations in vision-language models without retraining, weight editing, or extra decoding passes.","keywords":["Object Hallucination","Large Vision-Language Models","Hallucination Mitigation","Training-Free Decoding","CLIP","Entropy-Based Fusion","Token Validation","Multi-Modal AI"],"falsifier":"Construct an image that contains object A but not object B, with both among the model's near-tied top-k object candidates and with the image-text classifier scoring 'a photo of B' higher than 'a photo of A' (as can happen for commonly co-occurring objects). If TTH then raises B above A in the generated output, the method has replaced a correct token with a hallucination; counting how often this flip occurs across a set of such images, for example in the negative-object setting of OPOPE, would settle whether the central claim holds.","tokens_in":16948,"feed_emoji":"🖼️","tokens_out":12693,"duration_ms":113561,"temperature":0.7,"pith_summary":"Object hallucinations—models confidently naming objects that are not in the image—make large vision-language models unreliable in practice. This paper proposes a training-free fix called TTH: at each decoding step it takes the top candidate object tokens, checks each one against the image with a zero-shot image-text classifier using prompts like 'a photo of dog,' and mixes those verification scores into the model's own next-token scores. The mixing weight is set by the entropy of the model's prediction, so uncertain object choices are corrected more strongly while confident ones are left alone. The paper reports that TTH lowers hallucination rates on CHAIR, improves accuracy and F1 on OPOPE, and raises accuracy ratings on LLaVA-Bench across the LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2 models, with less computational overhead than contrastive decoding methods.","feed_headline":"Check each object name against the image to cut hallucinations","feed_subtitle":"Training-free token validator beats prior decoders on CHAIR and OPOPE while keeping generation fast.","key_machinery":"The load-bearing mechanism is the token-validator module: a zero-shot image-text classifier (CLIP) that scores an object token $w_i$ by the cosine similarity between the image embedding and the text embedding of 'a photo of $w_i$.' WordNet supplies the object filter that decides which top-$k$ candidate tokens are objects, Eq. (7) rescales the CLIP scores into the range of the model's logits, and Eq. (9) fuses the two with weight $H_t$, the normalized entropy of the model's distribution over the object candidates. The entropy term makes the intervention adaptive: at high entropy the validator's ground-truth signal takes over, while at low entropy the model's own prediction stands.","core_discovery":"The central claim is that object hallucination can be reduced at test time by treating each decoding step as a small verification problem: identify the object words the model is weighing, ask an external zero-shot image-text classifier whether each word matches the image, and replace the candidate logits with an entropy-weighted mixture of the model's own logits and the validator's scores. The correction is surgical because only object tokens in the candidate set are touched, and it preserves fluency because the language model's own prediction continues to dominate whenever its confidence is high. The experiments are designed to show that the improvement holds across three architecture families and three benchmarks, with the main comparisons against Nullu, OPERA, and HALC.","pith_inferences":["A natural testable extension is to swap the validator for a stronger or ensemble image-text model: the method's ceiling is set by the validator's zero-shot reliability, so gains should track validator accuracy.","The same entropy-gated fusion could be applied to other token classes, such as attributes or actions, by replacing the WordNet object filter with a part-of-speech filter; an experiment analogous to CHAIR with color or verb hallucinations would test this.","Because CLIP similarity is trained on broad internet image-text pairs, it may favor frequent objects; stratifying CHAIR results by object frequency would reveal whether the validator's bias becomes the method's bias.","TTH does not modify weights, so it could be composed with training-based debiasing; an additive-gain experiment would show whether the two correction mechanisms overlap or stack."],"forward_implications":["Any off-the-shelf vision-language model can receive a hallucination reduction by adding a frozen image-text classifier and a lexical lookup, with no fine-tuning and no access to internal weights.","Because TTH needs only a single generation pass, it can be deployed in latency-sensitive settings where contrastive or multi-generation decoders are too slow.","Only uncertain object tokens are reweighted, so the model's language prior is preserved; the reported BLEU scores stay near or above the unmodified greedy baseline.","The entropy-based weighting is itself the defense mechanism: fixed fusion weights are shown to be sensitive to their chosen coefficient, while the adaptive rule improves CHAIR scores on both sentence and instance level.","The reported gains across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2 indicate the mechanism transfers across different vision-language architectures and connector designs."],"supporting_citations":[{"why":"Supplies CLIP, the zero-shot image-text classifier used as the token validator.","marker":"[45]"},{"why":"Supplies WordNet, the lexical resource that identifies object words among candidates.","marker":"[42]"},{"why":"Documents that low-confidence predictions are prone to hallucination, motivating the entropy-based weighting.","marker":"[25]"},{"why":"Defines the CHAIR metric for hallucinated objects in captions.","marker":"[46]"},{"why":"Supplies the MSCOCO validation images used for the CHAIR and OPOPE evaluations.","marker":"[33]"},{"why":"Provides the LLaVA-1.5 model and the LLaVA-Bench benchmark used in the experiments.","marker":"[37]"},{"why":"Provides the Nullu baseline, a weight-editing method that TTH compares against without requiring weight access.","marker":"[59]"},{"why":"Provides the OPERA decoding baseline and its anchor-pattern hallucination signal.","marker":"[21]"},{"why":"Provides the VCD contrastive decoding baseline that requires extra decoding, which TTH aims to beat with a single generation.","marker":"[30]"},{"why":"Provides the HALC baseline and the OPOPE evaluation protocol used to measure object presence accuracy.","marker":"[6]"}],"fun_headline_variants":["Zero-shot token validator cuts vision-language hallucinations","Token-level check: image-grounded logits reduce hallucination","Surgical token check: image-validated logits reduce hallucinations","Entropy-weighted fusion of LVLM and image classifier tames hallucinations","Test-time token validation grounds LVLM predictions in the image"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method depends on the external image-text classifier's score for 'a photo of {word}' being a trustworthy measure of whether the object is present in the image, even after the scores are rescaled to fit the model's much narrower logit range; if that score is wrong or biased for a given image or object, the entropy weighting can steer the decoder toward the wrong word.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot token validator cuts vision-language hallucinations","Token-level check: image-grounded logits reduce hallucination","Surgical token check: image-validated logits reduce hallucinations","Entropy-weighted fusion of LVLM and image classifier tames hallucinations","Test-time token validation grounds LVLM predictions in the image"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001106,"raw_usage":{"total_tokens":4593,"prompt_tokens":911,"completion_tokens":3682,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":3599}},"tokens_in":527,"tokens_out":3682,"duration_ms":22777,"temperature":1.0,"reasoning_tokens":3599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:12:26.622041+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct an image that contains object A but not object B, with both among the model's near-tied top-k object candidates and with the image-text classifier scoring 'a photo of B' higher than 'a photo of A' (as can happen for commonly co-occurring objects). If TTH then raises B above A in the generated output, the method has replaced a correct token with a hallucination; counting how often this flip occurs across a set of such images, for example in the negative-object setting of OPOPE, would settle whether the central claim holds.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the HALC baseline and the OPOPE evaluation protocol used to measure object presence accuracy."}],"review_version":1}