{"id":"3a0f24a2-cd63-43d5-a2d4-ca1854c4ac01","arxiv_id":"2509.00371","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Omission and fabrication hallucinations in MLLMs are claimed to have distinct causes, and the new VPFC method reduces omissions without adding fabrications.","lead":"This paper argues that AI vision-language models make two different kinds of object errors: missing objects that are actually present, and inventing objects that are absent. It introduces a plug-in calibration method, VPFC, that reduces the first kind while avoiding the second, and reports state-of-the-art results among training-free fixes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VPFC rests on an unquantified HCVR-dispersion heuristic; if the heuristic does not hold, the method's stated mechanism collapses.","rationale":"The reader's weakest_assumption identifies exactly the same linchpin: the spatially concentrated vs. dispersed HCVR heuristic in Section 4. My read of the paper confirms this is the single most load-bearing concern. The paper's two-cause thesis is supported mainly by attention-intervention experiments and case studies, but the VPFC method—which is a key part of the claimed contribution—depends on an unquantified empirical regularity. Without quantitative evidence that HCVR dispersion reliably separates absent from present objects, the method's mechanism is not established, even though the benchmark numbers might still improve for other reasons (e.g., a positional bias toward image centers in COCO). The ablation in Figure 7 is suggestive but does not isolate the dispersion assumption. Therefore, the paper should be accepted only conditionally on providing such evidence, matching the reader's verdict. I am not proposing a different verdict because the concern is the same one the reader already identified; it reinforces CONDITIONAL rather than moving the verdict.","tokens_in":12195,"tokens_out":5746,"duration_ms":71590,"concrete_test":"Compute HCVR dispersion on POPE-COCO for LLaVA-1.5-7B using the same heads/layers selected by VPFC. For each question, extract attention maps, define HCVRs as the top 25% of tokens by attention, and compute a dispersion index such as the area of the minimum bounding box, mean pairwise distance, or spatial variance of HCVR token positions. Split by ground-truth presence vs. absence, separately for the Random, Popular, and Adversarial subsets, and report the separation (e.g., AUROC or Cohen's d). If the AUROC is below about 0.7 or the distributions overlap heavily, the asserted 'consistent pattern' does not hold at the population level, and VPFC's mechanism is not supported by its own stated rationale.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is the asserted 'consistent pattern' in Section 4: when the queried object is absent, HCVRs tend to be spatially dispersed, and when it is present, HCVRs tend to be spatially concentrated. This premise is the sole basis for VPFC's decision to enhance only a square region around the centroid of the top-25% attention tokens. Without it, VPFC has no principled way to choose whether to reinforce evidence of presence or absence. Yet the paper provides no quantitative support: no dispersion metric, no distributions for present vs. absent objects, no head/layer aggregation details, and the visual evidence is limited to two single-image case studies (Figures 2-3). Figure 4 shows only binary intervention outcomes, not the dispersion pattern. The ablation in Figure 7 ('VPFC w/o Centroid') does not test the dispersion assumption; it changes the intervention region from a centroid-centered square to the raw top-25% token set, confounding the dispersion hypothesis with the effect of spatially smoothing the intervention. If the dispersion prior is weak or dataset-specific, the mechanism fails in both directions: for absent images with concentrated HCVRs (e.g., a salient distractor), the centroid square would enhance high-confidence evidence and increase fabrications; for present images with dispersed HCVRs (e.g., small or multiple objects), the square may miss the object and fail to reduce omissions. The central claim that VPFC 'reduces omission hallucinations without introducing additional fabrication hallucinations' therefore rests on an untested empirical regularity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that omission and fabrication hallucinations in MLLMs have distinct causes: omissions arise from insufficient confidence when mapping already-captured visual features to language, while fabrications arise from spurious cross-modal associations driven by training-set statistical biases. It introduces the Visual-Semantic Attention Potential Field (VSAPF) as a conceptual account of how visual evidence is weighted, and proposes Visual Potential Field Calibration (VPFC), a training-free, plug-and-play method that enhances attention in a centroid-centered region of HCVR tokens to reduce omissions without increasing fabrications. Experiments on POPE, CHAIR, MME, and LLaVA-Bench report consistent improvements over VCD, SID, and MemVR.","tokens_in":12609,"tokens_out":3302,"duration_ms":42906,"significance":"If the mechanistic claims hold, the paper would be a valuable correction to the common assumption that all object hallucinations share one cause, and VPFC is a simple, effective training-free intervention. The benchmark results are promising: VPFC improves over strong baselines on multiple datasets and two backbones. However, the paper's central causal story and the mechanism of VPFC rest on qualitative, single-image evidence and on an unquantified dispersion heuristic. The contribution is therefore conditionally significant: the method may work, but the paper as written does not yet establish the proposed explanation, and the ablation evidence is too thin to separate the mechanism from generic smoothing effects.","major_comments":[{"comment":"The central premise of VPFC is the 'consistent pattern' that HCVRs are spatially concentrated when the queried object is present and dispersed when it is absent. This premise determines whether the centroid square enhances HCVRs or partially overlaps LCVRs, and it is never quantitatively validated. No dispersion metric, no distributions for present vs. absent objects, no head/layer aggregation details, and no statistics are provided. The paper must supply quantitative support across a representative sample of images and queries, including failure cases (e.g., small objects, multiple objects, salient distractors). Without this, the mechanism collapses even if the benchmark numbers improve.","section":"Section 4, 'Focused Region for Visual Potential Calibration'"},{"comment":"The intervention results that justify the HCVR/LCVR confidence account are described only qualitatively ('enhancing attention to the HCVRs leads the model to increasingly judge that the queried object is present'). No numbers, sample sizes, or consistency rates are reported. This is foundational evidence for the VSAPF concept and for the design of VPFC. The authors should report the distribution of model answers under HCVR enhancement, LCVR enhancement, and baseline, separately for present and absent objects, and ideally for multiple models and layers.","section":"Section 3.3, Figure 4"},{"comment":"The claim that SID's contrastive decoding is 'essentially equivalent' to enhancing attention to HCVRs is not established. The two equations have similar algebraic forms, but they involve different quantities: SID contrasts the original logits with logits from low-importance visual tokens vlow, while the proposed enhancement contrasts original logits with logits from high-importance visual tokens vhigh. No derivation shows that these become equivalent for any α and β, and the 'appropriately set' condition is not specified. The paper needs either a precise mathematical equivalence or an empirical comparison showing that the two decoding distributions are interchangeable in effect.","section":"Section 3.4, Equations (2)-(3)"},{"comment":"The abstract and Section 5.2 repeatedly claim that VPFC 'reduces omission hallucinations without introducing additional fabrication hallucinations.' However, none of the reported metrics (accuracy, F1, CHAIR, MME scores) decompose errors into omission vs. fabrication. POPE, for instance, is a binary yes/no task; the paper should report the false-'no' rate (omissions) and false-'yes' rate (fabrications) separately for each method. Figure 1 provides such counts for VCD but the corresponding decomposition for VPFC and the other baselines is missing. Without this, the central claim is not directly evidenced.","section":"Section 5.2, Tables 1-4"},{"comment":"The ablation 'VPFC w/o Centroid' is presented only as a bar chart without numeric values, error bars, or statistical tests. More importantly, it does not isolate the dispersion heuristic: replacing the centroid square with the raw top-25% token set changes both the spatial support and the smoothing of the intervention, confounding the hypothesis. A proper ablation should vary the dispersion criterion directly (e.g., using a dispersion threshold to choose between enhancing HCVRs vs. LCVRs) and report outcomes for images where the heuristic succeeds and fails.","section":"Section 5.3, Figure 7"}],"minor_comments":[{"comment":"The limitation section honestly states that VPFC does not explicitly suppress fabrications. This should be reflected in the abstract and conclusion, where the phrase 'reduces omission hallucinations without introducing additional fabrication hallucinations' is currently presented as a complete solution rather than as a balanced trade-off.","section":"Appendix D"},{"comment":"Several tables have inconsistent formatting: Table 1 repeats the 'Random Popular Adversarial' header in the text, and Table 4 lacks column alignment. Please standardize all tables and include error bars or confidence intervals where feasible.","section":"Section 5.1 / Tables"},{"comment":"The ablation figures in Appendix C have no captions and no axis labels in the main text. Additionally, the ablation results are only reported on COCO-Random; report results on Popular and Adversarial subsets as well.","section":"Figures 9 and 10"},{"comment":"The claim that 'fabrications significantly outnumber omissions' in the Adversarial subset is supported only by the authors' own prior work (Yin et al. 2025). Provide an independent analysis or additional citation to make this load-bearing statement stronger.","section":"References / Section 5.2"},{"comment":"Minor typographical issues: 'LLaV A-v1.5' is inconsistently spaced, 'Preprint,Underreview' appears in the header, and 'Kening Zheng' and 'Ken ing Zheng' are inconsistent in the references. Please proofread the manuscript.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's contribution is interesting but its central mechanistic claims are supported mainly by single-image case studies and an unquantified dispersion heuristic. The benchmark results are encouraging, but the authors need to provide quantitative evidence for the dispersion pattern and error decomposition before the paper can be accepted. I also note the heavy reliance on the authors' own prior work (Yin et al. 2025) for a key empirical claim; the editor may wish to verify the independence of that evidence. No code was provided; if the paper is revised, code release would substantially strengthen reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's the short version: this paper actually does something new. It splits object hallucination into omission and fabrication and argues they need different interventions, then introduces a training-free decoding method (VPFC) that gets consistent wins on POPE, CHAIR, MME, and LLaVA-Bench. The numbers look real, and VPFC is the only method that doesn't crater on the Adversarial POPE subset, where VCD, SID, and MemVR all lose accuracy. That is worth a serious look.\n\nWhat's genuinely new: the VSAPF framing (attention as a potential field, with high-credibility regions as wells) and the centroid-based steering idea. The ablation shows the centroid matters — removing it drops accuracy to VCD level. The method is plug-and-play, with a few hyperparameters that have a fairly wide stable range per the figures. The authors also state a clear limitation: they don't target fabrication suppression directly, which is honest and accurate.\n\nWeaknesses, in proportion. The mechanistic story rests on two case studies (spoon, toilet/sink) and a binary intervention plot with no numbers. The load-bearing premise — when the object is present HCVRs are concentrated, when absent they're dispersed — is asserted but never quantified. No distributions, no metric, no head/layer aggregation details. That's a real gap because the whole design (enhance a centroid-centered square) depends on it. The ablation 'w/o Centroid' doesn't isolate the dispersion hypothesis; it compares a square to the raw top-25% token set, confounding spatial smoothing with concentration. The claimed equivalence with SID is also hand-wavy: they write two equations, say 'effectively equivalent,' and don't test it. Finally, all evaluation is aggregate accuracy; they never break out omission vs. fabrication error rates, which is odd given the paper's thesis.\n\nShould this be peer-reviewed? Yes. The empirical contribution is strong and the two-cause claim, if supported, would redirect a line of work. A good referee should demand a quantitative test of the dispersion heuristic and a per-type error breakdown on POPE. Without those, the mechanism claim gets downgraded to a reinterpretation that happens to motivate a method that works.\n\nI'd bring it to the reading group and cite it in the context of decoding-time hallucination methods. It's not a desk reject; it's a revise-and-resubmit with teeth.","headline":"Useful two-cause framing and a solid plug-and-play decoder intervention, but the load-bearing spatial-concentration heuristic is never quantitatively tested.","tokens_in":13016,"tokens_out":2690,"would_cite":true,"duration_ms":29086,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that omission and fabrication hallucinations in multimodal LLMs have distinct mechanisms, and that its training-free VPFC method reduces omissions without adding fabrications.","keywords":["object hallucination","multimodal large language models","omission hallucination","fabrication hallucination","visual attention intervention","visual-semantic attention potential field","training-free mitigation","visual potential field calibration"],"falsifier":"Measure the spatial dispersion (e.g., area of the convex hull or sum of pairwise distances) of the top-attention visual tokens for present versus absent objects across a dataset with ground-truth object presence. If present-object queries do not yield significantly more concentrated HCVRs than absent-object queries on average, the centroid-enhancement mechanism loses its rationale.","tokens_in":12143,"feed_emoji":"👁️","tokens_out":4068,"duration_ms":42930,"temperature":0.7,"pith_summary":"Multimodal large language models sometimes fail to mention objects that are present (omission) and sometimes mention objects that are absent (fabrication). This paper argues that these two failure modes do not share one cause, as most prior mitigation methods assume. It claims omissions come from low confidence when the model maps visual features it already captured into words, while fabrications come from spurious associations between visual and textual concepts learned from co-occurrence statistics. The paper introduces a training-free intervention, Visual Potential Field Calibration (VPFC), that increases attention to spatially concentrated visual regions and steers hidden states accordingly, reporting reductions in omission errors without triggering new fabrication errors on several benchmarks. If right, it means hallucination mitigation should be split into two targeted problems instead of one.","feed_headline":"Hallucination fix cuts missed objects without inventing new ones","feed_subtitle":"A training-free method recalibrates visual attention so models mention real objects without fabricating absent ones.","key_machinery":"The Visual-Semantic Attention Potential Field (VSAPF) is a conceptual map in which each visual token sits in a landscape: High-Credibility Visual Regions (HCVRs), the top-attention tokens, sit at potential wells where the model readily affirms presence, and Low-Credibility Visual Regions (LCVRs) sit at peaks where the model tends toward negation. VPFC uses the observation that HCVRs are spatially concentrated when the queried object is present and dispersed when it is absent. It computes the centroid of HCVRs, enhances a square region around that centroid, computes a hidden-state steering direction from a mild attention boost, and applies this direction to selected attention heads chosen by","core_discovery":"The paper's central claim is that omission hallucinations and fabrication hallucinations have fundamentally different roots. Using attention maps from localization heads, the paper shows that when the model omits an object, it often has already attended to the correct visual region and captured the object's features; the failure is low confidence in mapping those features to the linguistic expression. In contrast, fabrication hallucinations occur when the model confidently maps visual features to a wrong semantic concept, such as interpreting part of a toilet as a sink, driven by co-occurrence biases in training data. Attention intervention experiments show that boosting attention to high-at","pith_inferences":["Beyond the paper: the HCVR-concentration heuristic could be tested directly by measuring the spatial dispersion of top-attention tokens on present versus absent object queries; if it does not hold on out-of-distribution objects or long-tail categories, VPFC would need a learned presence estimator.","Beyond the paper: the two-cause account suggests that benchmarks should report omission and fabrication rates separately at the item level, since aggregate accuracy can hide the exact tradeoff the paper identifies.","Beyond the paper: the same potential-field lens might extend to attribute-level hallucinations (color, position, count), which the MME results hint at but the paper does not analyze mechanistically.","Beyond the paper: the hidden-state steering direction computed by VPFC could be repurposed as a probe for object-presence confidence, enabling confidence-aware decoding that does not rely on centroid geometry."],"forward_implications":["VPFC is a plug-and-play, training-free method, so it can be applied to existing multimodal LLMs without retraining or fine-tuning.","Unified mitigation methods that indiscriminately boost high-attention regions should be expected to reduce omissions at the cost of increasing fabrications, because they amplify confidence in object presence regardless of whether the object is actually there.","Evaluation of hallucination mitigation should separate omission and fabrication errors rather than relying on aggregate accuracy or F1 scores, since a method can improve one while worsening the other.","A targeted fabrication-suppression strategy remains an open problem; VPFC only avoids adding fabrications while primarily addressing omissions.","The observed failure of contrastive-decoding baselines on the POPE Adversarial subset is explained as a consequence of treating both hallucination types with a single intervention."],"supporting_citations":[{"why":"Supplies the localization heads whose attention maps the paper uses to show that the model visually captures omitted objects.","marker":"[Kang et al. 2025]"},{"why":"The VCD baseline whose unified mitigation is shown in the motivation experiment to reduce omissions at the cost of fabrications.","marker":"[Leng et al. 2024]"},{"why":"SID baseline, which the paper formally compares to HCVR attention enhancement and shows to be equivalent at the decoding level.","marker":"[Huo et al. 2025]"},{"why":"POPE benchmark provides the main discriminative evaluation on random, popular, and adversarial subsets.","marker":"[Li et al. 2023c]"},{"why":"Saliency analysis used to select which attention heads receive the hidden-state steering in VPFC.","marker":"[Michel et al. 2019]"},{"why":"Supports the claim that model knowledge is encoded in hidden states, motivating direct hidden-state modification.","marker":"[Burns et al. 2022]"},{"why":"Supplies the observation that fabrications outnumber omissions on the POPE Adversarial subset, used to interpret VPFC's gains.","marker":"[Yin et al. 2025]"},{"why":"MemVR baseline compared against in the benchmark evaluation.","marker":"[Zou et al. 2025]"}],"fun_headline_variants":["Low confidence omits, spurious links fabricate—MLLM fix does both","Attention maps show why MLLMs miss objects and invent fake ones","A training-free fix for omission hallucinations that doesn't add fabrications","Two mechanisms behind object hallucinations: a new calibration method"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method's mechanism rests on the heuristic that when the queried object is actually present, the high-attention regions are spatially concentrated, and when it is absent they are dispersed; the paper reports no quantitative evidence for this relationship.","fun_headline_variants_meta":{"raw":{"variants":["Low confidence omits, spurious links fabricate—MLLM fix does both","Attention maps show why MLLMs miss objects and invent fake ones","A training-free fix for omission hallucinations that doesn't add fabrications","Two mechanisms behind object hallucinations: a new calibration method"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001015,"raw_usage":{"total_tokens":4092,"prompt_tokens":684,"completion_tokens":3408,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":428,"completion_tokens_details":{"reasoning_tokens":3341}},"tokens_in":428,"tokens_out":3408,"duration_ms":30664,"temperature":1.0,"reasoning_tokens":3341,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:38:51.882540+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the spatial dispersion (e.g., area of the convex hull or sum of pairwise distances) of the top-attention visual tokens for present versus absent objects across a dataset with ground-truth object presence. If present-object queries do not yield significantly more concentrated HCVRs than absent-object queries on average, the centroid-enhancement mechanism loses its rationale.","supporting_citations":[],"review_version":1}