{"id":"7a708c26-3df1-4439-9299-6f6bfa40bb93","arxiv_id":"2608.09344","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Per-instance adaptive projection onto clustered hallucination subspaces reduces LVLM hallucination on CHAIR and POPE benchmarks without fine-tuning.","lead":"This paper introduces a hallucination-suppression method for vision-language models that builds several 'hallucination subspaces' from paired truthful and false captions, then combines them differently for each test image. It matters because it tries to reduce image-grounded factual errors in captioning and visual QA without retraining the model, a practical goal for deploying large multimodal systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The per-instance adaptive weighting (Eq. 17) is never compared against a fixed or uniform mixture of the same subspaces; without that ablation, the paper's central claim that adaptivity drives the gains remains unsupported.","rationale":"The central scientific claim is not that multiple subspaces help—that could be a modest contribution—but that the model 'adaptively calculates weights reflecting each input's relationship to these subspaces' and that this per-instance selection suppresses the most probable hallucination directions. For that to be true, the probe in Eq. 15 must be a valid per-input signal of hallucination susceptibility, and the weights β_k must actually change the projection in a way that matters. Neither is tested. The missing ablation is the cleanest possible decisive experiment: a fixed-weight version of the same method. It is cheap, uses only already-computed quantities, and directly isolates the contribution of adaptivity. This concern is load-bearing because the rest of the pipeline (SVD, K-means, masked forward pass, weighted projection) could in principle be identical while the gains come from the richer subspace library rather than the per-instance weighting. The reader's weakest_assumption already pointed at the probe validity; my emphasis is on the missing fixed-weight baseline as the experiment that would settle it. I see no reason to reject the paper outright—the SVD/PSD algebra in Section 4.3 is internally consistent, the reported gains over Nullu are directionally consistent, and the deficiencies are addressable with one ablation and a clear definition of M. Therefore the conditional verdict stands; the condition should include demonstrating that adaptivity matters beyond a fixed mixture. I also note minor inconsistencies (abstract/conclusion claim six benchmarks and four LVLM families while experiments cover two benchmarks and three models; Table 3 versus Figure 3 show different chosen basis counts) but these do not change the main concern.","tokens_in":15086,"tokens_out":4822,"duration_ms":49669,"concrete_test":"Run the CHAIR and OPOPE evaluations with three variants on LLaVA-1.5: (i) adaptive β from Eq. 17 as in the paper; (ii) uniform β_k = 1/K (fixed mixture); (iii) fixed β_k estimated on the COCO training set, held out from test. Keep K, r, layers, mask, and decoding identical. Report CHAIR_I, CHAIR_S, BLEU, and OPOPE F-score with the same 10-run mean/std. If variants (ii)/(iii) match (i) within the reported std (~0.05 CHAIR_I), the per-instance adaptive weights are not responsible for the gains and the central claim fails; if (i) is consistently better across metrics, the concern is resolved. As a secondary check, compute β for shuffled/mismatched masks to see whether the weights are informative beyond the masked-image forward pass.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that per-instance adaptivity, not merely the multi-subspace construction, is what lowers hallucination. However, the only input-dependent component is the probe δ in Eq. 15: the subspaces V_k^r are built offline from caption-pair differences (Eqs. 4–9), and P in Eq. 18 is a weighted mixture of their projectors. The weights β_k (Eq. 17) come from projecting δ onto V_k^r. No ablation compares this adaptive β to any fixed baseline—e.g., β_k = 1/K, or β fit on COCO train—using the same K and r. If a fixed mixture achieves the same CHAIR_I and OPOPE F-score, the adaptive probe is not doing causal work and the method is just a tuned extension of Nullu with more subspaces. Two additional defects compound this. First, δ measures the effect of masking 70% of the image, while V_k^r encode text-induced hallucination shifts (hallucinated vs. truthful caption); the paper never establishes that these lie in the same directions, so the projection magnitudes could reflect generic image degradation. Second, the mask M in Eq. 14 is not defined ('semantically salient regions' is never operationalized), making the probe unreproducible. Figure 3(right) only compares perturbation types; it does not show that the resulting weights vary meaningfully across inputs or that they correlate with hallucination propensity. The ablation in Figure 3(left) and Table 3 vary K and r, not β, so the central mechanism is untested.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free, test-time method for mitigating hallucinations in large vision-language models (LVLMs). Offline, it builds a contrastive dataset of (image, truthful caption, hallucinated caption) triples, computes per-layer hidden-state differences between hallucinated and truthful captions, clusters these differences with K-means, and applies SVD within each cluster to obtain multiple low-rank orthonormal bases V_k^r. At inference, the method masks 70% of the input image, computes a hidden-state difference δ between the masked and original forward passes, projects δ onto each subspace to obtain alignment scores, converts these scores into per-instance weights β_k via softmax, and applies the weighted projector P = Σ β_k V_k^r (V_k^r)^T to hidden states at selected layers, effectively subtracting the principal hallucination-prone components. The authors evaluate on CHAIR and POPE/OPOPE benchmarks with LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, reporting consistent improvements over Nullu and other baselines. The central claim is that per-instance adaptivity, rather than merely having multiple subspaces, drives the improvement.","tokens_in":15370,"tokens_out":3818,"duration_ms":37735,"significance":"If substantiated, the contribution would be practically valuable: it is a training-free intervention that requires no parameter updates, adds only one extra forward pass per test input, and integrates with arbitrary decoding schemes. The core linear algebra in §4.3 is sound: since each V_k^r (V_k^r)^T is a symmetric positive semidefinite projector and β_k ≥ 0 with Σ β_k = 1, the resulting P is PSD with spectral norm at most 1, so I-P is a contraction and cannot amplify hidden-state components. The evaluation is broad in terms of model families and baselines, and the paper is generally clearly written. However, the paper's headline mechanism — input-dependent adaptive weighting — is never isolated experimentally. The reported gains could arise entirely from the multi-subspace construction or from a fixed mixture; without an ablation that holds the subspace library fixed and compares adaptive β to uniform or fixed weights, the central causal claim is not supported. The masked-image probe also has unresolved validity and reproducibility issues. These concerns are load-bearing for the paper's main contribution and require additional experiments before publication.","major_comments":[{"comment":"The central claim is that per-instance adaptivity, not merely the multi-subspace construction, drives hallucination reduction. This is never tested: no experiment compares the adaptive weights β_k from Eq. (17) with a fixed or uniform mixture (e.g., β_k = 1/K) or with weights fit on a training set, while holding K, r, and the layer set fixed. Figure 3(left) and Table 3 vary the number of clusters and basis vectors, respectively, and Figure 3(right) varies the perturbation type; none of these ablations varies how β is computed. Consequently, Tables 1 and 2 can support the usefulness of a multi-subspace projection, but not the adaptive mechanism advertised in the abstract and Section 4.3.","section":"§4.3, Eq. (17)"},{"comment":"The masked-image probe is not reproducible and its validity is unestablished. The mask M in Eq. (14) is described as zeroing out 70% of 'semantically salient regions,' but no algorithm is given for computing those regions, so the probe cannot be reimplemented from the text. More importantly, δ in Eq. (15) measures the hidden-state change caused by image degradation, whereas the subspaces V_k^r in Eqs. (9)-(12) are built from text-side hallucinated-versus-truthful caption differences under a fixed image; the paper supplies no evidence that these two kinds of shift occupy the same directions. Figure 3(right) shows only that masking outperforms Gaussian and blur perturbations, not that the projection magnitudes correlate with hallucination susceptibility or that the weights vary meaningfully across inputs.","section":"§4.3, Eqs. (14)-(15)"},{"comment":"There is an internal inconsistency in the ablation reporting. The text says that the optimal number of subspaces for LLaVA is 7 and then presents Table 3 as 'The influence of the number of subspaces,' but the table's rows are labeled by 'Number of basis' and its trend (larger basis lowers CHAIRS/CHAIRI and BLEU) concerns the subspace rank r, not the cluster count K. Implementation details in Section 5.3 report different K values (5 for mPLUG-Owl2, 11 for MiniGPT-4, and 7 for LLaVA), so it is unclear which hyperparameter the table actually varies and which values were used in the final comparisons.","section":"§5.6, Table 3"},{"comment":"The text states that for LLaVA-1.5 and MiniGPT-4 the method 'does not surpass Nullu in F-score or precision,' but Table 2 shows Ours with higher F-score and precision than Nullu for both models (e.g., 91.92 vs 91.79 and 92.32 vs 92.07). This contradiction undermines the narrative about architecture-dependent gains; the text and the table need to be reconciled.","section":"§5.5, Table 2"}],"minor_comments":[{"comment":"The improvements over Nullu are often small relative to the reported standard deviations (e.g., MiniGPT-4 CHAIR_I: 8.64±0.22 vs 8.99±0.36), and no significance tests or confidence intervals are provided; the authors should clarify whether the ten runs are over clustering randomness or decoding randomness and report paired comparisons where possible.","section":"§5.4, Table 1"},{"comment":"The notation δ_{ℓ,j} is used on the left, but the right-hand side is averaged over j and contains no j dependence; the index should be removed or the averaging should be made explicit.","section":"§4.3, Eq. (15)"},{"comment":"The conclusion claims evaluation on 'six benchmarks and four LVLM families,' but the experiments cover CHAIR and POPE/OPOPE on three LVLMs; this statement should match the reported results.","section":"§6, Conclusion"},{"comment":"Several references are duplicated with different citation keys (DoLa appears as [7] and [8], VCD as [16] and [17]); this should be cleaned up.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central scientific claim — that per-instance adaptivity is what yields the hallucination reduction — is untested. The missing ablation (adaptive β versus uniform or fixed β with the same subspace library) is essential and should be a condition for acceptance. Additionally, the undefined mask M and the text/table contradiction in Section 5.5 need to be fixed. These are fixable within the scope of the paper, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper extends Nullu's single HalluSpace to multiple clustered subspaces, combined at test time with weights computed from a masked-image probe. That combination is genuinely new in this line of work, and the linear algebra is handled correctly: P is PSD, has spectral norm at most 1, and I-P never amplifies. The writing is mostly clear, and the experiments cover three LVLM families and two standard benchmarks with fairly consistent (if small) gains over strong baselines. There is real engineering sense in clustering hallucination directions before SVD, and the method is training-free and cheap at inference.\n\nThe soft spots are serious, and the biggest one is exactly what the stress test flags: the paper never compares the adaptive weights to a fixed or uniform mixture of the same subspaces. Figure 3 varies the number of clusters and Table 3 varies the basis size, but neither isolates whether the per-instance β_k help at all. Without that ablation, the paper's central claim—that input-dependent weighting, not just having multiple subspaces—is simply unsupported. The probe itself is also under-specified: M in Eq. 14 is never defined, and the connection between the masked-image hidden-state difference and text-induced hallucination directions is asserted rather than demonstrated. So the adaptive signal could be noise, and the method could be reducing to a tuned multi-subspace version of Nullu with a fixed mixture.\n\nThere are also several smaller but real issues. The text in Section 5.5 says the method does not surpass Nullu on POPE F-score for LLaVA and MiniGPT-4, but Table 2 shows it does slightly; that is a factual misstatement. The conclusion claims \"six benchmarks and four LVLM families\" when only CHAIR/POPE and three models appear. The choice of hyperparameters is described as selected on the COCO training set, which is fine, but the same distribution is used for test, so the reported numbers are not fully out-of-distribution. Some implementation details (which layers L are edited, how the mask is computed) are missing. None of these are fatal on their own; together they point to a paper that needs revision more than rejection.\n\nFor a reader working on LVLM hallucination mitigation, this is worth a look as a plausible alternative to fixed subspace editing. It deserves a serious referee, because the idea is real and the missing ablation is readily addressable. I would not cite it yet, but I would ask the authors for the fixed-weight baseline and the mask details.","headline":"A clean, sensible extension of Nullu with a plausible per-instance weighting scheme, but the central claim that adaptivity drives the gains is untested and there are some sloppy overclaims.","tokens_in":15974,"tokens_out":2197,"would_cite":false,"duration_ms":23639,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that hallucination in large vision-language models spans several distinct activation subspaces, and that a per-image weighted blend of those subspaces suppresses hallucinated output better than any single fixed subspace…","keywords":["large vision-language models","hallucination mitigation","model editing","null-space projection","test-time adaptation","disentangled subspaces","subspace clustering","training-free"],"falsifier":"Compare the method's CHAIR_I against a version where the adaptive weights $\\beta_k$ are shuffled across test images, so each input keeps a valid weight vector but not its own, while the subspace library and projector construction stay the same; if hallucination suppression does not degrade, the per-instance probe is not what produces the gain. A second check is to take images whose ground-truth errors are respectively object-level and attribute-level and ask whether the learned weights concentrate on different clusters; if they do not, the subspaces are not behaving as disentangled modes.","tokens_in":14868,"feed_emoji":"🎯","tokens_out":15587,"duration_ms":138826,"temperature":0.7,"pith_summary":"The paper tries to establish that hallucination in large vision-language models is input-dependent and spans several distinct modes, and that a per-image weighted mixture of low-rank 'hallucination subspaces' suppresses it better than any single fixed subspace. It builds the subspaces offline from paired truthful and hallucinated captions by clustering hidden-state differences and applying SVD within each cluster. At test time it masks 70% of the image, reads how the model's internal activations shift, and uses that shift to weight a combined projection that removes the most likely hallucination directions while keeping image-grounded content. The paper reports lower hallucination rates than all baselines on CHAIR across three vision-language model families, and higher offline-POPE F-scores, without fine-tuning and with BLEU scores close to the unedited models. If correct, the method would be a drop-in reliability improvement for off-the-shelf vision-language models, needing only one extra forward pass per image.","feed_headline":"Per-image editing cuts vision-language hallucination below baselines","feed_subtitle":"Several disentangled hallucination modes, weighted per input, beat one fixed subspace with no fine-tuning.","key_machinery":"The central object is the set of Disentangled Hallucination Subspaces: $K$ orthonormal bases extracted by clustering contrastive hidden-state differences (truthful caption vs hallucinated caption) and taking the top-$r$ right singular vectors of each cluster via SVD. Each subspace $V_r^{(k)}$ is meant to isolate one hallucination mode, such as non-existent objects, wrong attributes, or distorted spatial relations. The mechanism that carries the argument at test time is the adaptive projector $P = \\sum_k \\beta_k V_r^{(k)} V_r^{(k)\\top}$, where the weights $\\beta_k$ come from projecting a masked-image hidden-state shift onto each subspace; the edited activations are $(I-P)z$. Because each $V_r^{(k)} V_r^{(k)\\top}$ is an orthogonal projector and the weights are nonnegative and sum to one, $P$ is symmetric positive semi-definite with spectral norm at most one, so $I-P$ is a (generalised) contraction and the intervention removes only the hallucination-aligned slice of each hidden state.","core_discovery":"The paper's central claim is that hallucination in large vision-language models is not a single failure direction: object, attribute, and relation errors occupy partially overlapping directions in activation space, so a fixed global subspace under-corrects some inputs and over-corrects others. The proposed remedy is a library of $K$ low-rank subspaces $V_r^{(k)}$, $k=1,\\dots,K$, each an orthonormal basis for one hallucination mode, together with a test-time probe that infers the mode mixture of the current image. The probe runs the model on the original image and on a version with 70% of semantically salient regions masked, forms the hidden-state difference, projects it onto each subspace, converts the projection magnitudes into softmax weights $\\beta_k$, and builds the combined projector $P = \\sum_k \\beta_k V_r^{(k)} V_r^{(k)\\top}$. It then replaces the hidden states by $(I-P)z$, removing the directions the input is most prone to hallucinate along while leaving the orthogonal complement untouched. On CHAIR the paper reports CHAIR_I values of 4.92, 8.64, and 5.46 for LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, all below the single-subspace Nullu baseline, and on the offline POPE variant it reports F-scores of 91.92, 92.32, and 91.68; because $P$ is a weighted sum of projectors with nonnegative weights summing to one, $I-P$ is a contraction that never amplifies any component of the representation.","pith_inferences":["The masked-image probe has a standalone use the paper does not develop: it scores how hallucination-prone a particular test image is under a given model, which could route high-risk inputs to external verification.","Because the combined projector is a weighted sum of projectors, the method defines a continuum of editing strengths; tuning the softmax temperature or capping the per-mode weights would let a deployer trade suppression against fluency, a knob the paper does not explore.","The subspace library's coverage is bounded by the hallucinated captions generated by a language-model oracle; if real test-time errors include modes the generation prompt did not enumerate, clustering cannot recover them and the adaptive probe cannot correct them. This limitation follows from the construction but is not tested in the paper.","The projection edits all token positions at the selected layers uniformly; an alternative the paper leaves implicit would be to weight token positions by their attention to visual tokens, which could suppress hallucination while leaving language-prior tokens untouched."],"forward_implications":["Any pretrained vision-language model can receive the method as a test-time wrapper: one extra forward pass on the masked image, no gradient updates, and compatibility with greedy, beam, or sampling decoding.","The subspace library is constructed once per backbone and then reused across benchmarks, so the offline clustering cost is amortised and the per-input overhead beyond the masked forward pass is small.","The per-instance weights give a continuous readout of how prone a given image is to each hallucination mode, which is a direct by-product of the method and not just a correction signal.","On CHAIR, the reported CHAIR_I values drop below every baseline on all three backbones while BLEU remains close to the unedited model, indicating that the suppression does not come at the expense of fluency.","On the offline POPE variant, the reported F-scores rise to 91.92, 92.32, and 91.68 across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, with the largest gains on mPLUG-Owl2."],"supporting_citations":[{"why":"Defines the single-subspace null-space projection (HalluSpace) that this paper extends by clustering into multiple subspaces; also supplies the evaluation protocol and the main baseline.","marker":"[29]"},{"why":"CHAIR benchmark, the caption-grounded hallucination metric on which the paper reports its largest reductions.","marker":"[25]"},{"why":"POPE benchmark, including the offline OPOPE variant used to measure object hallucination in generated captions.","marker":"[20]"},{"why":"LLaVA-1.5, one of the three pretrained vision-language backbones the method is applied to.","marker":"[22]"},{"why":"MiniGPT-4, the second backbone in the evaluation.","marker":"[35]"},{"why":"mPLUG-Owl2, the third backbone and the model with the largest reported offline-POPE gains.","marker":"[30]"}],"fun_headline_variants":["Per-instance subspaces beat global fixes for LVLM hallucinations","Weighted per-input hallucination subspaces cut LVLM errors","Test-time subspace blending suppresses LVLM hallucinations","No fine-tuning: per-image subspace weighting reduces hallucination","Dynamic per-input subspace combinations beat static editing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the shift in hidden states caused by masking 70% of the image is a trustworthy per-input signal of which hallucination mode is active; if that shift mostly reflects generic image degradation, the computed weights are noise and the method degenerates to a fixed mixture of the same subspaces, and the paper does not specify how the mask's 'semantically salient regions' are computed, so this premise cannot be checked from the text alone.","fun_headline_variants_meta":{"raw":{"variants":["Per-instance subspaces beat global fixes for LVLM hallucinations","Weighted per-input hallucination subspaces cut LVLM errors","Test-time subspace blending suppresses LVLM hallucinations","No fine-tuning: per-image subspace weighting reduces hallucination","Dynamic per-input subspace combinations beat static editing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000628,"raw_usage":{"total_tokens":2995,"prompt_tokens":1131,"completion_tokens":1864,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":747,"completion_tokens_details":{"reasoning_tokens":1787}},"tokens_in":747,"tokens_out":1864,"duration_ms":51028,"temperature":1.0,"reasoning_tokens":1787,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:55:53.788116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the method's CHAIR_I against a version where the adaptive weights $\\beta_k$ are shuffled across test images, so each input keeps a valid weight vector but not its own, while the subspace library and projector construction stay the same; if hallucination suppression does not degrade, the per-instance probe is not what produces the gain. A second check is to take images whose ground-truth errors are respectively object-level and attribute-level and ask whether the learned weights concentrate on different clusters; if they do not, the subspaces are not behaving as disentangled modes.","supporting_citations":[{"cited_title":"Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection","cited_arxiv_id":null,"evidence_quote":"Defines the single-subspace null-space projection (HalluSpace) that this paper extends by clustering into multiple subspaces; also supplies the evaluation protocol and the main baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"mPLUG-Owl2, the third backbone and the model with the largest reported offline-POPE gains."}],"review_version":1}