Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs

Ali Cheraghian; Hamed Barzamini; Hamidreza Dastmalchi; Hossein Rahmani; Mojtaba Golzan; Morteza Saberi; Shafin Rahman

REVIEW 4 major objections 4 minor 38 references

Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that hallucination in large vision-language models spans several distinct activation subspaces, and that a per-image weighted blend of those subspaces suppresses hallucinated output better than any single fixed subspace…

desk verdict A clean, sensible extension of Nullu with a plausible per-instance weighting scheme, but the central claim that adaptivity drives the gains is untested and there are some sloppy overclaims. read the letter →

arxiv 2608.09344 v1 pith:JEQFIEH6 submitted 2026-08-10 cs.CV

Ali Cheraghian , Hamidreza Dastmalchi , Hamed Barzamini , Morteza Saberi , Mojtaba Golzan , Shafin Rahman , Hossein Rahmani This is my paper · ORCID

classification cs.CV

keywords largevision-languagemodelshallucinationmitigationmodeleditingnull-spaceprojectiontest-timeadaptationdisentangledsubspacessubspaceclusteringtraining-free

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that hallucination in large vision-language models is input-dependent and spans several distinct modes, and that a per-image weighted mixture of low-rank 'hallucination subspaces' suppresses it better than any single fixed subspace. It builds the subspaces offline from paired truthful and hallucinated captions by clustering hidden-state differences and applying SVD within each cluster. At test time it masks 70% of the image, reads how the model's internal activations shift, and uses that shift to weight a combined projection that removes the most likely hallucination directions while keeping image-grounded content. The paper reports lower hallucination rates than all baselines on CHAIR across three vision-language model families, and higher offline-POPE F-scores, without fine-tuning and with BLEU scores close to the unedited models. If correct, the method would be a drop-in reliability improvement for off-the-shelf vision-language models, needing only one extra forward pass per image.

What carries the argument

The central object is the set of Disentangled Hallucination Subspaces: $K$ orthonormal bases extracted by clustering contrastive hidden-state differences (truthful caption vs hallucinated caption) and taking the top-$r$ right singular vectors of each cluster via SVD. Each subspace $V_r^{(k)}$ is meant to isolate one hallucination mode, such as non-existent objects, wrong attributes, or distorted spatial relations. The mechanism that carries the argument at test time is the adaptive projector $P = \sum_k \beta_k V_r^{(k)} V_r^{(k)\top}$, where the weights $\beta_k$ come from projecting a masked-image hidden-state shift onto each subspace; the edited activations are $(I-P)z$. Because each $V_r^{(k)} V_r^{(k)\top}$ is an orthogonal projector and the weights are nonnegative and sum to one, $P$ is symmetric positive semi-definite with spectral norm at most one, so $I-P$ is a (generalised) contraction and the intervention removes only the hallucination-aligned slice of each hidden state.

What would settle it

Compare the method's CHAIR_I against a version where the adaptive weights $\beta_k$ are shuffled across test images, so each input keeps a valid weight vector but not its own, while the subspace library and projector construction stay the same; if hallucination suppression does not degrade, the per-instance probe is not what produces the gain. A second check is to take images whose ground-truth errors are respectively object-level and attribute-level and ask whether the learned weights concentrate on different clusters; if they do not, the subspaces are not behaving as disentangled modes.

Watch

Extended reading notes

Core claim

The paper's central claim is that hallucination in large vision-language models is not a single failure direction: object, attribute, and relation errors occupy partially overlapping directions in activation space, so a fixed global subspace under-corrects some inputs and over-corrects others. The proposed remedy is a library of $K$ low-rank subspaces $V_r^{(k)}$, $k=1,\dots,K$, each an orthonormal basis for one hallucination mode, together with a test-time probe that infers the mode mixture of the current image. The probe runs the model on the original image and on a version with 70% of semantically salient regions masked, forms the hidden-state difference, projects it onto each subspace, converts the projection magnitudes into softmax weights $\beta_k$, and builds the combined projector $P = \sum_k \beta_k V_r^{(k)} V_r^{(k)\top}$. It then replaces the hidden states by $(I-P)z$, removing the directions the input is most prone to hallucinate along while leaving the orthogonal complement untouched. On CHAIR the paper reports CHAIR_I values of 4.92, 8.64, and 5.46 for LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, all below the single-subspace Nullu baseline, and on the offline POPE variant it reports F-scores of 91.92, 92.32, and 91.68; because $P$ is a weighted sum of projectors with nonnegative weights summing to one, $I-P$ is a contraction that never amplifies any component of the representation.

Load-bearing premise

The load-bearing premise is that the shift in hidden states caused by masking 70% of the image is a trustworthy per-input signal of which hallucination mode is active; if that shift mostly reflects generic image degradation, the computed weights are noise and the method degenerates to a fixed mixture of the same subspaces, and the paper does not specify how the mask's 'semantically salient regions' are computed, so this premise cannot be checked from the text alone.

Editorial extensions

If this is right

Any pretrained vision-language model can receive the method as a test-time wrapper: one extra forward pass on the masked image, no gradient updates, and compatibility with greedy, beam, or sampling decoding.
The subspace library is constructed once per backbone and then reused across benchmarks, so the offline clustering cost is amortised and the per-input overhead beyond the masked forward pass is small.
The per-instance weights give a continuous readout of how prone a given image is to each hallucination mode, which is a direct by-product of the method and not just a correction signal.
On CHAIR, the reported CHAIR_I values drop below every baseline on all three backbones while BLEU remains close to the unedited model, indicating that the suppression does not come at the expense of fluency.
On the offline POPE variant, the reported F-scores rise to 91.92, 92.32, and 91.68 across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, with the largest gains on mPLUG-Owl2.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The masked-image probe has a standalone use the paper does not develop: it scores how hallucination-prone a particular test image is under a given model, which could route high-risk inputs to external verification.
Because the combined projector is a weighted sum of projectors, the method defines a continuum of editing strengths; tuning the softmax temperature or capping the per-mode weights would let a deployer trade suppression against fluency, a knob the paper does not explore.
The subspace library's coverage is bounded by the hallucinated captions generated by a language-model oracle; if real test-time errors include modes the generation prompt did not enumerate, clustering cannot recover them and the adaptive probe cannot correct them. This limitation follows from the construction but is not tested in the paper.
The projection edits all token positions at the selected layers uniformly; an alternative the paper leaves implicit would be to weight token positions by their attention to visual tokens, which could suppress hallucination while leaving language-prior tokens untouched.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper extends Nullu's single HalluSpace to multiple clustered subspaces, combined at test time with weights computed from a masked-image probe. That combination is genuinely new in this line of work, and the linear algebra is handled correctly: P is PSD, has spectral norm at most 1, and I-P never amplifies. The writing is mostly clear, and the experiments cover three LVLM families and two standard benchmarks with fairly consistent (if small) gains over strong baselines. There is real engineering sense in clustering hallucination directions before SVD, and the method is training-free and cheap at inference.

The soft spots are serious, and the biggest one is exactly what the stress test flags: the paper never compares the adaptive weights to a fixed or uniform mixture of the same subspaces. Figure 3 varies the number of clusters and Table 3 varies the basis size, but neither isolates whether the per-instance β_k help at all. Without that ablation, the paper's central claim—that input-dependent weighting, not just having multiple subspaces—is simply unsupported. The probe itself is also under-specified: M in Eq. 14 is never defined, and the connection between the masked-image hidden-state difference and text-induced hallucination directions is asserted rather than demonstrated. So the adaptive signal could be noise, and the method could be reducing to a tuned multi-subspace version of Nullu with a fixed mixture.

There are also several smaller but real issues. The text in Section 5.5 says the method does not surpass Nullu on POPE F-score for LLaVA and MiniGPT-4, but Table 2 shows it does slightly; that is a factual misstatement. The conclusion claims "six benchmarks and four LVLM families" when only CHAIR/POPE and three models appear. The choice of hyperparameters is described as selected on the COCO training set, which is fine, but the same distribution is used for test, so the reported numbers are not fully out-of-distribution. Some implementation details (which layers L are edited, how the mask is computed) are missing. None of these are fatal on their own; together they point to a paper that needs revision more than rejection.

For a reader working on LVLM hallucination mitigation, this is worth a look as a plausible alternative to fixed subspace editing. It deserves a serious referee, because the idea is real and the missing ablation is readily addressable. I would not cite it yet, but I would ask the authors for the fixed-weight baseline and the mask details.

Referee Report

4 major / 4 minor

Summary. The paper proposes a training-free, test-time method for mitigating hallucinations in large vision-language models (LVLMs). Offline, it builds a contrastive dataset of (image, truthful caption, hallucinated caption) triples, computes per-layer hidden-state differences between hallucinated and truthful captions, clusters these differences with K-means, and applies SVD within each cluster to obtain multiple low-rank orthonormal bases V_k^r. At inference, the method masks 70% of the input image, computes a hidden-state difference δ between the masked and original forward passes, projects δ onto each subspace to obtain alignment scores, converts these scores into per-instance weights β_k via softmax, and applies the weighted projector P = Σ β_k V_k^r (V_k^r)^T to hidden states at selected layers, effectively subtracting the principal hallucination-prone components. The authors evaluate on CHAIR and POPE/OPOPE benchmarks with LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2, reporting consistent improvements over Nullu and other baselines. The central claim is that per-instance adaptivity, rather than merely having multiple subspaces, drives the improvement.

Significance. If substantiated, the contribution would be practically valuable: it is a training-free intervention that requires no parameter updates, adds only one extra forward pass per test input, and integrates with arbitrary decoding schemes. The core linear algebra in §4.3 is sound: since each V_k^r (V_k^r)^T is a symmetric positive semidefinite projector and β_k ≥ 0 with Σ β_k = 1, the resulting P is PSD with spectral norm at most 1, so I-P is a contraction and cannot amplify hidden-state components. The evaluation is broad in terms of model families and baselines, and the paper is generally clearly written. However, the paper's headline mechanism — input-dependent adaptive weighting — is never isolated experimentally. The reported gains could arise entirely from the multi-subspace construction or from a fixed mixture; without an ablation that holds the subspace library fixed and compares adaptive β to uniform or fixed weights, the central causal claim is not supported. The masked-image probe also has unresolved validity and reproducibility issues. These concerns are load-bearing for the paper's main contribution and require additional experiments before publication.

major comments (4)

[§4.3, Eq. (17)] The central claim is that per-instance adaptivity, not merely the multi-subspace construction, drives hallucination reduction. This is never tested: no experiment compares the adaptive weights β_k from Eq. (17) with a fixed or uniform mixture (e.g., β_k = 1/K) or with weights fit on a training set, while holding K, r, and the layer set fixed. Figure 3(left) and Table 3 vary the number of clusters and basis vectors, respectively, and Figure 3(right) varies the perturbation type; none of these ablations varies how β is computed. Consequently, Tables 1 and 2 can support the usefulness of a multi-subspace projection, but not the adaptive mechanism advertised in the abstract and Section 4.3.
[§4.3, Eqs. (14)-(15)] The masked-image probe is not reproducible and its validity is unestablished. The mask M in Eq. (14) is described as zeroing out 70% of 'semantically salient regions,' but no algorithm is given for computing those regions, so the probe cannot be reimplemented from the text. More importantly, δ in Eq. (15) measures the hidden-state change caused by image degradation, whereas the subspaces V_k^r in Eqs. (9)-(12) are built from text-side hallucinated-versus-truthful caption differences under a fixed image; the paper supplies no evidence that these two kinds of shift occupy the same directions. Figure 3(right) shows only that masking outperforms Gaussian and blur perturbations, not that the projection magnitudes correlate with hallucination susceptibility or that the weights vary meaningfully across inputs.
[§5.6, Table 3] There is an internal inconsistency in the ablation reporting. The text says that the optimal number of subspaces for LLaVA is 7 and then presents Table 3 as 'The influence of the number of subspaces,' but the table's rows are labeled by 'Number of basis' and its trend (larger basis lowers CHAIRS/CHAIRI and BLEU) concerns the subspace rank r, not the cluster count K. Implementation details in Section 5.3 report different K values (5 for mPLUG-Owl2, 11 for MiniGPT-4, and 7 for LLaVA), so it is unclear which hyperparameter the table actually varies and which values were used in the final comparisons.
[§5.5, Table 2] The text states that for LLaVA-1.5 and MiniGPT-4 the method 'does not surpass Nullu in F-score or precision,' but Table 2 shows Ours with higher F-score and precision than Nullu for both models (e.g., 91.92 vs 91.79 and 92.32 vs 92.07). This contradiction undermines the narrative about architecture-dependent gains; the text and the table need to be reconciled.

minor comments (4)

[§5.4, Table 1] The improvements over Nullu are often small relative to the reported standard deviations (e.g., MiniGPT-4 CHAIR_I: 8.64±0.22 vs 8.99±0.36), and no significance tests or confidence intervals are provided; the authors should clarify whether the ten runs are over clustering randomness or decoding randomness and report paired comparisons where possible.
[§4.3, Eq. (15)] The notation δ_{ℓ,j} is used on the left, but the right-hand side is averaged over j and contains no j dependence; the index should be removed or the averaging should be made explicit.
[§6, Conclusion] The conclusion claims evaluation on 'six benchmarks and four LVLM families,' but the experiments cover CHAIR and POPE/OPOPE on three LVLMs; this statement should match the reported results.
[References] Several references are duplicated with different citation keys (DoLa appears as [7] and [8], VCD as [16] and [17]); this should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the per-instance adaptive weights come from a masked-image probe, and the subspaces are fitted on an external contrastive set, so the evaluation does not reduce to the method's inputs.

full rationale

The derivation chain is self-contained and non-circular. The subspaces V_k^r are built in Eqs. (4)-(12) by running the LVLM on paired truthful/hallucinated captions from a GPT-4-generated contrastive dataset, clustering the per-sample difference vectors, and taking top-r singular vectors per cluster. These bases are fixed offline and do not use CHAIR/POPE labels or the test-time probe. The adaptive weights beta_k in Eqs. (15)-(17) are computed for each test input solely from the difference between hidden states under the original image and a 70%-masked image, projected onto those fixed subspaces; no benchmark label or fitted output enters the weight computation. The final projection P in Eq. (18) is a weighted sum of the precomputed projectors, and the edited states in Eq. (20) are just (I-P) applied to the original activations. Nothing in this chain is defined in terms of the evaluation target, and no fitted parameter is renamed as a prediction. The few self-citations in the reference list (e.g., [2] and prior test-time adaptation papers) are not load-bearing: the central construction explicitly extends the external Nullu work [29], and the cited items only support background statements. The main scientifically vulnerable point--that the masked-image probe may measure generic degradation rather than hallucination susceptibility, and that no fixed-mixture ablation isolates the benefit of adaptivity--is an empirical validation concern, not a circularity. Similarly, leaving the mask M in Eq. (14) unspecified affects reproducibility but does not make the result equivalent to its inputs. Therefore the paper warrants a circularity score of 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method's central claim rests on several domain assumptions about the geometry of hallucination representations and on hyperparameters fitted to the evaluation distribution. No genuinely new physical or mathematical entities are postulated.

free parameters (5)

Number of clusters K = 5 for mPLUG-Owl2 on CHAIR; 11 for MiniGPT-4 on CHAIR; 6 for LLaVA-1.5 on POPE; 11 for MiniGPT-4 on POPE
Selected per model and benchmark by 'preliminary experiments' balancing performance and computational efficiency (Section 5.3); the optimal number of subspaces is determined on the COCO training set and applied to the test set (Section 5.6).
Subspace rank r = 32 basis vectors for mPLUG-Owl2, 8 for MiniGPT-4 on CHAIR; 4 optimal for LLaVA-1.5 (Table 3)
Chosen to balance CHAIR_S and CHAIR_I against BLEU; selected on the COCO training set (Section 5.6); the values are benchmark-specific fits.
Temperature tau = Not reported
Used in Eq. (17) to convert alignment scores into weights; no value or sensitivity analysis is provided.
Mask ratio = 0.7
Chosen by hand in Section 4.3 to zero 70% of salient regions; no ablation over the ratio is reported.
Edited layer set L = Unspecified
Algorithm 1 edits 'selected layers L', but the paper never specifies which layers or how many are used.

assumptions (4)

domain assumption Token-averaged hidden-state differences between hallucinated and truthful captions are a valid proxy for hallucination direction.
Invoked in Section 4.2 to build difference matrices D_l and to justify clustering; no validation that averaging preserves the semantic shift rather than diluting it.
domain assumption The masked-image hidden-state difference encodes the test sample's hallucination susceptibility.
Central to Section 4.3 adaptivity; only supported by the perturbation ablation in Figure 3(right), which shows masking beats Gaussian and blur but does not show the difference correlates with the clustered hallucination modes.
domain assumption K-means on the averaged difference matrix yields semantically distinct hallucination modes.
The 'disentangled' claim rests on this; no analysis of cluster contents or separation is provided in Section 4.2.
domain assumption GPT-4-generated hallucinated captions on COCO produce hallucination directions that generalize to CHAIR and POPE benchmarks.
The contrastive dataset D is built with a GPT-4 perturbation oracle (Section 4.1); cross-benchmark transfer is assumed without evidence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs." pith.science (2026). https://pith.science/paper/JEQFIEH6

@misc{pith2026260809344,
  author       = {Pith},
  title        = {Pith review of: Beyond Global Editing: Per-Instance Disentangled Subspaces for Training-Free Hallucination Mitigation in LVLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JEQFIEH6}},
  note         = {Machine review of arXiv:2608.09344}
}

read the original abstract

Recent advances in large vision-language models (LVLMs) have enabled powerful multimodal reasoning by integrating visual encoders with large language models (LLMs). However, their reliability is frequently undermined by hallucinations, where generated text inaccurately describes the visual input. Although fine-tuning can mitigate this problem, it is computationally expensive and requires large, curated datasets, making training-free alternatives attractive. Among these, model editing is more promising than decoding-based approaches: decoding methods adapt outputs per input but introduce computational overhead and instability, whereas model editing modifies internal representations offline, providing a more efficient and stable solution. However, existing model-editing techniques typically rely on a single global subspace to correct hallucinations, treating all test samples identically and failing to capture diverse hallucination modes across inputs. To address this limitation, we propose a training-free hallucination mitigation framework for dynamic, per-instance suppression at test time. Our method first constructs a set of Disentangled Hallucination Subspaces, each isolating a distinct hallucination mode. During inference, the model adaptively calculates weights reflecting each input's relationship to these subspaces, guiding a dynamically combined projection that selectively suppresses the most probable hallucination directions while preserving image-grounded semantics. Extensive experiments across multiple vision-language benchmarks and LVLM families demonstrate consistent improvements, highlighting the robustness, generalizability, and efficiency of our approach.

Figures

Figures reproduced from arXiv: 2608.09344 by the authors.

**Figure 1.** (Left): Existing model editing methods [29] derive a single subspace of hallucination directions (via SVD) and apply fixed editing to the LVLM, using the same edited model for all inputs. (Right): Our adaptive method identifies multiple subspaces from different hallucination modes and adaptively adjusts their contributions to edit the model based on the input image, enabling more flexible and context-aware hallucin… view at source ↗

**Figure 2.** Illustration of our test-time hallucination mitigation framework. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** (left) The impact of the number of subspaces in the LLaVA-1.5 model within our proposed method. (right) We assess different perturbation strategies—masking, Gaussian, and blurring. grounding signals already discarded during encoding. In contrast, mPLUG-Owl2’s modality collaboration design retains richer visual context throughout the decoding pipeline, providing a more receptive substrate for our sample-specific sup… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 20 canonical work pages

[1]

Foundation model-powered 3d few- shot class incremental learning via training-free adaptor

Sahar Ahmadi, Ali Cheraghian, Morteza Saberi, Md.Towsif Abir, Hamidreza Dast- malchi, Farookh Hussain, and Shafin Rahman. Foundation model-powered 3d few- shot class incremental learning via training-free adaptor. In Minsu Cho, Ivan Laptev, Du Tran, Angela Yao, and Hongbin Zha, editors,Computer Vision – ACCV 2024, pages 178–195, Singapore, 2025. Springer ...

work page 2024
[2]

Hime: Mitigating object hallucinations in lvlms via hallucination in- sensitivity model editing, 2026

Ahmed Akl, Abdelwahed Khamis, Ali Cheraghian, Zhe Wang, Sara Khalifa, and Kewen Wang. Hime: Mitigating object hallucinations in lvlms via hallucination in- sensitivity model editing, 2026. URLhttps://arxiv.org/abs/2602.18711

work page arXiv 2026
[3]

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Jun- yang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

arXiv 2023
[4]

Halc: Object hallucination reduction via adaptive focal-contrast decoding.arXiv preprint arXiv:2403.00425, 2024

Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Halc: Object hallucination reduction via adaptive focal-contrast decoding.arXiv preprint arXiv:2403.00425, 2024

arXiv 2024
[5]

Synthesized feature based few-shot class-incremental learning on a mixture of subspaces

Ali Cheraghian, Shafin Rahman, Sameera Ramasinghe, Pengfei Fang, Christian Si- mon, Lars Petersson, and Mehrtash Harandi. Synthesized feature based few-shot class-incremental learning on a mixture of subspaces. In2021 IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pages 8641–8650, 2021. doi: 10.1109/ICCV48922.2021.00854

arXiv 2021
[6]

Canonical shape projection is all you need for 3d few-shot class incremental learning

Ali Cheraghian, Zeeshan Hayder, Sameera Ramasinghe, Shafin Rahman, Javad Ja- faryahya, Lars Petersson, and Mehrtash Harandi. Canonical shape projection is all you need for 3d few-shot class incremental learning. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol, editors,Computer Vi- sion – ECCV 2024, pages 36–53...

work page 2024
[7]

Dola: Decoding by contrasting layers improves factuality in large language models

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883, 2023

arXiv 2023
[8]

Glass, and Pengcheng He

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large lan- guage models. InThe Twelfth International Conference on Learning Representations,

work page

Show all 38 references

[9]

Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023

2023
[10]

Etta: Efficient test-time adaptation for vision-language models through dynamic embedding updates

Hamidreza Dastmalchi, Aijun An, and Ali Cheraghian. Etta: Efficient test-time adaptation for vision-language models through dynamic embedding updates. In36th British Machine Vision Conference 2025, BMVC 2025, Sheffield, UK, November 24-27,

2025
[11]

Test-time adaptation of 3d point clouds via denoising diffusion models

Hamidreza Dastmalchi, Aijun An, Ali Cheraghian, Shafin Rahman, and Sameera Ra- masinghe. Test-time adaptation of 3d point clouds via denoising diffusion models. In2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1566–1576, 2025. doi: 10.1109/W A...

2025
[12]

Fighting hallucinations with counterfactuals: Diffusion-guided perturbations for lvlm halluci- nation suppression

Hamidreza Dastmalchi, Aijun An, Ali Cheraghian, and Hamed Barzamini. Fighting hallucinations with counterfactuals: Diffusion-guided perturbations for lvlm halluci- nation suppression. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p...

2026
[13]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiao- hua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

2021
[14]

Beam search strategies for neural machine translation.arXiv preprint arXiv:1702.01806, 2017

Markus Freitag and Yaser Al-Onaizan. Beam search strategies for neural machine translation.arXiv preprint arXiv:1702.01806, 2017

2017 arXiv
[15]

Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation

Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. InProceed- ings of the IEEE/CVF Conferen...

2024
[17]

Mitigating object hallucinations in large vision-language models through visual contrastive decoding

Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

2024
[18]

Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. InProceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org, 2023

2023
[19]

Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

2023
[20]

Evaluating object hallucination in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

2023 arXiv
[21]

Visual instruction tuning, 2023

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. CHERAGHIAN ET AL: DISENTANGLED SUBSPACES FOR L VLMS17

2023
[22]

Visual instruction tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023

2023
[23]

Reducing hallucinations in vision- language models via latent space steering.arXiv preprint arXiv:2410.15778, 2024

Sheng Liu, Haotian Ye, Lei Xing, and James Zou. Reducing hallucinations in vision- language models via latent space steering.arXiv preprint arXiv:2410.15778, 2024

2024 arXiv
[24]

Learning transferable visual models from natural lan- guage supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural lan- guage supervision. In Marina Meila ...

2021
[25]

Object hallucination in image captioning.arXiv preprint arXiv:1809.02156, 2018

Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning.arXiv preprint arXiv:1809.02156, 2018

2018 arXiv
[26]

Aligning large multimodal models with factually augmented rlhf

Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, and Trevor Darrell. Aligning large multimodal models with factually augmented rlhf. 2023

2023
[27]

Mitigating hallucina- tions in large vision-language models with instruction contrastive decoding

Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucina- tions in large vision-language models with instruction contrastive decoding. InFind- ings of the Association for Computational Linguistics ACL 2024, pages 15840–15853,

2024
[28]

Detecting and mitigating hallucination in large vision language models via fine-grained ai feedback

Wenyi Xiao, Ziwei Huang, Leilei Gan, Wanggui He, Haoyuan Li, Zhelun Yu, Fangxun Shu, Hao Jiang, and Linchao Zhu. Detecting and mitigating hallucination in large vision language models via fine-grained ai feedback. InProceedings of the AAAI Con- ference on Artificial Intelligen...

2025
[29]

Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection

Le Yang, Ziwei Zheng, Boxu Chen, Zhengyu Zhao, Chenhao Lin, and Chao Shen. Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2025

2025
[30]

URLhttps://aclanthology.org/2024.findings-acl.937

2024
[31]

Woodpecker: Hallucination correction for multimodal large language models.Science China Information Sciences, 67(12): 220105, 2024

Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models.Science China Information Sciences, 67(12): 220105, 2024

2024
[32]

Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. InProceed- ings of the IEEE/CVF Conference on C...

2024
[33]

mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration

Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. InProceedings of the ieee/cvf conference on com- puter vision and pattern recognition, ...

2024
[34]

Analyzing and mitigating object hallucina- tion in large vision-language models

Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucina- tion in large vision-language models. InThe Twelfth International Conference on Learning Representations, 2024. URLhttps://o...

2024
[35]

Minigpt-4: Enhancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

2023 arXiv
[36]

Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754, 2023

Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754, 2023

arXiv
[39]

Vasparse: Towards efficient visual hallucination mitigation via visual-aware token sparsification

Xianwei Zhuang, Zhihong Zhu, Yuxin Xie, Liming Liang, and Yuexian Zou. Vasparse: Towards efficient visual hallucination mitigation via visual-aware token sparsification. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 4189–4199, 2025

2025
[2024]

URLhttps://openreview.net/forum?id=Th6NyL07na
[2025]

URLhttps://bmva-archive.org.uk/bmvc/2025/ assets/papers/Paper_1137/paper.pdf

BMV A, 2025. URLhttps://bmva-archive.org.uk/bmvc/2025/ assets/papers/Paper_1137/paper.pdf. 16CHERAGHIAN ET AL: DISENTANGLED SUBSPACES FOR L VLMS

2025

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Foundation model-powered 3d few- shot class incremental learning via training-free adaptor

Sahar Ahmadi, Ali Cheraghian, Morteza Saberi, Md.Towsif Abir, Hamidreza Dast- malchi, Farookh Hussain, and Shafin Rahman. Foundation model-powered 3d few- shot class incremental learning via training-free adaptor. In Minsu Cho, Ivan Laptev, Du Tran, Angela Yao, and Hongbin Zha, editors,Computer Vision – ACCV 2024, pages 178–195, Singapore, 2025. Springer ...

work page 2024

[2] [2]

Hime: Mitigating object hallucinations in lvlms via hallucination in- sensitivity model editing, 2026

Ahmed Akl, Abdelwahed Khamis, Ali Cheraghian, Zhe Wang, Sara Khalifa, and Kewen Wang. Hime: Mitigating object hallucinations in lvlms via hallucination in- sensitivity model editing, 2026. URLhttps://arxiv.org/abs/2602.18711

work page arXiv 2026

[3] [3]

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Jun- yang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

arXiv 2023

[4] [4]

Halc: Object hallucination reduction via adaptive focal-contrast decoding.arXiv preprint arXiv:2403.00425, 2024

Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Halc: Object hallucination reduction via adaptive focal-contrast decoding.arXiv preprint arXiv:2403.00425, 2024

arXiv 2024

[5] [5]

Synthesized feature based few-shot class-incremental learning on a mixture of subspaces

Ali Cheraghian, Shafin Rahman, Sameera Ramasinghe, Pengfei Fang, Christian Si- mon, Lars Petersson, and Mehrtash Harandi. Synthesized feature based few-shot class-incremental learning on a mixture of subspaces. In2021 IEEE/CVF Inter- national Conference on Computer Vision (ICCV), pages 8641–8650, 2021. doi: 10.1109/ICCV48922.2021.00854

arXiv 2021

[6] [6]

Canonical shape projection is all you need for 3d few-shot class incremental learning

Ali Cheraghian, Zeeshan Hayder, Sameera Ramasinghe, Shafin Rahman, Javad Ja- faryahya, Lars Petersson, and Mehrtash Harandi. Canonical shape projection is all you need for 3d few-shot class incremental learning. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol, editors,Computer Vi- sion – ECCV 2024, pages 36–53...

work page 2024

[7] [7]

Dola: Decoding by contrasting layers improves factuality in large language models

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883, 2023

arXiv 2023

[8] [8]

Glass, and Pengcheng He

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large lan- guage models. InThe Twelfth International Conference on Learning Representations,

work page