Pith. sign in

REVIEW 4 major objections 6 minor 3 references

This paper proposes choosing in-context examples for hallucination detection by learning the geometry of a frozen LLM's latent space—not by lexical similarity—and reports that the resulting demonstrations improve factual-consistency judgmen

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:55 UTC pith:J3TJS66C

load-bearing objection Novel combination of manifold learning and proxy-anchor metric learning for ICL demonstration selection, but the headline HaluEval gains are suspect because the retriever trains on evaluation-set labels; the FEVER results are cleaner and modest. the 4 major comments →

arxiv 2601.06196 v3 pith:J3TJS66C submitted 2026-01-08 cs.LG cs.AIcs.CL

Geometry-Aware Hallucination Detection in Large Language Models

Bodla Krishna Vamshi , Rohan Bhatnagar , Haizhao Yang This is my paper
classification cs.LG cs.AIcs.CL
keywords hallucination detectionin-context learningdemonstration selectionmanifold learningmetric learningproxy anchor lossfrozen language modelsfactual verification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that the choice of examples used to prompt a frozen LLM to judge factual consistency can be made geometrically rather than lexically. It proposes MB-ICL, which trains a small projection head on the LLM's final-layer token embeddings, models them as piecewise-linear manifolds with class prototypes, and then selects in-context demonstrations nearest to a learned momentum prototype. Across FEVER and HaluEval, the selected demonstrations improve hallucination-detection accuracy over BM25, perplexity, clustering, KNN, and SA-ICL in most settings, with the largest gains in dialogue and summarization. If right, this offers a training-light way to improve factual-consistency judgments without modifying LLM parameters.

Core claim

The paper proposes MB-ICL (named GA-ICL in the abstract) and claims that latent representations extracted from a frozen LLM—specifically attention-mask-weighted mean-pooled final-layer token embeddings—carry enough geometric structure to pick in-context examples for hallucination detection. By jointly optimizing a proxy-anchor loss (class prototypes in a learned space) and a manifold point-to-point loss (preserving local piecewise-linear structure), the method learns a lower-dimensional sampling space. After training, a momentum-updated prototype is used to retrieve the closest transformed representations as the ICL demonstrations. The authors report accuracy gains in the majority of compari

What carries the argument

The central machinery is a joint training objective over a lightweight projection head hθ: (1) a proxy-anchor loss with Euclidean distance that pulls embeddings of a class toward class prototypes and pushes negatives away; and (2) a manifold point-to-point loss that enforces distances in the transformed space to match manifold-induced dissimilarities, computed from local PCA bases of piecewise-linear submanifolds around anchor points. The momentum-updated prototype θm acts as the selector: demonstrations are the transformed embeddings closest to it.

Load-bearing premise

The load-bearing premise is that the mean-pooled final-layer token embeddings of a frozen LLM lie on low-dimensional, locally linear manifolds whose geometry aligns with factual-consistency labels; if that alignment fails, the learned prototypes will select demonstrations no better than random.

What would settle it

Train the same projection head and prototypes but replace the manifold-loss term with a randomized projection of identical dimension; if detection accuracy does not drop when the manifold term is removed, the geometric structure is not what is driving the gains. Alternatively, evaluate on a 70B+ model, which the paper itself leaves open.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • In two-shot ICL settings, geometry-selected demonstrations improve hallucination detection over BM25, perplexity, clustering, KNN, and SA-ICL in the majority of configurations, including up to 9.9 accuracy points on Mistral-7B dialogue and summarization.
  • The method is training-light: a thin projection head trained once per dataset-model (about 640 MB GPU memory, 3–4 hours) reaches accuracy comparable to LoRA-SFT on Vicuna-7B while keeping the LLM frozen.
  • Selected prompts are stable under temperature changes from 0 to 1, unlike BM25 and SA-ICL, suggesting the method narrows the model's predictive distribution.
  • Detection accuracy rises with the number of demonstrations up to about ten shots and then saturates; compact prototype dimensionality is the most influential hyperparameter.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The discrepancy between the abstract's Phi-14B/Qwen3-32B claims and the body's statement in Section 5 that evaluation is limited to models up to 8B means the scaling result should be treated as unverified until the large-model evaluations appear in the text.
  • Because the momentum prototype yields a single fixed set of demonstrations for all test queries, the method is not query-adaptive; a natural extension is per-query prototype retrieval, which should improve further if the learned geometry is genuinely informative.
  • The paper reports lower perplexity for its selected prompts but disclaims a causal link; testing perplexity-only selection would isolate whether the manifold loss adds anything beyond model-likelihood fit.
  • A transferred application: the same manifold-plus-prototype selection could be tested for other reliability labels (e.g., safety or instruction-following quality), where demonstration choice is known to shift behavior.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MB-ICL (also called GA-ICL in the abstract), a method for selecting in-context learning demonstrations for hallucination detection. It extracts attention-mask-weighted mean-pooled hidden states from frozen LLMs, trains a lightweight projection network h_theta plus class prototypes using a proxy-anchor loss and a manifold point-to-point loss, and then samples the two nearest transformed representations to a momentum prototype as demonstrations. The authors evaluate on FEVER and HaluEval (dialogue, summarization, QA) across six LLMs, comparing with BM25, perplexity, KNN, clustering, and SA-ICL. They report that MB-ICL outperforms baselines in the majority of settings, with especially large gains on HaluEval dialogue and summarization, and that it is more stable under temperature variation. The central claim is that geometry-aware prototype selection in the frozen-LLM latent space improves ICL demonstration selection for hallucination detection.

Significance. If the claimed results held under a clean experimental protocol, the paper would make a useful contribution: it demonstrates that a lightweight, label-aware retriever can be trained on frozen-LLM representations to select in-context demonstrations for factuality classification, without modifying the LLM. The FEVER results with a proper train/test split do show consistent small improvements (roughly 0.7–7.7 points), and the computational comparison in Appendix B (640 MB, 3–4 hours for a thin head) is a legitimate strength. The temperature-robustness experiments, though limited to two models and two tasks, are a reasonable diagnostic. However, the headline HaluEval gains are not trustworthy because the retriever is trained on the same data and labels used for evaluation, and the hyperparameter selection appears to be test-set informed. The paper's main empirical claim therefore currently rests on a transductive protocol, and the evidence is weakened by the absence of uncertainty estimates and by the absence of strong supervised-retriever baselines.

major comments (4)
  1. [Section 4.2, Algorithm Table 1, Eq. (1)] The HaluEval evaluation protocol is transductive and leaks test labels into retriever training. For HaluEval, Section 4.2 states there are no explicit splits and demonstrations are 'sampled directly from the dataset and subsequently removed prior to evaluation to prevent data leakage.' However, Algorithm Table 1 trains h_theta and the prototypes on the full D_Ft using the proxy-anchor loss in Eq. (1) and the manifold loss in Eq. (2), both of which use the labels and the test queries themselves. Thus the projection h_theta and momentum prototype are fit with access to the very inputs and labels that are later scored. Baselines such as BM25, perplexity, KNN, and clustering are unsupervised and do not have this access. The HaluEval gains (e.g., Mistral dialogue 69.6 vs BM25 57.0, Mistral summarization 63.2 vs BM25 56.0) may be artifacts of test-label supervision rather than manifold geometr
  2. [Section 4.2, Section 5, Appendix C] Hyperparameters appear to be selected using the evaluation set. The text states that after a baseline configuration, 'an ablation study is conducted on the above parameters for LLMs that exhibited comparatively lower performance than competing methods' (Appendix C). Figure 5 and Figure 8 show accuracy as a function of prototype size Z' and other hyperparameters, but no validation split is described. Tuning hyperparameters on models that underperformed, after seeing test results, invalidates the comparison. The paper also identifies Z' as the most influential hyperparameter and shows that reducing it improves accuracy, which is a post hoc selection. Please provide a pre-specified validation split, report all hyperparameter choices for each model/task before test evaluation, and give the complete grid results including failures.
  3. [Section 4, Table 3] Only single-run accuracies are reported, with no error bars, confidence intervals, or significance tests. The claim of determinism applies to decoding at temperature 0, but training h_theta involves random initialization, batch order, and optimizer stochasticity; different seeds will produce different demonstrations and different accuracies. Many reported gains are small (e.g., 1–2 points on Qwen and Falcon), so it is impossible to tell whether the 'majority of settings' claim is robust. Please report multiple seeds or bootstrap CIs, and use a paired test (e.g., McNemar) over the evaluation instances for the main comparisons. This is necessary to support the central claim.
  4. [Section 4.3, Table 3] The baseline set is too weak for the claim 'outperforms standard ICL selection baselines.' KNN and clustering are near chance on several HaluEval tasks (e.g., 5.8–14.7 on QA/dialogue), which suggests the underlying embedding space or the ICL prompt is poorly aligned with the task. Because MB-ICL is a supervised retriever trained with labels, it should be compared with supervised or adaptive retrieval baselines such as EPR (Rubin et al., 2022), unified demonstration retriever (Li et al., 2023c), and AdaICL (Mavromatis et al., 2023), or at least with SA-ICL using a stronger first-stage retriever. Otherwise the gains may reflect the weakness of the baselines rather than the proposed geometry-aware selection.
minor comments (6)
  1. [Abstract] The abstract refers to the method as GA-ICL, while the body and title use MB-ICL. Please align terminology throughout.
  2. [Appendix C] Several hyperparameters are left unspecified: the number of anchor points n, the neighborhood size k, prototype initialization, and the number of proxy vectors. These are needed for reproducibility.
  3. [Appendix B, Table 4] The SFT comparison does not specify which dataset/task the reported accuracy corresponds to, nor the evaluation split. Please clarify.
  4. [Section 4.4] The temperature-robustness claim is based only on Qwen3-4B and Falcon3-3B, not all six models. Please either extend the analysis or soften the claim.
  5. [Throughout] Figures 2 and 3 show perplexity for only dialogue and QA; the text says 'across models' but should be specific. Also, 'self adaptive ICL sampling' in the text should be consistently hyphenated as SA-ICL.
  6. [Section 3.1] The manifold construction procedure uses PCA on local neighborhoods, but no details are given on how the number of anchors n or the neighborhood size k are chosen; please add these to Appendix C.

Circularity Check

1 steps flagged

HaluEval gains reduce to transductive label fit: retriever trained on the evaluation set's labels before 'predicting' it.

specific steps
  1. fitted input called prediction [Section 3 / Table 1; Section 4.2]
    "For the HaluEval tasks summarization, question answering, and dialogue the dataset does not provide explicit splits. In these cases, demonstrations were sampled directly from the dataset and subsequently removed prior to evaluation to prevent data leakage."

    The MB-ICL algorithm trains hθ and θq on the full dataset D_F_t with labels (Table 1: 'Input: Dataset D_F_t with labels l'; Eq. 1 proxy-anchor loss uses labels l). For HaluEval there is no train/test split, so D_F_t is the same set that is later scored. The learned prototype geometry is therefore fit to the labels of the evaluation examples; selecting demonstrations closest to θm and then evaluating on those same examples is a supervised/transductive fit, not an independent prediction of the test labels. The baselines (BM25, perplexity, KNN, clustering, SA-ICL) do not train on the evaluation labels, making the comparison asymmetric and the reported gains an artifact of label access.

full rationale

The FEVER experiments use a proper train/test split, so those results are not circular; the method's component-wise use of manifold learning and proxy-anchor losses is also not circular in itself. However, the headline HaluEval gains are not an independent prediction: the paper states that HaluEval provides no explicit splits, and that demonstrations are sampled from the dataset and removed before evaluation. But the same dataset, with labels, is used to train the projection head hθ and proxy prototypes via the proxy-anchor loss (Eq. 1). Thus the retriever has already seen the test queries and their labels before selecting demonstrations; accuracy on those queries reflects supervised fit rather than an ICL selection effect. The comparison is asymmetric because the baselines do not train on evaluation labels. This reduces the central claim for the largest-gain settings by construction, though it does not invalidate the FEVER results or temperature-robustness analysis.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 1 invented entities

The central claim rests on a learned projection head and prototype vectors trained with several hand-chosen hyperparameters. The manifold hypothesis and the pooled-embedding adequacy are domain assumptions that are plausible but not independently validated. No physical invented entities are introduced; the prototypes are learned parameters inside the method.

free parameters (8)
  • prototype size Z' = 1.3e3-3.8e3, tuned per model/task (Fig 5)
    Reported as the hyperparameter with the largest impact on performance; reducing dimensionality improves accuracy.
  • manifold dimension m = 3
    Chosen in the baseline configuration in Appendix C; controls the dimensionality of local linear submanifolds.
  • reconstruction threshold T = 90%
    Used in manifold expansion/validation to decide whether a candidate point lies on the local submanifold.
  • distance scaling delta = 2
    Sets an upper bound on separation between dissimilar samples in the manifold point-to-point loss.
  • similarity decay exponents N_alpha, N_beta = 4 and 0.5
    Control how similarity attenuates with orthogonal and projected distances; constraint N_alpha > N_beta is imposed.
  • proxy-anchor temperature alpha and margin epsilon = alpha=32, epsilon=0.1
    Control concentration and separation in the proxy-anchor loss.
  • momentum coefficient mu = 0.99
    Used to update the momentum prototype theta_m; ablation in Fig 8a shows accuracy varies with mu.
  • training schedule = lr=1e-3, decay=0.97, batch=128, epochs=200
    Adam optimizers with this schedule are used for both the projection network and proxy parameters.
axioms (5)
  • domain assumption The manifold hypothesis: latent representations from frozen LLMs can be locally approximated by low-dimensional linear submanifolds.
    Invoked in Section 3.1; the entire manifold construction and both loss terms depend on this geometric assumption.
  • domain assumption Attention-mask-weighted mean of final-layer token hidden states yields a fixed-dimensional vector that preserves task-relevant factual-consistency information.
    Section 3; this pooling is the only representation used for all selection methods, yet its adequacy is not independently validated.
  • domain assumption Proximity in the learned transformed space to class prototypes is a good criterion for selecting ICL demonstrations that improve target-LLM accuracy.
    Central to the sampling step in Table 1; if this transfer fails, the method fails even though the projection loss may converge.
  • domain assumption Proxy-anchor loss with Euclidean distance and two proxies is trainable and stable on pooled text embeddings for these datasets.
    Section 3.2; no convergence or stability analysis is provided, and the two-proxy setup assumes binary label structure.
  • standard math PCA reconstruction quality threshold reliably detects m-dimensional linear structure in the latent space.
    Used in manifold expansion and validation in Section 3.1; PCA is standard, but its local applicability on text embeddings is an assumption.
invented entities (1)
  • class prototype vectors theta_q and momentum prototype theta_m no independent evidence
    purpose: Represent class-level geometry in the transformed embedding space and serve as anchors for selecting nearest demonstrations.
    These are learned parameters, not independently evidenced entities; their utility is demonstrated only through downstream accuracy on the evaluated benchmarks.

pith-pipeline@v1.3.0-alltime-deepseek · 14563 in / 10294 out tokens · 106832 ms · 2026-08-03T11:55:48.069714+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) frequently generate factually incorrect or unsupported content, commonly referred to as hallucinations. Prior work has explored decoding strategies, retrieval augmentation, and supervised fine-tuning for hallucination detection, while recent studies show that in-context learning (ICL) can substantially influence factual reliability. However, existing ICL demonstration selection methods often rely on surface-level similarity heuristics and exhibit limited robustness across tasks and models. We propose GA-ICL, a geometry-aware demonstration sampling framework for selecting in-context demonstrations that leverages latent representations extracted from frozen LLMs. By jointly modeling local manifold structure and class-aware prototype geometry, GA-ICL selects demonstrations based on their proximity to learned prototypes rather than lexical or embedding similarity alone. Across factual verification (FEVER) and hallucination detection (HaluEval) benchmarks, GA-ICL outperforms standard ICL selection baselines in the majority of evaluated settings, with particularly strong gains on dialogue and summarization tasks. The method remains robust under temperature perturbations and model variation, indicating improved stability compared to heuristic retrieval strategies. While lexical retrieval can remain competitive in certain question-answering regimes at smaller model scales, our results demonstrate that geometry-aware prototype selection provides a reliable and training-light approach for hallucination detection without modifying LLM parameters. Extended evaluations on Phi-14B and Qwen3-32B confirm that GA-ICL scales effectively to larger models, outperforming all compared baselines including on QA tasks where smaller models show boundary-condition limitations, offering a principled direction for improved ICL demonstration selection.

Figures

Figures reproduced from arXiv: 2601.06196 by Bodla Krishna Vamshi, Haizhao Yang, Rohan Bhatnagar.

Figure 1
Figure 1. Figure 1: Accuracy comparisons of MB-ICL vs other methods in-context learning (ICL), where models perform downstream tasks by conditioning on prompts that include a small number of representative exam￾ples. Owing to its effectiveness and flexibility, ICL has become a widely adopted and efficient paradigm for utilizing PLMs. This paradigm has been successfully applied to a variety of tasks, in￾cluding reasoning and c… view at source ↗
Figure 2
Figure 2. Figure 2: Perplexity comparison across models on HaluEval dialogue task. Lower perplexity indicates better performance. bustness in dynamic or streaming scenarios. Qwen3-4B Llama3-8B Falcon-3-3B Base GPT-Neo-2.7B Vicuna-7B Mistral-7B 0 5 10 15 Models Perplexity SA-ICL BM25 MB-ICL (ours) [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance of MB-ICL under varying numbers of in-context demonstrations [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Accuracy vs Prototype Size Z ′ For the initial set of experiments, the hyperpa￾rameters for manifold construction and manifold point-to-point loss estimation are configured as fol￾lows: T = 90%, δ = 2, m = 3, Nα = 4, and Nβ = 0.5. The momentum constant for updating θm is set to µ = 0.99. For Proxy Anchor loss, we employ α = 32 and ϵ = 0.1. These settings serve as the baseline configuration; subsequently, a… view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy vs Temperature comparison for Qwen3 and Falcon3 models on HaluEval QA dataset [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Accuracy vs Temperature comparisons for Qwen3 and Falcon3 models on HaluEval Dialogue dataset [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Hyperparameter tuning for Qwen3-4B on Summarization task [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

3 extracted references · 3 linked inside Pith

  1. [3]

    You are an unbiased document - grounded fact checker

    An information-theoretic approach to prompt engineering without ground truth labels. InProceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Pa- pers), pages 819–862, Dublin, Ireland. Association for Computational Linguistics. Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and ...

    Pith/arXiv arXiv 2024

  2. [2022]

    Learning to retrieve prompts for in-context learning. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 2655–2671, Seattle, United States. Association for Computational Linguistics. Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sri- parna Saha, Vinija Ja...

    Pith/arXiv arXiv 2022

  3. [2023]

    Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, Huzefa Rangwala, Christos Faloutsos, and George Karypis

    Selfcheckgpt: Zero-resource black-box hal- lucination detection for generative large language models.Preprint, arXiv:2303.08896. Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, Huzefa Rangwala, Christos Faloutsos, and George Karypis. 2023. Which examples to annotate for in-context learning? towards effective and efficient selec...

    Pith/arXiv arXiv 2023