Pith. sign in

REVIEW 4 major objections 4 minor 11 references

Relational linearity predicts whether a language model hallucinates or refuses when asked about an invented entity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 09:59 UTC pith:JDK6GGKT

load-bearing objection A plausible mechanism linking relation linearity to hallucination, with a careful evaluation pipeline, but the evidence rests on only six relations, an abstract/text mismatch, and a circular control for output priors. the 4 major comments →

arxiv 2601.11429 v2 pith:JDK6GGKT submitted 2026-01-16 cs.CL cs.AI

Relational Linearity is a Predictor of Hallucinations

Yuetian Lu , Yihong Liu , Sebastian Gerstner , Lea Hirlimann , Jonas Rohweder , Hinrich Sch\"utze This is my paper
classification cs.CL cs.AI
keywords hallucinationrelational linearityΔcoslanguage modelsknowledge self-assessmentsynthetic entitiesrefusal behavior
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a language model's tendency to fabricate an answer for an entity it has never seen is predictable from how linearly the model stores the underlying relation. Using six relations probed with 6,000 synthetic, unknown subjects, four instruction-tuned models hallucinated—gave a concrete value instead of refusing—far more often for relations whose subject-to-object mapping is well captured by a single added vector, with correlations between the linearity score and hallucination rate of about r = 0.78–0.82. The proposed reason is that a linear relation supplies a reusable translation that produces a plausible object even when no specific triple was learned, so the model cannot tell that it does not know; nonlinear relations require more instance-specific storage, which makes refusal easier. If this is right, part of the blame for hallucinations shifts from missing knowledge to the format in which knowledge is stored, and mitigation can target high-linearity relations with explicit known-fact checks.

Core claim

On the paper's own terms, the discovery is that relational linearity is a strong relation-level predictor of hallucination on prompts where the subject is unknown by construction. The authors generate synthetic entities across six relations, ask four instruction-tuned models for the relation's object, and label each answer as refusal or hallucination. Separately, they measure linearity with Δcos: on natural subject–object triples, they take the average subject-to-object difference vector, add it to held-out subject representations, and record the improvement in cosine similarity to the true object. Hallucination rate rises with Δcos for all four models (Pearson r between .78 and .82), and th

What carries the argument

The load-bearing measure is Δcos, computed with a deliberately simple translation-only probe: for each model and relation, subject and object hidden states are extracted from natural triples at fixed layers (mid-layer for subjects, third-from-last for objects); the relation direction d̄r is the mean object-minus-subject vector on a training split; the object is predicted as s + d̄r on held-out subjects; and linearity is the expected cosine improvement over using the subject itself. A high Δcos means one shared direction reproduces many triples—the relation is stored 'abstractly.' That same direction is what lets the model generate a plausible object for a never-seen subject, which is the pro

Load-bearing premise

The load-bearing premise is that Δcos—how much one averaged subject-to-object difference vector improves object prediction on held-out natural triples—captures how abstractly the model stores the relation; if Δcos mostly tracks answer-output concentration, layer choice, or object frequency instead, the correlation would not support the proposed storage-format mechanism.

What would settle it

Intervention test: take a nonlinear relation on which models mostly refuse for unknown subjects (e.g., father's first name), add the relation's average difference vector to the subject representation at the probed layer, and decode greedily. If refusals persist, linearity is not the causal mechanism; if they turn into confident hallucinations, the translation-vector story is confirmed. A complementary check is to recompute Δcos with a full affine map (allowing W not equal to identity) and see whether the r≈0.8 correlation survives.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Hallucination risk for a relation can be estimated from representation geometry on ordinary known triples, before the model is ever asked about an unknown entity.
  • For highly linear relations, a confident fabrication does not require the model to 'know' anything: the translation vector supplies an object automatically, so refusal behavior is not a straightforward honesty signal.
  • Mitigations should add explicit known-triple verification for linear relations; supplementing abstract representations with evidence markers is the concrete target the authors propose.
  • Linearity is a continuum, not a binary: intermediately linear relations show intermediate hallucination rates, so interventions should be graded.
  • On natural (known) triples the correlation reverses—more linear relations are more accurate—so the effect is specific to genuinely unknown subjects and evaluations must keep the two regimes separate.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is an intervention: add the learned relation direction to an unknown subject's representation and check whether refusals turn into hallucinations; this would convert the correlation into a causal test.
  • The authors' output-concentration controls are suggestive but not definitive; a matched-pair design that pairs relations with similar answer entropy but different Δcos would cleanly separate the storage-format story from a prior-concentration story.
  • The reversal on natural triples hints that linearity partly tracks training frequency: common relations become linear and accurate. Controlling for pretraining counts across relations would tell whether linearity adds predictive power beyond frequency.
  • If the mechanism generalizes, a practical diagnostic could rank relations by Δcos on a small probe set and use it to decide where refusal guardrails are needed before deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes that the degree of relational linearity in LLM representations—measured by Δcos on natural triples—predicts whether the model hallucinates an object for a synthetic unknown subject versus refuses to answer. The authors introduce SyntHal, a dataset of 6000 synthetic entities across six relations, and measure hallucination rates for four instruction-tuned LLMs. They report a strong positive correlation (r≈.78–.82) between relation-level Δcos and hallucination rate, and interpret this as evidence that linearly stored relations make knowledge self-assessment harder. The paper also includes a natural-triple analysis (LRE) showing the opposite correlation, explained as a relation difficulty effect.

Significance. If the central claim holds, the paper identifies a representational property—relational linearity—as a predictor of hallucination behavior, with potential implications for knowledge self-assessment and targeted mitigation. The measurement pipeline is careful and reproducible in several respects: Wilson confidence intervals for hallucination rates, exact permutation p-values, leave-one-relation-out ranges, a regex judge baseline with 96.4% agreement, and human validation of 200 judge labels. The use of external natural triples for linearity and separate synthetic entities for behavior reduces direct circularity. However, the evidence is correlational and rests on only six hand-picked relations, and the proposed storage-format mechanism is not cleanly separated from output-space priors. The paper is transparent about these limitations, but they are load-bearing for the interpretation.

major comments (4)
  1. [§4, Figure 1, Table 6] The central evidence is a correlation over n=6 hand-picked relations. Two-sided exact permutation p-values are all >0.05 (0.071, 0.064, 0.089, 0.086); only one-sided p-values are below .05. The Fisher combined p=0.0085 combines non-independent samples (the same six relations across four models) and is not a valid significance test. Because relations were selected to maximize linearity spread, the correlation may be inflated. Please add more relations (e.g., all LRE relations with sufficient triples), report confidence intervals for r, and use a test that accounts for relation identity.
  2. [Appendix G, Limitations] The output-space concentration control is partly circular: Top-1 share and normalized entropy are computed from the model's own hallucinated outputs—the very behavior being predicted. If answer spaces are concentrated, both high hallucination rate and high Δcos can result without storage-format abstraction being the cause. Please provide an independent measure of answer-space concentration (e.g., from a corpus or from the model's prior over known subjects) or use interventions (patching/steering) that hold answer-space entropy fixed. As it stands, the data do not separate the proposed storage-format mechanism from an output-space-prior confound.
  3. [Abstract vs. main text] The abstract states SyntHal contains 15 relations and reports r∈[.58,.84]; the main text (Abstract, §2.2, Figure 1) says six relations and r∈[.78,.82]. These are different claims. The manuscript must be reconciled—readers cannot assess which result is being claimed.
  4. [§3, Eq. (1)–(2), Appendix D] The linearity proxy uses a translation-only probe (W=I) and fixed layers ℓs=⌊L/2⌋, ℓo=L−2, with no robustness analysis. If Δcos is sensitive to these choices, the correlation could be a probe artifact. Please report Δcos across a range of layers and with an unconstrained affine probe (W free) for the same relations to show the main result is stable.
minor comments (4)
  1. [Figure 1] The x-axis label reads 'LRE cosine improvement ( cos)' but should be 'Δcos' for consistency with Eq. (3).
  2. [References] The entry 'Lindsey and 1 others. 2025' is unprofessional; either list the full author list or use standard 'et al.' formatting.
  3. [Table 2] The relation name 'father’s first name' is awkward as a relation label; consider a canonical form such as 'father_first_name'.
  4. [Appendix F] The statement that ignoring covariance is 'slightly conservative' is correct only because the covariance is typically positive; this could be made explicit.

Circularity Check

0 steps flagged

No significant circularity: the linearity measure and hallucination rates come from separate data sources, and no fitted parameter is renamed as a prediction.

full rationale

The paper's central claim is a relation-level correlation between Δcos (Eqs. 1–3), computed from external natural LRE triples using a fixed translation-only estimator (W=I, b=mean difference), and hallucination rates measured on a separate synthetic benchmark (SyntHal) with LLM-as-a-judge labels validated against human annotations on 200 examples. No parameter is fitted to hallucination outcomes; the Δcos values would be identical even if the behavioral results were different. The selection of six relations based on expected linearity is a sampling choice, not a fit. The Appendix G output-concentration control computes proxies from hallucinated outputs, but the paper transparently presents this as a robustness check and reports both unadjusted and partial correlations; this is a potential confounding/over-control concern, not a circular derivation. The Limitations explicitly state that Δcos is 'a practical proxy' and that the analyses are 'correlational,' disclaiming causal mechanism. The key method (LRE/Δcos) is attributed to external work by Hernandez et al. (2024), and no load-bearing self-citation is used. The abstract/main-text discrepancy (15 relations vs. 6 relations; r ranges [.58,.84] vs. [.78,.82]) is an internal consistency issue, not circularity. The derivation chain is therefore self-contained with respect to the prediction claim.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The central claim depends on (i) the construct validity of the Δcos proxy, (ii) the unknownness of the synthetic entities, and (iii) enough statistical power from six hand-picked relations. No entities are invented; the behavioral and representational measurements are conducted on separate datasets, which keeps the circularity burden low.

free parameters (3)
  • probe layer indices (ℓs, ℓo) = ℓs=⌊L/2⌋, ℓo=L−2 (e.g., 14/26 for 28-layer; 16/30 for 32-layer)
    Hand-chosen per model; Δcos values and the resulting correlation depend on where subject and object representations are read. No sensitivity analysis across layers is provided (Appendix D).
  • relation set (6 relations) = instrument, sport (linear); father, company_ceo (nonlinear); company_hq, country_language (intermediate)
    The six relations were deliberately selected to span the linearity spectrum (§2.2), not sampled; LOO analysis tests removal of single relations but cannot test the choice of the pool. This design artifact can inflate r.
  • translation probe constraint W=I = W fixed to identity, only b=mean difference fitted
    The Δcos proxy uses a constrained affine family; a full linear map could change which relations appear linear (Limitations). This is an acknowledged modelling simplification with consequences for the central correlation.
axioms (4)
  • ad hoc to paper Δcos (translation-only) is a valid proxy for 'abstraction' of a relation's storage.
    The paper's mechanism story assumes that high Δcos implies the model can generate objects via an affine map during generation, but Δcos is measured on natural triples at fixed layers with W=I; no causal or generation-time evidence (Limitations).
  • domain assumption Synthetic entities are truly unknown to the models.
    Entities come from ChatGPT-5.2 token pools (Appendix A) with no-per-collision constraints; no per-entity verification that the model has no prior. If any synthetic entity resembles a known entity, responses are not purely 'unsupported'.
  • ad hoc to paper Relation-level aggregation with six points is sufficient for a stable correlation.
    Within-model correlations rest on n=6; p-values exceed .05 for three of four models; the Fisher combined test ignores dependence between models sharing the same six relations.
  • domain assumption Cosine geometry is comparable across the four model families at the chosen layers.
    Hidden-state scales vary by an order of magnitude (e.g., Qwen, Appendix E); the paper uses scale-invariant cosine and relative changes, but cross-model comparability of Δcos is assumed for the pooled claims.

pith-pipeline@v1.3.0-alltime-deepseek · 12991 in / 16583 out tokens · 163266 ms · 2026-08-03T09:59:48.762095+00:00 · methodology

0 comments
read the original abstract

Hallucination is a central failure mode of language models (LMs). We focus on hallucinations in response to questions like: "Which instrument did Glenn Gould play?", but we ask these questions for synthetic entities designed to be unknown to the model. We find that LMs like Gemma-7B-IT frequently hallucinate, i.e., they have difficulty recognizing that the hallucinated fact is not part of their knowledge. Based on the idea of linear relational embeddings, we put forward the following hypothesis. (i) Due to the abstract scheme that is used to represent them, LMs can easily produce plausible objects for non-existing subjects of linear relations, which can lead to hallucinations. (ii) For a nonlinear relation, this mechanism for producing an object is not available and so a hallucination is easier to avoid. To test this hypothesis, we create SyntHal, a synthetic unknown-entity benchmark for 15 relations. We find that across four instruction-tuned models, relational linearity is a strong predictor of models hallucinating an object for an unknown subject vs refusing to give an answer, with correlations $r \in [.58, .84]$.

Figures

Figures reproduced from arXiv: 2601.11429 by Hinrich Sch\"utze, Jonas Rohweder, Lea Hirlimann, Sebastian Gerstner, Yihong Liu, Yuetian Lu.

Figure 1
Figure 1. Figure 1: Hallucination rate (# hallucinations divided by [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Two model generations and LLM-as-a-judge [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-relation behavioral outcomes under the standardized inference interface: hallucination (value [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

11 extracted references · 6 linked inside Pith

  1. [4]

    arXiv preprint arXiv:2407.21783

    The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Javier Ferrando, Oscar Balcells Obeso, Senthooran Ra- jamanoharan, and Neel Nanda

    Pith/arXiv arXiv

  2. [5]

    Mistral 7b.Preprint, arXiv:2310.06825. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, and 17 others

    Pith/arXiv arXiv

  3. [7]

    InThe 2023 Conference on Empirical Methods in Natural Language Processing

    G-eval: NLG evaluation using gpt-4 with better human align- ment. InThe 2023 Conference on Empirical Methods in Natural Language Processing. Samuel Marks and Max Tegmark

    2023

  4. [9]

    Preprint, arXiv:2502.04520

    Linear correlation in lm’s compositional generalization and hallucination. Preprint, arXiv:2502.04520. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, and 24 others

    Pith/arXiv arXiv

  5. [10]

    Stefan F

    Qwen2.5 technical report.Preprint, arXiv:2412.15115. Stefan F. Schouten, Peter Bloem, Ilia Markov, and Piek V ossen

    Pith/arXiv arXiv

  6. [11]

    Preprint, arXiv:2403.08295

    Gemma: Open models based on gemini research and technology. Preprint, arXiv:2403.08295. Edwin B. Wilson

    Pith/arXiv arXiv

  7. [2020]

    InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, On- line

    On faithfulness and factu- ality in abstractive summarization. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, On- line. Association for Computational Linguistics. Jack Merullo, Noah A. Smith, Sarah Wiegreffe, and Yanai Elazar

    1906

  8. [2022]

    Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg

    Language models (mostly) know what they know.Preprint, arXiv:2207.05221. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg

    Pith/arXiv arXiv

  9. [2023]

    InFind- ings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, Singapore

    The internal state of an LLM knows when it’s lying. InFind- ings of the Association for Computational Linguistics: EMNLP 2023, pages 967–976, Singapore. Associa- tion for Computational Linguistics. David Chanin, Anthony Hunter, and Oana-Maria Cam- buru

    2023

  10. [2024]

    Identifying linear relational concepts in large language models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (V olume 1: Long Papers), pages 1524–1535, Mexico City, Mexico. Association for Computational Linguistics. Miranda Anna Christ, Adri??n Csisz??rik,...

    2024

  11. [2025]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others

    The structure of relation decoding linear operators in large language models.Preprint, arXiv:2510.26543. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others

    arXiv