Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Linear Correlation in LM's Compositional Generalization and Hallucination

T0 review · 3 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that next-token predictions for related prompts are connected by a fixed affine map in logit space that survives fine-tuning and determines whether knowledge updates generalize or hallucinate.

desk verdict A reproducible logit-level linearity finding with a causal story that outruns the evidence. read the letter →

arxiv 2502.04520 v1 pith:M2GMPVLU submitted 2025-02-06 cs.CL

Letian Peng , Chenyang An , Shibo Hao , Chengyu Dong , Jingbo Shang This is my paper · ORCID
classification cs.CL
keywords linearcorrelationknowledgecompositioncompositionalgeneralizationhallucinationnext-tokenpredictionlogitspacevocabularyrepresentationsfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Language models can often compose related facts, knowing which city someone lives in points to their country, but they also fail at composition and hallucinate. This paper tries to explain both behaviors with one mechanism: a fixed affine map $(W,b)$ that sends the next-token logits of a source prompt, e.g., "X lives in the city of", to those of a related target prompt, e.g., "X lives in the country of", for every input $X$. The paper claims that $(W,b)$ is stable under large-scale fine-tuning, so an update to source knowledge propagates to target knowledge; when $W$ weights the correct real-world pairs this is compositional generalization, and when it weights wrong pairs this is compositional hallucination. If the claim is correct, the fitted $W$ gives a pre-hoc identifier of whether a knowledge edit will generalize or hallucinate, and vocabulary representations, rather than the deep architecture, are the main carrier of the effect.

What carries the argument

The central object is the fitted affine map $(W,b)$ between output logits of a source and a target knowledge prompt, restricted to meaningful vocabulary subdomains; each entry of $W$ acts as a compositional weight, e.g., $W_{\text{France},\text{Paris}}$ connects the probability of the source answer with the probability of the target answer. The paper fits $(W,b)$ on half of a sample of arbitrary inputs and evaluates it on the rest, then uses the same $(W,b)$ to estimate target gradients from source gradients. The load-bearing mechanism is that $(W,b)$ is resilient to post-training, so it works as a fixed channel through which a parameter update on source knowledge changes target knowledge.

What would settle it

Fine-tune a model on many source facts (e.g., "X lives in the city of Shanghai" for varied X), record the actual parameter update, and measure the observed change in target logits ("X lives in the country of China") on held-out X. If the observed target-logit change does not track $W \cdot \Delta \text{logit}_{\text{source}}$, with $(W,b)$ fit before training, then the correlation is static rather than causal and the paper's explanation of generalization and hallucination collapses.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the logits of next-token predictions across related knowledge prompts are not independent: for inputs $X$, $F_{\text{Country}}(X) \approx W \cdot F_{\text{City}}(X) + b$, and the same affine map is fit from arbitrary inputs and tested on held-out inputs. The fitted $W$ contains explicit weights for real-world pairs ($\text{Paris}\to\text{France}$), and those weights survive large-scale post-training, so a gradient update to source knowledge propagates to target knowledge. When both the correlation intensity and $W$'s precision are high, this yields compositional generalization; when correlation is high but $W$ is imprecise, the same propagation yields compositional hallucination, such as learning $\text{Indianapolis}\to\text{India}$. The paper further shows that a bag-of-words feedforward model with pretrained vocabulary representations reproduces the generalization behavior, suggesting the correlation is carried by the vocabulary embeddings rather than by the deep transformer stack.

Load-bearing premise

The load-bearing assumption is that the same linear map that fits the model's next-token logits also fits how the model's parameters change during training, so updating source knowledge moves target knowledge along that same map.

Editorial extensions

If this is right

  • Fine-tuning a piece of source knowledge will also change target knowledge along the fixed linear map even after large-scale post-training, which is why some edits ripple successfully.
  • Compositional generalization occurs only when both the logit correlation is high and $W$'s precision is high; with high correlation but low precision the same propagation produces hallucinations such as learning Indianapolis as a city and getting India as the country.
  • Because $(W,b)$ can be fit before editing and stays stable, the weights can serve as a diagnostic for whether a planned knowledge update will generalize or produce a compositional hallucination.
  • The linear correlation can be reproduced by a mean-pooling layer plus a single feedforward network when vocabulary mappings are preserved, so the deep transformer stack is not needed to create it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested consequence is that $W$ could be used as a pre-edit filter for knowledge editing: reject edits whose ground-truth pair has low weight in $W$, since those are predicted to hallucinate rather than generalize.
  • A sharper, untested prediction is that a single fine-tuning step should satisfy $\Delta \text{logit}_{\text{target}} \approx W \, \Delta \text{logit}_{\text{source}}$; measuring that equality directly on held-out inputs would turn the paper's static correlation into a causal claim.
  • The vocabulary-representation account suggests a testable intervention: reshaping or replacing the tied input/output embeddings should shift which knowledge pairs generalize, independently of the depth of the transformer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper reports an empirical study of a linear relationship between the next-token prediction logits of related prompts in language models. The authors fit a linear transformation W,b mapping logits from a source knowledge prompt (e.g., 'X lives in the city of') to a target knowledge prompt (e.g., 'X lives in the country of') over a curated output subdomain, evaluate on held-out inputs, and find high Pearson correlations for semantically related pairs across LLaMA-3 sizes and other models. They further report that the transformation persists after large-scale post-training, that gradient vectors on source and target prompts are correlated, that fine-tuning source knowledge transfers to target knowledge when the fitted W is precise and hallucinates when it is imprecise, and that a simplified bag-of-words model with pretrained embeddings can learn similar composition behavior.

Significance. The paper's core measurement—a held-out linear map between logits of related prompts—is a useful empirical regularity and is evaluated cleanly across several model families and sizes, with code released. If the causal link to learning dynamics were established, the result would be relevant to knowledge editing and hallucination diagnosis. The authors also provide a careful label-wise vs instance-wise distinction and acknowledge limitations in Appendix B. However, the central causal story currently rests on an unsupported assumption about gradient transfer, and the generalization experiments are confounded by token priors; these issues must be resolved before the paper's stronger claims can be accepted.

major comments (3)
  1. [Section 4.1] The claim that the same linear map W fitted on logits also governs gradient propagation is asserted, not derived. Even if z_t = W z_s + b at the current parameters, cross-entropy gradients are ∇θ L = J^T (softmax(z) − onehot), and the chain rule gives ∇θ L_t = J_s^T W^T e_t, which is not generally equal to W J_s^T e_s. Table 4 reports only Pearson correlations between gradient vectors; these can be high due to the shared input token X and do not demonstrate the specific identity ∇θ L_t ≈ W ∇θ L_s. The fine-tuning experiments in Table 5 measure success rates but not whether the transfer follows the fitted W. The authors should either derive the conditions under which the identity holds, directly measure parameter-space updates and compare them to W-transformed source updates, or substantially weaken the causal claim to a conjecture.
  2. [Section 5, Tables 5 and 6] The attribution of generalization and hallucination to W precision is confounded by token prior probabilities. In Table 6, Karnataka→India is counted as generalization despite India not having the top W weight, and the authors explain it by India's high prior; conversely, Helsinki→Finland is counted as hallucination despite a high W weight, attributed to Finland's low prior. Thus the reported success/generalization rates do not isolate the effect of W, and the claim that W precision is 'an underlying factor' is not cleanly supported. A controlled analysis (e.g., stratifying by prior probability of the target token, or reporting generalization conditioned on both W weight and prior) is needed.
  3. [Section 6] The conclusion that vocabulary representations are the key cause of the linear correlation is stronger than the evidence. Replacing the LM internals with a mean-pooling layer and a single feedforward network changes multiple components at once, and training this shallow model on 1,024 paired texts for 1,000 epochs is far from the original training distribution. The generalization performance of this toy model (Table 7) does show that the phenomenon can emerge in a simpler setting, but it does not identify the causal factor within the original LM. The text says 'hypothesize,' yet the abstract and conclusion state that 'LM generalization heavily relies on' vocabulary representations. Please either rephrase as a hypothesis or add experiments that more directly isolate the vocabulary representation's role.
minor comments (7)
  1. [Section 1] The word 'archecture' should be 'architecture'.
  2. [Table 2] The spacing in numbers like '0 .45' appears to be a formatting artifact and should be fixed.
  3. [Table 3] The row 'Indonesia' in the City column is not a city; presumably a typo, and the example is confusing.
  4. [Table 5] The 'Random' baseline is not defined; please specify how it was computed.
  5. [Figure 1] The labels y1, y2, W1, W2 are not defined in the caption; please add a legend or definitions.
  6. [Tables 4 and 5] No confidence intervals or significance tests are reported for the correlation values, making it hard to judge the stability of the differences.
  7. [Appendix D] The relationship between instance-wise correlation (Figure 10) and label-wise correlation (main text) could be clarified, especially since the choice of metric affects the interpretation of the bias term b.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular reduction found; the logit fit is held-out regression, and the gradient-transfer step is unsupported rather than circular.

full rationale

The paper's empirical chain is not circular. In Section 3.2, (W, b) is fitted on half of the collected source/target logit pairs and evaluated with Pearson correlation on the other half, so the reported logit-level correlation is an ordinary fitted-then-tested regression rather than a prediction equal to its own fit. W precision in Section 3.5 is scored against externally enumerated real-world pairs (e.g., City→Country), which are independent of the fine-tuning outcomes in Section 5; the generalization results in Table 5 are measured by actual post-fine-tuning target predictions, not by reusing W to define success. The vocabulary-representation ablation in Section 6 trains a simplified bag-of-words plus feedforward model on paired texts and then tests on new subjects, so it is a sufficiency demonstration rather than a definitional identity. The one potentially load-bearing weak link is Section 4.1, where the paper asserts that the same W fitted to static logits can estimate target gradients from source gradients; a linear relation between current logits does not formally imply the corresponding parameter-space Jacobian or loss-gradient relation. However, this is an evidentiary gap, not a circular reduction — Appendix B explicitly disclaims a formal theory ('We do not provide a formal theory explaining why resilient linear correlations emerge'). The only self-citation (Peng et al. 2024b, Appendix E) supports a subdomain-construction convenience and is not load-bearing for the central claims. Overall, the core correlation and generalization measurements are self-contained against held-out data, so the paper should not be scored as circular; the gradient bridge is a correctness-risk concern, not a definitional circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper's central quantities W,b are fitted to data, and the causal interpretation depends on assumptions about gradient transfer and the faithfulness of a shallow proxy model. No new physical entities are introduced.

free parameters (2)
  • W,b linear map per knowledge pair = fit on 10,000 logit pairs, half train, half test
    The core object is obtained by linear regression on logits; it is not derived from theory. Different pairs give different W,b, and the results depend on this fit.
  • Curated output subdomains = size ~100 per domain, from gpt-4o and search engines
    The evaluation only considers tokens in hand-built domains (e.g., 242 city names, 128 country names). The choice of domains affects the correlation and precision numbers.
assumptions (4)
  • domain assumption Next-token logits equal the dot product of a context representation with tied vocabulary embeddings (Eq. 1).
    Used to argue that linearity in hidden states implies linearity in logits. The paper tests Mistral with untied embeddings in Appendix F, but the main derivation assumes this.
  • domain assumption A single affine map W,b explains the relation for all inputs X in the source domain.
    The entire method fits one W,b per pair; if the true map were nonlinear or input-dependent, the correlation metric would be low. The paper validates this only for selected pairs.
  • domain assumption The logit-level W,b also applies to gradient updates of the parameters.
    Section 4.1 uses W to estimate target gradients from source gradients; this is the bridge to generalization and is only supported by correlation numbers.
  • ad hoc to paper A mean-pooled bag-of-words model with a single feedforward layer is a representative proxy for the LM's composition behavior.
    Section 6 uses this architecture to attribute the effect to vocabulary representations; there is no argument that it preserves the mechanisms of the original Transformer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Linear Correlation in LM's Compositional Generalization and Hallucination." pith.science (2026). https://pith.science/paper/M2GMPVLU

@misc{pith2026250204520,
  author       = {Pith},
  title        = {Pith review of: Linear Correlation in LM's Compositional Generalization and Hallucination},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M2GMPVLU}},
  note         = {Machine review of arXiv:2502.04520}
}
abstract

The generalization of language models (LMs) is undergoing active debates, contrasting their potential for general intelligence with their struggles with basic knowledge composition (e.g., reverse/transition curse). This paper uncovers the phenomenon of linear correlations in LMs during knowledge composition. For explanation, there exists a linear transformation between certain related knowledge that maps the next token prediction logits from one prompt to another, e.g., "X lives in the city of" $\rightarrow$ "X lives in the country of" for every given X. This mirrors the linearity in human knowledge composition, such as Paris $\rightarrow$ France. Our findings indicate that the linear transformation is resilient to large-scale fine-tuning, generalizing updated knowledge when aligned with real-world relationships, but causing hallucinations when it deviates. Empirical results suggest that linear correlation can serve as a potential identifier of LM's generalization. Finally, we show such linear correlations can be learned with a single feedforward network and pre-trained vocabulary representations, indicating LM generalization heavily relies on the latter.

Figures

Figures reproduced from arXiv: 2502.04520 by the authors.

Figure 1
Figure 1. Demonstration of our main discoveries. 1) We can fit a linear transformation between the output of source and target knowledge prompts, which is resilient against fine-tuning. 2) Updating the source knowledge will generalize to the target one via resilient linearity, causing compositional generalization/hallucination. post-training, which shows W fitted before post-training to retain the estimation ability for the L… view at source ↗
Figure 2
Figure 2. Our hypothesis and questions about how LMs compose knowledge by learning (W, b). esis that LMs learn such linear transformation inside the parameters to compose knowledge. The roadmap of our exploration is presented in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: The scaling-up of the precision of W with model size. Relation Pair Logit Correlation Grad. Correlation City→Country 0.89 0.79 CEO→Company 0.55 0.47 Cityen→Cityes 0.70 0.79 Cityen→Cityzh 0.58 0.46 Fruit→Color 0.48 0.46 Food→Taste 0.47 0.47 X+1→X+2 0.93 0.87 X+1→X*2 0.7…
Figure 5
Figure 5. Figure 5: The effect of W weights on generalization. City Reference Generalized Wref Wgen Wmax Shanghai China China 0.50 0.50 0.50 NYC USA USA 0.58 0.58 0.58 Copenhagen Denmark Denmark 0.47 0.47 0.47 Karnataka India India 0.34 0.34 0.56 Indianapolis USA India −0.05 0.15 0.17 Dre…
Figure 7
Figure 7. Figure 7: The linear correlation between NTP logits of llama-3-8b in math operations. +1 +2 +3 +4 -1 -2 -3 -4 *2 *3 *4 *5 /2 /3 /4 /5 Then X has attribute... /5 /4 /3 /2 *5 *4 *3 *2 -4 -3 -2 -1 +4 +3 +2 +1 If X has attribute... Math 0.0 0.2 0.4 0.6 0.8 1.0 A. Results for Main Co…
Figure 8
Figure 8. Figure 8: The linear correlation between NTP logits of llama-3-8b before and after large-scale post-training. birthplace language city country continent company ceo jobmother father gender gender father mother job ceo company continent country city language birthplace (Source) I…
Figure 9
Figure 9. Figure 9: The linear correlation between NTP logits in math operations before and after large-scale post-training. +1 +2 +3 +4 -1 -2 -3 -4 *2 *3 *4 *5 /2 /3 /4 /5 Then X has attribute... /5 /4 /3 /2 *5 *4 *3 *2 -4 -3 -2 -1 +4 +3 +2 +1 If X has attribute... Math 0.0 0.2 0.4 0.6 0…
Figure 10
Figure 10. Figure 10: The instance-wise correlation between NTP logits of llama3-8b (attribute as an example). 17 [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: The attribute correlation between NTP logits of gpt2-medium. E. Subdomain Building Procedure To build the subdomains, we do not simply collect the top predictions from the next token predictions because many predictions are introduced by the frequency and similarity b…
Figure 12
Figure 12. Figure 12: The attribute correlation between NTP logits of llama-3.2-1b. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: The attribute correlation between NTP logits of llama-3.2-3b. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: The attribute correlation between NTP logits of llama-3-8b. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]
Figure 15
Figure 15. Figure 15: The attribute correlation between NTP logits of llama-3-70b. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: The attribute correlation between NTP logits of deepseek-r1-distll-qwen-7B. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: The attribute correlation between NTP logits of mistral-7b-v0.3. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: The linear correlation between NTP logits of llama-3.2-3b. Relation Pair Fruit-Color Food-Taste Gem-Color Name-Country Animal-Size Correlation 48.37 46.95 50.48 78.83 69.43 Relation Pair Object-Genre Object-Heat Object-Size Object-Price Object-Color Correlation 81.92 …
Figure 19
Figure 19. Figure 19: The linear correlation between NTP logits of llama-3.2-3b before and after large-scale post-training. G. More Resilient Correlation in Larger LMs In [PITH_FULL_IMAGE:figures/full_fig_p026_19.png]
Figure 20
Figure 20. Figure 20: The correlation becomes more resilient in larger LMs. birthplace city country continent language company landmark ceo mother father job personality pet sport food drink gender vehicle color music hobby flower vacation vacation flower hobby music color vehicle gender d…
Figure 21
Figure 21. Figure 21: The correlation between logits from mistral-7b-v0.3 before and after post-training. birthplace city country continent language company landmark ceo mother father job personality pet sport food drink gender vehicle color music hobby flower vacation Then X has attribute…
Figure 22
Figure 22. Figure 22: The comparison between Aya and LLaMA in cross-lingual correlation. H. Multilingual LM Figure H demonstrates the cross-lingual correlation of the multilingual LM, aya-expanse-8b, which outperforms LLaMA-3 in multilingual tasks but still lags behind in English (Ust ¨ un…
Figure 24
Figure 24. Figure 24: The std of correlation distribution between logits before and after large-scale post-training. birthplace city country continent language company landmark ceo mother father job personality pet sport food drink gender vehicle color music hobby flower vacation Then X ha…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Relational Linearity is a Predictor of Hallucinations

    cs.CL 2026-01 conditional novelty 6.0 of 10

    Across six hand-picked relations and four instruction-tuned models, a relation's measured linearity (Δcos) positively correlates with hallucination rates on synthetic unknown entities (r≈.78–.82), though p-values most...

Reference graph

Works this paper leans on

42 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    a is b" fail to learn

    Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T., and Evans, O. The reversal curse: Llms trained on "a is b" fail to learn "b is a". In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=GPKTIktA0k

    work page 2024

  3. [3]

    Hallucination detection: Robustly discerning reliable answers in large language models, 2024

    Chen, Y., Fu, Q., Yuan, Y., Wen, Z., Fan, G., Liu, D., Zhang, D., Li, Z., and Xiao, Y. Hallucination detection: Robustly discerning reliable answers in large language models, 2024. URL https://arxiv.org/abs/2407.04121

    arXiv 2024

  4. [4]

    Evaluating the ripple effects of knowledge editing in language models

    Cohen, R., Biran, E., Yoran, O., Globerson, A., and Geva, M. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12: 0 283--298, 2024

    work page 2024

  5. [5]

    Knowledge neurons in pretrained transformers

    Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., and Wei, F. Knowledge neurons in pretrained transformers. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022 , pp.\ 8493--8502. Association for Com...

    doi:10.18653/v1/2022.acl-long.581 2022

  6. [6]

    Stolen Probability: A Structural Weakness of Neural Language Models

    Demeter, D., Kimmel, G., and Downey, D. Stolen probability: A structural weakness of neural language models. arXiv preprint arXiv:2005.02433, 2020

    work page Pith review arXiv 2005

  7. [7]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  8. [8]

    Detecting hallucinations in large language models using semantic entropy

    Farquhar, S., Kossen, J., Kuhn, L., and Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature, 630 0 (8017): 0 625--630, 2024

    2024

Show all 42 references
  1. [9]

    C., Neyshabur, B., and Sedghi, H

    Garg, S., Balakrishnan, S., Lipton, Z. C., Neyshabur, B., and Sedghi, H. Leveraging unlabeled data to predict out-of-distribution performance, 2022. URL https://arxiv.org/abs/2201.04234

    2022 arXiv

  2. [10]

    Transformer feed-forward layers are key-value memories

    Geva, M., Schuster, R., Berant, J., and Levy, O. Transformer feed-forward layers are key-value memories. In Moens, M., Huang, X., Specia, L., and Yih, S. W. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Even...

    2021 doi

  3. [11]

    R., and Goldberg, Y

    Geva, M., Caciularu, A., Wang, K. R., and Goldberg, Y. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Langu...

    2022 doi

  4. [12]

    P., Bhagia, A., Kinney, R., Tafjord, O., Jha, A

    Groeneveld, D., Beltagy, I., Walsh, E. P., Bhagia, A., Kinney, R., Tafjord, O., Jha, A. H., Ivison, H., Magnusson, I., Wang, Y., Arora, S., Atkinson, D., Authur, R., Chandu, K. R., Cohan, A., Dumas, J., Elazar, Y., Gu, Y., Hessel, J., Khot, T., Merrill, W., Morrison, J., Muenn...

    2024

  5. [13]

    K., Zhao, W., Li, X

    Gupta, A., Mondal, D., Sheshadri, A. K., Zhao, W., Li, X. L., Wiegreffe, S., and Tandon, N. Editing common sense in transformers. arXiv preprint arXiv:2305.14956, 2023

    2023 arXiv

  6. [14]

    S., Haklay, T., Meng, K., Wattenberg, M., Andreas, J., Belinkov, Y., and Bau, D

    Hernandez, E., Sharma, A. S., Haklay, T., Meng, K., Wattenberg, M., Andreas, J., Belinkov, Y., and Bau, D. Linearity of relation decoding in transformer language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, ...

    2024

  7. [15]

    Detecting edit failures in large language models: An improved specificity benchmark

    Hoelscher-Obermaier, J., Persson, J., Kran, E., Konstas, I., and Barez, F. Detecting edit failures in large language models: An improved specificity benchmark. arXiv preprint arXiv:2305.17553, 2023

    2023 arXiv

  8. [16]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., and Liu, T. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, November 2024. ISSN 1...

    2024 doi

  9. [17]

    K., Lincoln, P., Bastian, N

    Jha, S., Jha, S. K., Lincoln, P., Bastian, N. D., Velasquez, A., and Neema, S. Dehallucinating large language models using formal methods guided iterative prompting. In 2023 IEEE International Conference on Assured Autonomy (ICAA), pp.\ 149--152. IEEE, 2023

    2023

  10. [18]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  11. [19]

    On large language models' hallucination with regard to known facts, 2024

    Jiang, C., Qi, B., Hong, X., Fu, D., Cheng, Y., Meng, F., Yu, M., Zhou, B., and Zhou, J. On large language models' hallucination with regard to known facts, 2024. URL https://arxiv.org/abs/2403.20009

    2024 arXiv

  12. [20]

    What do learning dynamics reveal about generalization in llm reasoning?, 2024

    Kang, K., Setlur, A., Ghosh, D., Steinhardt, J., Tomlin, C., Levine, S., and Kumar, A. What do learning dynamics reveal about generalization in llm reasoning?, 2024. URL https://arxiv.org/abs/2411.07681

    2024 arXiv

  13. [21]

    Lampinen, A. K. and McClelland, J. L. Transforming task representations to perform novel tasks. Proc. Natl. Acad. Sci. USA , 117 0 (52): 0 32970--32981, 2020. doi:10.1073/PNAS.2008852117. URL https://doi.org/10.1073/pnas.2008852117

    2020 doi

  14. [22]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  15. [23]

    Locating and editing factual associations in GPT

    Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in GPT . In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information...

    2022

  16. [24]

    Locating and editing factual associations in gpt

    Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35: 0 17359--17372, 2022 b

    2022

  17. [25]

    Fine-grained hallucination detection and editing for language models, 2024

    Mishra, A., Asai, A., Balachandran, V., Wang, Y., Neubig, G., Tsvetkov, Y., and Hajishirzi, H. Fine-grained hallucination detection and editing for language models, 2024. URL https://arxiv.org/abs/2401.06855

    2024 arXiv

  18. [26]

    J., Padmanabhan, S., Durrett, G., and Choi, E

    Onoe, Y., Zhang, M. J., Padmanabhan, S., Durrett, G., and Choi, E. Can lms learn new entities from descriptions? challenges in propagating injected knowledge. arXiv preprint arXiv:2305.01651, 2023

    2023 arXiv

  19. [27]

    Peng, B., Narayanan, S., and Papadimitriou, C. H. On limitations of the transformer architecture. CoRR, abs/2402.08164, 2024 a . doi:10.48550/ARXIV.2402.08164. URL https://doi.org/10.48550/arXiv.2402.08164

    2024 doi arXiv

  20. [28]

    Correlation and navigation in the vocabulary key representation space of language models

    Peng, L., An, C., and Shang, J. Correlation and navigation in the vocabulary key representation space of language models. CoRR, abs/2410.02284, 2024 b . doi:10.48550/ARXIV.2410.02284. URL https://doi.org/10.48550/arXiv.2410.02284

    2024 doi arXiv

  21. [29]

    and Wolf, L

    Press, O. and Wolf, L. Using the output embedding to improve language models. In Lapata, M., Blunsom, P., and Koller, A. (eds.), Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, ...

    2017 doi

  22. [30]

    Detecting hallucinations in large language model generation: A token probability approach, 2024

    Quevedo, E., Yero, J., Koerner, R., Rivas, P., and Cerny, T. Detecting hallucinations in large language model generation: A token probability approach, 2024. URL https://arxiv.org/abs/2405.19648

    2024 arXiv

  23. [31]

    Unsupervised real-time hallucination detection based on the internal states of large language models, 2024

    Su, W., Wang, C., Ai, Q., HU, Y., Wu, Z., Zhou, Y., and Liu, Y. Unsupervised real-time hallucination detection based on the internal states of large language models, 2024. URL https://arxiv.org/abs/2403.06448

    2024 arXiv

  24. [32]

    S., Love, J., et al

    Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi \`e re, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  25. [33]

    Limits of transformer language models on learning to compose algorithms

    Thomm, J., Camposampiero, G., Terzic, A., Hersche, M., Sch \"o lkopf, B., and Rahimi, A. Limits of transformer language models on learning to compose algorithms. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  26. [34]

    L., Sharma, A

    Todd, E., Li, M. L., Sharma, A. S., Mueller, A., Wallace, B. C., and Bau, D. Function vectors in large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openrevie...

    2024

  27. [35]

    X., Ko, W., D'souza, D., Onilude, G., Bhandari, N., Singh, S., Ooi, H., Kayid, A., Vargus, F., Blunsom, P., Longpre, S., Muennighoff, N., Fadaee, M., Kreutzer, J., and Hooker, S

    \" U st \" u n, A., Aryabumi, V., Yong, Z. X., Ko, W., D'souza, D., Onilude, G., Bhandari, N., Singh, S., Ooi, H., Kayid, A., Vargus, F., Blunsom, P., Longpre, S., Muennighoff, N., Fadaee, M., Kreutzer, J., and Hooker, S. Aya model: An instruction finetuned open-access multili...

    2024

  28. [36]

    H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., and Fedus, W

    Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., and Fedus, W. Emergent abilities of large language models. Trans. Mach. Learn. Res., 2022, 2022. UR...

    2022

  29. [37]

    Predicting out-of-distribution error with the projection norm, 2022

    Yu, Y., Yang, Z., Wei, A., Ma, Y., and Steinhardt, J. Predicting out-of-distribution error with the projection norm, 2022. URL https://arxiv.org/abs/2202.05834

    2022 arXiv

  30. [38]

    T., Bi, W., Shi, F., and Shi, S

    Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., Wang, L., Luu, A. T., Bi, W., Shi, F., and Shi, S. Siren's song in the ai ocean: A survey on hallucination in large language models, 2023. URL https://arxiv.org/abs/2309.01219

    2023 arXiv

  31. [39]

    Unveiling linguistic regions in large language models

    Zhang, Z., Zhao, J., Zhang, Q., Gui, T., and Huang, X. Unveiling linguistic regions in large language models. In Ku, L., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2...

    2024 doi

  32. [40]

    D., Potts, C., and Chen, D

    Zhong, Z., Wu, Z., Manning, C. D., Potts, C., and Chen, D. Mquake: Assessing knowledge editing in language models via multi-hop questions. arXiv preprint arXiv:2305.14795, 2023

    2023 arXiv

  33. [41]

    I., Jiao, J., Tian, Y., and Russell, S

    Zhu, H., Huang, B., Zhang, S., Jordan, M. I., Jiao, J., Tian, Y., and Russell, S. Towards a theoretical understanding of the 'reversal curse' via training dynamics. CoRR, abs/2405.04669, 2024. doi:10.48550/ARXIV.2405.04669. URL https://doi.org/10.48550/arXiv.2405.04669

    2024 doi arXiv

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.