Pith. sign in

REVIEW 5 major objections 6 minor 43 references

One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a new uncertainty-aware attribution method, MATRICE, identifies feed-forward neurons that store a fact in a language-independent way, so that manipulating them edits the fact across all tested languages.

desk verdict Useful benchmark and a plausible uncertainty-weighted localization method, but the storage claim rests on a self-referential test and weak locality numbers. read the letter →

arxiv 2411.17401 v1 pith:UTJEC5A6 submitted 2024-11-26 cs.CL

Pengfei Cao , Yuheng Chen , Zhuoran Jin , Yubo Chen , Kang Liu , Jun Zhao This is my paper · ORCID
classification cs.CL
keywords largelanguagemodelsknowledgeneuronslanguage-agnosticlocalizationintegratedgradientsmultilingualcross-lingualeditingRML-LAMA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that factual knowledge in multilingual large language models is stored, at least in part, in specific neurons whose association with a fact does not depend on the language of the query. It proposes MATRICE, an attribution method that scores every feed-forward neuron by its integrated-gradient contribution to the correct answer, then averages those scores over several paraphrased queries in each language while penalizing high variance across queries and languages. On a new seven-language benchmark, RML-LAMA, the paper shows that suppressing or amplifying the selected neurons changes the expression of the corresponding fact in all tested languages. If true, this gives a practical handle: a single edit can update, strengthen, or inject a fact across languages, including languages the model handles poorly.

What carries the argument

The load-bearing object is the language-agnostic knowledge neuron, defined as an FFN neuron whose attribution to a fact's correct answer remains high and low-variance when the fact is queried through many paraphrases in many languages. The method that carries the argument is MATRICE: Sequential Adapted Integrated Gradients computes per-query attribution by integrating gradients along a path from a baseline sentence (each token replaced by $\langle \text{PAD} \rangle$) to the original query; Uncertainty Quantification then forms a language-agnostic score $S_i^l = \beta_1 E[\bar{s}_i^l] - \beta_2 \sqrt{\mathrm{Var}[\bar{s}_i^l]}$, and a dynamic per-fact threshold picks the final neuron set. This design is what converts a single-query, two-language intersection into a seven-language, paraphrase-robust localization.

What would settle it

Localize neurons for a set of facts with MATRICE, then suppress them and check whether the probability of the fact drops only for those facts and in every language, while unrelated facts and randomly selected neurons matched for activation produce no comparable effect; a failure on held-out paraphrases or languages would show the localization is query-sensitive rather than storage-based.

Watch

Extended reading notes

Core claim

The central claim is that MATRICE accurately localizes language-agnostic knowledge neurons: feed-forward neurons whose high attribution to a fact is stable across semantically equivalent queries and across languages. The method computes sequential adapted integrated gradients for each query, replaces tokens one at a time with a padding token to build low-information baseline vectors, then combines mean and variance of attribution scores first across paraphrases and then across languages, and finally selects neurons above a dynamic threshold. Manipulation experiments show that setting these neurons to zero lowers the probability of the correct answer in all seven languages, while doubling their activations raises it; the same neurons also serve as targets for cross-lingual erasure and update editing, for improving accuracy on low-resource languages, and for fine-tuning that injects new knowledge with less forgetting. The paper concludes that these neurons store facts in a language-independent form and that localizing them well is the key to multilingual knowledge control.

Load-bearing premise

The method assumes that a neuron's integrated-gradient attribution to the probability of the correct answer identifies where the fact is actually stored, rather than merely identifying high-leverage switches that can change that probability.

Editorial extensions

If this is right

  • Editing the localized neurons once in English changes the corresponding fact in all other tested languages, with reliability gains over the earlier intersection-based method of roughly 13 to 22 points depending on the model.
  • Amplifying these neurons raises accuracy on previously wrong queries in low-resource languages, and the gain grows as the underlying model becomes larger and more capable.
  • Fine-tuning only the localized neurons lets a model acquire new facts while retaining old knowledge far better than full fine-tuning, with old-knowledge accuracy often staying near 90 percent.
  • Most language-agnostic knowledge neurons concentrate in the last few layers, and this pattern holds across both auto-encoding and auto-regressive transformer architectures.
  • Removing either uncertainty component degrades localization, and the query-level uncertainty component matters more than the language-level component.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same uncertainty-weighted attribution pipeline could apply to facts expressed without cloze templates by defining the target probability over generated tokens, but the paper does not test that setting.
  • Editorial inference: if attribution identifies high-leverage control points rather than true storage sites, the editing, enhancement, and injection applications could still work, but the storage interpretation would need independent causal evidence such as causal tracing.
  • Editorial inference: applying the method to a model trained almost entirely on one language and checking whether it still finds language-agnostic neurons would help separate shared multilingual representations from simple training-corpus overlap.
  • Editorial inference: the benchmark and method could be extended to non-factual knowledge such as commonsense or procedural knowledge, with the caveat that the attribution target would need to be redefined.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes MATRICE, a method for localizing language-agnostic knowledge neurons in multilingual LLMs. It introduces RML-LAMA, a multilingual cloze benchmark with 7,849 facts across seven languages, computes per-neuron attribution scores via integrated gradients with token-replacement baselines, aggregates scores over paraphrased queries and languages using mean-variance uncertainty penalties, and selects neurons with a per-fact dynamic threshold. The paper evaluates the method by suppressing or enhancing selected neurons, by cross-lingual knowledge editing, by improving low-resource-language predictions, and by targeted fine-tuning for knowledge injection, reporting gains over AMIG and random baselines.

Significance. If the localization claim held, the paper would be a substantial methodological contribution: a reproducible multilingual benchmark, an uncertainty-aware attribution procedure, and evidence that selected neurons support cross-lingual editing and enhancement. The ablation in Table II shows that both uncertainty components help under the paper's own metric, and the breadth across four model families is a strength. However, the central claim that MATRICE identifies where a fact is stored, rather than merely identifying high-leverage control units of the prediction function, is validated mainly by manipulating the same probability used to construct the attribution scores. The independent signal available in the locality metrics is weak, with only a 2-3 point gap over the random baseline, which supports the concern that the selected neurons may be generic influential units rather than fact-specific storage locations. The paper is therefore a useful engineering contribution, but the scientific claim of fact-specific storage needs re-grounding.

major comments (5)
  1. [§IV-D, Eq. (14)] The main validation is circular. Attribution scores in Eqs. (1)-(5) are path integrals of the gradient of P(y*|q) with respect to neuron activations, and Eq. (14) measures the change in exactly this P(y*|q) after zeroing or doubling the selected neurons. Any neuron with high integrated gradient is, by construction, one whose perturbation should move P(y*|q) in the direction of the gradient, so the suppression and enhancement results in Figure 5 are partly entailed by the selection objective. Those experiments demonstrate that MATRICE finds influential neurons, not that these neurons are the storage location of the fact. An independent test, such as checking whether the same neurons are required for an unrelated fact with the same answer or whether a different causal intervention removes the fact-specific effect, is needed before the storage interpretation can be accepted.
  2. [Tables III and IV, Eq. (18)] The locality numbers weaken the fact-specific interpretation. Averaged across languages and models, MATRICE locality is roughly 7.2-10.1 for erasure and 8.5-11.9 for update, while the Random baseline is roughly 5.1-7.1 and 5.6-7.8; the gap is only 2-3 points. This means editing MATRICE-selected neurons disrupts unrelated factual predictions almost as much as editing random neurons. If the selected units were fact-specific storage, one would expect substantially higher locality relative to random. The observed pattern is consistent with generic high-leverage units. The paper should report locality on a per-fact matched basis and include a baseline of high-attribution neurons for unrelated facts.
  3. [§IV-F, Eqs. (16)-(17)] The reliability and generality metrics remain self-referential for the storage question. Both metrics measure whether the edited model outputs the edited answer for queries that express the edited fact in the target language. They show that the edit transfers across languages, but they do not distinguish editing a fact's storage location from editing a general-purpose control path that affects that fact's expression. A discriminating test would edit neurons selected for fact A and measure effects on a held-out fact B that shares the subject or relation, comparing against neurons selected for fact B.
  4. [§IV-D and Tables III/IV] The comparison with AMIG is confounded by the number of edited neurons. The paper acknowledges in §IV-D that AMIG's intersection set shrinks as the number of languages grows, which explains its weaker manipulation effects, but the editing tables do not report neuron counts per method. If MATRICE edits a larger set, higher reliability and generality with comparable or slightly worse locality would be expected even without better localization. Please report the distribution of selected-neuron counts for each method and include an AMIG variant matched to MATRICE's neuron count.
  5. [§IV-G and §IV-H] The knowledge enhancement and injection experiments use queries the model initially answers incorrectly (Qerror and Qnew). For such queries, the correct-answer probability P(y*|q) in Eq. (1) is not the model's expressed prediction, and the meaning of attributing that probability to 'knowledge neurons' is unclear; if the model has not stored the fact, there may be no fact-specific storage to localize. The paper should clarify how MATRICE behaves when P(y*|q) is low or near zero, and should evaluate new-knowledge injection using facts whose neurons were selected on the pre-fine-tuned model's correct predictions rather than on the same initially unknown facts.
minor comments (6)
  1. [§III-A, Eqs. (4)-(5)] The notation is inconsistent in Eqs. (4)-(5) (w'_i vs. w'_j, Attri(nl_j) vs. Attr(nl_j)), and the denominator of Eq. (5) is undefined if the sum of attribution scores over a layer is zero; since integrated gradients can be negative, this normalization needs a justification or a numerical safeguard.
  2. [§IV-A] The construction of RML-LAMA relies on Claude 3 Opus with automatic regeneration, but no inter-annotator agreement, human validation statistics, or public release link are provided; please add a data-quality section with sample counts, agreement rates, and a URL.
  3. [Algorithm 1] The hyperparameters alpha1, alpha2, beta1, beta2, tau, M, lambda1, and lambda2 are listed but no values or sensitivity analysis are given; at a minimum, report the chosen values and vary tau and M over a small grid to show stability.
  4. [Figures 5 and 6] The figures report a single run with no error bars or significance tests; given that several comparisons are small in magnitude, run-to-run variance should be reported.
  5. [§IV-D, Fig. 5] The explanation that Random's nonzero effect is partly due to selecting neurons also chosen by the other methods applies equally to MATRICE; a more informative control would be random neurons drawn from the same layer distribution as MATRICE.
  6. [§III-A, Eq. (3)] Using the EOS token as an information-free baseline for auto-regressive models is questionable because EOS has a trained positional and functional role; this choice should be justified empirically with a comparison to alternative baselines.

Circularity Check

2 steps flagged · score 5.0 of 10

Central localization validation is self-referential: attribution scores and the manipulation metric both use the same correct-answer probability P(y*|q), and the only partly independent signal, locality, shows a small gap over random.

  1. self definitional [Section III-A Eqs. (1)-(5) and Section IV-D Eq. (14), Figure 5]
    "For the neuron n_i^l, its attribution score is calculated as follows: Attr(n_i^l) = Δw_i^l ∫_0^1 ∂P(w'_i^l + αΔw_i^l)/∂w̄_i^l dα ... We measure performance using the probability change rate (ΔP) of correct answers before and after manipulation as follows: ΔP = (Pafter − Pbefore)/Pbefore"

    The neuron scores are defined as integrated gradients of P(y*|q) with respect to neuron activations, so high-scoring neurons are, by construction, the neurons whose path integral contributes to P(y*|q). The validation then suppresses or enhances those selected neurons and measures ΔP of the same P(y*|q) from Eq. (14). A significant ΔP therefore confirms that the selected neurons influence the exact probability used to select them; it does not test whether the fact is stored there. Generic high-leverage control points of P(y*|q) would behave the same way, so the experiment cannot distinguish 'storage location' from 'influential unit'.

  2. self definitional [Section IV-G, Knowledge Enhancement, Figure 6]
    "we filter the queries from the new RML-LAMA dataset and obtain some queries that LLMs originally answer incorrectly, denoted as Qerror. Thus, the original accuracy of LLMs on the dataset Qerror is exactly 0. Then, we perform the enhancement operation on language-agnostic knowledge neurons corresponding to these queries, and calculate the increase in accuracy of modified LLMs on the Qerror."

    The enhanced neurons are localized using MATRICE attribution scores for the same Qerror queries, i.e., using gradients of the correct-answer probability P(y*|q) for those exact queries. The success metric is the accuracy increase on the same Qerror after doubling those neurons' activations. Because positive attribution means that raising the neuron's value increases P(y*|q) along the integrated path, amplifying these neurons is expected to raise P(y*|q) for those same queries. The reported accuracy gain is therefore partly entailed by the selection objective rather than being independent evidence that the neurons are the storage locus of the fact.

full rationale

MATRICE's attribution scores are integrals of ∂P(y*|q)/∂neuron (Eqs. 1-5), and the main manipulation experiment (Sec. IV-D) evaluates ΔP of the same P(y*|q) (Eq. 14), so the principal evidence for 'accurate localization' reduces to showing that neurons selected for influencing P(y*|q) do influence P(y*|q). The knowledge-enhancement experiment (Sec. IV-G) has the same structure: it evaluates on the same Qerror queries used for localization. The locality metric is the only partly independent signal, but the gap over Random is small (e.g., erasure locality 8.46 vs 5.94 and update locality 9.73 vs 6.39 on mGPT), which is consistent with generic high-leverage units rather than fact-specific storage. The knowledge-injection experiment (Sec. IV-H) is not circular in the same way and provides some independent evidence that the localized units are editable, though it does not by itself establish the storage claim. Self-citations to prior knowledge-neuron work [13,15] are present, but no load-bearing uniqueness theorem is imported from those citations; the circularity here is in the validation logic, not the citation chain.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the existing knowledge-neuron ontology, five unreported hyperparameters, and the assumption that LLM-generated multilingual paraphrases are semantically aligned. No new physical or formal entities are invented, but the paper's localization requires accepting these premises.

free parameters (5)
  • alpha_1, alpha_2 = not reported
    Coefficients combining mean and variance of attribution scores across queries (Eq. 8). Values are hand-chosen hyperparameters; no tuning or sensitivity analysis is reported.
  • beta_1, beta_2 = not reported
    Coefficients combining mean and variance of normalized attribution scores across languages (Eq. 11). Values are not reported.
  • tau = not reported
    Scaling factor for the dynamic neuron-selection threshold (Eq. 12). No value or sensitivity analysis is provided.
  • M = not reported
    Number of Riemann approximation steps in the integrated gradients computation (Eq. 4). Not specified in the experiments.
  • lambda_1, lambda_2 = not reported
    Coefficients in the FFN weight update for knowledge editing (Eq. 15). Not reported or analyzed.
assumptions (5)
  • domain assumption Factual knowledge is stored in individual FFN neurons ('knowledge neurons').
    Inherited from Geva et al. (2021) and Dai et al. (2022); the paper does not test this storage hypothesis, it assumes it and localizes neurons within it.
  • domain assumption Integrated gradients over neuron activations reveal the neurons responsible for expressing a fact.
    The attribution score is used as the ground-truth relevance signal (Section III-A); the manipulation test validates influence on P(y*|q), which is the same quantity used to define relevance.
  • ad hoc to paper Replacing each token with mask/eos yields an information-free baseline for integrated gradients.
    The baseline definition in Eq. (3) is motivated by the claim that PAD tokens 'do not have a specific meaning', but this is not empirically verified in the paper.
  • domain assumption Claude-generated rephrased queries are semantically equivalent across languages.
    The RML-LAMA benchmark construction (Section IV-A) relies on the LLM-generated queries being parallel and correct; no human evaluation or inter-annotator agreement is reported.
  • ad hoc to paper A single dynamic threshold per fact (Eq. 12) selects comparable neuron sets.
    The threshold form is chosen for convenience; no justification or robustness check for tau is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models." pith.science (2026). https://pith.science/paper/UTJEC5A6

@misc{pith2026241117401,
  author       = {Pith},
  title        = {Pith review of: One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UTJEC5A6}},
  note         = {Machine review of arXiv:2411.17401}
}
read the original abstract

Large language models (LLMs) have learned vast amounts of factual knowledge through self-supervised pre-training on large-scale corpora. Meanwhile, LLMs have also demonstrated excellent multilingual capabilities, which can express the learned knowledge in multiple languages. However, the knowledge storage mechanism in LLMs still remains mysterious. Some researchers attempt to demystify the factual knowledge in LLMs from the perspective of knowledge neurons, and subsequently discover language-agnostic knowledge neurons that store factual knowledge in a form that transcends language barriers. However, the preliminary finding suffers from two limitations: 1) High Uncertainty in Localization Results. Existing study only uses a prompt-based probe to localize knowledge neurons for each fact, while LLMs cannot provide consistent answers for semantically equivalent queries. Thus, it leads to inaccurate localization results with high uncertainty. 2) Lack of Analysis in More Languages. The study only analyzes language-agnostic knowledge neurons on English and Chinese data, without exploring more language families and languages. Naturally, it limits the generalizability of the findings. To address aforementioned problems, we first construct a new benchmark called Rephrased Multilingual LAMA (RML-LAMA), which contains high-quality cloze-style multilingual parallel queries for each fact. Then, we propose a novel method named Multilingual Integrated Gradients with Uncertainty Estimation (MATRICE), which quantifies the uncertainty across queries and languages during knowledge localization. Extensive experiments show that our method can accurately localize language-agnostic knowledge neurons. We also further investigate the role of language-agnostic knowledge neurons in cross-lingual knowledge editing, knowledge enhancement and new knowledge injection.

Figures

Figures reproduced from arXiv: 2411.17401 by the authors.

Figure 1
Figure 1. An example of LLMs answering multilingual queries correctly. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The process of obtaining language-agnostic knowledge neurons [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The architecture of our proposed multilingual integrated gradients with uncertainty estimation ( [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The distribution of language-agnostic knowledge neurons in four multilingual LLMs, including mGPT, LLaMA2, LLaMA3 and mBERT. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Results of suppressing or enhancing language-agnostic knowledge neurons experiment. The “Probability Change Rate” denotes the probability [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Results of knowledge enhancement experiment in low resource languages (i.e., Finnish (fi), Hungarian (hu), Korean (ko) and Japanese (ja)). We [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Two examples to qualitatively illustrate the effectiveness of [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 28 canonical work pages

  1. [1]

    What is a knowledge representa- tion?

    R. Davis, H. Shrobe, and P. Szolovits, “What is a knowledge representa- tion?” AI magazine, vol. 14, no. 1, pp. 17–17, 1993

    work page 1993

  2. [2]

    Some expert systems need common sense,

    J. McCarthy, “Some expert systems need common sense,” Annals of the New York Academy of Sciences , vol. 426, no. 1, pp. 129–137, 1984

    work page 1984

  3. [3]

    How knowledge works,

    J. Hyman, “How knowledge works,” The philosophical quarterly, vol. 49, no. 197, pp. 433–451, 1999

    work page 1999

  4. [4]

    The life cycle of knowledge in big language models: A survey,

    B. Cao, H. Lin, X. Han, and L. Sun, “The life cycle of knowledge in big language models: A survey,” Machine Intelligence Research, vol. 21, no. 2, pp. 217–238, 2024

    work page 2024

  5. [5]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    arXiv 2023

  6. [6]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    arXiv 2023

  7. [7]

    Language models as knowledge bases?

    F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y . Wu, and A. Miller, “Language models as knowledge bases?” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , 2019, pp. 2463–2473

    work page 2019

  8. [8]

    How multilingual is multilingual bert?

    T. Pires, E. Schlinger, and D. Garrette, “How multilingual is multilingual bert?” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4996–5001

    work page 2019

Show all 43 references
  1. [9]

    mgpt: Few-shot learners go multilingual,

    O. Shliazhko, A. Fenogenova, M. Tikhonova, V . Mikhailov, A. Kozlova, and T. Shavrina, “mgpt: Few-shot learners go multilingual,”arXiv preprint arXiv:2204.07580, 2022

    2022 arXiv

  2. [10]

    Multilingual large language model: A survey of resources, taxonomy and frontiers,

    L. Qin, Q. Chen, Y . Zhou, Z. Chen, Y . Li, L. Liao, M. Li, W. Che, and P. S. Yu, “Multilingual large language model: A survey of resources, taxonomy and frontiers,” arXiv preprint arXiv:2404.04925 , 2024

    2024 arXiv

  3. [11]

    Measuring and improving consistency in pretrained language models,

    Y . Elazar, N. Kassner, S. Ravfogel, A. Ravichander, E. Hovy, H. Schütze, and Y . Goldberg, “Measuring and improving consistency in pretrained language models,” Transactions of the Association for Computational Linguistics, vol. 9, pp. 1012–1031, 2021

    2021

  4. [12]

    Kola: Carefully benchmarking world knowledge of large language models,

    J. Yu, X. Wang, S. Tu, S. Cao, D. Zhang-Li, X. Lv, H. Peng, Z. Yao, X. Zhang, H. Li et al., “Kola: Carefully benchmarking world knowledge of large language models,” arXiv preprint arXiv:2306.09296 , 2023

    2023 arXiv

  5. [13]

    Knowledge neurons in pretrained transformers,

    D. Dai, L. Dong, Y . Hao, Z. Sui, B. Chang, and F. Wei, “Knowledge neurons in pretrained transformers,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 8493–8502

    2022

  6. [14]

    Locating and editing factual associations in gpt,

    K. Meng, D. Bau, A. Andonian, and Y . Belinkov, “Locating and editing factual associations in gpt,” Advances in Neural Information Processing Systems, vol. 35, pp. 17 359–17 372, 2022

    2022

  7. [15]

    Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons,

    Y . Chen, P. Cao, Y . Chen, K. Liu, and J. Zhao, “Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 16, 2024, ...

    2024

  8. [16]

    Promptbench: Towards evaluating the robustness of large language models on adversarial prompts,

    K. Zhu, J. Wang, J. Zhou, Z. Wang, H. Chen, Y . Wang, L. Yang, W. Ye, Y . Zhang, N. Z. Gong et al. , “Promptbench: Towards evaluating the robustness of large language models on adversarial prompts,” arXiv preprint arXiv:2306.04528, 2023

    2023 arXiv

  9. [17]

    Language-specific neurons: The key to multilingual capabilities in large language models,

    T. Tang, W. Luo, H. Huang, D. Zhang, X. Wang, X. Zhao, F. Wei, and J.-R. Wen, “Language-specific neurons: The key to multilingual capabilities in large language models,” arXiv preprint arXiv:2402.16438 , 2024

    2024 arXiv

  10. [18]

    Test-time augmentation for factual probing,

    G. Kamoda, B. Heinzerling, K. Sakaguchi, and K. Inui, “Test-time augmentation for factual probing,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , 2023, pp. 3650–3661

    2023

  11. [19]

    Readprompt: A readable prompting method for reliable knowledge probing,

    Z. Wang, L. Ye, H. Wang, W. C. Kwan, D. Ho, and K.-F. Wong, “Readprompt: A readable prompting method for reliable knowledge probing,” in Findings of the Association for Computational Linguistics: EMNLP 2023, 2023, pp. 7468–7479

    2023

  12. [20]

    X- factr: Multilingual factual knowledge retrieval from pretrained language models,

    Z. Jiang, A. Anastasopoulos, J. Araki, H. Ding, and G. Neubig, “X- factr: Multilingual factual knowledge retrieval from pretrained language models,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 5943–5959

    2020

  13. [21]

    Multilingual lama: Investigating knowledge in multilingual pretrained language models,

    N. Kassner, P. Dufter, and H. Schütze, “Multilingual lama: Investigating knowledge in multilingual pretrained language models,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , 2021, pp. 3250–3258

    2021

  14. [22]

    T-rex: A large scale alignment of natural language with 14 knowledge base triples,

    H. Elsahar, P. V ougiouklis, A. Remaci, C. Gravier, J. Hare, F. Laforest, and E. Simperl, “T-rex: A large scale alignment of natural language with 14 knowledge base triples,” in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 201...

    2018

  15. [23]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  16. [24]

    Zhujiu- knowledge: A fairer platform for evaluating multiple knowledge types in large language models,

    P. Du, S. Liang, B. Zhang, P. Cao, Y . Chen, K. Liu, and J. Zhao, “Zhujiu- knowledge: A fairer platform for evaluating multiple knowledge types in large language models,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational L...

    2024

  17. [25]

    Gpt understands, too,

    X. Liu, Y . Zheng, Z. Du, M. Ding, Y . Qian, Z. Yang, and J. Tang, “Gpt understands, too,” AI Open, 2023

    2023

  18. [26]

    Factual probing is [mask]: Learning vs. learning to recall,

    Z. Zhong, D. Friedman, and D. Chen, “Factual probing is [mask]: Learning vs. learning to recall,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2021, pp. 5017–5033

    2021

  19. [27]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) ,...

    2021

  20. [28]

    Prompt waywardness: The curious case of discretized interpretation of continuous prompts,

    D. Khashabi, X. Lyu, S. Min, L. Qin, K. Richardson, S. Welleck, H. Ha- jishirzi, T. Khot, A. Sabharwal, S. Singh et al., “Prompt waywardness: The curious case of discretized interpretation of continuous prompts,” in Proceedings of the 2022 Conference of the North American Chap...

    2022

  21. [29]

    Knowledge mechanisms in large language models: A survey and perspective,

    M. Wang, Y . Yao, Z. Xu, S. Qiao, S. Deng, P. Wang, X. Chen, J.-C. Gu, Y . Jiang, P. Xie et al., “Knowledge mechanisms in large language models: A survey and perspective,” arXiv preprint arXiv:2407.15017 , 2024

    2024 arXiv

  22. [30]

    Transformer feed-forward layers are key-value memories,

    M. Geva, R. Schuster, J. Berant, and O. Levy, “Transformer feed-forward layers are key-value memories,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 5484– 5495

    2021

  23. [31]

    Mass-editing memory in a transformer,

    K. Meng, A. S. Sharma, A. J. Andonian, Y . Belinkov, and D. Bau, “Mass-editing memory in a transformer,” in The Eleventh International Conference on Learning Representations , 2022

    2022

  24. [32]

    The da vinci code of large pre-trained language models: Deciphering degenerate knowledge neurons,

    Y . Chen, P. Cao, Y . Chen, Y . Wang, S. Liu, K. Liu, and J. Zhao, “The da vinci code of large pre-trained language models: Deciphering degenerate knowledge neurons,” arXiv preprint arXiv:2402.13731 , 2024

    2024 arXiv

  25. [33]

    Measures of degeneracy and redundancy in biological networks,

    G. Tononi, O. Sporns, and G. M. Edelman, “Measures of degeneracy and redundancy in biological networks,” Proceedings of the National Academy of Sciences , vol. 96, no. 6, pp. 3257–3262, 1999

    1999

  26. [34]

    Degeneracy: Demystifying and destigmatizing a core concept in systems biology,

    P. H. Mason, “Degeneracy: Demystifying and destigmatizing a core concept in systems biology,” Complexity, vol. 20, no. 3, pp. 12–21, 2015

    2015

  27. [35]

    Dissecting recall of factual associations in auto-regressive language models,

    M. Geva, J. Bastings, K. Filippova, and A. Globerson, “Dissecting recall of factual associations in auto-regressive language models,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 12 216–12 235

    2023

  28. [36]

    Knowledge localization: Mission not accomplished? enter query localization!

    Y . Chen, P. Cao, Y . Chen, K. Liu, and J. Zhao, “Knowledge localization: Mission not accomplished? enter query localization!” arXiv preprint arXiv:2405.14117, 2024

    2024 arXiv

  29. [37]

    How to explain individual classification decisions,

    D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R. Müller, “How to explain individual classification decisions,” The Journal of Machine Learning Research , vol. 11, pp. 1803–1831, 2010

    2010

  30. [38]

    Deep inside convolutional networks: Visualising image classification models and saliency maps,

    K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolutional networks: Visualising image classification models and saliency maps,” arXiv preprint arXiv:1312.6034 , 2013

    2013 arXiv

  31. [39]

    Axiomatic attribution for deep networks,

    M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in International conference on machine learning . PMLR, 2017, pp. 3319–3328

    2017

  32. [40]

    Discretized integrated gradients for explaining language models,

    S. Sanyal and X. Ren, “Discretized integrated gradients for explaining language models,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021, pp. 10 285–10 299

    2021

  33. [41]

    Sequential integrated gradients: a simple but effective method for explaining language models,

    J. Enguehard, “Sequential integrated gradients: a simple but effective method for explaining language models,” in Findings of the Association for Computational Linguistics: ACL 2023 , 2023, pp. 7555–7565

    2023

  34. [42]

    The effective coalitions of shapley value for integrated gradients,

    S. Liu, C. Fan, Y . Xiong, M. Wang, Y . Hu, T. Lv, Z. Chen, R. Wu, and Y . Gao, “The effective coalitions of shapley value for integrated gradients,” 2022

    2022

  35. [43]

    Introducing meta llama 3: The most capable openly available llm to date,

    A. Meta, “Introducing meta llama 3: The most capable openly available llm to date,” Meta AI, 2024

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.