{"id":"e407c533-bb4e-41d8-9d3f-676fab4f4312","arxiv_id":"2411.17401","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MATRICE localizes language-agnostic knowledge neurons by combining integrated-gradient attributions with uncertainty weighting across paraphrased queries and seven languages, supported by a new multilingual benchmark RML-LAMA.","lead":"This paper presents a method to locate the specific neurons inside large language models that hold a fact across many languages, along with a new multilingual test set. It shows that editing, amplifying, or fine-tuning these neurons can update or boost knowledge across languages at once.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central storage claim is validated only by self-referential manipulation of the same correct-answer probability, and the reported editing locality is within ~2-3 points of random, which is consistent with generic high-leverage neurons rather than fact-specific storage.","rationale":"The reader identified the weakest assumption as the attribution-to-storage inference being validated only through manipulation of the same P(y*|q). I agree with that core concern and extend it with the paper's own locality numbers: after cross-lingual editing, MATRICE's locality is only 2-3 points above a random-neuron baseline across the four models (Tables III and IV), meaning the edited neurons have little fact-specificity. That makes the central claim more than just under-validated; it suggests the localization may be selecting generic high-leverage units. However, this is an empirical question that a held-out-fact control can settle, and the paper has enough methodological structure (uncertainty weighting, new benchmark, application studies) that the appropriate disposition is still conditional acceptance pending that control and the missing code/data, rather than outright rejection. I therefore keep the reader's CONDITIONAL verdict unchanged, but flag that the specificity test should be a required revision, not a suggestion.","tokens_in":23298,"tokens_out":6207,"duration_ms":60913,"concrete_test":"Run a held-out-fact control for Section IV-D. For each fact f, compute MATRICE's neuron set N_f, then suppress N_f and measure the probability change for a matched set of unrelated facts G (different subject and relation, same language and query format). Compute the specificity ratio S = mean(|Delta P| on f) / mean(|Delta P| on G), and compare with the same ratio for Random neurons of equal size. If S is close to 1, or if Delta P on G is comparable to Delta P on f, the selected neurons are generic control points and the storage interpretation fails. A reasonable pass criterion is S >= 2-3 and a Random-vs-MATRICE gap substantially larger than the 2-3 point locality gap reported in Tables III and IV.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's core claim is that MATRICE identifies where a fact is stored, not merely which neurons influence P(y*|q). The main validation in Section IV-D (Eq. 14) measures exactly that probability before and after suppressing or enhancing the selected neurons, using the same query and answer that generated the attribution scores. This cannot distinguish storage from control: by construction, an integrated-gradient-selected neuron is one whose path integral contributes to P(y*|q), so perturbing it should move P(y*|q). The cross-lingual editing experiments (Sections IV-F to IV-H) remain largely self-referential because reliability and generality are also measured on the same fact and answer (Eqs. 16-17). The only non-self-referential signal, the locality metric in Tables III and IV, is troubling: MATRICE's locality is roughly 7-12, while the Random baseline is 5-10, leaving only a 2-3 point gap. In other words, editing the localized neurons destroys a comparable fraction of unrelated factual predictions as editing random neurons. That is the expected signature of generic high-leverage units, not of fact-specific storage locations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MATRICE, a method for localizing language-agnostic knowledge neurons in multilingual LLMs. It introduces RML-LAMA, a multilingual cloze benchmark with 7,849 facts across seven languages, computes per-neuron attribution scores via integrated gradients with token-replacement baselines, aggregates scores over paraphrased queries and languages using mean-variance uncertainty penalties, and selects neurons with a per-fact dynamic threshold. The paper evaluates the method by suppressing or enhancing selected neurons, by cross-lingual knowledge editing, by improving low-resource-language predictions, and by targeted fine-tuning for knowledge injection, reporting gains over AMIG and random baselines.","tokens_in":23586,"tokens_out":6752,"duration_ms":70157,"significance":"If the localization claim held, the paper would be a substantial methodological contribution: a reproducible multilingual benchmark, an uncertainty-aware attribution procedure, and evidence that selected neurons support cross-lingual editing and enhancement. The ablation in Table II shows that both uncertainty components help under the paper's own metric, and the breadth across four model families is a strength. However, the central claim that MATRICE identifies where a fact is stored, rather than merely identifying high-leverage control units of the prediction function, is validated mainly by manipulating the same probability used to construct the attribution scores. The independent signal available in the locality metrics is weak, with only a 2-3 point gap over the random baseline, which supports the concern that the selected neurons may be generic influential units rather than fact-specific storage locations. The paper is therefore a useful engineering contribution, but the scientific claim of fact-specific storage needs re-grounding.","major_comments":[{"comment":"The main validation is circular. Attribution scores in Eqs. (1)-(5) are path integrals of the gradient of P(y*|q) with respect to neuron activations, and Eq. (14) measures the change in exactly this P(y*|q) after zeroing or doubling the selected neurons. Any neuron with high integrated gradient is, by construction, one whose perturbation should move P(y*|q) in the direction of the gradient, so the suppression and enhancement results in Figure 5 are partly entailed by the selection objective. Those experiments demonstrate that MATRICE finds influential neurons, not that these neurons are the storage location of the fact. An independent test, such as checking whether the same neurons are required for an unrelated fact with the same answer or whether a different causal intervention removes the fact-specific effect, is needed before the storage interpretation can be accepted.","section":"§IV-D, Eq. (14)"},{"comment":"The locality numbers weaken the fact-specific interpretation. Averaged across languages and models, MATRICE locality is roughly 7.2-10.1 for erasure and 8.5-11.9 for update, while the Random baseline is roughly 5.1-7.1 and 5.6-7.8; the gap is only 2-3 points. This means editing MATRICE-selected neurons disrupts unrelated factual predictions almost as much as editing random neurons. If the selected units were fact-specific storage, one would expect substantially higher locality relative to random. The observed pattern is consistent with generic high-leverage units. The paper should report locality on a per-fact matched basis and include a baseline of high-attribution neurons for unrelated facts.","section":"Tables III and IV, Eq. (18)"},{"comment":"The reliability and generality metrics remain self-referential for the storage question. Both metrics measure whether the edited model outputs the edited answer for queries that express the edited fact in the target language. They show that the edit transfers across languages, but they do not distinguish editing a fact's storage location from editing a general-purpose control path that affects that fact's expression. A discriminating test would edit neurons selected for fact A and measure effects on a held-out fact B that shares the subject or relation, comparing against neurons selected for fact B.","section":"§IV-F, Eqs. (16)-(17)"},{"comment":"The comparison with AMIG is confounded by the number of edited neurons. The paper acknowledges in §IV-D that AMIG's intersection set shrinks as the number of languages grows, which explains its weaker manipulation effects, but the editing tables do not report neuron counts per method. If MATRICE edits a larger set, higher reliability and generality with comparable or slightly worse locality would be expected even without better localization. Please report the distribution of selected-neuron counts for each method and include an AMIG variant matched to MATRICE's neuron count.","section":"§IV-D and Tables III/IV"},{"comment":"The knowledge enhancement and injection experiments use queries the model initially answers incorrectly (Qerror and Qnew). For such queries, the correct-answer probability P(y*|q) in Eq. (1) is not the model's expressed prediction, and the meaning of attributing that probability to 'knowledge neurons' is unclear; if the model has not stored the fact, there may be no fact-specific storage to localize. The paper should clarify how MATRICE behaves when P(y*|q) is low or near zero, and should evaluate new-knowledge injection using facts whose neurons were selected on the pre-fine-tuned model's correct predictions rather than on the same initially unknown facts.","section":"§IV-G and §IV-H"}],"minor_comments":[{"comment":"The notation is inconsistent in Eqs. (4)-(5) (w'_i vs. w'_j, Attri(nl_j) vs. Attr(nl_j)), and the denominator of Eq. (5) is undefined if the sum of attribution scores over a layer is zero; since integrated gradients can be negative, this normalization needs a justification or a numerical safeguard.","section":"§III-A, Eqs. (4)-(5)"},{"comment":"The construction of RML-LAMA relies on Claude 3 Opus with automatic regeneration, but no inter-annotator agreement, human validation statistics, or public release link are provided; please add a data-quality section with sample counts, agreement rates, and a URL.","section":"§IV-A"},{"comment":"The hyperparameters alpha1, alpha2, beta1, beta2, tau, M, lambda1, and lambda2 are listed but no values or sensitivity analysis are given; at a minimum, report the chosen values and vary tau and M over a small grid to show stability.","section":"Algorithm 1"},{"comment":"The figures report a single run with no error bars or significance tests; given that several comparisons are small in magnitude, run-to-run variance should be reported.","section":"Figures 5 and 6"},{"comment":"The explanation that Random's nonzero effect is partly due to selecting neurons also chosen by the other methods applies equally to MATRICE; a more informative control would be random neurons drawn from the same layer distribution as MATRICE.","section":"§IV-D, Fig. 5"},{"comment":"Using the EOS token as an information-free baseline for auto-regressive models is questionable because EOS has a trained positional and functional role; this choice should be justified empirically with a comparison to alternative baselines.","section":"§III-A, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"This paper extends the same group's AMIG work (Chen et al., AAAI 2024), and the incremental contribution should be stated more crisply. The benchmark and the uncertainty formulation are useful, but the current validation does not support the strong storage claim. The scientific claim should either be re-grounded with the independent tests described in the major comments or toned down to 'influential neurons' in the abstract and conclusion. The journal fit is appropriate for an NLP and interpretability venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: worth reading for the RML-LAMA benchmark and the MATRICE method, but the central storage-location claim is oversold. The main validation, Eq. 14, measures the probability change of the same correct-answer P(y*|q) that the integrated-gradient attribution was computed on. Selecting neurons by attribution to P(y*|q) and then showing that perturbing them moves P(y*|q) is close to a tautology. It tells you these are high-leverage control points, not where the fact is stored. The cross-lingual editing results are still partly self-referential because reliability and generality are measured on the same facts and answers. The strongest independent evidence would be locality, and that is where the paper is weakest: in Tables III and IV, MATRICE's locality is only 2-3 points above Random (e.g., 8.46 vs 5.94 on mGPT erasure). That is the pattern you would expect from generic influential neurons, not fact-specific storage.\n\nCredit where due: RML-LAMA, with rephrased queries across 7 languages, is a solid resource. The uncertainty weighting across queries and languages (Eqs. 6-11) is a sensible extension of prior intersection-based localization, and the ablation shows both components contribute. Experiments are extensive — four models, seven languages. The knowledge enhancement and injection results suggest the identified neurons have practical utility even if the storage interpretation is shaky. The writing is clear and the method is described well enough to reimplement.\n\nSoft spots besides circularity: hyperparameters (alpha, beta, tau, M, lambda) are never reported, and code and data are not released. The benchmark was generated with an API but has no human evaluation reported. These are fixable. The paper is for researchers working on knowledge editing, multilingual interpretability, and anyone who wants a benchmark for cross-lingual factual recall.\n\nBottom line: a serious referee should engage with this, but the paper needs major revision. The authors should either add non-circular validation — e.g., show that editing these neurons does not affect held-out facts beyond a random baseline, or align with circuit-level analyses — or soften the storage claim to 'control points for cross-lingual expression.' The benchmark and uncertainty method are contributions on their own.","headline":"Useful benchmark and a plausible uncertainty-weighted localization method, but the storage claim rests on a self-referential test and weak locality numbers.","tokens_in":24072,"tokens_out":3440,"would_cite":true,"duration_ms":32116,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a new uncertainty-aware attribution method, MATRICE, identifies feed-forward neurons that store a fact in a language-independent way, so that manipulating them edits the fact across all tested languages.","keywords":["large language models","knowledge neurons","language-agnostic knowledge neurons","knowledge localization","integrated gradients","multilingual knowledge","cross-lingual knowledge editing","RML-LAMA"],"falsifier":"Localize neurons for a set of facts with MATRICE, then suppress them and check whether the probability of the fact drops only for those facts and in every language, while unrelated facts and randomly selected neurons matched for activation produce no comparable effect; a failure on held-out paraphrases or languages would show the localization is query-sensitive rather than storage-based.","tokens_in":23125,"feed_emoji":"🧠","tokens_out":5510,"duration_ms":55993,"temperature":0.7,"pith_summary":"The paper is trying to establish that factual knowledge in multilingual large language models is stored, at least in part, in specific neurons whose association with a fact does not depend on the language of the query. It proposes MATRICE, an attribution method that scores every feed-forward neuron by its integrated-gradient contribution to the correct answer, then averages those scores over several paraphrased queries in each language while penalizing high variance across queries and languages. On a new seven-language benchmark, RML-LAMA, the paper shows that suppressing or amplifying the selected neurons changes the expression of the corresponding fact in all tested languages. If true, this gives a practical handle: a single edit can update, strengthen, or inject a fact across languages, including languages the model handles poorly.","feed_headline":"One neuron set stores a fact in seven languages","feed_subtitle":"Localizing these neurons lets one edit, strengthen, or inject a fact across all languages at once.","key_machinery":"The load-bearing object is the language-agnostic knowledge neuron, defined as an FFN neuron whose attribution to a fact's correct answer remains high and low-variance when the fact is queried through many paraphrases in many languages. The method that carries the argument is MATRICE: Sequential Adapted Integrated Gradients computes per-query attribution by integrating gradients along a path from a baseline sentence (each token replaced by $\\langle \\text{PAD} \\rangle$) to the original query; Uncertainty Quantification then forms a language-agnostic score $S_i^l = \\beta_1 E[\\bar{s}_i^l] - \\beta_2 \\sqrt{\\mathrm{Var}[\\bar{s}_i^l]}$, and a dynamic per-fact threshold picks the final neuron set. This design is what converts a single-query, two-language intersection into a seven-language, paraphrase-robust localization.","core_discovery":"The central claim is that MATRICE accurately localizes language-agnostic knowledge neurons: feed-forward neurons whose high attribution to a fact is stable across semantically equivalent queries and across languages. The method computes sequential adapted integrated gradients for each query, replaces tokens one at a time with a padding token to build low-information baseline vectors, then combines mean and variance of attribution scores first across paraphrases and then across languages, and finally selects neurons above a dynamic threshold. Manipulation experiments show that setting these neurons to zero lowers the probability of the correct answer in all seven languages, while doubling their activations raises it; the same neurons also serve as targets for cross-lingual erasure and update editing, for improving accuracy on low-resource languages, and for fine-tuning that injects new knowledge with less forgetting. The paper concludes that these neurons store facts in a language-independent form and that localizing them well is the key to multilingual knowledge control.","pith_inferences":["Editorial inference: the same uncertainty-weighted attribution pipeline could apply to facts expressed without cloze templates by defining the target probability over generated tokens, but the paper does not test that setting.","Editorial inference: if attribution identifies high-leverage control points rather than true storage sites, the editing, enhancement, and injection applications could still work, but the storage interpretation would need independent causal evidence such as causal tracing.","Editorial inference: applying the method to a model trained almost entirely on one language and checking whether it still finds language-agnostic neurons would help separate shared multilingual representations from simple training-corpus overlap.","Editorial inference: the benchmark and method could be extended to non-factual knowledge such as commonsense or procedural knowledge, with the caveat that the attribution target would need to be redefined."],"forward_implications":["Editing the localized neurons once in English changes the corresponding fact in all other tested languages, with reliability gains over the earlier intersection-based method of roughly 13 to 22 points depending on the model.","Amplifying these neurons raises accuracy on previously wrong queries in low-resource languages, and the gain grows as the underlying model becomes larger and more capable.","Fine-tuning only the localized neurons lets a model acquire new facts while retaining old knowledge far better than full fine-tuning, with old-knowledge accuracy often staying near 90 percent.","Most language-agnostic knowledge neurons concentrate in the last few layers, and this pattern holds across both auto-encoding and auto-regressive transformer architectures.","Removing either uncertainty component degrades localization, and the query-level uncertainty component matters more than the language-level component."],"supporting_citations":[{"why":"Defines knowledge neurons and the attribution framework that MATRICE extends.","marker":"[13]"},{"why":"Discovers language-independent knowledge neurons using a two-language intersection, the baseline MATRICE refines.","marker":"[15]"},{"why":"Supplies the key-value memory view of feed-forward layers that motivates treating FFN neurons as knowledge storage.","marker":"[30]"},{"why":"Provides the integrated gradients technique that the paper adapts into Sequential Adapted Integrated Gradients.","marker":"[39]"},{"why":"Provides the multilingual LAMA benchmark that RML-LAMA extends with paraphrased queries.","marker":"[21]"},{"why":"Supplies the original LAMA cloze-style probing setup and English rephrased queries used to build RML-LAMA.","marker":"[7]"},{"why":"Motivates the uncertainty quantification by showing that localization is sensitive to query wording.","marker":"[36]"}],"fun_headline_variants":["Neurons that know facts in any language","One set of neurons, every language","Cross-lingual fact editing: neurons that speak all tongues","Language-agnostic neurons: the universal key to facts","MATRICE: pinpointing neurons behind multilingual facts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a neuron's integrated-gradient attribution to the probability of the correct answer identifies where the fact is actually stored, rather than merely identifying high-leverage switches that can change that probability.","fun_headline_variants_meta":{"raw":{"variants":["Neurons that know facts in any language","One set of neurons, every language","Cross-lingual fact editing: neurons that speak all tongues","Language-agnostic neurons: the universal key to facts","MATRICE: pinpointing neurons behind multilingual facts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1548,"prompt_tokens":1029,"completion_tokens":519,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":444}},"tokens_in":645,"tokens_out":519,"duration_ms":5832,"temperature":1.0,"reasoning_tokens":444,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:08:19.346841+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Localize neurons for a set of facts with MATRICE, then suppress them and check whether the probability of the fact drops only for those facts and in every language, while unrelated facts and randomly selected neurons matched for activation produce no comparable effect; a failure on held-out paraphrases or languages would show the localization is query-sensitive rather than storage-based.","supporting_citations":[{"cited_title":"Knowledge neurons in pretrained transformers,","cited_arxiv_id":null,"evidence_quote":"Defines knowledge neurons and the attribution framework that MATRICE extends."},{"cited_title":"Journey to the center of the knowledge neurons: Discoveries of language-independent knowledge neurons and degenerate knowledge neurons,","cited_arxiv_id":null,"evidence_quote":"Discovers language-independent knowledge neurons using a two-language intersection, the baseline MATRICE refines."},{"cited_title":"Transformer feed-forward layers are key-value memories,","cited_arxiv_id":null,"evidence_quote":"Supplies the key-value memory view of feed-forward layers that motivates treating FFN neurons as knowledge storage."},{"cited_title":"Multilingual lama: Investigating knowledge in multilingual pretrained language models,","cited_arxiv_id":null,"evidence_quote":"Provides the multilingual LAMA benchmark that RML-LAMA extends with paraphrased queries."},{"cited_title":"Language models as knowledge bases?","cited_arxiv_id":null,"evidence_quote":"Supplies the original LAMA cloze-style probing setup and English rephrased queries used to build RML-LAMA."}],"review_version":1}