Pith. sign in

REVIEW 5 major objections 5 minor 31 references

Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that gradient-attribution scores can pick a sparse set of neurons—around 1% of parameters—whose update edits a factual association in a large language model while keeping unrelated knowledge intact.

desk verdict A plausible locate-and-edit variant that can't be reproduced as written, but the mapping-layer finding and the missing random-control critique are the two things to argue about. read the letter →

arxiv 2412.20637 v1 pith:4HHTPADN submitted 2024-12-30 cs.CL

Yongchang Li , Yujin Zhu , Tao Yan , Shijian Fan , Gang Wu , Liang Xu This is my paper · ORCID
classification cs.CL
keywords knowledgeeditinglargelanguagemodelsneuronalensemblegradientattributionparameterlocalizationlocate-then-editfeed-forwardnetworksbatch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Knowledge Neuronal Ensemble (KNE), a locate-then-edit method for changing a large language model's factual knowledge without retraining the whole model. KNE computes a gradient-attribution score for every neuron in every layer, keeps the top-scoring ~1% of neurons as a 'knowledge neuronal ensemble,' and backpropagates through only those parameters. The paper claims this yields higher edit success than five baselines on ZsRE, WikiData counterfact, and WikiData recent, with portability and locality that match or exceed the best baselines on most benchmarks. A secondary claim is that factual knowledge is not confined to FFN value layers; editing mapping layers gives better locality, and localizing only a quarter of the facts can suffice.

What carries the argument

The load-bearing object is the 'knowledge neuronal ensemble' (KNE): the set of neurons whose token-level gradient-attribution score, summed over answer tokens and approximated by a Riemann sum, lies above the top-1-p% quantile across all layers. The paper uses these scores to build a sparse update matrix $W_{\text{kne}}$, maps it into a zero matrix $\Delta W$ at the selected indices, and updates the weights with $W \leftarrow W + \frac{\alpha}{\sqrt{n}} \Delta W$. The dynamic-interaction claim rests on computing losses and gradients only over this ensemble and backpropagating through it.

What would settle it

Run the KNE edit with the same number of neurons chosen from the lowest attribution scores instead of the highest; if edit success and locality stay roughly unchanged, the attribution ranking is not doing the causal work. A second test is to compare an edit that updates only neurons above the threshold against an edit that updates the same count of parameter positions chosen at random, where the claim predicts a decisive gap.

Watch

Extended reading notes

Core claim

The central claim is that a small set of neurons selected by gradient-attribution scores, rather than a fixed layer, can act as a distributed ensemble that carries a fact, and that updating only this ensemble with gradients flowing across layers edits the fact while preserving unrelated knowledge. On the three benchmarks the paper reports Edit Success of roughly 97-99%, Portability of 53-63%, and Locality that is competitive with or better than the baselines on two of the three datasets. The paper also argues that knowledge storage is more distributed than the key-value memory view: editing mapping layers (gate and up projections) can match or beat editing value layers on locality and fluency.

Load-bearing premise

Everything reported depends on the premise that the gradient-attribution score actually locates the parameters that causally carry the fact, so that editing the top-scoring ~1% is both sufficient to install the new fact and safe for other knowledge.

Editorial extensions

If this is right

  • If the reported numbers hold, a 7-billion-parameter model can have a single fact changed with roughly 1% of weights updated, making repeated edits cheap enough for ongoing deployment.
  • The partial-localization result implies that a model may not need every fact localized before editing; a representative subset of similar facts can locate the region, cutting localization cost by about 75%.
  • Batch editing is possible with modest degradation as batch size grows, which extends the method beyond single-fact editors.
  • The layer analysis suggests practitioners can trade edit success for locality by choosing which projection to edit, with mapping layers offering safer localized edits.
  • The method's low parameter footprint could make knowledge editing practical on resource-constrained or on-device models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the localization premise holds, KNE's attribution scores could be reused as a diagnostic probe, and the overlap between ensembles for semantically close facts could predict whether batch edits will conflict.
  • Editorial inference: The finding that only 200-300 localized facts suffice hints that similar facts share storage regions; an external test would be to perturb one region and measure whether all related facts change together.
  • Editorial inference: The method's reliance on answer-token-level gradients suggests it may extend to low-resource or multilingual settings where only a few correct answer tokens are available, though the paper does not test this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Knowledge Neuronal Ensemble (KNE), a locate-then-edit method for knowledge editing in LLMs. KNE computes token-level gradient attribution scores for FFN parameters (Eq. 5), selects the highest-scoring parameters across layers into a 'knowledge neuronal ensemble' (Eqs. 6-7), and updates only those parameters via a scaled gradient update (Eqs. 8-11). The authors evaluate KNE on ZsRE, WikiData counterfact, and WikiData recent using Llama-2-7b-chat and GPT-J-6B, reporting high Edit Success and competitive Portability/Locality in Table 1 and Table 2. The paper also includes layer-wise storage analyses, batch-editing experiments, and an ablation varying the number of edited parameters. A brief limitations section acknowledges the need for better knowledge-set selection and for theoretical grounding of the layer-choice findings.

Significance. If its empirical claims were reproducible, KNE would be a practically useful and simple baseline for batch knowledge editing, since it edits roughly 1% of parameters and reports strong portability. The paper formulates knowledge editing as constrained optimization and extends token-level gradient attribution to multiple tokens, which is a sensible direction. However, the manuscript is currently not reproducible: the update rule in Eq. (10) has a dimensional inconsistency, the hyperparameters p, m, and alpha are never specified, and no code is provided. More importantly, the central claim that the gradient-attribution ensemble is what drives the gains is not causally validated against a random-parameter control. The significance of the reported numbers is therefore conditional on substantial revisions.

major comments (5)
  1. [§3.2.3, Eqs. (8)–(10)] The update rule as written cannot be executed: Eq. (8) defines W_kne as an n×d1 matrix, Eq. (9) defines Delta W as d2×d1, and Eq. (10) assigns Delta W[:, M_kne] = W_kne. Since M_kne has length n and its entries are stated to be natural numbers less than d2, the left-hand side has shape d2×n, so the assignment requires W_kne to be d2×n, not n×d1. If the intent is to select rows, the assignment should instead be Delta W[M_kne, :] = W_kne. This dimensional inconsistency must be corrected, and the indexing convention (row vs. column neurons, and 0- vs. 1-based indexing) must be stated explicitly.
  2. [§3.2.2, Eqs. (6)–(7), and §4.5] The central claim that the gradient-attribution ensemble drives the reported gains is not tested. Section 4.5 varies only the number of edited parameters; it does not compare the attributed ensemble with a same-size random subset of FFN parameters or with any identity/control selection. Without such a control, the high Edit Success and Portability figures in Table 1 are equally compatible with the hypothesis that any sufficiently large restricted gradient update on FFN weights improves these metrics. Because the novelty of KNE is precisely the attribution-based localization, this missing control is load-bearing for the paper's main conclusion.
  3. [§4.1, Appendix A, and Eq. (11)] The hyperparameters p (selection quantile in Eq. (6)), m (Riemann steps in Eq. (5)), and alpha (scaling factor in Eq. (11)) are never specified, and no code or repository is provided. The subset-selection procedure in Section 4.4, where 200 localized knowledge points outperform full-dataset localization, is also not described. These omissions make the numerical results in Tables 1 and 2 unreproducible, and the paper reports no error bars or significance tests to support the word 'significantly' in the abstract.
  4. [§3.2.2, Eq. (6)] The quantile notation is self-contradictory: the text says KNE selects the top 1−p% of neurons, but Eq. (6) defines t_p = Quantile_{1−p}(...), and the following sentence says this quantile function 'calculates the value corresponding to the top p%'. This ambiguity makes it impossible to determine the fraction of parameters actually edited; please clarify the intended selection rule and report the fraction used in each experiment.
  5. [Abstract and Table 1] The abstract claims KNE 'achieves, or even exceeds, the performance of the best baseline methods in portability and locality metrics,' but Table 1 contradicts this for locality on WikiData recent: KNE obtains 37.58, well below ROME (66.2), MEMIT (64.78), and FT-L (63.7). The claim should be revised to per-dataset and per-metric accuracy, and the large locality drop on this dataset deserves a specific explanation.
minor comments (5)
  1. [§5, Conclusion] The phrase 'Knowledge Neurona’l Ensemble' contains a typographical error ('Neurona’l') that should be corrected to 'Neuronal'.
  2. [§3, notation] The symbol m is overloaded: it denotes the number of Riemann steps in Eq. (5) and the number of neurons in a layer in Definition 3. Please use distinct symbols for these two quantities.
  3. [Figure 2 and Appendix B] Figure 2 is referenced as illustrating layer-wise findings, but the text does not walk through panels (a)–(d); adding a sentence per panel would make the conclusions in Section 4.3 easier to verify. The appendix figures B.3–B.5 would also benefit from explicit descriptions of the plotted axes and error bars, if any.
  4. [Table 2] Table 2 reports KNE results on two models but does not include baseline comparisons for GPT-J-6B, so it only shows KNE's own consistency across models and cannot support cross-model claims about relative performance.
  5. [Appendix A.3] The paper states that baseline results were sourced from the EasyEdit GitHub repository, but no repository version, commit, or experiment configuration is given; please provide exact dataset splits and baseline hyperparameter details for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the KNE edit pipeline is self-contained against external baselines, and no evaluated quantity is equivalent to its input by construction; the localization premise is unvalidated rather than circular.

full rationale

The derivation chain is: (i) gradient attribution scores are computed per Eq. (5) on the target answer tokens of each edit; (ii) a top-quantile threshold selects the Knowledge Neuronal Ensemble per Eqs. (6)-(7); (iii) a masked gradient update modifies only the KNE parameters per Eqs. (8)-(11); (iv) the edited model is evaluated on Edit Success, Portability, Locality, and Fluency against five external baselines (FT, FT-L, AdaLoRA, ROME, MEMIT) on ZsRE, WikiData counterfact, and WikiData recent. No step reduces to its own input by construction: the attribution selection does not itself produce the edit, since the restricted gradient update must still succeed on the selected subnetwork, and Portability and Locality are measured on paraphrase, reasoning, and unrelated inputs that never enter the attribution computation. The token-level gradient attribution localization is cited to external prior work (Dai et al. [6], ACL 2022), not to the present authors, and no uniqueness theorem or load-bearing self-citation is invoked. The only non-independence is that Edit Success, by definition (A.2), includes the edited prompt whose target tokens were used for attribution; this is inherent to the supervised training-and-evaluation structure shared by all locate-then-edit methods and is not a circularity. The central premise that attributed neurons, rather than the restricted gradient update itself, drive the gains is under-supported by the absence of a same-size random-parameter control in Section 4.5 (which varies parameter count, not parameter identity), and the reported WikiData-recent Locality of 37.58 versus ROME's 66.2 sits uneasily with the abstract's 'achieves, or even exceeds, the best baseline methods in portability and locality metrics' claim; the Appendix A.4 note that baseline results were sourced from the EasyEdit GitHub repository also raises comparison-fairness questions. These are correctness, verification, and reproducibility concerns, not circularity per the definition used here.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The central claim rests on the untested premise that gradient attribution localizes knowledge in a causally sufficient way, plus several unreported hyperparameters. No free parameter is justified by external benchmarks, and the invented 'ensemble' entity has no independent evidence.

free parameters (3)
  • p
    Percentage threshold for selecting the top-attribution neurons (Eq. 6). Value not reported; controls the size of the ensemble and the edit-locality trade-off.
  • alpha
    Scaling factor in the update rule (Eq. 11). Value not reported; controls the magnitude of weight changes and thus the balance between edit success and preservation of unrelated knowledge.
  • m
    Number of Riemann steps in the gradient attribution approximation (Eq. 5). Value not reported; affects the fidelity of the integrated-gradient estimates.
assumptions (3)
  • domain assumption Gradient attribution scores identify parameters that are causally sufficient for the target edit.
    Section 3.2 assumes that neurons with high attribution scores are the ones that must be updated to change the fact, but no causal mediation analysis or ablation validates this.
  • domain assumption Factual knowledge is localized in a small subset of FFN parameters.
    The locate-then-edit paradigm, inherited from Dai et al. and ROME, assumes that editing a small subset of weights can implement a fact change without retraining the whole model.
  • ad hoc to paper The update rule W_hat = W + (alpha / sqrt(n)) * Delta W preserves out-of-scope behavior.
    Eq. (11) provides no derivation or guarantee that the scaled delta preserves locality; it is a heuristic design choice.
invented entities (1)
  • Knowledge Neuronal Ensemble (KNE)
    purpose: A set of neurons across all layers selected by attribution threshold, used as the target for masked parameter updates.
    The concept is defined operationally by the paper's own method; there is no falsifiable handle outside the paper that would support or refute its existence as a distinct neural structure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble." pith.science (2026). https://pith.science/paper/4HHTPADN

@misc{pith2026241220637,
  author       = {Pith},
  title        = {Pith review of: Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4HHTPADN}},
  note         = {Machine review of arXiv:2412.20637}
}
read the original abstract

As real-world knowledge is constantly evolving, ensuring the timeliness and accuracy of a model's knowledge is crucial. This has made knowledge editing in large language models increasingly important. However, existing knowledge editing methods face several challenges, including parameter localization coupling, imprecise localization, and a lack of dynamic interaction across layers. In this paper, we propose a novel knowledge editing method called Knowledge Neuronal Ensemble (KNE). A knowledge neuronal ensemble represents a group of neurons encoding specific knowledge, thus mitigating the issue of frequent parameter modification caused by coupling in parameter localization. The KNE method enhances the precision and accuracy of parameter localization by computing gradient attribution scores for each parameter at each layer. During the editing process, only the gradients and losses associated with the knowledge neuronal ensemble are computed, with error backpropagation performed accordingly, ensuring dynamic interaction and collaborative updates among parameters. Experimental results on three widely used knowledge editing datasets show that the KNE method significantly improves the accuracy of knowledge editing and achieves, or even exceeds, the performance of the best baseline methods in portability and locality metrics.

Figures

Figures reproduced from arXiv: 2412.20637 by the authors.

Figure 1
Figure 1. The framework of KNE method. Since calculating the continuous gradient integral is computationally expensive, we apply a Riemann approximation as follows: Attr( g w (l) k ) = w (l) k m Xs j=1 Xm i=1 ∂Py ∗ j |x  i mw (l) k  ∂w(l) k (5) Where m represents the number of discrete steps used to approximate the continu￾7 [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Visualization of Knowledge Storage Deviations in Large Language Models [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 18 canonical work pages

  1. [1]

    S. Wang, Y . Zhu, H. Liu, Z. Zheng, C. Chen, J. Li, Knowledge editing for large language models: A survey, ACM Comput. Surv. 57 (3) (Nov. 2024). doi:10.1145/3698590. 15

    doi:10.1145/3698590 2024

  2. [2]

    Z. Han, C. Gao, J. Liu, J. Zhang, S. Q. Zhang, Parameter-efficient fine-tuning for large models: A comprehensive survey (2024). arXiv:2403.14608. URL https://arxiv.org/abs/2403.14608

    arXiv 2024

  3. [3]

    Y . Gao, Y . Xiong, X. Gao, K. Jia, J. Pan, Y . Bi, Y . Dai, J. Sun, M. Wang, H. Wang, Retrieval-augmented generation for large language models: A survey (2024). arXiv:2312.10997. URL https://arxiv.org/abs/2312.10997

    arXiv 2024

  4. [4]

    Y . Yao, P. Wang, B. Tian, S. Cheng, Z. Li, S. Deng, H. Chen, N. Zhang, Editing large language models: Problems, methods, and opportunities, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Singapore, 2023, pp. 10222–10240. doi:10. 18...

    work page 2023

  5. [5]

    Zhang, Y

    N. Zhang, Y . Yao, B. Tian, P. Wang, S. Deng, M. Wang, Z. Xi, S. Mao, J. Zhang, Y . Ni, S. Cheng, Z. Xu, X. Xu, J.-C. Gu, Y . Jiang, P. Xie, F. Huang, L. Liang, Z. Zhang, X. Zhu, J. Zhou, H. Chen, A comprehensive study of knowledge editing for large language models (2024). arXiv:2401.01286. URL https://arxiv.org/abs/2401.01286

    arXiv 2024

  6. [6]

    D. Dai, L. Dong, Y . Hao, Z. Sui, B. Chang, F. Wei, Knowledge neurons in pretrained transformers, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, 2022, pp. 8493–8502

    work page 2022

  7. [7]

    M. Geva, R. Schuster, J. Berant, O. Levy, Transformer feed-forward layers are key-value memories, in: M. Moens, X. Huang, L. Specia, S. W. Yih (Eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, Association for Computational Linguis...

    work page 2021

  8. [8]

    M. Geva, A. Caciularu, K. R. Wang, Y . Goldberg, Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space, in: Y . Goldberg, Z. Kozareva, Y . Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, Associat...

    work page 2022

Show all 31 references
  1. [9]

    Mitchell, C

    E. Mitchell, C. Lin, A. Bosselut, C. D. Manning, C. Finn, Memory-based model editing at scale, in: Proceedings of the 39th International Conference on Machine Learning, V ol. 162, PMLR, 2022, pp. 15817–15831

    2022

  2. [10]

    Zheng, L

    C. Zheng, L. Li, Q. Dong, Y . Fan, Z. Wu, J. Xu, B. Chang, Can we edit factual knowledge by in-context learning?, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-...

    2023 doi

  3. [11]

    W. Wang, B. Haddow, A. Birch, Retrieval-augmented multilingual knowl- edge editing, in: L. Ku, A. Martins, V . Srikumar (Eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis- tics (V olume 1: Long Papers), ACL 2024, Bangkok, Thailand, Augu...

    2024 doi

  4. [12]

    Y . Shi, Q. Tan, X. Wu, S. Zhong, K. Zhou, N. Liu, Retrieval-enhanced knowledge editing in language models for multi-hop question answering, in: E. Serra, F. Spezzano (Eds.), Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 202...

    2024

  5. [13]

    Madaan, N

    A. Madaan, N. Tandon, P. Clark, Y . Yang, Memory-assisted prompt editing to improve GPT-3 after deployment, in: Y . Goldberg, Z. Kozareva, Y . Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Ar...

    2022 doi

  6. [14]

    Murty, C

    S. Murty, C. D. Manning, S. M. Lundberg, M. T. Ribeiro, Fixing model bugs with natural language patches, in: Y . Goldberg, Z. Kozareva, Y . Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab ...

    2022 doi

  7. [15]

    Huang, Y

    Z. Huang, Y . Shen, X. Zhang, J. Zhou, W. Rong, Z. Xiong, Transformer- patcher: One mistake worth one neuron, in: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net, 2023

    2023

  8. [16]

    Q. Dong, D. Dai, Y . Song, J. Xu, Z. Sui, L. Li, Calibrating factual knowledge in pretrained language models, in: Y . Goldberg, Z. Kozareva, Y . Zhang (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11,...

    2022 doi

  9. [17]

    Hartvigsen, S

    T. Hartvigsen, S. Sankaranarayanan, H. Palangi, Y . Kim, M. Ghassemi, Aging with GRACE: lifelong model editing with discrete key-value adaptors, in: A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems 36: An...

    2023

  10. [18]

    L. Yu, Q. Chen, J. Zhou, L. He, MELO: enhancing model editing with neuron-indexed dynamic lora, in: M. J. Wooldridge, J. G. Dy, S. Natarajan (Eds.), Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artif...

    2024 doi

  11. [19]

    N. D. Cao, W. Aziz, I. Titov, Editing factual knowledge in language models, in: M. Moens, X. Huang, L. Specia, S. W. Yih (Eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, ...

    2021 doi

  12. [20]

    Mitchell, C

    E. Mitchell, C. Lin, A. Bosselut, C. Finn, C. D. Manning, Fast model editing at scale, in: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, OpenReview.net, 2022

    2022

  13. [21]

    C. Tan, G. Zhang, J. Fu, Massive editing for large language models via meta learning, in: The Twelfth International Conference on Learning Repre- 18 sentations, ICLR 2024, Vienna, Austria, May 7-11, 2024, OpenReview.net, 2024

    2024

  14. [22]

    K. Meng, D. Bau, A. Andonian, Y . Belinkov, Locating and editing factual associations in GPT, in: Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9...

    2022

  15. [23]

    K. Meng, A. S. Sharma, A. J. Andonian, Y . Belinkov, D. Bau, Mass-editing memory in a transformer, in: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net, 2023. URL https://openreview.net/forum?id=MkbcAHIYgyS

    2023

  16. [24]

    X. Li, S. Li, S. Song, J. Yang, J. Ma, J. Yu, PMET: precise model editing in a transformer, in: M. J. Wooldridge, J. G. Dy, S. Natarajan (Eds.), Thirty- Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artifici...

    2024

  17. [25]

    Gueta, E

    A. Gueta, E. Venezian, C. Raffel, N. Slonim, Y . Katz, L. Choshen, Knowledge is a region in weight space for fine-tuned language models, in: H. Bouamor, J. Pino, K. Bali (Eds.), Findings of the Association for Computational Lin- guistics: EMNLP 2023, Singapore, December 6-10, ...

    2023

  18. [26]

    Zhang, M

    Q. Zhang, M. Chen, A. Bukharin, P. He, Y . Cheng, W. Chen, T. Zhao, Adap- tive budget allocation for parameter-efficient fine-tuning, in: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net, 2023

    2023

  19. [27]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bash- lykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V . Goswami, N. Goyal, A. Hartshorn, S. ...

    2023 arXiv

  20. [28]

    meta llama, Inference code for Llama models, https://github.com/ meta-llama/llama (2023)

    2023

  21. [29]

    B. Wang, A. Komatsuzaki, GPT-J-6B: A 6 Billion Parameter Au- toregressive Language Model, https://github.com/kingoflolz/ mesh-transformer-jax (May 2021)

    2021

  22. [30]

    Wang, Mesh-Transformer-JAX: Model-Parallel Implementation of Trans- former Language Model with JAX, https://github.com/kingoflolz/ mesh-transformer-jax (May 2021)

    B. Wang, Mesh-Transformer-JAX: Model-Parallel Implementation of Trans- former Language Model with JAX, https://github.com/kingoflolz/ mesh-transformer-jax (May 2021)

    2021

  23. [31]

    P. Wang, N. Zhang, X. Xie, Y . Yao, B. Tian, M. Wang, Z. Xi, S. Cheng, K. Liu, G. Zheng, et al., Easyedit: An easy-to-use knowledge editing framework for large language models, arXiv preprint arXiv:2308.07269 (2023). Appendix A. Settings Appendix A.1. Baselines We compare agai...

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.