Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

This paper claims that a layer-scoped variant of AdaLoRA, guided by per-layer gradient importance, can correct deprecated API recommendations in code LLMs with little collateral damage, and introduces the EDAPIBench benchmark to measure the

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:43 UTC pith:CRR5WZQT

load-bearing objection Solid benchmark study; AdaLoRA-L improves specificity, but the layer-selection story is under-tested. the 4 major comments →

arxiv 2511.21022 v2 pith:CRR5WZQT submitted 2025-11-26 cs.SE

Don't Use a Cannon to Kill a Fly: Lightweight Model Editing for LLMs to Correct Deprecated API Recommendations

Guancheng Lin , Xiao Yu , Jacky Keung , Xing Hu , Xin Xia , Alex X. Liu This is my paper
classification cs.SE
keywords deprecated APImodel editingcode completionLLM knowledge editingEDAPIBenchAdaLoRAparameter-efficient fine-tuningspecificity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the right fix for a code-assistant's outdated API suggestions is a small, targeted model edit rather than retraining, and that the real obstacle is collateral damage: the most effective editing method, AdaLoRA, installs the new API but also rewrites unrelated knowledge. To support the argument it introduces EDAPIBench, an automatically constructed benchmark of more than 3,000 editing instances covering eight Python libraries, and shows that ten editing methods spread across four families either fail to install the up-to-date API or degrade unrelated completions. It then proposes AdaLoRA-L, which scores every layer by the average squared gradient of the loss on the target API token, freezes layers important to all APIs, and restricts edits to layers important only for the API being updated. Reported results are large specificity gains — relative AEM improvements of 836.2%, 33.5%, and 310.2% on the three tested LLMs — with effectiveness, generalization, and portability staying comparable to unconstrained AdaLoRA. If the claim holds, coding assistants can be kept current with evolving libraries in seconds per fix instead of with expensive retraining.

Core claim

The central claim is that deprecated-API knowledge in code LLMs can be updated by editing only the layers that carry API-specific knowledge, leaving 'common' layers that matter for every API untouched. On EDAPIBench, a fully automated benchmark of over 3,000 real-world editing instances across eight Python libraries, the parameter-efficient method AdaLoRA outperforms nine other editors on effectiveness, generalization, and portability but disturbs unrelated API completions. AdaLoRA-L instead computes, per layer, the mean squared gradient of the loss on the target up-to-date API token; layers with high importance across all APIs are frozen as Common API Layers, and only layers with high impor

What carries the argument

The load-bearing mechanism is layer-importance localization. For each editing instance, AdaLoRA-L computes the loss only on the tokens of the up-to-date API, backpropagates to the editable parameters, and scores each layer as the mean squared gradient magnitude, S_i = (1/n) Σ_j (∇_{w_{i,j}} L)^2. Scores are averaged over all instances of the same API; layers that score high for every API are declared Common API Layers and frozen, while layers that score high only for the target API become Specific API Layers, the only layers AdaLoRA is allowed to edit. This converts a knowledge-editing problem into a question of where API-specific knowledge lives in the transformer, and it is the component t

Load-bearing premise

The load-bearing premise is that every one of the 145 deprecated-to-up-to-date API mappings is correct and that the line containing the up-to-date API is the right single-line completion target for every editing instance; if either fails, the benchmark labels and all reported scores are uninterpretable.

What would settle it

Manually audit a random sample of the 145 deprecated-to-up-to-date API mappings against official library release notes, and for each sampled instance verify that the up-to-date API line is the natural single-line completion of the prompt; any wrong or context-dependent mapping would make the benchmark labels uninterpretable and the reported gains artifacts.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the layer-localization claim holds, a code LLM's deprecated-API mistakes can be corrected per API in seconds on a single 24GB GPU, instead of by retraining or rewriting completions at inference time.
  • Because each edit is confined to API-specific layers, unrelated API knowledge mostly survives, so the same model could absorb many independent API updates without the usual catastrophic interference.
  • The number of frozen common layers and the number of edited specific layers form a practical dial: more freezing raises specificity but lowers portability, while more edited layers boosts accuracy but lowers specificity.
  • Since EDAPIBench is constructed by an automated pipeline, it can be regenerated for newly released LLMs and newly deprecated APIs, giving an evaluation platform that tracks library evolution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same gradient-mask recipe could plausibly extend to other single-line code-knowledge corrections — renamed functions, new optional parameters, security patches — so library maintainers might precompute per-API editing recipes and ship them with releases; the paper does not explore this.
  • The common/specific layer partition implies a model can host many targeted updates simultaneously, each touching a different subset of layers; whether edits accumulate without interference is a natural test the paper leaves open.
  • EDAPIBench measures single-line completions, so an editor's inference is that a multi-line, functionally grounded extension would be the strictest next test of whether the specificity gains reflect preserved behavior rather than preserved token patterns.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces EDAPIBench, a benchmark for evaluating model editing of deprecated API knowledge in three code LLMs. It applies ten existing editing methods and finds the parameter-efficient fine-tuning method AdaLoRA strong on Effectiveness, Generalization, and Portability but weak on Specificity. To remedy this, the authors propose AdaLoRA-L, which uses gradient-based layer importance scores to designate 'Common API Layers' (excluded from editing) and 'Specific API Layers' (edited). They report large Specificity improvements while preserving other metrics, and attribute the gain to restricting edits to API-specific layers.

Significance. If the central claim holds, the paper would make a useful contribution: a systematically constructed, needs-editing-filtered benchmark for deprecated-API editing, a broad comparison of ten methods, and a lightweight layer-restriction strategy with a concrete mechanism. The benchmark construction is careful in several respects: the need-to-edit filter, per-instance construction of Generalization/Portability/Specificity data, API-exact-match metric, and median-of-five reporting. The proposed method is also simple and computationally cheap. However, the evidence for the mechanistic claim—that the Common/Specific layer distinction, rather than the reduced parameter-update budget or test-set-tuned hyperparameters, drives the Specificity improvement—is not yet established. The reported gains are therefore plausible but currently underdetermined.

major comments (4)
  1. [§5.1 and Table 6] The headline improvements are selected on the evaluation set. §5.1 describes sweeping the number of frozen Common API Layers (0–12) and the number of edited Specific API Layers (1–10) on EDAPIBench and choosing final values by balancing all metrics; Table 6 then reports AdaLoRA-L on the same benchmark. Thus the Specificity gains of 836.2%, 33.5%, and 310.2% are fitted values rather than independent estimates. In addition, §4.3.1 computes layer importance scores on the same editing instances used for the Effectiveness/Generalization/Portability evaluation. Please re-select layers on a held-out split or via nested cross-validation and report performance on instances not used for layer selection, or explicitly show that conclusions are unchanged under a random split.
  2. [§4.3.2 and Table 6] No random-layer control is provided. AdaLoRA-L restricts edits to 8 layers while AdaLoRA edits all layers, so even a random selection of 8 layers may improve Specificity by reducing the total parameter-update budget. A comparison with the same number of randomly selected layers (and, ideally, with an equal-budget AdaLoRA using lower rank) is necessary to support the qualitative interpretation that 'Common API Layers store general knowledge' and that the Specificity gain is due to editing the 'right' layers rather than simply editing fewer layers.
  3. [§4.3.1] The identification of Common and Specific API Layers is not specified precisely enough to be replicated or tested. The paper states that Common layers are those with 'high importance across all APIs' and Specific layers are those with 'high importance only for the target API,' but it does not define a threshold, a ranking criterion, or an algorithmic procedure for classifying layers. Because this classification is the core of the proposed method, please formalize it (e.g., define the score distribution, a percentile or gap-based threshold, and the exact selection rule for each target API).
  4. [§4.3.2 and Table 6] The statistical-significance claims are broader than the reported results. In Table 6, several Portability improvements are not marked as significant—e.g., Qwen2.5-Coder Portability AEM 76.3 vs 71.4 has no asterisk, and DeepSeek-Coder Portability AEM 75.3 vs 58.0 has no asterisk—yet the text says 'in both Portability and Specificity ... statistical tests confirm that these improvements are significant.' Please report per-cell significance and effect sizes, and soften the claim to match the starred entries.
minor comments (6)
  1. [Abstract and §3.2] The third LLM is named inconsistently: the abstract says 'CodeGemma,' while the introduction, §3.2, and all tables use 'StarCoder2.' Please correct the abstract.
  2. [Tables 3–5] The header 'Pre-editd' should be 'Pre-edit' or 'Pre-edited.'
  3. [§3.3 Step 2] The filtering step says the model completes each prompt 'three times with temperature set to 0 (using greedy sampling).' Temperature 0 is deterministic, so three completions are identical; clarify whether a nonzero temperature or nucleus sampling was actually used, or state that the three repetitions are a safeguard against implementation-level nondeterminism.
  4. [§3.3 Step 3] Minor wording: 'randomly select another editing instances' should be 'another editing instance.' Also, please state the fraction of instances for which Portability data could not be constructed because only one instance existed for the target API.
  5. [§3.3 and Figure 2] The description of GPT-4.1 rephrasing is clear, but the figure labels '2.1' and '2.2' are not explained in the caption. Please align the caption with the numbered steps.
  6. [§3.3] The paper calls EDAPIBench construction 'fully automated,' but it adopts manual API mappings from Wang et al. [49] and uses GPT-4.1 for rephrasing. Clarify that the automation applies to the filtering and instance-construction pipeline after those inputs are supplied.

Circularity Check

2 steps flagged

No self-citation circularity, but AdaLoRA-L's headline gains are partly fitted to the evaluation benchmark: layer selection uses the same EDAPIBench instances and the freeze/edit layer counts are tuned on the full benchmark before being reported as improvements.

specific steps
  1. fitted input called prediction [Section 4.3.1 (Gradient Computation / Layer Importance Scoring) vs. Section 4.3.2 (Table 6)]
    "For each editing instance, we have the LLM perform code completion on the editing input. We then calculate the loss based on the output—specifically, we calculate the loss on the target API token ... Next, we backpropagate this loss to calculate the gradient for each editable parameter across all layers. ... For each editing instance, we calculate the importance scores of all layers. Then, for each API, we average the scores across all its editing instances to obtain API-specific layer importance scores"

    The layer-importance scores that determine which layers AdaLoRA-L edits are computed by backpropagating the loss on the target API token for each EDAPIBench editing instance. The same editing instances (and their per-API averages) are the Effectiveness data on which Table 6 reports near-100% AEM, and the Generalization/Portability inputs are derived from or share the same target-API instances. Thus the reported gains are not out-of-sample predictions: the edit locations are fitted to the very instances and target tokens used as ground truth in the evaluation. This does not make the result equal by construction (gradient magnitude does not guarantee post-edit success), but it removes the independence needed to interpret the comparison as evidence for the 'Common/Specific layer' claim.

  2. fitted input called prediction [Section 5.1 (hyperparameter selection) vs. Section 4.3.2/Table 6]
    "Based on these experimental results and balancing all performance dimensions, we finalize the hyperparameters for the three models as follows: the number of frozen Common API Layers is set to 8 for DeepSeek-Coder and Qwen2.5-Coder, and 10 for StarCoder2; the number of edited Specific API Layers is uniformly set to 8 across all models."

    The number of frozen Common API Layers and edited Specific API Layers is selected after sweeping on the full EDAPIBench (Figures 7 and 8) using the same Effectiveness, Generalization, Portability, and Specificity metrics that Table 6 then reports as AdaLoRA-L's improvements. The headline Specificity gains (836.2%, 33.5%, 310.2%) are therefore partly produced by selecting the configuration that maximizes or balances those same metrics; the evaluation is not a held-out test of the configuration. This is a fitted parameter presented as a discovered result rather than a forced mathematical equivalence.

full rationale

Most of the derivation chain is non-circular: EDAPIBench's labels come from real GitHub functions and externally published deprecated-API mappings; the ten baseline results are independent; and AdaLoRA-L's gradient scoring is a well-defined procedure not defined in terms of the evaluation metrics. There are no load-bearing self-citations and no imported uniqueness or ansatz. The circularity is concentrated in the evaluation of AdaLoRA-L: the importance scores and layer counts are fitted on the same instances and metrics used to report the headline gains. A random-layer control and a held-out configuration selection would resolve the ambiguity; their absence is primarily a validity threat rather than a definitional collapse. Score 4 reflects partial in-sample fitting rather than full circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The central claim rests on the quality of EDAPIBench labels (inherited API mappings), on the assumption that gradient importance localizes API-specific knowledge, and on several benchmark-construction choices (GPT-4.1 rephrasing, CodeBERT similarity, greedy-completion filters). The two layer-count hyperparameters are fitted to the evaluation set.

free parameters (5)
  • Number of frozen Common API Layers (per model) = 8, 8, 10 for Qwen2.5-Coder, DeepSeek-Coder, StarCoder2
    Selected by sweeping 0–12 on EDAPIBench (Section 5.1, Fig. 7) then used for the headline results in Table 6; the choice directly tunes the specificity/portability trade-off.
  • Number of edited Specific API Layers = 8 for all three models
    Selected by sweeping 1–10 on EDAPIBench (Section 5.1, Fig. 8); increasing this improves Effect/Gen/Port but lowers Specificity.
  • Number of non-target specificity inputs per editing instance = 5
    Hand-chosen in Section 3.3 Step 3 to construct Specificity data; larger/smaller values would change the specificity estimates.
  • Number of greedy completions required for 'needs editing' filter = 3
    Hand-chosen filtering rule (Section 3.3 Step 2); a stricter/looser threshold changes which instances enter the benchmark.
  • Rephrasing rule count for Generalization data = 14
    Adopted from Yu et al. [57] minus 4 inapplicable rules; GPT-4.1 applies these to create generalization inputs (Section 3.3 Step 3).
axioms (6)
  • domain assumption The 145 deprecated→up-to-date API mappings from Wang et al. [49] are correct and complete for the 8 libraries and version ranges studied.
    Adopted as ground truth in Section 3.3 Step 1 without independent verification; any mapping error propagates to all labels and scores.
  • domain assumption The layer importance score (average squared gradient magnitude on the target API token) identifies layers whose edit transfers API-specific knowledge rather than general knowledge.
    Section 4.3.1; assumed from pruning literature [35,63]; the distinction between Common and Specific API Layers is not validated by an external criterion.
  • domain assumption GPT-4.1 rephrasing with 14 code-transformation rules preserves the semantic equivalence of code while increasing syntactic difference.
    Generalization data rely on this (Section 3.3 Step 3); no automated semantic-equivalence check is described.
  • domain assumption CodeBERT embedding distance is a valid proxy for 'unrelatedness' of code inputs for Specificity.
    Specificity inputs are the 5 nearest CodeBERT neighbors of each editing input (Section 3.3 Step 3); if embedding similarity does not align with output-overlap relevance, the specificity measure is mis-calibrated.
  • domain assumption The target up-to-date API line in each real-world function is the intended completion; the preceding lines provide sufficient context for a single-line completion.
    The benchmark design (trimming functions to lines before the API call, Section 3.3 Step 1) assumes the completion task is well-posed from the prompt alone.
  • domain assumption Greedy decoding with three repeats identifies stable 'deprecated' behavior.
    Step 2 filtering; the paper itself reports non-determinism even at temperature 0 (Threats of Validity), so this filter may include/exclude borderline instances arbitrarily.

pith-pipeline@v1.3.0-alltime-deepseek · 24521 in / 22803 out tokens · 198077 ms · 2026-08-04T06:43:45.880636+00:00 · methodology

0 comments
read the original abstract

Pre-trained or fine-tuned on large code corpora, Large Language Models (LLMs) have demonstrated strong performance in code completion tasks. However, their embedded knowledge is constrained by the timeliness of training data, which often includes code using deprecated APIs. Consequently, LLMs frequently generate deprecated APIs that will no longer be supported in future versions of third-party libraries. While retraining LLMs on updated codebases could refresh their API knowledge, this approach is computationally expensive. Recently, lightweight model editing methods have emerged to efficiently correct specific knowledge in LLMs. However, it remains unclear whether these methods can effectively update deprecated API knowledge and enable edited models to generate up-to-date APIs. To address this gap, we conduct the first systematic study applying 10 state-of-the-art model editing techniques to update deprecated API knowledge in three LLMs: Qwen2.5-Coder, CodeGemma, and DeepSeek-Coder. We introduce EDAPIBench, a dedicated benchmark featuring over 70 deprecated APIs from 8 popular Python libraries, with more than 3,000 editing instances. Our results show that the parameter-efficient fine-tuning method AdaLoRA achieves the best performance in enabling edited models to generate correct, up-to-date APIs, but falls short in Specificity (i.e., the editing influences untargeted knowledge). To resolve this, we propose AdaLoRA-L, which defines "Common API Layers" (layers within the LLMs with high importance across all APIs, storing general knowledge and excluded from editing) and restricts edits exclusively to "Specific API Layers" (layers with high importance only for the target API, storing the API-specific knowledge). Experimental results demonstrate that AdaLoRA-L significantly improves Specificity while maintaining comparable performance across other evaluation metrics.

Figures

Figures reproduced from arXiv: 2511.21022 by Alex X. Liu, Guancheng Lin, Jacky Keung, Xiao Yu, Xing Hu, Xin Xia.

Figure 1
Figure 1. Figure 1: The construction process of EDAPIBench. 2024, covering source code, text-code grounding data, synthetic data, and other relevant datasets. StarCoder2 (3B) [29]: Developed by BigCode and released in January 2024, it is trained on 17 programming languages from The Stack v2 [29] up to September 2023, using the fill-in-the-middle objective and covering over 3 trillion tokens. DeepSeek-Coder (1.3B) [10]: Develo… view at source ↗
Figure 2
Figure 2. Figure 2: The construction process of generalization data and specificity data in [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The distribution of the number of target APIs from different libraries in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The average time cost (seconds) of the model [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: The identification process of Specific API Layers and Common API Layers. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The impact of the number of frozen Common API Layers on AdaLoRA-L. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The impact of the number of edited Specific API Layers on AdaLoRA-L. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 13
Figure 13. Figure 13: The three benchmarks similar to our EDAPIBench. 7 Related Works 7.1 API Evolution and LLM API Knowledge Updating Prior studies [34, 41, 42, 44] have examined the motivations behind API deprecation and how client developers respond to these changes. Common reasons for deprecating APIs include improving code readability, reducing redundancy, addressing poor coding practices, and fixing functional bugs. Depr… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Q-ARE: An Evaluation Dataset for Query Based API Recommendation

    cs.SE 2026-05 unverdicted novelty 7.0

    Q-ARE dataset and metrics reveal that existing API recommendation methods and LLMs degrade sharply on multi-level invocation chains.

  2. Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs

    cs.SE 2026-06 unverdicted novelty 6.0

    CURE applies contrastive unlearning to reduce deprecated API usage in code LLMs and improve correct replacements on a benchmark dataset while preserving general performance.

Reference graph

Works this paper leans on

67 extracted references · 21 linked inside Pith · cited by 2 Pith papers

  1. [1]

    d.].API Lifecycle Stages

    [n. d.].API Lifecycle Stages. https://developers.meetmarigold.com/engage/terms/versioning-deprecation/#api-lifecycle- stages

  2. [2]

    d.].GPT-4.1

    [n. d.].GPT-4.1. https://openai.com/index/gpt-4-1/

  3. [3]

    d.].Our source code

    [n. d.].Our source code. https://figshare.com/s/f18cf1ee0f8558b84d97

  4. [4]

    d.].Pytorch: A python package that provides tensor computation and deep neural networks

    [n. d.].Pytorch: A python package that provides tensor computation and deep neural networks. https://pytorch.org/

  5. [5]

    Aakash Bansal, Sakib Haque, and Collin McMillan. 2021. Project-level encoding for neural source code summarization of subroutines. In2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). IEEE, 253–264

    2021

  6. [6]

    Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. 2024. Evaluating the ripple effects of knowledge editing in language models.Transactions of the Association for Computational Linguistics12 (2024), 283–298

    2024

  7. [7]

    Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2024. Alphaedit: Null-space constrained knowledge editing for language models.arXiv preprint arXiv:2410.02355(2024)

    Pith/arXiv arXiv 2024

  8. [8]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. arXiv:2002.08155 [cs.CL]

    Pith/arXiv arXiv 2020

  9. [9]

    Jian Gu, Aldeida Aleti, Chunyang Chen, and Hongyu Zhang. 2024. Neuron Patching: Semantic-based Neuron-level Language Model Repair for Code Generation. arXiv:2312.05356 [cs.SE] https://arxiv.org/abs/2312.05356

    Pith/arXiv arXiv 2024

  10. [10]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence. arXiv preprint arXiv:2401.14196(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. 2023. Aging with grace: Lifelong model editing with discrete key-value adaptors.Advances in Neural Information Processing Systems36 (2023), 47934–47959

    2023

  12. [12]

    2013.Benjamini–Hochberg Method

    Winston Haynes. 2013.Benjamini–Hochberg Method. Springer New York, New York, NY, 78–78

    2013

  13. [13]

    Jiwoo Hong, Noah Lee, and James Thorne. 2024. Orpo: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07691(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    André Hora, Romain Robbes, Nicolas Anquetil, Anne Etien, Stéphane Ducasse, and Marco Tulio Valente. 2015. How do developers react to api evolution? the pharo ecosystem case. In2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 251–260

    2015

  15. [15]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al

  16. [16]

    Mingzhe Hu and Yu Zhang. 2023. An empirical study of the Python/C API on evolution and bug patterns.Journal of Software: Evolution and Process35, 2 (2023), e2507. , Vol. 1, No. 1, Article . Publication date: November 2026. 20 G. Lin et al

    2023

  17. [17]

    Baixiang Huang, Canyu Chen, Xiongxiao Xu, Ali Payani, and Kai Shu. 2024. Can Knowledge Editing Really Correct Hallucinations?arXiv preprint arXiv:2410.16251(2024)

    Pith/arXiv arXiv 2024

  18. [18]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2. 5-Coder Technical Report.arXiv preprint arXiv:2409.12186(2024)

    Pith/arXiv arXiv 2024

  19. [19]

    Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. Codesearchnet challenge: Evaluating the state of semantic code search.arXiv preprint arXiv:1909.09436(2019)

    Pith/arXiv arXiv 2019

  20. [20]

    Maliheh Izadi, Jonathan Katzy, Tim Van Dam, Marc Otten, Razvan Mihai Popescu, and Arie Van Deursen. 2024. Language models for code completion: A practical evaluation. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

    2024

  21. [21]

    Raula Gaikovina Kula, Ali Ouni, Daniel M German, and Katsuro Inoue. 2018. An empirical study on the impact of refactoring activities on evolving client-used apis.Information and Software Technology93 (2018), 186–199

    2018

  22. [22]

    Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension.arXiv preprint arXiv:1706.04115(2017)

    Pith/arXiv arXiv 2017

  23. [23]

    Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. 2024. Pmet: Precise model editing in a transformer. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18564–18572

    2024

  24. [24]

    Xiaopeng Li, Shangwen Wang, Shasha Li, Jun Ma, Jie Yu, Xiaodong Liu, Jing Wang, Bin Ji, and Weimin Zhang. 2025. Model Editing for LLMs4Code: How Far are we? (2025), 937–949

    2025

  25. [25]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. InText summarization branches out. 74–81

    2004

  26. [26]

    Mario Linares-Vásquez, Gabriele Bavota, Carlos Bernal-Cárdenas, Massimiliano Di Penta, Rocco Oliveto, and Denys Poshyvanyk. 2013. Api change and fault proneness: A threat to the success of android apps. InProceedings of the 2013 9th joint meeting on foundations of software engineering. 477–487

    2013

  27. [27]

    Shuhan Liu, Xing Hu, Kerui Huang, Xiaohu Yang, David Lo, and Xin Xia. 2025. Improving Code LLM Robustness to Prompt Perturbations via Layer-Aware Model Editing.arXiv preprint arXiv:2507.16407(2025)

    arXiv 2025

  28. [28]

    Zeyu Leo Liu, Shrey Pandit, Xi Ye, Eunsol Choi, and Greg Durrett. 2024. Codeupdatearena: Benchmarking knowledge editing on api updates.arXiv preprint arXiv:2407.06249(2024)

    Pith/arXiv arXiv 2024

  29. [29]

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, et al . 2024. StarCoder 2 and The Stack v2: The Next Generation. arXiv:2402.19173 [cs.SE]

    Pith/arXiv arXiv 2024

  30. [30]

    Tyler McDonnell, Baishakhi Ray, and Miryung Kim. 2013. An empirical study of api stability and adoption in the android ecosystem. In2013 IEEE International Conference on Software Maintenance. IEEE, 70–79

    2013

  31. [31]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. Advances in neural information processing systems35 (2022), 17359–17372

    2022

  32. [32]

    Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022. Mass-editing memory in a transformer.arXiv preprint arXiv:2210.07229(2022)

    Pith/arXiv arXiv 2022

  33. [33]

    Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems37 (2024), 124198–124235

    2024

  34. [34]

    Ariana Mirian, Nikunj Bhagat, Caitlin Sadowski, Adrienne Porter Felt, Stefan Savage, and Geoffrey M Voelker. 2019. Web feature deprecation: a case study for chrome. In2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 302–311

    2019

  35. [35]

    Pavlo Molchanov, Arun Mallya, Stephen Tyree, Iuri Frosio, and Jan Kautz. 2019. Importance estimation for neural network pruning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11264–11272

    2019

  36. [36]

    Shuyin Ouyang, Jie M Zhang, Mark Harman, and Meng Wang. 2023. LLM is Like a Box of Chocolates: the Non- determinism of ChatGPT in Code Generation.arXiv e-prints(2023), arXiv–2308

    2023

  37. [37]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318

    2002

  38. [38]

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4.arXiv preprint arXiv:2304.03277(2023)

    Pith/arXiv arXiv 2023

  39. [39]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

    2023

  40. [40]

    Romain Robbes, Mircea Lungu, and David Röthlisberger. 2012. How do developers react to API deprecation? The case of a Smalltalk ecosystem. InProceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. 1–11

    2012

  41. [41]

    Anand Ashok Sawant, Mauricio Aniche, Arie van Deursen, and Alberto Bacchelli. 2018. Understanding developers’ needs on deprecation as a language feature. InProceedings of the 40th international conference on software engineering. 561–571. , Vol. 1, No. 1, Article . Publication date: November 2026. Lightweight Model Editing for LLMs to Correct Deprecated A...

    2018

  42. [42]

    Anand Ashok Sawant, Guangzhe Huang, Gabriel Vilen, Stefan Stojkovski, and Alberto Bacchelli. 2018. Why are features deprecated? an investigation into the motivation behind deprecation. In2018 IEEE international conference on software maintenance and evolution (ICSME). IEEE, 13–24

    2018

  43. [43]

    Anand Ashok Sawant, Romain Robbes, and Alberto Bacchelli. 2016. On the reaction to deprecation of 25,357 clients of 4+ 1 popular Java APIs. In2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 400–410

    2016

  44. [44]

    Anand Ashok Sawant, Romain Robbes, and Alberto Bacchelli. 2019. To react, or not to react: Patterns of reaction to API deprecation.Empirical Software Engineering24, 6 (2019), 3824–3870

    2019

  45. [45]

    Chenmien Tan, Ge Zhang, and Jie Fu. 2024. Massive Editing for Large Language Models via Meta Learning. arXiv:2311.04661 [cs.CL] https://arxiv.org/abs/2311.04661

    Pith/arXiv arXiv 2024

  46. [46]

    Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S Yu. 2018. Improving automatic source code summarization via deep reinforcement learning. InProceedings of the 33rd ACM/IEEE international conference on automated software engineering. 397–407

    2018

  47. [47]

    Chenlong Wang, Zhaoyang Chu, Zhengxiang Cheng, Xuyi Yang, Kaiyue Qiu, Yao Wan, Zhou Zhao, Xuanhua Shi, and Dongping Chen. 2025. CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale. arXiv preprint arXiv:2502.16645(2025)

    Pith/arXiv arXiv 2025

  48. [48]

    Chaozheng Wang, Junhao Hu, Cuiyun Gao, Yu Jin, Tao Xie, Hailiang Huang, Zhenyu Lei, and Yuetang Deng. 2023. How practitioners expect code completion?. InProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1294–1306

    2023

  49. [49]

    Chong Wang, Kaifeng Huang, Jian Zhang, Yebo Feng, Lyuye Zhang, Yang Liu, and Xin Peng. 2024. Llms meet library evolution: Evaluating deprecated api usage in llm-based code completion.arXiv preprint arXiv:2406.09834(2024)

    Pith/arXiv arXiv 2024

  50. [50]

    Chaozheng Wang, Zezhou Yang, Shuzheng Gao, Cuiyun Gao, Ting Peng, Hailiang Huang, Yuetang Deng, and Michael Lyu. 2025. RAG or Fine-tuning? A Comparative Study on LCMs-based Code Completion in Industry. InProceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 93–104

    2025

  51. [51]

    Jiawei Wang, Li Li, Kui Liu, and Haipeng Cai. 2020. Exploring how deprecated python library apis are (not) handled. In Proceedings of the 28th acm joint meeting on european software engineering conference and symposium on the foundations of software engineering. 233–244

    2020

  52. [52]

    Peng Wang, Ningyu Zhang, Bozhong Tian, Zekun Xi, Yunzhi Yao, Ziwen Xu, Mengru Wang, Shengyu Mao, Xiaohan Wang, Siyuan Cheng, et al. 2023. Easyedit: An easy-to-use knowledge editing framework for large language models. arXiv preprint arXiv:2308.07269(2023)

    Pith/arXiv arXiv 2023

  53. [53]

    Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. 2024. Knowledge editing for large language models: A survey.Comput. Surveys57, 3 (2024), 1–37

    2024

  54. [54]

    Ying Wang, Bihuan Chen, Kaifeng Huang, Bowen Shi, Congying Xu, Xin Peng, Yijian Wu, and Yang Liu. 2020. An empirical study of usages, updates and risks of third-party libraries in java projects. In2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 35–45

    2020

  55. [55]

    Robert F Woolson. 2005. Wilcoxon signed-rank test.Encyclopedia of biostatistics8 (2005)

    2005

  56. [56]

    Pengcheng Yin, Bowen Deng, Edgar Chen, Bogdan Vasilescu, and Graham Neubig. 2018. Learning to mine aligned code and natural language pairs from stack overflow. InProceedings of the 15th international conference on mining software repositories. 476–486

    2018

  57. [57]

    Shiwen Yu, Ting Wang, and Ji Wang. 2022. Data augmentation by program transformation.Journal of Systems and Software190 (2022), 111304

    2022

  58. [58]

    Xiao Yu, Lei Liu, Xing Hu, Jacky Wai Keung, Jin Liu, and Xin Xia. 2024. Fight Fire with Fire: How Much Can We Trust ChatGPT on Source Code-Related Tasks?IEEE Transactions on Software Engineering50, 12 (2024), 3435–3453

    2024

  59. [59]

    Xian Zhan, Tianming Liu, Lingling Fan, Li Li, Sen Chen, Xiapu Luo, and Yang Liu. 2021. Research on third-party libraries in android apps: A taxonomy and systematic literature review.IEEE Transactions on Software Engineering48, 10 (2021), 4181–4213

    2021

  60. [60]

    Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen

  61. [61]

    Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, et al. 2024. A Comprehensive Study of Knowledge Editing for Large Language Models.arXiv preprint arXiv:2401.01286(2024)

    Pith/arXiv arXiv 2024

  62. [62]

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. Adalora: Adaptive budget allocation for parameter-efficient fine-tuning.arXiv preprint arXiv:2303.10512 (2023)

    Pith/arXiv arXiv 2023

  63. [63]

    Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2022. Platon: Pruning large transformer models with upper confidence bound of weight importance. InInternational conference on , Vol. 1, No. 1, Article . Publication date: November 2026. 22 G. Lin et al. machine learning. PMLR, 26809–26823

    2022

  64. [64]

    Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts, and Danqi Chen. 2023. Mquake: Assessing knowledge editing in language models via multi-hop questions.arXiv preprint arXiv:2305.14795(2023)

    Pith/arXiv arXiv 2023

  65. [65]

    Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. 2020. Modifying memories in transformer models.arXiv preprint arXiv:2012.00363(2020). , Vol. 1, No. 1, Article . Publication date: November 2026

    Pith/arXiv arXiv 2020

  66. [2022]

    Lora: Low-rank adaptation of large language models.ICLR1, 2 (2022), 3

    2022

  67. [2023]

    InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2471–2484

    2023