{"id":"a0ea1c58-fc99-415e-8d35-49554213e61c","arxiv_id":"2501.13669","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A Synaptic-Intelligence-style importance penalty, scaled per layer, reduces catastrophic forgetting during LoRA fine-tuning of GPT-J and LLaMA-3 on three domain tasks.","lead":"This paper proposes a fine-tuning method that records which adapter parameters matter most for a model's general abilities, then applies a weighted penalty to keep those parameters stable while learning a domain task. It reports better retention of general knowledge and about 20 times faster importance estimation than a Fisher-matrix baseline.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core claim is supported only by Pile PPL on the same corpus used to compute Ω_i; lower PPL may just reflect preservation of the Pile-tuned state, not of general pretrained knowledge.","rationale":"The reader's CONDITIONAL verdict is appropriate. The primary weakness is not a formal inconsistency in the optimization but a circularity in the evaluation: general ability is measured with Pile PPL, and Pile is also the corpus on which Ω_i is recorded and toward which θ_ν is optimized. This makes the reported PPL gains expected even if the method only preserves a Pile-adapted state rather than broad pretrained knowledge. The proposed test is feasible and directly separates 'preserving the Pile distribution' from 'preserving general knowledge.' If it fails, the abstract's claim should be weakened to in-distribution Pile perplexity preservation; if it passes, the conditional can be lifted.","tokens_in":16852,"tokens_out":7439,"duration_ms":73243,"concrete_test":"For LLaMA-3-3B and GPT-J, evaluate HLoRA, Base, LoRA(µ), and LoRA(ν+µ) on held-out general-knowledge benchmarks not used in the ν stage: MMLU (5-shot), HellaSwag, WinoGrande, and ARC-Challenge, with 3 seeds and report means/standard errors. Also report the θ_ν checkpoint's PPL/accuracy on these benchmarks before µ tuning. If HLoRA's external accuracies are statistically indistinguishable from or worse than LoRA(µ) while its Pile PPL is lower, the central forgetting-mitigation claim is an artifact of optimizing the Pile corpus; if HLoRA retains significantly higher external accuracy, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical core of the paper is Table 1, where 'general ability' is PPL on Pile (Section 5.2). But Pile is not an independent evaluator of general knowledge: the element-wise importance Ω_i in Eq. 13 is accumulated from gradients of a LoRA fine-tuning run on Pile (Eqs. 6 and 12), and Eq. 14 regularizes toward θ_ν, the checkpoint reached after that same Pile run. Thus minimizing L_reg is essentially minimizing deviation from a Pile-optimized adapter, so a low final Pile PPL is partly by construction. This is especially visible for LLaMA-3: Ours reports PPL 4.64 versus Base 4.94, but the method has been trained on Pile in the ν stage while Base has not; the comparison conflates continued training on Pile with retention of pretrained knowledge. No held-out benchmark (MMLU, HellaSwag, etc.) is reported. The result therefore does not yet establish mitigation of catastrophic forgetting; it may establish only preservation of the Pile distribution and the importance-recording loss.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HLoRA, a regularization method for LoRA-based fine-tuning of LLMs. It first records element-wise parameter importance during a preliminary fine-tuning stage on a general corpus (Pile), using a path-integral/Synaptic-Intelligence-style formula adapted to the LoRA parameterization. During subsequent domain fine-tuning, it adds a quadratic regularization term that penalizes deviations from the post-ν checkpoint, weighted element-wise by the recorded importance and layer-wise by an L2-norm softmax coefficient. Experiments on GPT-J-6B and LLaMA-3-3B over SciQ, PiQA, and MedMCQA compare the method with LoRA, rsLoRA, EWCLoRA, and sequential fine-tuning baselines, and report lower Pile perplexity, competitive or better domain accuracy, and large speed and storage savings for the importance computation relative to EWCLoRA.","tokens_in":17081,"tokens_out":5873,"duration_ms":52719,"significance":"If the empirical results hold, the method offers a cheap alternative to Fisher-based regularization for LoRA fine-tuning: parameter importance is obtained as a byproduct of the ν-stage training rather than through an additional backward pass, and the storage requirement is per-LoRA-element rather than a full Fisher matrix. The derivation in Eqs. (6)-(13) is a reasonable adaptation of Synaptic Intelligence to the LoRA parameterization, and the layer-wise reweighting in Eq. (15) is a simple, interpretable addition. The main weakness is that the current experimental design does not isolate retention of pretrained general knowledge from continued training on the importance-recording distribution, so the central claim of mitigating catastrophic forgetting is not yet fully established.","major_comments":[{"comment":"The general-ability metric is PPL on Pile, which is exactly the corpus used in the ν importance-recording stage, and Eq. (14) regularizes toward θ_ν, the checkpoint obtained after Pile fine-tuning. Thus the reported improvement for Ours (LLaMA-3 PPL 4.64 vs. Base 4.94) may simply reflect additional training on Pile during the ν stage rather than preservation of pretrained general knowledge. This is especially problematic because Base is never trained on Pile, so the comparison is not apples-to-apples. The paper should report at least one held-out general benchmark that is not used in the ν stage (e.g., MMLU, HellaSwag, ARC, or a disjoint language-modeling corpus); without such evidence, the central claim of mitigating catastrophic forgetting is not established.","section":"Section 5.2, Table 1"},{"comment":"The claimed 20x speedup compares the EWCLoRA Fisher computation on 20,000 Pile samples with the authors' 5-epoch ν-stage LoRA training, but the paper does not state how many Pile tokens or examples were used in the 5-epoch ν stage for HLoRA. If the two computations use very different amounts of data, the ratio is not a fair comparison. Please report the data volume, batch size, total optimizer steps, and hardware for both methods, and specify the numerical precision used for storing the importance matrix Ω, since the storage comparison (10%-15%) depends on this.","section":"Section 6.2, Figure 5"},{"comment":"The notation Σ_{ν<t<µ} is not defined and appears to mix task indices with time indices; if the sum is intended to accumulate per-step contributions during µ training, it should be written as a sum over optimization steps t. Additionally, in LoRA the full weight θ_t differs from θ_ν only through the product B_tA_t, so the regularization should explicitly state whether the sum runs over entries of the LoRA matrices or over entries of the full weight matrix, since this affects how Ω_i is indexed, stored, and compared with methods such as EWCLoRA.","section":"Eq. (14), Section 4.2"},{"comment":"No seed information or error bars are reported for Table 1, so it is unclear whether the point estimates are means over multiple runs. Figure 3 reports independent-samples t-tests, but the text does not state what the samples in the violin plots are (e.g., multiple seeds, multiple test batches, or bootstrap resamples); if they are test-batch perplexities from a single run, a t-test does not support a claim about method-level differences. Please report the number of seeds, standard deviations, and confidence intervals for the main results.","section":"Section 5.2, Table 1, Figure 3"}],"minor_comments":[{"comment":"The notation y_k should be y_{i,k} (or similar) to match the example index i in the same equation; as written, the index k is unclear.","section":"Eq. (2)"},{"comment":"The summation Σ_ν is confusing because ν denotes the single general task; clarify that the sum runs over the optimization steps or epochs of the ν-stage training.","section":"Eq. (13)"},{"comment":"The phrase 'we take Pile as the evaluation datasets for LLM general ability' should be justified, since Pile is a training corpus; using the training distribution as an evaluation set inflates apparent retention and should at least be acknowledged as a limitation.","section":"Section 5.2"},{"comment":"The text states that p-values are below 0.05 for several comparisons but does not report the actual p-values; please include them in the caption or text.","section":"Figure 3"},{"comment":"The heading 'Time Complexity' measures wall-clock time, not asymptotic complexity; rename it to something like 'Computation Time' to avoid confusion.","section":"Section 6.2"},{"comment":"The ablation row '- layer, element' is equivalent to the LoRA(ν+µ) baseline in Table 1; state this explicitly so the ablation table is self-contained.","section":"Table 2"},{"comment":"The model is described as 'LLaMA-3' in the abstract but as 'LLaMA 3.2-3B' in Section 5.2; use one consistent name and specification.","section":"Section 5.1, Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is a fairly direct adaptation of Synaptic Intelligence to LoRA with a layer-wise softmax weighting. The contribution is incremental but potentially useful if the efficiency claim is made fair and the evaluation is strengthened. The main risk is that the evaluation uses Pile perplexity, which is the same distribution used for importance recording, so the headline result may be partly by construction. If the authors add held-out general benchmarks and seed-level statistics, and if they clarify the data volumes behind the 20x speedup claim, I would be willing to accept a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a workmanlike incremental paper, not a breakthrough. The novel bit is adapting Synaptic Intelligence importance to LoRA's low-rank updates and adding a layer-wise softmax weight; the derivation in Eqs. 7-13 is internally consistent and the layer weighting is a simple, sensible extension. The efficiency advantage over a Fisher-based method is real in kind: SI needs one pass to accumulate gradients while Fisher needs sampling. So if the method works, it is a useful cheap regularizer for LoRA fine-tuning.\n\nThe problem is the evidence for \"works.\" General ability is measured only as PPL on Pile, and Pile is exactly the corpus used in the ν stage to fine-tune the model and record importance. Eq. 14 anchors the final parameters near θ_ν, the Pile-tuned checkpoint. So a low final Pile PPL is partly by construction. The LLaMA-3 result is especially misleading: Ours reports 4.64 PPL, lower than Base's 4.94, but Ours has seen Pile during ν and Base has not. That is continued training on the test distribution, not forgetting prevention. On GPT-J, Ours (3.35) is actually worse than Base (3.28), which undercuts the \"mitigates forgetting\" claim even on the paper's own metric. No held-out benchmark such as MMLU or HellaSwag is reported. The stress-test note lands: the central claim is not yet established.\n\nOther soft spots are smaller. Table 1 has no error bars or seed counts, though Figure 3 provides some t-test violin plots for the EWCLoRA comparison. The speed/storage comparison is only against EWCLoRA, and the setup is not transparent enough—EWCLoRA is described as using 20k samples, while Ours uses 5 epochs, so the 20x figure may not be apples-to-apples. Code is promised but not provided. The ablation shows the layer-wise component helps, but the gains are a few tenths of PPL and less than 2 points of accuracy, and there is no variance estimate.\n\nTo be clear, none of this is fatal. The mechanism is plausible, the math is a legitimate extension of SI, and the efficiency story is believable. But the paper currently asks the reader to take the headline claim on a circular metric. With a held-out benchmark, seeds, and code, this could be a solid contribution.\n\nI would send this to peer review—it deserves referee time—and the first reviewer request should be: redo the general-ability eval on data the ν stage never saw, and make the complexity comparison fair. I would not cite it yet in my own work.","headline":"A plausible SI-for-LoRA adaptation whose main empirical claim rests on a circular Pile-PPL metric; needs held-out evaluation and code before it can be trusted.","tokens_in":17618,"tokens_out":4421,"would_cite":false,"duration_ms":39561,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A cheap importance-recording pass can keep a large language model's general knowledge intact while it is fine-tuned to a specialized domain.","keywords":["catastrophic forgetting","large language models","LoRA fine-tuning","parameter importance","synaptic intelligence","layer-wise regularization","perplexity","continual learning"],"falsifier":"Run the same two-phase procedure but evaluate the fine-tuned model on a broad held-out suite of general-knowledge benchmarks that do not overlap with Pile; if accuracy there drops by the same amount as unregularized LoRA while Pile perplexity stays low, the claim that importance-based regularization protects general knowledge is falsified.","tokens_in":16623,"feed_emoji":"🧠","tokens_out":11773,"duration_ms":89713,"temperature":0.7,"pith_summary":"The paper proposes a way to keep a large language model's general knowledge intact while fine-tuning it on a specialized domain. Its core move is to record, during a short pass over general text, how much each LoRA parameter contributed to reducing the language-modeling loss, and then, during domain fine-tuning, to penalize movement of the parameters that mattered most. A layer-wise weighting term lets the model protect some layers more than others. On scientific, medical, and physics QA tasks with GPT-J and LLaMA-3, the authors report lower general perplexity than unregularized LoRA and than the Fisher-based EWCLoRA baseline, with comparable or better domain accuracy, and they estimate the importance-recording step is about 20 times faster and needs 10–15% of the storage. This makes protecting general knowledge during LoRA fine-tuning a cheap, practical step.","feed_headline":"Keep LLM general knowledge through fine-tuning, 20x cheaper","feed_subtitle":"A ~1-hour importance recording replaces a ~25-hour Fisher computation and keeps general ability intact.","key_machinery":"The load-bearing object is the per-parameter importance score $\\Omega_i^{\\nu} = \\frac{\\omega_i^{\\nu}}{(\\Delta_i^{\\nu})^2 + \\xi}$, where $\\omega_i^{\\nu} = -\\int_{t_0}^{t_1} \\tilde{g}_i(t) \\tilde{\\theta}'_i(t)\\,dt$ is a path integral of the loss gradient over the LoRA update trajectory, and $\\Delta_i^{\\nu}$ is the total change of that parameter during the general-knowledge recording phase. The argument carries through a derived 'hypothetical gradient' $\\tilde{g}(t) = g_B(t)A(t) + B(t)g_A(t) - \\eta g_B(t)g_A(t)$ for the LoRA update rule, which lets the path-integral idea from full fine-tuning apply to low-rank adapters. This importance score then defines a per-layer regularizer $L_{\\text{reg},l} = \\sum_i \\Omega_i^{\\nu} (\\theta_i^t - \\theta_i^{\\nu})^2$, and the layer-wise coefficients $\\text{softmax}(\\|\\Omega_l^{\\nu}\\|_2)$ decide how strongly each layer is held near its recorded value. The whole mechanism is a quadratic safeguard around the $\\nu$-task solution, weighted by how much each parameter contributed to learning that task.","core_discovery":"The central claim is that catastrophic forgetting during LoRA fine-tuning can be alleviated without a Fisher matrix, by computing a Synaptic-Intelligence-style importance score for each LoRA parameter during a general-language recording phase and then regularizing the fine-tuning objective with a weighted quadratic penalty on those parameters. The paper further claims that treating each layer's regularization strength separately, via a softmax over the L2 norms of per-layer importance, yields better retention than a uniform penalty. Empirically, the method—called HLoRA—achieves the lowest perplexity on the Pile corpus among all compared methods on three domain QA datasets across two backbone models, while improving or matching domain accuracy; the importance-recording phase is reported to take about 1.2 hours versus roughly 26 hours for EWCLoRA and to store 1.3–3.5 GB rather than 12–23 GB.","pith_inferences":["A natural next test is whether importance scores recorded on the Pile corpus transfer to protecting general knowledge measured on entirely different benchmarks; the paper does not establish this transfer, since Pile is used both for recording and for evaluating perplexity.","The reported importance heatmaps suggest a cheaper variant: freeze or down-weight the value-projection LoRA parameters in the first four and last layers during domain fine-tuning, instead of computing full importance matrices.","The quadratic regularizer assumes the ν-task loss landscape is locally quadratic; one could test whether higher-order terms matter by checking whether the optimal regularization coefficient changes systematically as the distance between the recorded parameters and the domain-tuned parameters grows.","The layer-wise coefficients are computed from the same importance matrices, so the method could be extended to per-module or per-head regularization, e.g., protecting only attention blocks or only MLP blocks, which the paper does not explore."],"forward_implications":["A single short recording pass over general text yields importance scores that can then regularize fine-tuning on any domain task.","Removing the Fisher matrix cuts the storage footprint of importance-based regularization from 12–23 GB to 1.3–3.5 GB and the recording time from roughly 26 hours to about 1.2 hours.","Layer-wise softmax weighting means different layers receive different regularization strengths, with value-projection layers in early and final blocks flagged as most important for general knowledge.","The method retains or improves domain accuracy while lowering perplexity, so the protection of general knowledge does not come at the cost of domain adaptation.","Because the importance-recording phase is a standard LoRA training run rather than a Fisher computation, it requires no additional backpropagation pass and no samples from the model distribution."],"supporting_citations":[{"why":"Supplies the path-integral definition of per-parameter importance that the element-wise scores are based on.","marker":"Zenke et al., 2017"},{"why":"Defines the EWCLoRA baseline and the Fisher-matrix time/storage costs the paper compares against.","marker":"Xiang et al., 2024"},{"why":"Defines the LoRA parameterization and update rule used to derive the hypothetical gradient.","marker":"Hu et al., 2021"},{"why":"Provides the Pile corpus used as the ν task for importance recording and for general-ability perplexity evaluation.","marker":"Gao et al., 2020"},{"why":"Provides the GPT-J-6B backbone used in one set of experiments.","marker":"Wang & Komatsuzaki, 2021"},{"why":"Provides the LLaMA-3 backbone used in the other set of experiments.","marker":"Dubey et al., 2024"},{"why":"Provides the rsLoRA baseline compared in the general/domain ability table.","marker":"Kalajdzievski, 2023"},{"why":"Supplies the MedMCQA medical QA dataset used as a µ task.","marker":"Pal et al., 2022"},{"why":"Supplies the SciQ scientific QA dataset used as a µ task.","marker":"Welbl et al., 2017"},{"why":"Supplies the PiQA physics commonsense QA dataset used as a µ task.","marker":"Bisk et al., 2020"}],"fun_headline_variants":["LLM fine-tuning without forgetting, 20x faster","New regularizer keeps LLM knowledge, slashes costs","Say goodbye to catastrophic forgetting in fine-tuning","Element-wise importance stops LLM forgetting cheaply","Hierarchical penalties preserve general LLM ability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The empirical claim depends on Pile perplexity being a faithful measure of general ability, and it is the same corpus used to record importance, so the results do not by themselves show that other general capabilities survive fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["LLM fine-tuning without forgetting, 20x faster","New regularizer keeps LLM knowledge, slashes costs","Say goodbye to catastrophic forgetting in fine-tuning","Element-wise importance stops LLM forgetting cheaply","Hierarchical penalties preserve general LLM ability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000259,"raw_usage":{"total_tokens":1577,"prompt_tokens":931,"completion_tokens":646,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":573}},"tokens_in":547,"tokens_out":646,"duration_ms":6281,"temperature":1.0,"reasoning_tokens":573,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:43:03.375647+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same two-phase procedure but evaluate the fine-tuned model on a broad held-out suite of general-knowledge benchmarks that do not overlap with Pile; if accuracy there drops by the same amount as unregularized LoRA while Pile perplexity stays low, the claim that importance-based regularization protects general knowledge is falsified.","supporting_citations":[],"review_version":1}