{"id":"fd196651-2e8b-415c-a367-c105bec0a71f","arxiv_id":"2501.10979","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Control LLM pairs frozen and trainable transformer blocks with interpolation and a divergence loss, letting Llama-3.1 learn new math, code, and Chinese skills with less catastrophic forgetting.","lead":"The paper introduces Control LLM, a training method that adds small trainable branches to a frozen large language model and blends their outputs to reduce forgetting. The authors report gains on math, coding, and Chinese tasks with less degradation on general knowledge than full fine-tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim is undermined by comparisons against models trained from different base checkpoints; same-base superiority is not established.","rationale":"The reader's weakest_assumption focuses on the training objective L = L_task + lambda * L_divergence and the unreported lambda. That is a legitimate hyperparameter concern, but the ablation study (Table 5) already shows that even without the divergence loss, Lerp interpolation preserves MMLU far better than full-parameter tuning (58.8 vs 7.3), so the method does not critically depend on the exact lambda value. The more consequential weakness is the mismatch in base checkpoints between Control LLM and the open-source models it claims to surpass. The abstract and Section 4.2 explicitly state 'same base model', yet OpenMath2-Llama3.1-8B is from the non-instruct base, OpenCoder-8B-Instruct is from a different pretrained base, and Llama-3-SynE is from Llama-3. This makes the headline SOTA claim unverified. A concrete same-base comparison is feasible because the code and models are released and the training protocol is specified; running it would settle the claim. The reader already flagged the unfair comparison in the rationale, so this is a partial agreement rather than a new concern. The verdict remains CONDITIONAL: the architecture is plausible and well-ablated, but the central comparative claim needs a same-base experiment before it can be accepted as stated.","tokens_in":16743,"tokens_out":3673,"duration_ms":41614,"concrete_test":"Re-run the CSFT math experiment starting from Llama-3.1-8B (non-instruct) using the exact Control LLM protocol and compare directly with OpenMath2-Llama3.1-8B, Full Param Tune, and Stack Expansion initiated from the same non-instruct base. Separately, fine-tune Llama-3.1-8B-Instruct on OpenMath2 with a standard full-parameter or LoRA recipe and compare against Control LLM-Instruct. If Control LLM does not beat OpenMath2 from the non-instruct base, or if the Instruct-base baseline matches or exceeds Control LLM-Instruct, the 'SOTA among open-source models tuned from the same base model' claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims 'SOTA among open-source models tuned from the same base model', but the comparison tables do not hold the base model fixed. Control LLM is initialized from Llama-3.1-8B-Instruct (Section 4.1.2), while OpenMath2-Llama3.1-8B is tuned from the non-instruct Llama-3.1-8B base (Section 4.2.1), OpenCoder-8B-Instruct is built on the separately pre-trained OpenCoder base, and Llama-3-SynE is derived from Llama-3 rather than Llama-3.1. Starting from an instruction-tuned model confers a substantial advantage on the 'Original Capabilities' metrics (MMLU, ARC, GPQA) and can also affect new-task performance through better instruction following. The paper further selects the best checkpoint on new-task performance (Section 3.4, item 5), which can compound the comparison bias. Thus the central claim of surpassing existing methods on the same base model is not actually demonstrated by Tables 1-3. If the base-model mismatch is corrected, the reported SOTA margin may shrink or disappear, so this is a load-bearing issue for the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Control LLM, a model-upcycling method that adds a trainable expanded transformer block in parallel with each frozen pre-trained block (one every N-1 layers) and fuses their hidden states through interpolation (Lerp, Dlerp, DlerpIn, Plerp, MoE) with an optional divergence loss. The aim is to mitigate catastrophic forgetting during continuous pre-training (CPT) and continuous supervised fine-tuning (CSFT) while still learning new skills. The authors evaluate on math (OpenMath2), coding (OpenCoder), and Chinese multilingual (Llama3-SynE) tasks using Llama-3.1-8B/Instruct, reporting improved new-task performance with much smaller degradation on original capabilities than full-parameter tuning. The paper also includes step-level training curves, an ablation study of interpolation mechanisms, divergence losses, and expansion strategies, and it releases training/evaluation code and models.","tokens_in":16950,"tokens_out":4037,"duration_ms":44073,"significance":"If the central claim is valid, Control LLM would be a practical alternative to full fine-tuning for continual post-training, with useful properties for production settings: retention of broad capabilities, reduced data and compute, and no reliance on replay of original data. The paper has several concrete strengths: it reports training curves at 1k-step granularity, ablates interpolation strategies and divergence losses, compares against full-parameter and partial-parameter tuning, and releases code and models. These assets make the empirical findings checkable. However, the headline claim of state-of-the-art performance among open-source models tuned from the same base model is not supported by the current comparison tables, because key external baselines start from different base checkpoints. The internal same-base comparisons (Full Param Tune, Partial Param Tune, Stack Expansion) are informative and do support the forgetting-mitigation benefit, but the broader SOTA claim needs either new controlled experiments or a substantially narrowed statement.","major_comments":[{"comment":"The claim that Control LLM 'surpasses existing methods and achieves SOTA among open-source models tuned from the same base model' is not established by the reported comparisons. In Table 1, OpenMath2-Llama3.1 is fine-tuned from the non-instruct Llama-3.1-8B base, whereas Control LLM starts from Llama-3.1-8B-Instruct (Section 4.2.1). In Table 2, OpenCoder-8B-Instruct is built on the separately pre-trained OpenCoder base. In Table 3, Llama-3-SynE is derived from Llama-3, not Llama-3.1-8B. Starting from an instruction-tuned model can substantially raise original-capability scores (MMLU, ARC, GPQA) and may also affect new-task performance. The paper needs either same-base external baselines or a revision of the SOTA claim to what the data actually show: superiority over the internally implemented full/partial/stack baselines on the same starting checkpoint.","section":"Abstract; Section 4.1.2; Tables 1-3"},{"comment":"The checkpoint-selection protocol is a source of comparison bias. The text says 'Pick best-performing checkpoint on new tasks' after training 2-6 epochs, while OpenMath2 is reported as a single released checkpoint and the paper explicitly notes that OpenMath2 uses checkpoint averaging. Selecting the best new-task checkpoint for Control LLM, but not applying the same selection to external baselines, can inflate the reported margin. The authors should either apply the same best-checkpoint selection to every comparison model or report results at a fixed training budget with a clear protocol.","section":"Section 3.4, item 5; Tables 1-3"},{"comment":"The divergence-loss weight lambda in L = L_task + lambda * L_divergence is never reported, despite being a central hyperparameter of the method. The ablation names such as Lerp8MSE0 suggest lambda=0 in one configuration, but no lambda value is given for the main results. Since the authors release code, this is partially mitigated, but the paper should state the lambda values used for each experiment group, together with the divergence metric (MSE vs cosine) for each reported row.","section":"Section 3.4, Eq. (1); Section 4.2.3"},{"comment":"The paper's explanation that hidden-state alignment is the cause of catastrophic-forgetting mitigation is presented as a finding, but the evidence is correlational: probing analogous sentences and comparing nearest-neighbor distances shows that well-performing configurations have better-aligned hidden states. The training objective explicitly enforces such alignment, so the observed correlation is expected by construction. This does not invalidate the method, but the causal language in the contributions ('Highlight the role of hidden-state alignment in retaining prior knowledge') should be softened to a hypothesis, or supported by an intervention study that varies alignment independently of the training objective.","section":"Section 3.1; Appendix B"}],"minor_comments":[{"comment":"The sentence 'We also compare to open-source models trained on similar datasets and same base model' is inaccurate for the three listed models, as none of them is trained from the same base checkpoint as Control LLM. This should be corrected to avoid confusing readers.","section":"Section 4.1.2"},{"comment":"The table captions state 'Avg.' and the text says results are 'size-weighted averages within each task group, along with the overall mean of these group averages,' but the reader cannot verify which columns are averaged and how the size weights are computed. Please include the exact averaging formula in a footnote or appendix.","section":"Tables 1-3"},{"comment":"The ablation table uses compact names such as Lerp8MSE0, Lerp8MSE0*, Lerp8MSE0*M, and Lerp8MSEα without a key. A one-sentence explanation of each suffix (e.g., '0' means alpha=0 during training, '*' means inference at alpha=0.5, 'M' means merged weights) would greatly improve readability.","section":"Section 4.2.3"},{"comment":"There are several typos and formatting issues: 'harvness' for 'harness', 'Con-trol LLM' line breaks, and inconsistent use of 'full-parameter' vs 'full parameter'. These should be cleaned up before publication.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical core—that Control LLM mitigates forgetting relative to full-parameter tuning on the same starting checkpoint—is plausible and supported by the internal baselines and training curves. The problem is the overreach in the abstract and Section 4.2: the comparison set does not hold the base model fixed, so the SOTA claim is not backed by the evidence. This is fixable by rerunning external baselines from the same base checkpoint or by revising the claims to the internally demonstrated comparisons. I would not reject the paper on this basis, but the revision must be substantive, not cosmetic. I also recommend that the authors report lambda values and the exact checkpoint-selection protocol, since both are load-bearing for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here’s the short version: the core architecture is genuinely useful and the controlled experiments support it, but the abstract oversells the result by comparing against models trained from different base checkpoints. If you take the headline at face value you’ll be misled; if you read the tables carefully, the method still looks worth trying.\n\nWhat’s new: parallel frozen and expanded transformer blocks with interpolation (Lerp/Dlerp) and a divergence loss that keeps the expanded branch’s hidden states close to the frozen branch. That combination is not in LLaMA Pro, SOLAR, or LLaMA-MoE. The paper systematically ablates interpolation strategies, expansion placement, loss choices, and number of layers, and it ships code and models. That is real work.\n\nThe solid part: within the same base model (Llama-3.1-8B-Instruct), Control LLM clearly beats full-parameter tuning and stack expansion on the retention/learning trade-off. The training curves in the appendix show MMLU staying above 67% while Math-Hard climbs to ~38%, where full fine-tuning collapses MMLU to single digits. That is a real and reproducible effect. The multilingual CPT result on Llama-3.1-8B also shows improvement over the base without catastrophic loss.\n\nThe soft spots are in the framing. Table 1 compares against OpenMath2-Llama3.1, which starts from the non-instruct Llama-3.1-8B; Table 2 compares against OpenCoder-8B-Instruct, which is a separately pre-trained model; Table 3 compares against Llama-3-SynE, which is from Llama-3, not Llama-3.1. None of these hold the base model fixed, so the claim of \"SOTA among open-source models tuned from the same base model\" is not supported by the tables. The stress-test note is right. Also, model selection picks the best checkpoint on new-task performance (Section 3.4 item 5), which can bias results in your favor, and no error bars are reported. And lambda, the divergence-loss weight, is never reported, which hurts reproducibility.\n\nAre these fatal? No. The controlled same-base comparisons stand, and the method looks useful for anyone doing continued training or SFT without wanting to destroy existing capabilities. But the SOTA sentence needs to be rewritten or backed by same-base comparisons.\n\nMy recommendation: send it to peer review. A serious referee will ask for the same-base comparison and the lambda value, but the method and the controlled evidence deserve referee time. I’d cite it if I were working on continual learning. It’s not a paradigm shift, but it’s a solid, reproducible increment.","headline":"Useful method for mitigating catastrophic forgetting, but the SOTA claim rests on comparisons against models trained from different bases; the controlled same-base evidence is solid.","tokens_in":17513,"tokens_out":2608,"would_cite":true,"duration_ms":25847,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Control LLM proposes that inserting trainable transformer blocks beside frozen ones and aligning their hidden states with interpolation and a divergence loss lets an LLM learn new skills while retaining old ones.","keywords":["catastrophic forgetting","continuous pre-training","continuous supervised fine-tuning","model expansion","hidden-state alignment","interpolation","divergence loss","large language models"],"falsifier":"Sweep the divergence weight lambda over a wide range (from zero to very large) on a fixed base model and new-task dataset, and record new-task accuracy alongside MMLU. If no lambda value yields both near-full-fine-tuning new-task accuracy and MMLU within 5 points of the base model, then the claimed 'learn more, forget less' balance does not hold as stated. A simpler check: train on a synthetic task deliberately far from the base distribution, where hidden-state alignment would force the trainable branch to stay close to representations that cannot express the task; if the method fails there, its applicability is limited to tasks near the original distribution.","tokens_in":16523,"feed_emoji":"🧠","tokens_out":8354,"duration_ms":78821,"temperature":0.7,"pith_summary":"Control LLM is a training architecture for updating a large language model on new data without catastrophic forgetting. Instead of unfreezing the whole model, it inserts trainable copies of selected transformer blocks beside frozen originals, blends the two branches' hidden states with an interpolation weight, and adds a divergence loss that keeps the trainable branch's representations close to the frozen branch's. The paper reports that this 'learn more, forget less' recipe works for continuous pre-training and supervised fine-tuning alike: on Llama-3.1-8B it lifts math, coding, and Chinese benchmarks while holding MMLU within 4.3 points of the base model, where full fine-tuning loses more than 35 points. If the recipe holds, it offers a cheaper path to specializing models without retraining from scratch.","feed_headline":"Parallel blocks let LLMs add skills without erasing old ones","feed_subtitle":"Control LLM keeps MMLU loss under 4.3% while adding math, code, and Chinese capabilities.","key_machinery":"The central object is a dual-branch transformer block: for each selected layer, a frozen copy of the original block runs in parallel with a trainable copy, and their hidden states are fused by an interpolation function (linear, dynamic, input-conditioned, progressive, or hard MoE gating). A divergence loss, averaged over layers and weighted per token, penalizes the distance between the two branches' hidden states. This combination is what the paper claims carries the anti-forgetting effect: the interpolation keeps the trainable branch plugged into the frozen model's information flow, while the divergence loss prevents the expanded branch from drifting into representations that erase old skills.","core_discovery":"The paper's central claim is that catastrophic forgetting in LLM updates is driven by drift in hidden-state representations, and that drift can be controlled by running every selected layer as two parallel branches: a frozen pre-trained block that preserves old knowledge and a trainable expanded block that absorbs new data, with their outputs combined by interpolation and a per-layer divergence loss that penalizes separation. The authors argue this alignment is what allows the model to 'refine rather than overwrite' prior knowledge. They support the claim with experiments on math (OpenMath2), coding (OpenCoder), and multilingual (Llama-3-SynE) updates, showing that Control LLM matches or exceeds full-parameter tuning on new-task benchmarks while keeping MMLU, ARC, and GPQA near base-model levels, and that the best configuration is Concat expansion with Lerp interpolation and MSE divergence loss.","pith_inferences":["Editorial extension: A testable consequence follows from the paper's own Lerp8MSE0* ablation: training with alpha=0 and inferring at alpha=0.5 preserves old skills while learning new ones, suggesting the interpolation weight could be tuned at inference time as a retention-versus-adaptation dial without retraining.","Editorial extension: If hidden-state alignment is the active mechanism, the same frozen-plus-trainable parallel block design could transfer to vision-language or multimodal continual learning, where representation drift is also implicated in forgetting.","Editorial extension: The paper leaves the divergence weight lambda unreported; a natural follow-up is to map the lambda-versus-new-task-accuracy curve to see whether there is a principled operating point, or whether the method's success depends on a narrow range.","Editorial extension: The reported production deployment suggests the added parameters from the parallel blocks are acceptable in low-latency serving settings, which is a consequence the paper states but does not quantify."],"forward_implications":["Specializing an LLM for math, code, or a new language no longer requires retraining the whole model or replaying the original training data.","The approach works in both continuous pre-training and supervised fine-tuning, so a deployed model can be refreshed on new data without a full retraining cycle.","Original capabilities degrade by less than 4.3% on MMLU while new-task gains reach +14.4% on Math-Hard and +10% on MBPP-PLUS, a retention profile that standard fine-tuning does not approach.","Concat expansion with Lerp interpolation and MSE divergence loss is the recommended default, with the ablation showing that alignment and interpolation together, not either alone, produce the effect.","Because only the expanded blocks and interpolators are trained, the method uses substantially less compute and data than full retraining while matching open-source models tuned from the same base."],"supporting_citations":[{"why":"Defines catastrophic interference, the problem the paper targets.","marker":"(McCloskey and Cohen, 1989)"},{"why":"Provides the EWC regularization baseline that the method positions against conceptually.","marker":"(Kirkpatrick et al., 2017)"},{"why":"LLaMA Pro stack expansion is the structural baseline Control LLM adapts and compares against.","marker":"(Wu et al., 2024)"},{"why":"OpenMathInstruct-2 supplies the math CSFT dataset and the OpenMath2 open-source comparison model.","marker":"(Toshniwal et al., 2024)"},{"why":"OpenCoder supplies the coding CSFT dataset and an open-source comparison model.","marker":"(Huang et al., 2024)"},{"why":"Llama-3-SynE supplies the multilingual CPT dataset and a replay-based comparison.","marker":"(survivi, 2024)"},{"why":"Llama-3.1-8B and its Instruct variant are the base models all experiments start from.","marker":"(Dubey et al., 2024)"},{"why":"Motivates hidden representation alignment as the site where catastrophic forgetting occurs.","marker":"(Ramasesh et al., 2020)"},{"why":"LoRA represents the PEFT baseline that the method positions against for limited learning capacity.","marker":"(Hu et al., 2021)"}],"fun_headline_variants":["LLM upgrade: Add math, code, Chinese without forgetting old skills","New method prevents LLM forgetting during continued training","Control LLM: Evolve LLMs without catastrophic forgetting","Parallel blocks preserve LLM knowledge while adding new capabilities","Add math, code, and Chinese to LLMs without losing base skills"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that penalizing the trainable branch's hidden states to stay close to the frozen branch's states will not prevent the trainable branch from learning the new task; the paper provides no formal condition for this trade-off, only empirical curves, and the value of the penalty weight lambda is never reported.","fun_headline_variants_meta":{"raw":{"variants":["LLM upgrade: Add math, code, Chinese without forgetting old skills","New method prevents LLM forgetting during continued training","Control LLM: Evolve LLMs without catastrophic forgetting","Parallel blocks preserve LLM knowledge while adding new capabilities","Add math, code, and Chinese to LLMs without losing base skills"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000898,"raw_usage":{"total_tokens":3926,"prompt_tokens":1062,"completion_tokens":2864,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":2780}},"tokens_in":678,"tokens_out":2864,"duration_ms":23958,"temperature":1.0,"reasoning_tokens":2780,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:45:13.209786+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sweep the divergence weight lambda over a wide range (from zero to very large) on a fixed base model and new-task dataset, and record new-task accuracy alongside MMLU. If no lambda value yields both near-full-fine-tuning new-task accuracy and MMLU within 5 points of the base model, then the claimed 'learn more, forget less' balance does not hold as stated. A simpler check: train on a synthetic task deliberately far from the base distribution, where hidden-state alignment would force the trainable branch to stay close to representations that cannot express the task; if the method fails there, its applicability is limited to tasks near the original distribution.","supporting_citations":[],"review_version":1}