Pith. sign in

REVIEW 2 major objections 2 minor 104 references

Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning

T0 review · 2 major / 2 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DualSelect jointly selects task samples and safety references to preserve LLM safety during fine-tuning.

desk verdict DualSelect introduces a joint minimax selection of task samples and safety references with lazy refresh, claiming over 5 points safety gains on 1B-8B models, but the abstract keeps experimental details thin. read the letter →

arxiv 2606.09866 v1 pith:NASLEKJR submitted 2026-06-01 cs.LG cs.AI

Xinrui Chen , Jianhao Zhang , Ou Wu , Di Gao This is my paper
classification cs.LGcs.AI
keywords LLMfine-tuningsafetyalignmenttaskselectionreferenceDualSelectcontinuallearningpreservation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that fine-tuning safety-aligned LLMs on downstream tasks risks eroding safety, and that existing approaches using fixed safety examples or one-sided filtering fail to address how task updates create varying safety constraints. It introduces DualSelect, a framework that refreshes safety references conditioned on the current task and then selects only compatible task samples to align with the reference direction. Experiments on 1B to 8B parameter models show this coupled selection maintains safety scores while keeping task performance intact. Readers would care because the method offers a concrete way to adapt models to new data without undoing prior safety training. The same joint-selection logic is claimed to extend to retention-focused continual learning.

What carries the argument

DualSelect, the coupled framework that refreshes task-conditioned safety references before filtering whole task samples compatible with the induced reference direction.

What would settle it

On the same 1B-8B model benchmarks and judges, DualSelect produces safety averages no higher than the strongest baseline while still matching task utility.

Watch

Extended reading notes

Core claim

DualSelect selects safety references that have high preservation loss and task conflict, together with compatible task samples, through entropy-regularized scoring surrogates, lazy reference refresh, and gradient correction. On 1B-8B LLMs it preserves safety without losing task utility; using the REDORCA judge it improves Safety Avg. over the strongest baseline by at least 5.10 points and remains highest in Safety Avg. across judges with moderate overhead. This view extends to retention focused continual learning.

Load-bearing premise

Task updates expose different safety constraints that require joint selection of references and task samples rather than handling them separately.

Editorial extensions

If this is right

  • Safety Avg. rises by at least 5.10 points over the strongest baseline on the REDORCA judge.
  • Safety Avg. stays highest across multiple judges while task utility is retained.
  • The method incurs only moderate overhead.
  • The same coupled selection logic applies to retention-focused continual learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The minimax formulation might link to other game-theoretic selection problems in machine learning.
  • The refresh-and-filter pattern could be tested on alignment dimensions beyond safety, such as factual consistency.
  • Scaling the approach to models larger than 8B would show whether the safety gains persist.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes DualSelect, a coupled task-reference selection framework for safe LLM fine-tuning. It motivates the approach by noting that task updates expose varying safety constraints, then frames selection as a minimax problem solved via entropy-regularized scoring surrogates, lazy reference refresh, and gradient correction. The method refreshes task-conditioned safety references before filtering compatible task samples. On 1B-8B LLMs it reports preserving safety while retaining task utility, with a Safety Avg. improvement of at least 5.10 points over the strongest baseline under the REDORCA judge and top-ranked safety across multiple judges, at moderate overhead; the approach is also positioned as applicable to retention-focused continual learning.

Significance. If the empirical claims hold under full experimental scrutiny, the work offers a practical, adaptive alternative to fixed safety examples or one-sided filtering by explicitly coupling reference and task selection. The quantified margin on multiple model scales and judges, together with the extension to continual learning, would make the contribution relevant to the safe-adaptation literature.

major comments (2)
  1. [Experiments] Experimental section: the abstract states a ≥5.10 point Safety Avg. gain on REDORCA but supplies no information on baseline implementations, number of random seeds, variance, or statistical tests; without these the reported margin cannot be assessed for robustness and is load-bearing for the central empirical claim.
  2. [Methods] Methods: the minimax framing and entropy-regularized surrogates are presented at a high level; explicit equations showing how the surrogates are computed from the preservation-loss and conflict terms, and how lazy refresh plus gradient correction are applied, are required to verify that the procedure is not circular or task-specific by construction.
minor comments (2)
  1. Define the REDORCA judge and all other acronyms on first use in the abstract and main text.
  2. The abstract would benefit from a one-sentence statement of the number of tasks, model sizes, and evaluation judges used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and the recommendation for minor revision. We address the two major comments point by point below.

read point-by-point responses

  1. Referee: [Experiments] Experimental section: the abstract states a ≥5.10 point Safety Avg. gain on REDORCA but supplies no information on baseline implementations, number of random seeds, variance, or statistical tests; without these the reported margin cannot be assessed for robustness and is load-bearing for the central empirical claim.

    Authors: We agree that the current experimental section does not provide sufficient detail on these aspects to allow independent assessment of robustness. In the revised manuscript we will expand the experimental section to explicitly describe baseline implementations, the number of random seeds, variance across runs, and any statistical tests performed. revision: yes

  2. Referee: [Methods] Methods: the minimax framing and entropy-regularized surrogates are presented at a high level; explicit equations showing how the surrogates are computed from the preservation-loss and conflict terms, and how lazy refresh plus gradient correction are applied, are required to verify that the procedure is not circular or task-specific by construction.

    Authors: We acknowledge that the methods presentation remains at a high level. In the revision we will insert the explicit equations for the entropy-regularized scoring surrogates (derived from the preservation-loss and conflict terms), the lazy reference refresh schedule, and the gradient correction step, together with a short argument showing that the procedure is not circular or task-specific by construction. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper introduces DualSelect as a new coupled selection procedure motivated by diagnostics on task-induced safety constraints, framed under a minimax view with entropy-regularized surrogates, lazy refresh, and gradient correction. The load-bearing claims are empirical (Safety Avg. gains of ≥5.10 points on 1B-8B models versus baselines, using named judges), with no equations, fitted parameters renamed as predictions, self-definitional reductions, or load-bearing self-citations that collapse the central result to its own inputs. The derivation remains self-contained against external benchmarks and does not exhibit any of the enumerated circular patterns.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated. The DualSelect framework itself is the introduced method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning." pith.science (2026). https://pith.science/paper/NASLEKJR

@misc{pith2026260609866,
  author       = {Pith},
  title        = {Pith review of: Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NASLEKJR}},
  note         = {Machine review of arXiv:2606.09866}
}
read the original abstract

Fine-tuning safety aligned large language models (LLMs) on downstream data improves adaptation but may erode learned safety behavior. Existing methods use fixed safety examples, global constraints, or one-sided task filtering. Our diagnostics show task updates expose different safety constraints, motivating joint selection of relevant references and compatible task samples. We propose DualSelect, a coupled framework for task and reference selection that refreshes task conditioned safety references before filtering whole task samples compatible with the induced reference direction. Under a minimax view, DualSelect selects safety references with high preservation loss and task conflict, together with compatible task samples, through entropy-regularized scoring surrogates, lazy reference refresh, and gradient correction. On 1B-8B LLMs, DualSelect preserves safety without losing task utility; using the REDORCA judge, it improves Safety Avg. over the strongest baseline by at least 5.10 points and remains highest in Safety Avg. across judges with moderate overhead. This view extends to retention focused continual learning.

Figures

Figures reproduced from arXiv: 2606.09866 by the authors.

Figure 1
Figure 1. Paradigm shift in safety-reference use: from [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. DualSelect overview. Fixed safety references may weakly constrain task-specific updates. DualSelect couples task–reference selection: selecting safety-critical safe-response references to constrain updates, filtering reference-compatible samples, and applying reference-gradient correction to improve safety–utility trade-offs. 3 Methodology [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Task-conditioned reference diagnostic on Llama-3-8B-Instruct under REDORCA. Panel (a) reports cross [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Efficiency comparison. models while maintaining comparable task accu￾racy. For stronger models, GSM8K utility differs marginally across methods and stays near Stan￾dard SFT. Results indicate that task-conditioned reference selection improves safety preservation under c…
Figure 5
Figure 5. Figure 5: Main performance of component ablations. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: REDORCA sensitivity to ρ, q tok new, and qref. 4.6 Additional Analyses Reweighting and correction variants. We include SOT-style reweighting and SPF-style preservation as mechanism variants. SOT-style uses global safe/harmful reference-aware weights, whereas SPF-style …
Figure 7
Figure 7. Figure 7: Robustness to fixed scoring constants. Update-alignment diagnostics. We report RawCos/+Cos diagnostics for the correction￾strength sweep in Sec. 4.5. Definitions follow Ap￾pendix D.2; RawCos measures pre-correction task￾reference geometry, while +Cos measures post￾corr…
Figure 8
Figure 8. Figure 8: Cross-judge robustness on REDORCA. We report Safety Avg. and Utility under three judge models. [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

104 extracted references · 17 canonical work pages

  1. [1]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  2. [2]

    Journal of machine learning research , volume=

    Palm: Scaling language modeling with pathways , author=. Journal of machine learning research , volume=

  3. [3]

    International conference on machine learning , pages=

    Parameter-efficient transfer learning for NLP , author=. International conference on machine learning , pages=. 2019 , organization=

    2019

  4. [4]

    Edward J Hu and yelong shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo

  5. [5]

    Findings of the association for computational linguistics: EMNLP 2020 , pages=

    Realtoxicityprompts: Evaluating neural toxic degeneration in language models , author=. Findings of the association for computational linguistics: EMNLP 2020 , pages=

    2020

  6. [6]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    Red teaming language models with language models , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  7. [7]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    work page Pith review arXiv

  8. [8]

    Advances in neural information processing systems , volume=

    Training language models to follow instructions with human feedback , author=. Advances in neural information processing systems , volume=

Show all 104 references
  1. [9]

    arXiv preprint arXiv:2508.07172 , year=

    Gradient surgery for safe llm fine-tuning , author=. arXiv preprint arXiv:2508.07172 , year=

  2. [10]

    2004 , publisher=

    Convex optimization , author=. 2004 , publisher=

    2004

  3. [11]

    Mathematical programming , volume=

    An analysis of approximations for maximizing submodular set functions-I , author=. Mathematical programming , volume=. 1978 , publisher=

    1978

  4. [12]

    50 Years of Integer Programming 1958-2008: from the Early Years to the State-of-the-Art , pages=

    Reducibility among combinatorial problems , author=. 50 Years of Integer Programming 1958-2008: from the Early Years to the State-of-the-Art , pages=. 2009 , publisher=

    1958

  5. [13]

    International Conference on Learning Representations , year=

    Fast is better than free: Revisiting adversarial training , author=. International Conference on Learning Representations , year=

  6. [14]

    International Conference on Machine Learning , pages=

    Towards Stable and Efficient Adversarial Training against l\_1 Bounded Adversarial Attacks , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  7. [15]

    Efficient Lifelong Learning with A-

    Arslan Chaudhry and Marc'Aurelio Ranzato and Marcus Rohrbach and Mohamed Elhoseiny , booktitle=. Efficient Lifelong Learning with A-

  8. [16]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    A comprehensive survey of continual learning: Theory, method and application , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2024 , publisher=

    2024

  9. [17]

    Annals of operations research , volume=

    An overview of bilevel optimization , author=. Annals of operations research , volume=. 2007 , publisher=

    2007

  10. [18]

    International conference on machine learning , pages=

    Hyperparameter optimization with approximate gradient , author=. International conference on machine learning , pages=. 2016 , organization=

    2016

  11. [19]

    International conference on artificial intelligence and statistics , pages=

    Optimizing millions of hyperparameters by implicit differentiation , author=. International conference on artificial intelligence and statistics , pages=. 2020 , organization=

    2020

  12. [20]

    Proceedings of the national academy of sciences , volume=

    Overcoming catastrophic forgetting in neural networks , author=. Proceedings of the national academy of sciences , volume=. 2017 , publisher=

    2017

  13. [21]

    arXiv preprint arXiv:1606.04671 , year=

    Progressive neural networks , author=. arXiv preprint arXiv:1606.04671 , year=

    arXiv

  14. [22]

    Advances in neural information processing systems , volume=

    Gradient episodic memory for continual learning , author=. Advances in neural information processing systems , volume=

  15. [23]

    Advances in Neural Information Processing Systems , volume=

    Keeping llms aligned after fine-tuning: The crucial role of prompt templates , author=. Advances in Neural Information Processing Systems , volume=

  16. [24]

    Advances in Neural Information Processing Systems , volume=

    Navigating the safety landscape: Measuring risks in finetuning large language models , author=. Advances in Neural Information Processing Systems , volume=

  17. [25]

    Advances in Neural Information Processing Systems , volume=

    Vaccine: Perturbation-aware alignment for large language models against harmful fine-tuning attack , author=. Advances in Neural Information Processing Systems , volume=

  18. [26]

    Advances in Neural Information Processing Systems , volume=

    Lisa: Lazy safety alignment for large language models against harmful fine-tuning attack , author=. Advances in Neural Information Processing Systems , volume=

  19. [27]

    The Thirteenth International Conference on Learning Representations , year =

    Booster: Tackling Harmful Fine-tuning for Large Language Models via Attenuating Harmful Perturbation , author=. The Thirteenth International Conference on Learning Representations , year =

  20. [28]

    The Thirteenth International Conference on Learning Representations , year =

    SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection , author=. The Thirteenth International Conference on Learning Representations , year =

  21. [29]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Layer-aware representation filtering: Purifying finetuning data to preserve llm safety alignment , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  22. [30]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track , pages=

    How to Fine-Tune Safely on a Budget: Model Adaptation Using Minimal Resources , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track , pages=

    2025

  23. [31]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

    Adaptive Defense against Harmful Fine-Tuning for Large Language Models via Bayesian Data Scheduler , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

  24. [32]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

    Shape it Up! Restoring LLM Safety during Finetuning , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

  25. [33]

    Forty-second International Conference on Machine Learning , year =

    Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack , author=. Forty-second International Conference on Machine Learning , year =

  26. [34]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

    Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year =

  27. [35]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Data advisor: Dynamic data curation for safety alignment of large language models , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  28. [36]

    arXiv preprint arXiv:2510.10085 , year=

    Pharmacist: Safety alignment data curation for large language models against harmful fine-tuning , author=. arXiv preprint arXiv:2510.10085 , year=

  29. [37]

    Token-level Data Selection for Safe

    Yanping Li and Zhening Liu and Zijian Li and Zehong Lin and Jun Zhang , booktitle=. Token-level Data Selection for Safe

  30. [38]

    The Fourteenth International Conference on Learning Representations , year=

    Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence , author=. The Fourteenth International Conference on Learning Representations , year=

  31. [39]

    Shuhao Chen and Weisen Jiang and Yeqi Gong and Shengda Luo and Chengxiang Zhuo and Zang Li and James Kwok and Yu Zhang , year=

  32. [40]

    arXiv preprint arXiv:2605.14194 , year=

    GradShield: Alignment Preserving Finetuning , author=. arXiv preprint arXiv:2605.14194 , year=

    arXiv

  33. [41]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Separate the wheat from the chaff: A post-hoc approach to safety re-alignment for fine-tuned language models , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  34. [42]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Lssf: Safety alignment for large language models through low-rank safety subspace fusion , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  35. [43]

    Advances in Neural Information Processing Systems , volume=

    Smalltolarge (s2l): Scalable data selection for fine-tuning large language models by summarizing training trajectories of small models , author=. Advances in Neural Information Processing Systems , volume=

  36. [44]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Davir: Data selection via implicit reward for large language models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  37. [45]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Data whisperer: Efficient data selection for task-specific llm fine-tuning via few-shot in-context learning , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  38. [46]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Hft: Half fine-tuning for large language models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  39. [47]

    Forty-second International Conference on Machine Learning , year=

    Boosting Multi-Domain Fine-Tuning of Large Language Models through Evolving Interactions between Samples , author=. Forty-second International Conference on Machine Learning , year=

  40. [48]

    Token Cleaning: Fine-Grained Data Selection for

    Jinlong Pang and Na Di and Zhaowei Zhu and Jiaheng Wei and Hao Cheng and Chen Qian and Yang Liu , booktitle=. Token Cleaning: Fine-Grained Data Selection for

  41. [49]

    Mengzhou Xia and Sadhika Malladi and Suchin Gururangan and Sanjeev Arora and Danqi Chen , booktitle=

  42. [50]

    GIST: Targeted Data Selection for Instruction Tuning via Coupled Optimization Geometry , author=

  43. [51]

    Zichun Yu and Spandan Das and Chenyan Xiong , booktitle=

  44. [52]

    Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM , author=

  45. [53]

    Yichen Yan and Ming Zhong and Qi Zhu and Xiaoling Gu and Jinpeng Chen and Huan Li , booktitle=. Co

  46. [54]

    Diversity as a Reward: Fine-Tuning

    Zhenqing Ling and Daoyuan Chen and Liuyi Yao and Qianli Shen and Yaliang Li and Ying Shen , booktitle=. Diversity as a Reward: Fine-Tuning

  47. [55]

    arXiv preprint arXiv:2601.09195 , year=

    ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection , author=. arXiv preprint arXiv:2601.09195 , year=

    arXiv

  48. [56]

    The Thirteenth International Conference on Learning Representations , year=

    Data Shapley in One Training Run , author=. The Thirteenth International Conference on Learning Representations , year=

  49. [57]

    Reza Shirkavand and Peiran Yu and Qi He and Heng Huang , booktitle=. Bilevel

  50. [58]

    International conference on machine learning , pages=

    A generic first-order algorithmic framework for bi-level programming beyond lower-level singleton , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  51. [59]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    A general descent aggregation framework for gradient-based bi-level optimization , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2022 , publisher=

    2022

  52. [60]

    International conference on machine learning , pages=

    Model-agnostic meta-learning for fast adaptation of deep networks , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  53. [61]

    International conference on machine learning , pages=

    Bilevel programming for hyperparameter optimization and meta-learning , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  54. [62]

    Hanxiao Liu and Karen Simonyan and Yiming Yang , booktitle=

  55. [63]

    Yang Yu and Kai Han and Hang Zhou and Yehui Tang and Kaiqi Huang and Yunhe Wang and Dacheng Tao , booktitle=

  56. [64]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  57. [65]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Compress Large Language Models via Collaboration Between Learning and Matrix Approximation , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  58. [66]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Recurrent knowledge identification and fusion for language model continual learning , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  59. [67]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Continual Learning Using Only Large Language Model Prompting , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

  60. [68]

    International Conference on Machine Learning , pages=

    Learning Dynamics in Continual Pre-Training for Large Language Models , author=. International Conference on Machine Learning , pages=. 2025 , organization=

    2025

  61. [69]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  62. [70]

    arXiv preprint arXiv:2509.23893 , year=

    Dynamic Orthogonal Continual Fine-tuning for Mitigating Catastrophic Forgettings , author=. arXiv preprint arXiv:2509.23893 , year=

  63. [71]

    arXiv preprint arXiv:2510.15103 , year=

    Continual learning via sparse memory finetuning , author=. arXiv preprint arXiv:2510.15103 , year=

  64. [72]

    arXiv e-prints , pages=

    The llama 3 herd of models , author=. arXiv e-prints , pages=

  65. [73]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  66. [74]

    Qwen3.5: Accelerating Productivity with Native Multimodal Agents , author =

  67. [75]

    Proceedings of the 2013 conference on empirical methods in natural language processing , pages=

    Recursive deep models for semantic compositionality over a sentiment treebank , author=. Proceedings of the 2013 conference on empirical methods in natural language processing , pages=

    2013

  68. [76]

    Advances in neural information processing systems , volume=

    Character-level convolutional networks for text classification , author=. Advances in neural information processing systems , volume=

  69. [77]

    Hugging Face dataset repository , year=

    Openorca: An open dataset of gpt augmented flan reasoning traces , author=. Hugging Face dataset repository , year=

  70. [78]

    arXiv preprint arXiv:2306.02707 , year=

    Orca: Progressive learning from complex explanation traces of gpt-4 , author=. arXiv preprint arXiv:2306.02707 , year=

    arXiv

  71. [79]

    arXiv preprint arXiv:2209.07858 , year=

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned , author=. arXiv preprint arXiv:2209.07858 , year=

    arXiv

  72. [80]

    Hex-phi: Human-extended policy-oriented harmful instruction benchmark , author=

  73. [81]

    The Twelfth International Conference on Learning Representations , year=

    Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! , author=. The Twelfth International Conference on Learning Representations , year=

  74. [82]

    Data Selection for

    Jia Zhang and Yao Liu and Chen-Xi Zhang and Yi Liu and Yi-Xuan Jin and Lan-Zhe Guo and Yu-Feng Li , booktitle=. Data Selection for

  75. [83]

    Safety-Tuned

    Federico Bianchi and Mirac Suzgun and Giuseppe Attanasio and Paul Rottger and Dan Jurafsky and Tatsunori Hashimoto and James Zou , booktitle=. Safety-Tuned

  76. [84]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    ToolSafety: A Comprehensive Dataset for Enhancing Safety in LLM-Based Agent Tool Invocations , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  77. [85]

    arXiv preprint arXiv:2601.07200 , year=

    Safeguarding LLM Fine-tuning via Push-Pull Distributional Alignment , author=. arXiv preprint arXiv:2601.07200 , year=

  78. [86]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Data Selection for Language Models via Importance Resampling , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  79. [87]

    arXiv preprint arXiv:2601.10141 , year=

    Understanding and Preserving Safety in Fine-Tuned LLMs , author=. arXiv preprint arXiv:2601.10141 , year=

  80. [88]

    2023 , publisher =

    SlimOrca: An Open Dataset of GPT-4 Augmented FLAN Reasoning Traces, with Verification , author =. 2023 , publisher =

    2023

  81. [89]

    Rethinking Safety in

    Minseon Kim and Jin Myung Kwak and Lama Alssum and Bernard Ghanem and Philip Torr and David Krueger and Fazl Barez and Adel Bibi , booktitle=. Rethinking Safety in

  82. [90]

    arXiv preprint arXiv:2601.22313 , year=

    Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment , author=. arXiv preprint arXiv:2601.22313 , year=

  83. [91]

    Safe Delta: Consistently Preserving Safety when Fine-Tuning

    Ning Lu and Shengcai Liu and Jiahao Wu and Weiyu Chen and Zhirui Zhang and Yew-Soon Ong and Qi Wang and Ke Tang , booktitle=. Safe Delta: Consistently Preserving Safety when Fine-Tuning

  84. [92]

    arXiv preprint arXiv:2510.21885 , year=

    Preventing Catastrophic Forgetting: Behavior-Aware Sampling for Safer Language Model Fine-Tuning , author=. arXiv preprint arXiv:2510.21885 , year=

  85. [93]

    arXiv preprint arXiv:2604.08297 , year=

    Towards Identification and Intervention of Safety-Critical Parameters in Large Language Models , author=. arXiv preprint arXiv:2604.08297 , year=

    arXiv

  86. [94]

    Neurips Safe Generative AI Workshop 2024 , year=

    Safety-Aware Fine-Tuning of Large Language Models , author=. Neurips Safe Generative AI Workshop 2024 , year=

    2024

  87. [95]

    , journal =

    Yang, Shuo and Zhang, Qihui and Liu, Yuyang and Huang, Yue and Jia, Xiaojun and Ning, Kun-Peng and Yao, Jia-Yu and Wang, Jigang and Hailiang, Dai and Song, Yibing and et al. , journal =. 2026 , month =

    2026

  88. [96]

    arXiv preprint arXiv:2605.04572 , year=

    From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning , author=. arXiv preprint arXiv:2605.04572 , year=

    arXiv

  89. [97]

    Mitigating Fine-tuning Risks in

    Chengcan Wu and Zhixin Zhang and Zeming Wei and Yihao Zhang and Meng Sun , booktitle=. Mitigating Fine-tuning Risks in

  90. [98]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Continual gradient low-rank projection fine-tuning for LLMs , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  91. [99]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Controlled low-rank adaptation with subspace regularization for continued training on large language models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  92. [100]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  93. [101]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  94. [102]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Orthogonal subspace learning for language model continual learning , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  95. [103]

    Journal of machine learning research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. Journal of machine learning research , volume=

  96. [104]

    Advances in neural information processing systems , volume=

    Experience replay for continual learning , author=. Advances in neural information processing systems , volume=

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.