{"id":"c03d7dd3-fcf7-47e3-a807-067e8e13787b","arxiv_id":"2508.12531","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Fine-tuning safety loss is largely avoidable: better hyperparameter selection and a parameter-space EMA reduce unsafe responses from 16% to 5% while preserving utility.","lead":"This paper tests whether fine-tuning language models necessarily makes them less safe, and argues that poor optimization choices, not an inherent trade-off, are the main cause. It finds that tuning learning rate, batch size, and gradient steps, or using a parameter-space EMA, can cut unsafe responses from about 16% to 5% while keeping utility.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Keyword-matching safety metric may not measure genuine safety; the 16%→5% reduction could be an artifact of this proxy.","rationale":"The reader's verdict is UNVERDICTED because the full text was not available and the abstract alone is insufficient. My stress-test identifies the same load-bearing assumption: the keyword-matching safety metric. The central claim is precisely about safety, and the only quantitative evidence for it is measured by keyword matching. If that metric is unreliable, the conclusion does not follow. This is an external-validity concern rather than an internal inconsistency, and it is concrete and testable. The recommendation is UNCHANGED because the reader already marked the paper UNVERDICTED; my concern reinforces that the abstract's headline result requires verification but does not by itself move the verdict to ACCEPT or REJECT without seeing the full experimental protocol. I agree with the reader's weakest assumption assessment.","tokens_in":722,"tokens_out":1200,"duration_ms":16686,"concrete_test":"Take the same fine-tuned checkpoints and the same adversarial prompt set used in the paper, and rescore the generated responses with independent human annotators (or, as a scalable proxy, a well-calibrated strong LLM judge such as GPT-4 with a rubric for refusal and harmfulness) alongside the original keyword matching. Then compute the unsafe rates under both measurements for the baseline and EMA fine-tuned models. If the judge-based unsafe rate is not close to 5% for the EMA model, or if the reduction from 16% to 5% does not reproduce under the judge, the central claim is an artifact of the keyword-matching metric. A minimal version: report confusion matrices and correlation between keyword matching and human/LLM judgments on a random sample of 500 responses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that poor optimization choices, not inherent trade-offs, cause safety degradation and that EMA preserves safety—rests entirely on the abstract's measurement: 'unsafe model responses from 16% to approximately 5%, as measured by keyword matching.' This is a load-bearing assumption because the conclusion is about safety, but the operationalization is a keyword-matching heuristic on responses to adversarial prompts. If the keyword list is too narrow, superficial rephrasing, hedging, or non-verbal unsafe completions that evade the keyword list would be counted as safe. Conversely, benign responses containing a banned word would be counted as unsafe. The abstract provides no evidence that this proxy correlates with human judgments of harmfulness, no list of keywords, no definition of 'adversarial prompts,' and no error rates. Without validation of the metric, the 16%→5% reduction and the broader claim that 'safety problems during fine-tuning can largely be avoided' do not necessarily describe real safety improvements; they may describe improvements only in matching a particular string-matching rule. Since the abstract's headline quantitative result is defined through this proxy, the proxy is the single most load-bearing unverified element.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper challenges the prevailing assumption that fine-tuning a safety-aligned LLM necessarily degrades its safety, arguing instead that poor optimization choices (learning rate, batch size, gradient steps) are the primary cause. The abstract reports a reduction in unsafe responses to adversarial prompts from 16% to ~5% by tuning these hyperparameters, and proposes an exponential moving average (EMA) of model parameters as a simple intervention to preserve pretrained safety properties. Experiments across Llama-family models and three datasets (Dolly, Alpaca, ORCA) are claimed to show that safety problems can be largely avoided without specialized safety data.","tokens_in":1047,"tokens_out":3223,"duration_ms":41134,"significance":"If the empirical claim holds, the paper is practically significant: it reframes safety degradation during fine-tuning as an optimization pitfall rather than an inherent trade-off, offers directly actionable hyperparameter guidance, and contributes a low-cost EMA technique that may reduce the need for safety-specific data or auxiliary losses. The proposal is simple, easy to adopt, and testable. However, the evidence as presented in the abstract is not yet convincing: the headline metric is keyword matching, no validation against human judgment or standard safety benchmarks is provided, and the statistical robustness of the 16%→5% reduction is not established.","major_comments":[{"comment":"The central quantitative claim (16%→5% reduction in unsafe responses) is defined solely by 'keyword matching' on adversarial prompts. The abstract provides no keyword list, no description of the adversarial prompt set, and no evidence that this proxy agrees with human assessments or established safety benchmarks (e.g., HarmBench, JailbreakBench). Without such validation, the reported reduction is ambiguous: it could reflect a drop in surface-level banned words while genuine unsafe behavior (e.g., reframed or hedged harmful content) remains unchanged, or conversely flag benign content containing keywords. This metric is load-bearing for the paper's main conclusion, so the paper must either validate the proxy or explicitly restrict its claims to keyword-matched refusal behavior.","section":"Abstract (measurement)"},{"comment":"The phrase 'by properly selecting key training hyper-parameters' suggests tuning. If the same evaluation metric used to report safety also guided the hyperparameter search, the reduction may be an artifact of overfitting to the keyword-matching rule. The paper should disclose the selection protocol: the hyperparameter grid, whether a held-out validation set was used, the number of configurations tried, and whether the reported numbers are from a single run or a test set not used for selection. This is essential to assess whether the result generalizes across random seeds and datasets.","section":"Abstract (hyperparameter selection)"},{"comment":"The report gives single point estimates (16% and 5%) without error bars, number of runs, or significance tests. Given the modest absolute reduction (11 percentage points), run-to-run variance could be large, especially across dataset and model-family combinations. The paper should report the spread across multiple fine-tuning runs (e.g., different seeds) and provide confidence intervals or statistical tests to support the claim that hyperparameter choices reliably produce this improvement.","section":"Abstract (statistical robustness)"}],"minor_comments":[{"comment":"The term 'systematic testing' is vague; specify what was varied and how many configurations were evaluated.","section":"Abstract"},{"comment":"'approximately 5%' would benefit from a precise value and the number of test prompts; a percentage alone is not reproducible.","section":"Abstract"},{"comment":"The phrase 'Llama families' is imprecise; state which Llama versions (e.g., Llama-2, Llama-3) and sizes were used.","section":"Abstract"},{"comment":"The comparison to 'existing approaches that require additional safety data' is not enumerated or referenced; this claim should be backed by a citation or a brief description in the body.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is potentially important, but it cannot be evaluated from the abstract alone. The authors may well have performed careful experiments in the full text, including validation of the keyword-matching metric against more robust safety measures. If that validation is absent or inadequate, the contribution would be substantially weakened. I recommend requesting the full experimental details, especially the safety-evaluation protocol and hyperparameter-selection methodology, before any decision. The keyword-matching proxy is the single most load-bearing element and deserves the most scrutiny."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: this paper makes a bold, simple claim—fine-tuning safety loss is mostly an optimization problem, not an inherent trade-off—and backs it with a concrete recipe: pick the right LR, batch size, gradient steps, and use a parameter-space EMA. The reported drop from 16% to ~5% unsafe responses is the headline. If it holds up, it's a practical win for practitioners who want to fine-tune without safety collapse.\n\nWhat's good: the paper challenges a lazy assumption and offers a low-cost intervention. The EMA in parameter space is a nice twist, and the experiments cover Llama families across Dolly, Alpaca, ORCA. The abstract is clear about the metric: keyword matching on adversarial prompts. That transparency is helpful, but it also invites the central problem.\n\nThe soft spot is the measurement. Keyword matching is a coarse proxy for safety. The stress-test note is on target: the 16%→5% may partly reflect the proxy's blind spots—phrasings that sidestep the keyword list, or benign responses that trip on a banned word. Without validation against human judgments or a broader safety eval, we can't tell whether the drop is real safety improvement or just rule-following. The paper also likely tunes hyperparameters and the EMA coefficient on the same evaluation benchmarks, so the numbers may be conditional on those choices. The abstract gives no error bars or dataset splits. And the sweeping conclusion that 'safety problems can largely be avoided' goes further than the demonstrated scope.\n\nThat said, these are concerns about what we couldn't see. We only had the abstract; the full text may address them. The central idea deserves a serious look. The authors seem to know what they're doing, and the claim is falsifiable. A referee should demand metric validation, hyperparameter sensitivity analysis, and error bars, but the paper should get that chance.\n\nBottom line: send it to peer review. It's not desk-reject material. I'd read the full version carefully before citing it, but it's worth a reading group slot and a referee's time.\n\nBest.","headline":"The paper's central claim is plausible and important, but the keyword-matching metric and hyperparameter tuning make the 16%→5% number conditional; deserves peer review.","tokens_in":1449,"tokens_out":2958,"would_cite":false,"duration_ms":30262,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that safety degradation during LLM fine-tuning is largely an optimization artifact, not an inherent safety–utility trade-off, and that choosing learning rate, batch size, and gradient steps plus a parameter-space EMA cuts","keywords":["LLM fine-tuning","safety alignment","hyperparameter selection","parameter-space EMA","adversarial prompts","keyword matching","Llama models","optimization trajectory"],"falsifier":"Run the recommended hyperparameter settings and parameter-space EMA on a held-out set of human-written adversarial prompts and jailbreak attacks, then compare unsafe-response rates judged by human raters; if the fine-tuned models are not meaningfully safer than a standard fine-tuned baseline, the central claim fails. A second check is to vary batch size beyond the tested range and see whether the safety benefit disappears, which would indicate the effect is specific to the tested configuration rather than a general optimization principle.","tokens_in":695,"feed_emoji":"🛡️","tokens_out":4411,"duration_ms":49369,"temperature":0.7,"pith_summary":"Fine-tuning a language model on harmless data is widely believed to erode its safety, forcing extra safety measures. This paper challenges that belief: it argues that safety failures during fine-tuning are mostly an optimization artifact, not an unavoidable trade-off. Using the Llama family across the Dolly, Alpaca, and ORCA datasets, the authors show that careful selection of learning rate, batch size, and number of gradient steps reduces unsafe responses to adversarial prompts from about 16% to about 5%, measured by keyword matching, while utility is maintained. They also introduce a parameter-space exponential moving average that stabilizes the optimization path and preserves the pretrained model's safety behavior. If right, the paper means safe fine-tuning is largely a matter of optimization discipline rather than specialized safety interventions.","feed_headline":"Right hyperparameters cut unsafe LLM responses from 16% to 5%","feed_subtitle":"A simple EMA momentum trick preserves pretrained safety, so safe fine-tuning may not need extra safety data.","key_machinery":"The load-bearing mechanism is the exponential moving average (EMA) of model parameters: during fine-tuning, the method maintains a running average of the weights and uses the averaged model for evaluation. EMA smooths the optimization trajectory and prevents transient weight updates from carrying the model into regions where harmful responses become likely. It works together with deliberate hyperparameter selection—small learning rate, moderate batch size, limited gradient steps—to keep the model near the pretrained safety region while still learning the new task. The named identity of the argument is 'parameter-space EMA', an average of weights rather than of gradients or outputs.","core_discovery":"The paper's central claim is that the widely observed safety drop in fine-tuned language models is not an inherent consequence of further training on harmless data; it is a side effect of poor optimization choices. The evidence is a controlled comparison: by selecting key training hyperparameters—learning rate, batch size, and gradient steps—the authors reduce unsafe responses to adversarial prompts from 16% to about 5% (keyword-matched) while keeping utility performance intact. To make the mechanism explicit, they propose exponential moving average (EMA) in parameter space, which averages model weights over the optimization trajectory and keeps the fine-tuned model close to the safety prope","pith_inferences":["If safety loss is an optimization artifact, other fine-tuning regressions—such as catastrophic forgetting or loss of instruction-following stability—may respond to the same trajectory-control treatment; this is a natural extension the paper does not test.","The keyword-matching safety metric is the key measurement assumption; showing the same pattern under human evaluation or stronger adversarial evaluation would materially strengthen the conclusion, and that test remains open.","The EMA mechanism suggests an implicit budget on distance from the pretrained weights; a practical extension would be to monitor parameter-space distance during training and halt before safety degrades.","Because the argument is framed as an optimization-geometry effect, direct loss-landscape measurements would be the cleanest way to confirm the mechanism behind the reported safety preservation."],"forward_implications":["Practitioners can avoid most fine-tuning safety loss by choosing small learning rates, moderate batch sizes, and few gradient steps, without needing safety-specific data.","Parameter-space EMA is a low-cost addition to fine-tuning pipelines that preserves safety by stabilizing the weight trajectory.","The common belief that harmless fine-tuning inevitably harms safety is, under this evidence, not the right model; safety and utility can be maintained together.","The reported reduction from 16% to about 5% unsafe keyword-matched responses gives a concrete evaluation target for safe fine-tuning.","The approach compares favorably to existing methods that require additional safety data, suggesting that optimization choices alone can be competitive with specialized interventions."],"supporting_citations":[],"fun_headline_variants":["Fine-tuning safety loss is a fixable optimization problem","Choosing right hyperparameters cuts unsafe LLM responses 16% to 5%","EMA momentum preserves pretrained safety during fine-tuning","Safety drop in fine-tuned LLMs is an optimization artifact","Optimization tweaks cut unsafe responses from 16% to 5%"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The paper's numbers depend on measuring safety by keyword matching of harmful responses to adversarial prompts; if that measurement is too noisy or too narrow, the reported reduction from 16% to about 5% may not represent a genuine improvement in safety.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuning safety loss is a fixable optimization problem","Choosing right hyperparameters cuts unsafe LLM responses 16% to 5%","EMA momentum preserves pretrained safety during fine-tuning","Safety drop in fine-tuned LLMs is an optimization artifact","Optimization tweaks cut unsafe responses from 16% to 5%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000501,"raw_usage":{"total_tokens":2266,"prompt_tokens":706,"completion_tokens":1560,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":450,"completion_tokens_details":{"reasoning_tokens":1472}},"tokens_in":450,"tokens_out":1560,"duration_ms":14160,"temperature":1.0,"reasoning_tokens":1472,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:24:26.380715+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the recommended hyperparameter settings and parameter-space EMA on a held-out set of human-written adversarial prompts and jailbreak attacks, then compare unsafe-response rates judged by human raters; if the fine-tuned models are not meaningfully safer than a standard fine-tuned baseline, the central claim fails. A second check is to vary batch size beyond the tested range and see whether the safety benefit disappears, which would indicate the effect is specific to the tested configuration rather than a general optimization principle.","supporting_citations":[],"review_version":1}