{"id":"e2fd4bfb-87aa-4b13-bd64-4be27c870d6f","arxiv_id":"2501.00517","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A multi-stage LLM-based attack/defense dataset pipeline improves reported safety scores of Llama-3.2-1B after SFT, but the evaluation is partly circular and lacks statistical baselines.","lead":"This paper constructs a Chinese safety alignment dataset by labeling attack prompts into intent categories, generating extra attack prompts, regenerating safe answers with Qwen models, and filtering with a Qwen-based safety reward model, then fine-tunes Llama 3.2 on it.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"New-Safety Benchmark gains may reflect alignment to the Qwen judge rather than to human-valued safety; only the SafetyBench +3.8 gain is independent.","rationale":"The central claim rests on two pieces of evidence: the external SafetyBench improvement (+3.8) and the New-Safety Benchmark improvement ('over 30%'). The New-Safety Benchmark, however, does not provide independent evidence because the same Qwen family is used to generate the training responses (Section 3.6), score and filter them (Section 3.7), and judge the final model's outputs (Section 4.3). SFT with Qwen-selected data is effectively training the 1B model to imitate Qwen's safety judgments, so a Qwen-based evaluator is partially measuring whether imitation succeeded. The manual review in Section 4.3 only applies to responses the API is uncertain about, so it does not independently validate the bulk of the scoring. The SafetyBench result survives this objection because it uses fixed multiple-choice answers and is not scored by Qwen; however, it is a single run with no confidence intervals and does not by itself demonstrate that the proposed multi-dimensional method is the cause. The paper also omits ablations (no Safety-Prompts-only SFT, no random sampling, no human evaluation), so the contribution of the multi-intent classification, diversity control, and safety reward filter is unquantified. Thus the paper is not rejectable outright, but it is only conditionally acceptable: the authors need to provide an independent judge or human evaluation, ablations, and ideally multiple seeds. This aligns with the reader's conditional verdict, so no change to the verdict is needed.","tokens_in":8843,"tokens_out":6490,"duration_ms":62531,"concrete_test":"Have independent human annotators, blind to model identity, score a random sample of at least 300 responses per model from the 14-category New-Safety Benchmark using the same safety definitions, and compute the base-versus-ours improvement. If the human-judged improvement is substantially smaller than the Qwen-turbo-judged 'over 30%' average (or near zero), the headline New-Safety Benchmark gain is an artifact of matching the training-time judge; the paper should then rest on SafetyBench alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the 'over 30%' improvement on the New-Safety Benchmark may measure agreement with the training-time judge rather than safety itself. Section 3.6 regenerates responses with Qwen2.5-7B-Instruct; Section 3.7 uses Qwen-turbo-API to score both instruction safety and response safety, and selects samples with low instruction scores and high response scores; Section 4.3 then uses Qwen-turbo-API to score the fine-tuned model's responses in the 14-category benchmark, with manual review only for responses the API flags as uncertain. Because the training data is filtered and generated by the same model family that later evaluates the output, the SFT model is explicitly optimized to satisfy Qwen-turbo's notion of safety. The reported gains on the 14 categories are therefore not independent evidence; they may be an artifact of style or value alignment to Qwen. The externally scored SafetyBench result (Table 5-1, +3.8) is not affected by this circularity, but it is a single number without error bars or multiple runs. Additionally, the 14-category evaluation set samples from CValues-Comparison and S-Eval (Section 4.2), while CValues-Comparison is collected and used for risk-classification expansion (Section 3.3); the paper gives no disclosure to rule out overlap between evaluation samples and the expanded training prompts. This second-order contamination would further erode the only benchmark supporting the 'multi-dimensional' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pipeline for constructing safety-aligned SFT data: it collects Chinese safety prompts (Safety-Prompts, SafetyBench, CValues-Comparison), classifies attack instructions into 100+ intent labels using Qwen2.5-7B-Instruct, expands prompt diversity by two generation schemes, regenerates responses with Qwen2.5-7B-Instruct, scores instruction/response safety with Qwen-turbo-API, and selects roughly 160k safety samples plus 40k general instructions to fine-tune Llama-3.2-1B-Instruct. The authors evaluate the resulting model on SafetyBench (7 categories), a newly constructed 14-category benchmark, CValues responsibility, and standard general benchmarks (gsm8k, mmlu, cmmlu, ceval, HumanEval). They report an average gain of +3.8 on SafetyBench, a rise in responsibility accuracy from about 0.576 to 0.599, and mixed but mostly flat general-capability scores.","tokens_in":9153,"tokens_out":3374,"duration_ms":33313,"significance":"If the claims are correct, the paper would provide a cheap, practical recipe for improving the Chinese-safety behavior of a small open-weight model via SFT on a filtered, diversified dataset, while preserving general capability. The pipeline is described concretely with prompts and training hyperparameters, and the authors make a good-faith attempt to construct a finer-grained 14-category evaluation. However, the load-bearing evidence is currently weakened by the fact that the model family used to generate and rank training data is also the judge of the new benchmark, by the absence of any statistical uncertainty, and by the lack of ablations isolating the proposed multi-dimensional diversity and defense components. The external SafetyBench gain is a useful existence proof but is a single point estimate.","major_comments":[{"comment":"The paper's central 'multi-dimensional' safety claim rests on the New-Safety Benchmark results, but that benchmark is scored by the same model family used to produce the training data. Section 3.6 regenerates responses with Qwen2.5-7B-Instruct; Section 3.7 uses Qwen-turbo-API to score both instruction safety and response safety and to select the final 200k-sample dataset; Section 4.3 then uses Qwen-turbo-API to judge the fine-tuned model's responses in the 14-category benchmark. This circularity means that the reported gains (e.g., IA +59%, OFF +53%, MI +51% in Figure 5-3) may largely measure the degree to which the SFT model has aligned to Qwen's safety preferences rather than to human-valued safety. To support the claim, the authors should provide an independent evaluation, for example human annotation of a representative subset or a judge from a different model family, and should show agreement between the Qwen judge and human labels.","section":"Sections 3.6, 3.7, and 4.3"},{"comment":"There is a potential contamination between training-related data and the evaluation set. Section 3.3 states that CValues-Comparison is used for 'safety scenario analysis and further expansion of risk classification,' and Section 4.2 builds the 1.5k evaluation set by sampling from CValues-Comparison and S-Eval. The paper never discloses whether the evaluation samples are disjoint from the prompts used in the expansion step or from the training set after expansion. If overlapping prompts appear in both stages, the New-Safety Benchmark results would be inflated. The authors should explicitly verify and report the absence of overlap, or remove the overlapping samples.","section":"Sections 4.2 and 3.3"},{"comment":"All reported results are single-run point estimates without error bars, standard deviations, or significance tests. The main external evidence is the +3.8 SafetyBench gain in Table 5-1, which could plausibly lie within run-to-run variance for a 1B-parameter model fine-tuned with a single seed. The authors should report results over at least three random seeds, with means and standard deviations, and ideally a paired significance test, to establish that the observed improvements are stable and not noise.","section":"Section 5.2.2 and Table 5-1"},{"comment":"The paper attributes its gains to the multi-dimensional attack/defense pipeline, but it contains no ablation isolating the contributions of the specific components: the multi-intent reclassification and diversity control (Section 3.5), the response regeneration by HighSafeLLM (Section 3.6), and the safety-reward-model filtering (Section 3.7). Without such ablations, the observed improvement could come simply from training on a larger, more diverse safety dataset or from the response-regeneration step alone. The authors should run and report ablations that remove or vary each of these components.","section":"Sections 3.4–3.8"}],"minor_comments":[{"comment":"The paper states that the learning rate and batch size were tuned by 'multiple experimental comparisons' but does not report the search range or the values tried; please provide this information for reproducibility.","section":"Section 5.2.1"},{"comment":"The text uses '3w' and '1w' without defining the units; presumably these are 30,000 and 10,000 samples respectively. Please spell these out.","section":"Section 4.2"},{"comment":"The 14-category results are only shown as a bar chart; the numerical safety scores per category are not listed in the text or a table. Please include the underlying numbers so readers can reproduce the average-over-30% claim in the conclusion.","section":"Figure 5-3"},{"comment":"There is a typo in 'innova-tively' with a broken hyphen; please correct it.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper assembles a standard safety-alignment data pipeline (intent classification, prompt expansion, response regeneration, reward-model filtering) and reports a +3.8 point gain on SafetyBench against Llama-3.2-1B-Instruct. That external number is the one thing I'd trust. The much larger gains on the self-built 14-category benchmark should be read as alignment to the Qwen judge, not to human-valued safety.\n\nWhat's genuinely useful: the 14-category taxonomy is a reasonable extension of SafetyBench, the pipeline is a clean recipe that others can copy, and the authors did check general capabilities (ceval actually improves). The SafetyBench number is not explained by circularity, and the responsibility gain is consistent with better Chinese-aligned responses.\n\nThe circular loop is real. Section 3.6 uses Qwen2.5-7B to write training responses, Section 3.7 uses Qwen-turbo to filter them, and Section 4.3 uses Qwen-turbo to grade the final model. Fine-tuning on data selected by Qwen to satisfy Qwen means the new-benchmark scores partly measure agreement with Qwen's style and values. The manual review only kicks in when Qwen is uncertain, so it doesn't break the loop. On top of that, all results are single-run point estimates with no error bars, and there are no ablations for the diversity-control or reward-filtering components. The evaluation set also samples from CValues-Comparison, which was used in Section 3.3 for risk-classification expansion; the paper doesn't rule out overlap, so even the 14-category numbers could be inflated by train/test leakage.\n\nThat said, the central SafetyBench result (+3.8) is a legitimate external signal, and the paper is coherent. The limitations are serious but fixable.\n\nSafety alignment practitioners working with small models or Chinese-language safety data will find the recipe useful; anyone using the new benchmark should independently score with a different judge.\n\nI'd send it to peer review, but with the expectation of major revision: add an independent safety judge (e.g., GPT-4 or human rating), run multiple seeds, add ablations, and release data and code. As is, I'd treat the 'over 30%' claim as unverified.","headline":"A usable safety-data recipe with one solid external result, but the new-benchmark gains mostly measure alignment to the Qwen judge.","tokens_in":9648,"tokens_out":2417,"would_cite":false,"duration_ms":22516,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Constructing a 200k-sample safety SFT set with fine-grained attack labels and score-filtered responses lifts a 1B model's safety by ~4 points on SafetyBench and >30% on a wider benchmark, without hurting general capability.","keywords":["large language model safety","safety alignment","supervised fine-tuning","attack instruction diversity","multi-intent classification","safety reward model","Chinese safety evaluation","adversarial robustness"],"falsifier":"Rescore the 1.5k New-Safety Benchmark responses with independent human raters or with a second judge from a different model family, comparing the base and fine-tuned models under identical prompts. If the fine-tuned model's judged safety advantage falls well below the reported >30% average gain, then the headline result largely reflects agreement with the original judge rather than a discoverable safety improvement. A simpler check: rerun the SafetyBench evaluation multiple times on held-out subsets and confirm the +3.8-point average gain is reproducible outside the exact training/evaluation split.","tokens_in":8626,"feed_emoji":"🛡️","tokens_out":10161,"duration_ms":89597,"temperature":0.7,"pith_summary":"This paper sets out to show that the weak link in safety alignment is the data, not the algorithm: if you build a supervised fine-tuning set that covers many kinds of attack instructions and pairs each with a high-quality safe answer, ordinary SFT can make a small model much harder to provoke into unsafe generation. The proposed pipeline classifies attack prompts into over one hundred intent themes, expands low-diversity categories with generated prompts, regenerates answers with a stronger safety model, and filters samples using perplexity plus safety scores. Fine-tuning Llama3.2-1B-Instruct on the resulting ~200k-sample set is reported to raise SafetyBench average from 56 to 59.8, lift the 14-category New-Safety Benchmark average by over 30%, and leave general benchmarks essentially flat. If these results hold, safety alignment becomes a data-construction problem rather than a training-cost problem, and the same dataset recipe could be reused for other base models and languages.","feed_headline":"Safety fine-tune on 200k samples lifts a 1B model past a 7B","feed_subtitle":"Paper reports +3.8 on SafetyBench and >30% on a wider safety benchmark, with general abilities flat.","key_machinery":"The load-bearing machinery is the safety-aligned SFT data pipeline, specifically its three-way sample filter. Each candidate example is scored on prompt perplexity, to remove degenerate or unnatural instructions; response safety, judged by a large safety model, to keep the most defensible answers among the original and regenerated versions; and instruction safety, scored low for genuinely aggressive attack prompts so that the model trains against hard attacks. Coupled with multi-intent classification and diversity control, this filter turns raw prompt collections into the ~200k-sample fine-tuning set. The same judge model is later reused as the evaluator for the 14-category benchmark.","core_discovery":"The central discovery claimed is that multi-dimensional attack–defense data construction—not a more powerful alignment algorithm—is what drives the safety gain. The author builds a safety SFT set in which each instruction carries a fine-grained safety-intent label (over 100 labels, later consolidated into 14 benchmark categories), diversifies rare categories through rejection sampling and prompt augmentation, replaces or supplements the original ChatGPT responses with answers regenerated by a safer large model using chain-of-thought reasoning, and then keeps only samples whose prompts are fluent, whose responses score high on safety, and whose instructions score low (i.e., are more attack-like). Applying full-parameter SFT to Llama3.2-1B-Instruct with this set, the paper reports +3.8 points average on SafetyBench, average gains above 30% on the new 14-category benchmark with instruction-attack, offense, and malicious-inducement categories rising more than 50%, improved CValues responsibility accuracy, and negligible change on GSM8K, MMLU, CMMLU, and HumanEval.","pith_inferences":["Because the same commercial judge model scores training data and grades the 14-category benchmark, part of the reported gain could be agreement with that judge; re-scoring with independent human raters or a second judge would isolate the true safety improvement.","The +3.8-point SafetyBench gain, measured on fixed multiple-choice questions, is the least judge-dependent result and could serve as the primary external check of the method.","A direct ablation—training on the original Safety-Prompts subset alone versus the full expanded, regenerated, and filtered set—would show how much of the gain comes from each pipeline stage.","The dataset recipe appears portable: because it only manipulates attack prompts, responses, and scores, it should transfer to other base models and non-Chinese languages, though the paper only verifies Chinese."],"forward_implications":["The reported results imply that a 1B model fine-tuned on this 200k-sample set can exceed a 7B Chinese chat model on the SafetyBench average (59.8 vs 59.2).","Constructing safety SFT data with fine-grained attack-intent labels, diversity expansion, safe-response regeneration, and score-based filtering is claimed to be a reusable recipe for alignment without RLHF.","General capability need not trade off against safety: the reported general benchmarks stay within 0.5 points except C-Eval, which rises 2.5 points.","The largest reported gains occur on categories the older benchmarks under-covered (instruction attack, malicious inducement, offense), suggesting that benchmark coverage itself was masking real safety weaknesses."],"supporting_citations":[{"why":"It supplies the 11,435-question multiple-choice benchmark and seven safety scenarios used as the external safety metric.","marker":"[16]"},{"why":"It provides the CValues evaluation scripts and responsibility data used for the responsibility accuracy metric and scenario expansion.","marker":"[17]"},{"why":"It supplies S-Eval risk prompts used in the 14-category evaluation set and the strategy for generating diverse attack prompts.","marker":"[18]"},{"why":"It provides the 40,000 general instruction samples mixed into the safety SFT set to preserve general capability.","marker":"[19]"},{"why":"It supplies the Safety-Prompts data, the core source of attack instructions and original ChatGPT responses.","marker":"[20]"},{"why":"It supplies additional Chinese comparison data used to expand the safety scenario taxonomy into 14 categories.","marker":"[21]"},{"why":"It is the training framework used to run the full-parameter SFT and to generate the reported evaluation numbers.","marker":"[24]"},{"why":"It is the evaluation harness used to compute the general-capability benchmark numbers including GSM8K, MMLU, and CMMLU.","marker":"[25]"},{"why":"It is the code-generation benchmark whose HumanEval score is reported to stay flat after safety tuning.","marker":"[26]"}],"fun_headline_variants":["Safety SFT on attack-defense data lifts 1B model past 7B","Multi-dimensional attack-defense data enhances LLM safety","Curated attack-defense set boosts safety, keeps general skills","1B model beats 7B on safety via diverse attack-defense tuning","Attack-defense alignment data lifts safety of small LLM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The commercial judge model's safety scores used to filter training data and to grade the 14-category benchmark are assumed to match human judgments; if they do not, the reported gains mostly show that the model learned to answer the way that judge prefers.","fun_headline_variants_meta":{"raw":{"variants":["Safety SFT on attack-defense data lifts 1B model past 7B","Multi-dimensional attack-defense data enhances LLM safety","Curated attack-defense set boosts safety, keeps general skills","1B model beats 7B on safety via diverse attack-defense tuning","Attack-defense alignment data lifts safety of small LLM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000658,"raw_usage":{"total_tokens":2991,"prompt_tokens":909,"completion_tokens":2082,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":2010}},"tokens_in":525,"tokens_out":2082,"duration_ms":13937,"temperature":1.0,"reasoning_tokens":2010,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:48:50.133379+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rescore the 1.5k New-Safety Benchmark responses with independent human raters or with a second judge from a different model family, comparing the base and fine-tuned models under identical prompts. If the fine-tuned model's judged safety advantage falls well below the reported >30% average gain, then the headline result largely reflects agreement with the original judge rather than a discoverable safety improvement. A simpler check: rerun the SafetyBench evaluation multiple times on held-out subsets and confirm the +3.8-point average gain is reproducible outside the exact training/evaluation split.","supporting_citations":[{"cited_title":"A Post-Training Enhanced Optimization Approach for Small Language Models","cited_arxiv_id":null,"evidence_quote":"It provides the 40,000 general instruction samples mixed into the safety SFT set to preserve general capability."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies additional Chinese comparison data used to expand the safety scenario taxonomy into 14 categories."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the training framework used to run the full-parameter SFT and to generate the reported evaluation numbers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the evaluation harness used to compute the general-capability benchmark numbers including GSM8K, MMLU, and CMMLU."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the code-generation benchmark whose HumanEval score is reported to stay flat after safety tuning."}],"review_version":1}