{"id":"dd7b2ae0-caf1-47e0-b997-c5a378afd953","arxiv_id":"2508.09224","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Safe-completions train models to maximize helpfulness within safety constraints using a composite reward, and the paper reports safety and helpfulness gains over refusal-based training in GPT-5.","lead":"OpenAI researchers replaced the usual 'refuse or comply' safety rule with a reward that grades each response for how safe and helpful it is, letting GPT-5 give partial, redirected answers in risky cases. The paper reports that this 'safe-completion' training improves safety on ambiguous dual-use prompts and makes the model more useful, backed by human reviews.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The controlled experiment may conflate the safe-completion reward with a change in the safety policy: Section 2.3 revises the policy to a 'meaningful facilitation' threshold, and the paper never states that CE-Refusal used the identical updated spec.","rationale":"The reader's weakest assumption was about evaluation-measure validity (autograders sharing the training policy, helpfulness rubric encoding the desired behavior). I agree those are concerns, but I identify a more fundamental threat to internal validity: the controlled experiment may not hold the policy constant. Section 2.3 describes a policy change as part of the safe-completions method, and the paper never confirms that CE-Refusal was trained with the same updated policy spec. If the policy differs, the experiment conflates two interventions. The human evaluation is reassuring but does not rescue the causal attribution, because the human reviewers also compare models that may differ in the policy they were trained under. The paper's central claim—that safe-completion training, rather than the policy update, drives the gains—therefore rests on an unverified assumption about the experimental design. This is a concrete, fixable issue: the authors should state the policy version used for each CE arm or release the training specs. The verdict remains CONDITIONAL, as the paper could still be correct; the concern does not change the reader's conditional assessment but highlights a different, more design-level weakness that should be addressed before acceptance.","tokens_in":13667,"tokens_out":6929,"duration_ms":76995,"concrete_test":"Obtain or reconstruct the training configurations for the CE pair: verify from the paper, appendix, or authors whether both CE-Refusal and CE-SafeComplete use the identical updated policy spec (including the 'meaningful facilitation' criterion). If the policy differs, rerun the controlled experiment with CE-Refusal retrained under the updated policy but with refusal-oriented objectives, and compare against CE-SafeComplete under the same policy. If the safety/helpfulness gaps shrink or vanish, the headline improvements are attributable to the policy update rather than to safe-completion training.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the safe-completion training paradigm, not a policy change, improves safety and helpfulness. However, Section 2.3 ('Policy Development') describes a substantive update to the illicit wrongdoing policy: the threshold shifts from binary 'advice or instructions' to 'meaningful facilitation.' This update is presented as part of the safe-completions method. The controlled experiment (Section 3.1) says the only intended difference is the safety-training strategy, but it never explicitly states that the CE-Refusal baseline was trained with the same updated policy spec as CE-SafeComplete. If CE-Refusal was trained under the previous, more restrictive policy, then the observed safety and helpfulness gains in Figures 4-6 could be driven by the policy itself (which now permits high-level, non-actionable guidance), not by the safe-completion reward design. The safety autograder is category-specific and policy-based, so evaluating both models with the updated policy would further bias the comparison in favor of CE-SafeComplete if the baseline was trained on an older policy. The production comparison is confounded by architecture, data, and policy differences. Thus the cleanest evidence for the causal claim is not currently verified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 'safe-completions,' an output-centric safety-training paradigm that replaces binary refusal training. The method builds on deliberative alignment: an SFT stage teaches spec-aware chain-of-thought with three response modes (direct answer, safe-completion, refuse with redirection), and an RL stage uses a composite reward r_i = h_i * s_i, where s_i is a policy-compliance score and h_i combines direct and indirect helpfulness. The paper also describes a policy update shifting the illicit-wrongdoing threshold from 'advice or instructions' to 'meaningful facilitation.' It evaluates two matched model pairs—a controlled ablation (CE-Refusal vs. CE-SafeComplete) and a production pair (o3 vs. gpt5-r)—using policy-based autograders and a human evaluation, plus a biorisk case study. The central claims are that safe-completion training improves safety on dual-use prompts, reduces the severity of residual safety failures, and substantially increases helpfulness.","tokens_in":13966,"tokens_out":4120,"duration_ms":47544,"significance":"If the causal effect is established, the contribution is significant: it articulates and empirically tests an output-centric training objective that addresses dual-use brittleness and over-refusal, and the harm-severity framing is a useful addition to typical binary safety metrics. The controlled ablation and the external human evaluation are genuine strengths, and the manuscript is explicit about many limitations. However, the cleanest evidence is weakened by an unstated relationship between the policy update described in Section 2.3 and the baseline used in the controlled experiment, and by evaluation metrics that may partially encode the behavior the method is designed to produce. The central claim is plausible and important, but the causal attribution needs to be pinned down before publication.","major_comments":[{"comment":"The controlled experiment is the primary evidence for the causal claim that the safe-completion reward design, rather than a policy change, drives the observed gains. Section 3.1 states that the only intended difference between CE-Refusal and CE-SafeComplete is the safety-training strategy, but Section 2.3 describes a substantive update to the illicit-wrongdoing policy: the threshold shifts from 'advice or instructions' to 'meaningful facilitation.' The paper never states that CE-Refusal was trained with the identical updated spec. If CE-Refusal used the older, more restrictive policy, then the safety and helpfulness improvements in Figures 4–6 could be due to the policy relaxation itself (which now permits high-level, non-actionable guidance) rather than to the safe-completion reward. Moreover, the safety autograder is category-specific and policy-based, so evaluating both models agains","section":"Section 2.3 vs. Section 3.1"},{"comment":"The helpfulness rubric appears to encode the very shift the paper aims to demonstrate. In Table 1, a hard refusal receives helpfulness 1, safety education/risk context receives 2, and concrete alternatives receive 3. Since safe-completion training explicitly optimizes indirect helpfulness and redirection, the reported 'substantial helpfulness gains' from moving away from hard refusals are partly baked into the rubric. The human evaluation is policy-free and thus partially addresses this concern, but it covers only 2,000 prompts, uses 3–5 reviewers per item, and its relative-helpfulness and balance questions may inherit the same preference for redirections over refusals. To make the helpfulness claim non-circular, report human helpfulness on a rubric that does not by construction rank safe-completions above refusals, or show that the autograder results are robust to re-scoring hard refusa","section":"Table 1 and Section 3.1.1"},{"comment":"The safety and harm-severity autograders are not validated against human judgment. The safety autograder uses the same internal content policy that was used in training, and the severity autograder, though described as spec-agnostic, has no reported calibration, inter-rater agreement, or error analysis. Because GPT-5 is optimized against this family of rewards, the autograder results could reflect policy-compliance rather than real-world safety. Please add a meta-evaluation: e.g., agreement between model graders and the human reviewers on a shared subset, and an error analysis on dual-use and malicious categories. Without this, the reported safety gains in Figures 4 and 5 remain difficult to interpret.","section":"Section 3.1.1 and Section 3.2.2"}],"minor_comments":[{"comment":"There are typographical and spacing errors: 'we proposesafe-completions' in the abstract, 'judgesafe-completion' in Section 3.4.2, and 'W ang' in the reference list. Please proofread.","section":"Abstract and throughout"},{"comment":"The text says gains are 'significant' and 'small but significant,' but no test statistics, effect sizes, or confidence intervals are reported in or next to Figure 4. Given the large prompt counts and multiple comparisons, report the relevant statistics.","section":"Figure 4 and Section 3.2.1"},{"comment":"The human evaluation would benefit from reporting inter-rater reliability (e.g., Krippendorff's alpha or Fleiss' kappa) and the exact number of responses per model pair. The current description ('3–5 human reviewers' on 2,000 prompts) leaves uncertainty about the stability of the win-rate estimates.","section":"Section 3.4.1"},{"comment":"The 'meaningful facilitation' policy update is described qualitatively. Include more concrete examples of what is allowed versus disallowed under the updated threshold, or point to a public version of the spec, so readers can assess how much of the behavior change is policy-driven.","section":"Section 2.3"},{"comment":"The abbreviation 'gpt5-r' is used without definition; if it denotes 'GPT-5 Thinking (reasoning)' it should be stated at first use. Also, Figure 5 bars are labeled with percentages but not with the number of unsafe responses per model/intent; add sample sizes.","section":"Section 3.1 and Appendix"}],"recommendation":"major_revision","confidential_remarks":"The load-bearing issue is the policy-change confound in the controlled experiment. If the authors confirm that CE-Refusal was trained under the same 'meaningful facilitation' spec as CE-SafeComplete, the first major comment can be resolved by a clarifying statement. If not, the central causal claim is unsupported by the current experiment, and a rerun or substantial re-analysis would be needed. The helpfulness-rubric circularity and the missing autograder validation also need to be addressed, but they are secondary to the policy confound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two takeaways. This is a real industrial contribution: the safe-completion reward (safety times helpfulness, with direct and indirect helpfulness paths) is a sensible alternative to binary refusal training, backed by controlled ablations, production comparisons, and a policy-free human eval. The main caveat: Section 2.3 changes the policy threshold from 'advice or instructions' to 'meaningful facilitation,' and the paper never says the refusal baseline used the same updated spec. That is a load-bearing confound.\n\nWhat's good: the controlled experiment is the right design, holding architecture and post-training fixed; they report safety and helpfulness separately, stratify by intent, and add a harm-severity analysis. The biorisk case study is a useful test. They also acknowledge related work fairly (DA, RBR, Constitutional AI, Claude 3.7).\n\nSoft spots, in order. (1) The policy-spec issue above. If CE-Refusal was trained under the older, stricter policy, the gains could be from the policy, not the reward. The paper should state explicitly that the spec was identical, or rerun the ablation. (2) The safety autograder is policy-based, so it shares the spec used in training. The severity grader is spec-agnostic, and the human eval is policy-free, which helps, but the human eval covers 2,000 of ~9,000 prompts with 3–5 reviewers and no inter-annotator agreement. (3) The helpfulness rubric scores redirections as more helpful than hard refusals—level 2 vs 1—so the helpfulness gain partly reflects the metric. (4) Production comparison is confounded by architecture and data, and there's a small regression on malicious illicit prompts that's downplayed.\n\nNone of this sinks the paper. The design is a genuine step toward output-centric safety, and the evidence is better than most industry system-card reports. But the central causal claim needs a cleaner comparison.\n\nFor peer review: send it out. A good referee will ask for clarification on the spec, inter-annotator agreement, and preferably the evaluation prompts. If those are addressed, this is a useful paper for both practitioners and researchers.","headline":"A serious and readable case for output-centric safety training, but the policy change in Section 2.3 may explain part of the gains, so the causal claim needs tightening.","tokens_in":14428,"tokens_out":3660,"would_cite":true,"duration_ms":39434,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training AI to give safe partial answers instead of blanket refusals makes models safer and more helpful on ambiguous requests.","keywords":["safe-completions","safety training","refusal boundary","dual-use","output-centric safety","helpfulness","deliberative alignment","reinforcement learning"],"falsifier":"Conduct an independent audit in which domain experts (e.g., biologists, security engineers) score the actionability and harm of model outputs on dual-use prompts without seeing either model’s output style or the policy autograder’s labels. If the safe-completion model’s outputs are found to be no less actionable or harmful than the refusal-trained model’s full compliance, the claimed safety improvement is an artifact of the grading rubric. Conversely, counting a redirection as a refusal on the helpfulness rubric would test whether the reported helpfulness gains are real.","tokens_in":13596,"feed_emoji":"🛡️","tokens_out":6176,"duration_ms":64730,"temperature":0.7,"pith_summary":"The paper argues that binary refusal training—classifying a prompt as safe or unsafe and then either fully complying or refusing—is brittle for dual-use requests whose danger depends on how actionable the answer is. It proposes safe-completions: train the model to maximize helpfulness subject to the constraint that the output itself is safe, using three response modes (direct answer, safe-completion with high-level non-operational guidance, refuse-with-redirection). Built into GPT-5 and compared with refusal-trained o3 and a controlled ablation, the approach reports improved safety on dual-use prompts, comparable safety on explicitly malicious requests, reduced severity of residual failures, and substantially higher helpfulness across intent categories. If correct, this reframes safety training as an output-safety scoring problem, which matters for domains like biology and cybersecurity where detailed information can cause harm.","feed_headline":"Safe-completions beat hard refusals in AI safety training","feed_subtitle":"New output-centric training on GPT-5 reduces harmful dual-use answers and boosts helpfulness over refusal models.","key_machinery":"The central mechanism is the safe-completion reward: in RL, final reward $r_i = h_i \\cdot s_i$ where $s_i \\in [0,1]$ is an output-safety score (1 for fully compliant, 0 for severe violations, intermediate for low-severity or borderline) and $h_i \\in [0,1]$ is a helpfulness score combining direct helpfulness (fulfilling the user’s stated task) with indirect helpfulness (informative refusals, risk framing, and constructive alternatives). Multiplying the two forces the model to be safe first, then as helpful as possible; when direct compliance would violate policy, the model earns reward via indirect helpfulness. The SFT stage precedes this by teaching three output modes—direct answer, safe-com","core_discovery":"The core claim is that replacing the refusal boundary with an output-centric objective—final reward $r_i = h_i \\cdot s_i$, the product of a helpfulness score $h_i$ and a safety-compliance score $s_i$—yields models that are safer and more helpful on ambiguous and dual-use prompts while maintaining safety on clearly malicious ones. The SFT stage teaches spec-aware chain-of-thought and three response modes; the RL stage penalizes unsafe outputs in proportion to severity and rewards both direct and indirect helpfulness. On roughly 9,000 production safety prompts the method “fails softer”: unsafe outputs shift from Moderate/High to Low/Negligible severity, and on the biorisk case study the produc","pith_inferences":["If this generalizes, safety policies themselves may be rewritten around “meaningful facilitation” thresholds rather than prompt categories, changing how content policy is specified.","The severity-weighted safety penalty suggests a natural upgrade: training directly on harm-severity labels as a cost signal, not just binary safe/unsafe, could further push residual failures to negligible.","The helpfulness rubric that scores redirections as more helpful than refusals is itself contestable; external value models that weight user autonomy may rank the same responses differently, so the helpfulness gain may not transfer to all user populations.","A testable implication: on dual-use categories where safe alternatives are rare, the model should show smaller helpfulness gains and more over-refusal; measuring that could isolate how much of the gain comes from redirection quality versus the safety reward."],"forward_implications":["Safety training can shift from classifying user intent to scoring assistant output, enabling graduated responses instead of binary comply-or-refuse.","Dual-use queries in biology and cybersecurity can be answered with high-level, non-actionable guidance without sacrificing safety.","Residual safety failures become less severe: unsafe outputs migrate from moderate/high harm toward low/negligible.","Helpfulness can rise without a safety trade-off, including on malicious prompts where redirections replace unhelpful hard refusals.","The approach works better on stronger models: production GPT-5 showed larger gains than the controlled ablation."],"supporting_citations":[{"why":"Provides the deliberative-alignment SFT pipeline and spec-aware chain-of-thought that safe-completions builds on.","marker":"[9]"},{"why":"Supplies the rule-based-rewards approach to structured safety feedback that the composite reward extends.","marker":"[10]"},{"why":"Documents the refusal-trained behavior of GPT-4o, the predecessor baseline for binary safety training.","marker":"[12]"},{"why":"Defines the o3/o4-mini refusal-trained production model and the training setup used for the controlled experiment pair.","marker":"[14]"},{"why":"Establishes the helpfulness–harmlessness balance objective and preference data used as a related-work baseline.","marker":"[5]"},{"why":"Provides the content-policy model spec that defines safety categories and specifications used in training and evaluation.","marker":"[13]"},{"why":"Demonstrates the brittleness of refusal boundaries under adversarial or ambiguous prompts, motivating output-centric training.","marker":"[21]"}],"fun_headline_variants":["Output-centric safety training beats refusals on GPT-5","Safe-completions: safer and more helpful than refusals","From refusals to safe-completions: AI safety improved","GPT-5's new safety training: fewer harmful answers, more help","Output-centric training reduces dual-use risks in AI"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The evaluation’s validity hinges on the autograders: the category-specific policy autograder’s binary safe/unsafe label, the 1–4 helpfulness rubric that scores redirections as more helpful than hard refusals, and the intent classifier’s Benign/Dual-use/Malicious labels must capture real-world safety and usefulness; if they systematically favor safe-completion behavior over genuine harm reduction, the reported gains would be artifacts. The human evaluation mitigates this but c","fun_headline_variants_meta":{"raw":{"variants":["Output-centric safety training beats refusals on GPT-5","Safe-completions: safer and more helpful than refusals","From refusals to safe-completions: AI safety improved","GPT-5's new safety training: fewer harmful answers, more help","Output-centric training reduces dual-use risks in AI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000175,"raw_usage":{"total_tokens":1111,"prompt_tokens":724,"completion_tokens":387,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":310}},"tokens_in":468,"tokens_out":387,"duration_ms":4505,"temperature":1.0,"reasoning_tokens":310,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:30:17.975507+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Conduct an independent audit in which domain experts (e.g., biologists, security engineers) score the actionability and harm of model outputs on dual-use prompts without seeing either model’s output style or the policy autograder’s labels. If the safe-completion model’s outputs are found to be no less actionable or harmful than the refusal-trained model’s full compliance, the claimed safety improvement is an artifact of the grading rubric. Conversely, counting a redirection as a refusal on the helpfulness rubric would test whether the reported helpfulness gains are real.","supporting_citations":[{"cited_title":"[14]OpenAI","cited_arxiv_id":null,"evidence_quote":"Establishes the helpfulness–harmlessness balance objective and preference data used as a related-work baseline."}],"review_version":1}