Pith. sign in

REVIEW 4 major objections 5 minor 14 references

Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Blocking one token flips censored AI refusals into answers

desk verdict A zero-parameter logit suppression that flips DeepSeek-R1 refusals into substantive answers; the finding looks real but the paper leaves key validations undone. read the letter →

arxiv 2505.23848 v1 pith:TLLOITWX submitted 2025-05-28 cs.CL cs.LG

Harvey Dam , Jonas Knochelmann , Vinu Joseph , Ganesh Gopalakrishnan This is my paper
classification cs.CLcs.LG
keywords logitsuppressionrefusalsubspacechain-of-thoughtRLHFalignmentDeepSeek-R1censorshipbypasstoken-levelinterventionoutputboundary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that one narrow, generation-time intervention can derail refusals in reasoning-style language models. In the DeepSeek-R1 distillation family, a refusal is typically signaled by a two-newline token appearing immediately after the chain-of-thought opener . Setting that token's raw probability score to zero at that single position blocks the model's path into what the authors call the refusal subspace, and substantially raises the share of answers that are both relevant and coherent. The method needs no new training data, no weight edits, and no prompt rewriting, and the authors report no loss on standard benchmarks. If right, this means some censored knowledge is not erased but gated behind a few predictable output tokens.

What carries the argument

The load-bearing object is the token transition <think> followed by \n\n: a boundary where the model's output distribution splits between a refusal subspace and a normal response subspace. The mechanism is a logits processor applied during autoregressive generation that waits for the <think> token and then sets the \n\n logit to $-\infty$ for exactly the next step, with an optional second processor that sets the end-of-sequence logit to $-\infty$ after </think>. It works by blocking the refusal trigger while preserving the model's natural continuation otherwise.

What would settle it

Count, on the same sampled CCP and DECCP prompts, how often a baseline refusal is actually preceded by <think>\n\n and how often a substantive answer avoids it. If the pattern is present in well under 90 percent of refusals, or if clamping the \n\n logit frequently produces incoherent or empty output instead of an answer, the claimed binary separation does not hold for these models.

Watch

Extended reading notes

Core claim

The central discovery is that censorship in the tested RLHF-aligned reasoning models is implemented, at least partly, at a punctuation-level branch point: when the generated sequence reaches <think>, the model almost always follows with \n\n before refusing or evading, whereas a substantive response usually proceeds without that double newline. Overriding that single transition by clamping the \n\n logit to $-\infty$ once the <think> token has been emitted diverts generation out of the refusal trajectory, greatly increasing the proportion of 'relevant coherent' chain-of-thought blocks and 'answered' final responses across four DeepSeek-R1 distillations (Qwen 7B/14B/32B and Llama 70B) on two censorship probe datasets. Optionally clamping the end-of-sequence logit after </think> further nudges the model toward a final answer, although this second step is not uniformly beneficial and can push some outputs toward refusal or evasive propaganda. Standard-benchmark performance is reported unchanged, so the intervention is claimed to redirect behavior without degrading general capability.

Load-bearing premise

The whole method hangs on the claim that, in these models, the exact two-character sequence <think> followed by two newlines is a near-certain warning of an upcoming refusal, and that removing that warning diverts the model into a real answer rather than into a different failure mode.

Editorial extensions

If this is right

  • On both censorship probe datasets, suppressing \n\n after <think> greatly increases the proportion of relevant coherent CoT blocks and answered responses across all four model sizes.
  • Because only token probabilities are changed during generation, the method requires no datasets, no parameter updates, and no prompt modification, making it applicable through a standard decoding hook.
  • The paper's stated limitation is that the technique only helps models whose refusals are tied to identifiable output patterns, and its effectiveness may vary across architectures and alignment strategies.
  • Suppressing \n\n alone can increase empty final responses; adding EOS suppression after </think> fixes most but not all of these, and sometimes raises refusal or evasive propaganda instead.
  • The authors claim no degradation on standard benchmarks, so the intervention is presented as redirecting refusal behavior rather than reducing general capability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the refusal trigger is this local, the same suppression idea could be made adaptive: a decoder that learns which token transitions most strongly predict refusals for a given model, rather than hardcoding <think>\n\n.
  • The increased empty-final-response rate after \n\n suppression suggests the refusal pressure can resurface one decision later, at </think>; a natural test is whether repeated suppression at several boundary points compounds or eventually destabilizes the answer.
  • The paper's evidence implies that 'unlearning' audits should include generation-time probing with token-level interventions, because knowledge judged erased through normal sampling may still be reachable through a single logit edit.
  • The right-to-be-forgotten consequence the paper flags in its impact statement is worth taking literally: if knowledge persists behind a token boundary, deletion compliance cannot be certified by behavioral refusal tests alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes a generation-time intervention to reduce refusals in DeepSeek-R1 distilled models. The method suppresses the logit of the double-newline token (\n\n) immediately after the <think> token, and optionally suppresses the EOS token after </think>, without changing model weights or prompts. Using GPT-4o mini as an automated judge, the authors report that suppressing \n\n significantly increases the proportion of 'answered' final responses and 'relevant coherent' chain-of-thought blocks across Qwen 7B/14B/32B and Llama 70B distillations on two censorship-probe datasets (CCP and DECCP). They also claim that standard benchmarks are unaffected, though no benchmark experiments are reported.

Significance. The proposed method is elegantly simple and, if the measured effects are genuine, provides evidence that some safety-aligned behaviors in CoT models are mediated by a small set of token-level transitions rather than by comprehensive knowledge erasure. This would be of value to researchers studying alignment robustness, model auditing, and machine unlearning. The intervention is a clean counterfactual manipulation against a no-intervention baseline, with no fitted parameters. However, the empirical significance is currently contingent on the validity of the GPT-4o mini judge, which is not established. The unsupported claim of unchanged benchmark performance further limits the paper's current strength.

major comments (4)
  1. [Section 4, Appendix A.1] The paper's central quantitative claim—that suppressing \n\n after <think> increases 'answered' responses and 'relevant coherent' CoT blocks (Figures 1–6)—rests entirely on labels produced by GPT-4o mini. The appendix provides the classifier prompts but no validation: no human agreement, no alternative judge, no error analysis. This is particularly concerning because the intervention changes the surface formatting of the CoT block (removing the initial double newline), and the same judge labels both the CoT block and the final response. A judge that is sensitive to formatting could produce exactly the observed pattern even if the model's substantive behavior is unchanged. Please provide classifier validation, such as human-labeled agreement or a blind re-judging of samples from baseline and intervention conditions, and report per-condition label distributions for the judge.
  2. [Abstract, Section 4] The abstract and Section 4 claim that the interventions increase substantive answers 'without affecting performance on standard benchmarks,' but no benchmark experiments are reported anywhere in the manuscript. This claim is load-bearing for the practical safety of the method. Either add the benchmark evaluation (with results) or remove/qualify the claim.
  3. [Section 3] The method is motivated by the assertion that '<think> followed by \n\n almost always produces a refusal or evasive answer' and that otherwise the model 'likely provides a substantive response.' No quantitative evidence is given for this binary separation. The authors should report the contingency between the presence of the \n\n token and the eventual response class across all prompts and models, and analyze cases where the intervention fails (e.g., \n\n suppressed but the final response is still a refusal or evasive). Without this, the proposed 'refusal subspace' mechanism remains an untested interpretation.
  4. [Section 4] There is an internal inconsistency about which interventions were reported: the text says 'only \n\n suppression was used in the final reported results,' yet Figures 1–6 show a third condition labeled '\n\n and EOS.' Please clarify whether EOS suppression was included in the reported comparisons and, if so, align the text with the figures.
minor comments (5)
  1. [Section 2, paragraph 3] The sentence 'Our method requires does not require any search or optimization over prompts' contains a doubled verb; please fix.
  2. [Section 4, first paragraph] The phrase 'The applied our interventions' should read 'We applied our interventions'; please fix.
  3. [References] The DeepSeek-R1 technical report reference lists a placeholder 'arXiv:2501.XXXXX, To appear'; the final arXiv identifier should be supplied.
  4. [Figures 5–6] The p-value heatmaps are difficult to read in log scale without significance thresholds; consider adding a conventional significance marker (e.g., * for p<0.05) or reporting the underlying contingency tables in a supplementary table.
  5. [Section 4, paragraph 2] The sentence about QwQ ('none except DeepSeek-R1 exhibited notable censorship') appears to be an informal observation; if it is not a formal evaluation, please label it as such or remove it.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: the core \n\n-suppression result is a genuine counterfactual intervention with no fitted parameters, though the EOS-suppression component has a minor by-construction effect on empty responses.

  1. other [Section 3 (Methodology, Step 2); Section 4 (Experiments); Appendix A.1 (classifier definitions)]
    "After the </think> token, we similarly set the logit of the EOS token to −∞. ... Additionally suppressing the EOS token after </think> was successful in most dataset-distillation pairs. ... failure empty: The model fails to generate meaningful text after internal reasoning (like a think block) or after the initial assistant token, resulting in an empty or near-empty final response."

    Setting the EOS logit to −∞ immediately after </think> makes it impossible for the model to terminate with an empty final response at that point, and Appendix A.1 defines 'failure empty' precisely as an empty or near-empty final response after the think block. Any reduction in the 'failure empty' class under the \n\n-and-EOS condition is therefore guaranteed by construction, not discovered empirically. This is a minor side effect rather than the core claim: the main reported increases in 'answered' and 'relevant coherent' come from \n\n suppression, and the paper's own p-values for '\n\n vs. \n\n and EOS' on 'answered' are mostly non-significant.

full rationale

The paper's central derivation is a counterfactual intervention: it observes that refusals in DeepSeek-R1 distillations are typically preceded by <think>\n\n, suppresses that token during generation, and measures the resulting class distributions against a no-intervention baseline. No parameter is fitted to maximize the 'answered' rate, no training is performed, and the outcome is not guaranteed—the model could plausibly continue with a differently worded refusal, evasive propaganda, or irrelevant content. The evaluation depends on a GPT-4o-mini classifier that is not validated against human labels, which is a serious threat to the measured effect's validity, but that is a benchmarking and robustness concern, not circularity: the labels are external to the intervention and the paper does not tune the classifier to produce its headline result. There are no load-bearing self-citations; the cited prior work (Azaria & Mitchell, Naseh et al., etc.) is external and not used to define the outcome. The only by-construction component is the optional EOS suppression: setting the EOS logit to −∞ immediately after </think> makes an immediate empty final response impossible, so any reduction in the 'failure empty' class under that condition is definitionally guaranteed rather than empirically discovered. This is a minor side effect, not the driver of the main '\n\n suppression' result, so the overall circularity score is low.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The method introduces no fitted numeric parameters, but the two token-suppression choices are hand-selected design elements. The central claims rest on the unproven assumption of a clean refusal subspace marked by <think>\n\n, on the unvalidated reliability of the GPT-4o mini judge, and on the unsupported assertion that standard benchmarks are unaffected.

free parameters (2)
  • Suppressed token: double newline after <think> = logit set to -infinity (no numeric value)
    Hard-coded intervention choice based on qualitative observation that refusals follow <think>\n\n in the same models being tested; no numeric fitting, but it is a hand-selected element of the method.
  • Suppressed token: EOS after </think> = logit set to -infinity (no numeric value)
    Optional second intervention chosen after seeing that Step 1 often led to premature EOS; not applied uniformly and its benefit varies by model and dataset.
assumptions (3)
  • domain assumption Refusal behavior occupies a separable output subspace marked by the <think>\n\n token transition.
    Section 3 states that if <think> is followed by \n\n the model almost always refuses, otherwise it likely responds substantively; this binary separation is stated without quantitative baseline evidence.
  • domain assumption GPT-4o mini classifier labels are accurate proxies for whether a response is a substantive answer.
    Section 4 uses GPT-4o mini as the sole judge; Appendix A.1 gives instructions but no human agreement or validation.
  • domain assumption Standard benchmark performance is unaffected by the interventions.
    The abstract asserts this, but Section 4 reports no benchmark evaluation; it is an unsupported premise of the broad claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models." pith.science (2026). https://pith.science/paper/TLLOITWX

@misc{pith2026250523848,
  author       = {Pith},
  title        = {Pith review of: Derailing Non-Answers via Logit Suppression at Output Subspace Boundaries in RLHF-Aligned Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TLLOITWX}},
  note         = {Machine review of arXiv:2505.23848}
}
read the original abstract

We introduce a method to reduce refusal rates of large language models (LLMs) on sensitive content without modifying model weights or prompts. Motivated by the observation that refusals in certain models were often preceded by the specific token sequence of a token marking the beginning of the chain-of-thought (CoT) block (<think>) followed by a double newline token (\n\n), we investigate the impact of two simple formatting adjustments during generation: suppressing \n\n after <think> and suppressing the end-of-sequence token after the end of the CoT block (</think>). Our method requires no datasets, parameter changes, or training, relying solely on modifying token probabilities during generation. In our experiments with official DeepSeek-R1 distillations, these interventions increased the proportion of substantive answers to sensitive prompts without affecting performance on standard benchmarks. Our findings suggest that refusal behaviors can be circumvented by blocking refusal subspaces at specific points in the generation process.

Figures

Figures reproduced from arXiv: 2505.23848 by the authors.

Figure 1
Figure 1. Main response class distribution for the CCP dataset across interventions. baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 7B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 14B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 32B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 70B relevant coherent relevant incoherent irrelevant empty [PITH_FULL_IMAGE:figures/… view at source ↗
Figure 2
Figure 2. CoT block class distribution for the CCP dataset across interventions. baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 7B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 14B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 32B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 70B answered refusal evasive propaganda failure empty [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 3
Figure 3. Main response class distribution for the DECCP dataset across interventions. baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 7B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 14B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 32B baseline \n\n \n\n and EOS 0.0 0.2 0.4 0.6 0.8 1.0 proportion 70B relevant coherent relevant incoherent irrelevant empty [PITH_FULL_IMAGE:figure… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: CoT block class distribution for the DECCP dataset across interventions. 3 [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: P-values from Fisher’s exact test between interventions on the CCP dataset. 7B 14B 32B 70B model 2.13e-16 3.36e-23 7.31e-02 2.57e-25 3.26e-22 5.13e-01 2.72e-30 2.70e-33 5.87e-01 1.48e-06 2.98e-08 5.31e-01 main output: p-values for "answered" proportion baseline vs. \n\…
Figure 6
Figure 6. Figure 6: P-values from Fisher’s exact test between interventions on the DECCP dataset. Results demonstrate that suppressing \n\n immediately af￾ter <think> was the most impactful intervention, which greatly increased the proportion of “relevant coherent” CoT blocks as well as “…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

14 extracted references · 5 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    AUGMXNT. Deccp. https://github.com/AUGMXNT/deccp, 2024

    work page 2024

  3. [3]

    and Mitchell, J

    Azaria, A. and Mitchell, J. C. Refusal in language models is mediated by a single direction. arXiv preprint arXiv:2406.11717, 2024

    arXiv 2024

  4. [4]

    F., Choquette-Choo, C

    Cooper, A. F., Choquette-Choo, C. A., Bogen, M., Jagielski, M., Filippova, K., Liu, K. Z., Chouldechova, A., Hayes, J., Huang, Y., Mireshghallah, N., Shumailov, I., Triantafillou, E., Kairouz, P., Mitchell, N., Liang, P., Ho, D. E., Choi, Y., Koyejo, S., Delgado, F., Grimmelmann, J., Shmatikov, V., De Sa, C., Barocas, S., Cyphert, A., Lemley, M., boyd, d....

    arXiv 2024

  5. [5]

    Deepseek-r1 technical report

    DeepSeek AI . Deepseek-r1 technical report. arXiv preprint arXiv:2501.XXXXX, 2025. To appear

    work page 2025

  6. [6]

    R., and Goldberg, Y

    Geva, M., Caciularu, A., Wang, K. R., and Goldberg, Y. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space, 2022. URL https://arxiv.org/abs/2203.14680

    arXiv 2022

  7. [7]

    L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Cand \`e s, E., and Hashimoto, T

    Muennighoff, N., Yang, Z., Shi, W., Li, X. L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Cand \`e s, E., and Hashimoto, T. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025

    arXiv 2025

  8. [8]

    Investigating local censorship in deepseek's R1 language model

    Naseh, A., Chaudhari, H., Roh, J., Wu, M., Oprea, A., and Houmansadr, A. Investigating local censorship in deepseek's R1 language model. arXiv preprint arXiv:2505.12625, 2025

    arXiv 2025

Show all 14 references
  1. [9]

    Using logit bias to alter token probability

    OpenAI . Using logit bias to alter token probability. https://help.openai.com/en/articles/5247780-using-logit-bias-to-alter-token-probability-with-the-openai-api, 2024. Accessed: 2024-05-20

    2024

  2. [10]

    Ccp-sensitive-prompts

    promptfoo. Ccp-sensitive-prompts. https://huggingface.co/datasets/promptfoo/CCP-sensitive-prompts, 2024

    2024

  3. [11]

    Qwq: Reflect deeply on the boundaries of the unknown, November 2024

    Team, Q. Qwq: Reflect deeply on the boundaries of the unknown, November 2024. URL https://qwenlm.github.io/blog/qwq-32b-preview/

    2024

  4. [12]

    Wang, B. et al. Iteratively prompt pre-trained language models for chain of thought. arXiv preprint arXiv:2203.08383, 2022

    2022 arXiv

  5. [13]

    L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Transformers: St...

    2020

  6. [14]

    Representation engineering: A top-down approach to AI transparency

    Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, D., Mazeika, M., Dombrowski, A.-K., et al. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2023

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.