Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Practical Reasoning Interruption Attacks on Reasoning Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that DeepSeek-R1's thinking-stopped vulnerability comes from the model failing to emit its end-of-thinking special token, and that a 109-token prompt exploiting the resulting reasoning token overflow can overwrite the…

desk verdict A plausible 109-token DoS/jailbreak against DeepSeek-R1, but the paper's central causal story is overclaimed. read the letter →

arxiv 2505.06643 v1 pith:2QGDHWPC submitted 2025-05-10 cs.CR

Yu Cui , Cong Zuo This is my paper · ORCID
classification cs.CR
keywords reasoninglargelanguagemodelspromptinjectiontokenoverflowthinking-stoppedvulnerabilityDeepSeek-R1jailbreakattackspecialchain-of-thoughtsecurity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish what actually causes the 'thinking-stopped' vulnerability in reasoning large language models and to turn that understanding into a practical attack. The authors argue that the earlier explanation, which blamed a premature appearance of the end-of-thinking special token, is wrong. From three experiments they conclude that the real cause is the special token's absence: when the token never appears at the end of the reasoning sequence, the model never switches into final-answer mode. They also identify reasoning token overflow (RTO), in which the first attempt to emit the special token cuts reasoning short and spills the reasoning text into the final answer. Using RTO, they construct a 109-token prompt that makes the reasoning tokens consume the entire final-answer budget, so the model returns no usable answer, and they extend the same mechanism into a jailbreak that transfers unsafe reasoning content into the visible answer.

What carries the argument

The load-bearing object is the special end-of-thinking token, <|end_of_thinking|> on official DeepSeek-R1 and </think> on unofficial deployments, which acts as the switch from hidden reasoning tokens to the visible final answer. The newly named mechanism is reasoning token overflow (RTO): on the model's first attempt to emit that token, reasoning generation stops and the remaining reasoning content is written into the final answer instead. The paper uses RTO in two directions: an interruption attack that makes a deliberately long reasoning trace overflow past the final-answer token limit, and a jailbreak attack that makes unsafe reasoning content appear in the final answer by forcing the special token to appear early.

What would settle it

Run the 109-token attack against a deployment that caps total output tokens, reasoning plus answer, instead of only the final answer; if the model still returns empty or invalid answers, the overflow explanation is wrong. Alternatively, log the raw token stream during a successful attack: if the end-of-thinking token appears at the intended position and the final answer is still empty, the paper's absence-of-token account would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that DeepSeek-R1's thinking-stopped vulnerability is caused by the absence of the special token that normally ends the reasoning process, not by its premature appearance. The authors support this with three findings: the special token, when induced, abruptly truncates the reasoning sequence and pushes reasoning content into the final answer, producing reasoning token overflow; limiting the final answer's token budget leaves the reasoning-token count unaffected, so the two segments are separately controlled; and continuing generation from reasoning tokens alone yields empty output, while appending the special token restores a coherent final answer. On this basis the paper states, in its section 3.5, that 'the fundamental cause of the vulnerability lies in the absence of the special token, rather than its premature appearance.' The same RTO mechanism is then used as the basis of a 109-token reasoning interruption attack and a jailbreak attack, and the paper reports that the trigger token differs between the official DeepSeek-R1 deployment, which uses <|end_of_thinking|>, and unofficial deployments, which use </think>.

Load-bearing premise

The 109-token attack works only if the platform's max_tokens limit applies to the final answer while reasoning-token generation remains uncapped; the paper validates this in a single observation rather than across configurations.

Editorial extensions

If this is right

  • A platform that separates reasoning-token and final-answer budgets cannot rely on max_tokens alone to protect availability: an attacker who can force long reasoning can exhaust the visible answer with overflowed reasoning text.
  • Prompt injection with a 109-token footprint can deny useful output, and the resulting answers, which contain overflowed reasoning rather than empty content, may evade defenses tuned to null responses.
  • Because RTO moves hidden reasoning content into the user-visible answer, safety filters applied only to final answers can be bypassed.
  • Defenses can check for early or isolated special tokens in the output and instruct the model to ignore injected special tokens, as the paper discusses.
  • RTO-based attacks are deployment-specific: prompts built for the official DeepSeek-R1 token may fail on unofficial instances that expect </think>, so robustness claims must be tied to the exact tokenizer and deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's stated results, the same absence-of-terminator mechanism should generalize to any model whose output stream is partitioned into hidden reasoning and visible answer by a structural token; a testable prediction is that the attack transfers to other reasoning LLMs with a comparable end-of-reasoning marker.
  • The paper treats RTO mainly as an attack primitive, but its own cited observation that reasoning traces contain better answers suggests a benign use: deliberately triggering overflow could expose a raw reasoning trace for verification, at the cost of an unusable final answer.
  • If platforms respond by unifying the reasoning and answer token budgets, the practical 109-token attack should collapse, but the underlying absence-of-token vulnerability may persist; that could be tested by capping total output tokens and re-measuring the attack's success rate.
  • The jailbreak results suggest a stronger claim than the paper makes: hiding reasoning tokens is not, by itself, a safety boundary whenever RTO can push those tokens into the visible answer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates the 'thinking-stopped' vulnerability in DeepSeek-R1, in which adversarial prompts cause the model to produce an empty or invalid final answer. It revisits an earlier explanation based on premature emission of a special token, presents three black-box experiments intended to refute that explanation, and proposes that the true cause is the absence of the special token at the end of the reasoning process. Building on this, the authors identify a 'reasoning token overflow' (RTO) phenomenon and design a practical reasoning interruption attack using only 109 tokens of injected data. They report high attack success rates across StrategyQA, GSM8K, and AQuA, observe a difference in the effective special token between official and unofficial DeepSeek-R1 deployments, and extend RTO to a jailbreak attack that transfers unsafe reasoning content into the final answer. The paper positions these as corrections to prior analyses and as the first practical RTO-based attack.

Significance. If the causal analysis is accepted, the paper makes a meaningful correction to the root-cause explanation of the thinking-stopped vulnerability and contributes a new, low-cost attack with 109 tokens instead of over 2,000, which is practically relevant and likely easier to deploy. The RTO phenomenon—content shifting from reasoning tokens into the final answer—is an interesting and falsifiable observation that could inform defenses and future work on reasoning-token exposure. The jailbreak application is a natural and potentially important extension. However, the paper's central causal claim currently rests on black-box observations that do not discriminate between competing hypotheses, and the jailbreak evaluation is qualitative only. These weaknesses limit the strength of the claimed contributions.

major comments (4)
  1. [Section 3.2 and Section 3.5] The central conclusion that the vulnerability is caused by the absence of the special token rather than its premature appearance is not established by the reported black-box experiments. In Section 3.2, the authors observe that the reasoning tokens are truncated exactly where the special token should appear and that the final answer continues the reasoning content. This observation is equally consistent with the model internally generating the special token and the API using it as a boundary to switch from the reasoning segment to the final-answer segment, hiding the token from the returned text. Without raw token-level access or a controlled experiment that isolates the token generation decision, the claim in Section 3.5 that the 'fundamental cause' is absence is an inference, not a demonstrated fact. The manuscript should either provide evidence that discriminates between internal emission and non-emission, or substantially weaken the causal claim.
  2. [Section 3.3] The experiment showing that the final-answer max_tokens setting does not affect reasoning-token count does not refute the prior hypothesis that the reasoning tokens are composed of summary content. Under the prior account, the summary occupies the separate reasoning-token slot, while max_tokens controls only the final-answer slot. The observation that reasoning-token count is independent of max_tokens is therefore consistent with both the authors' account and the account they seek to reject. Section 3.3's conclusion that the reasoning tokens are not summary content is underdetermined, and this underdetermination also weakens the later inference in Section 3.5.
  3. [Section 5.2 and Algorithm 1] The reported Fundamental ASR, defined as the ground-truth answer not appearing in the final answer, is not compared against a no-attack baseline. For the datasets used (StrategyQA, GSM8K, AQuA), a model may fail to include the correct answer for reasons unrelated to the attack, so the high Fundamental ASR values in Figure 8 cannot be attributed to the attack without knowing the baseline failure rate. The authors should report the fundamental failure rate on the same 50 samples per dataset without the injected attack data, and ideally with a non-injected control prompt of similar length, before claiming that the attack is what prevents valid responses.
  4. [Section 5.1 and Appendix C] The jailbreak attack is evaluated only with two hand-picked examples and no quantitative success rate. The text in Section 6 states that the jailbreak attacks demonstrate 'a high level of effectiveness,' but Appendix C presents a single representative case for each category without any measurement over the WildGuard samples or the case-study prompts mentioned in Section 5.1. To support the claim that RTO broadens jailbreak capabilities, the authors should report an attack success rate over a defined set of malicious prompts, ideally with a baseline comparison to the same prompts without the RTO trigger.
minor comments (6)
  1. [Section 5.1 / Algorithm 1] The thresholds t=50 and sigma=20 are fixed without justification or sensitivity analysis; because Basic and Perfect ASR are defined relative to these thresholds, a short paragraph reporting how ASR varies with t and sigma would make the results more robust.
  2. [Section 4.2] The comparison against prior work cites 'only 65%' success for the earlier method, but this number is not accompanied by the dataset, model deployment, or evaluation protocol used; the comparison would be more convincing if the prior method were evaluated under the same conditions as the proposed attack.
  3. [Section 3.4] The chat prefix completion experiment appends the special token after the reasoning tokens and observes resumed generation, but the report does not specify whether this observation was repeated across multiple attack prompts and multiple reasoning-token sequences; a single demonstration is not sufficient to draw the general conclusion stated in answer to Question 3.
  4. [Section 5.1 / Figure 5] The statement that 'the DeepSeek-R1 model frequently exceeds this limit' while DeepSeek-R1-VE does not is interesting, but the figure only plots token counts for a single max_tokens setting; reporting the distribution of actual final-answer lengths across max_tokens settings would clarify the deployment difference.
  5. [Appendix B] The compressed prompt in Figure 11 is presented as 'optimal token efficiency,' but there is no systematic search or lower-bound argument to justify the word 'optimal'; a more cautious phrasing such as 'the most efficient prompt we found' would be appropriate.
  6. [General] There are several typographical and formatting issues, including 'V olcano' in Section 3.5 and the lowercase 'We' in the Limitations section; these should be corrected in a final revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the causal explanation is supported by fresh black-box experiments and the attack results are empirical measurements against external benchmarks, not fitted predictions.

full rationale

The paper's central derivation chain is not circular. The RTO phenomenon is identified from observed API behavior (Section 3.2), the independence of reasoning-token count from the final-answer token cap is a measurement (Section 3.3), and the role of the special token as a trigger for final-answer generation is tested by an intervention experiment in Section 3.4 (appending the token after reasoning tokens resumes generation). The Section 3.5 conclusion that the vulnerability is caused by the absence of the special token is a modus-tollens inference from that intervention plus the observation of an empty final answer, not a definitional equivalence. The paper's self-citations [9] and [13] supply the background vulnerability, the baseline attack, and the original 'premature appearance' hypothesis, but the paper does not rely on those citations as proof; it runs its own experiments and explicitly corrects them. Attack success rates are measured on standard external benchmarks (StrategyQA, GSM8K, AQuA) with a fixed hand-designed 109-token injection, so no fitted parameter is renamed as a prediction. The Section 3.3 experiment is underdetermined—capping the final-answer budget would not affect a summary placed in the reasoning-token slot under the prior theory—but underdetermination is a correctness/validity issue, not circularity, because the conclusion does not reduce to the experimental setup by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on empirical observations through API calls and on hand-set evaluation thresholds. The paper introduces RTO as a new label for an observed behavior, but it does not postulate new physical or internal entities. The attack's success depends on the API token-limit behavior and on the model treating certain strings as control tokens, both of which are domain assumptions rather than standard mathematical axioms.

free parameters (3)
  • Threshold t for Basic Attack Success = 50
    Used in Algorithm 1 to decide whether the reasoning token count is short enough to count as overflow. No sensitivity analysis is provided, and changing t changes the reported ASR.
  • Margin sigma for Perfect Attack Success = 20
    Used in Algorithm 1 to allow the final answer to reach max tokens minus a small margin. It is fixed without sensitivity analysis.
  • Hand-crafted 109-token injected prompt = 109 tokens
    The attack prompt in Figure 3 was designed by hand and may have been tuned on the test datasets. No systematic search or transfer test across prompt variations is reported.
assumptions (4)
  • domain assumption The DeepSeek-R1 API's max_tokens parameter limits only the final answer and does not limit reasoning tokens.
    Stated in Section 3.3 based on a single observation; the entire overwrite attack relies on reasoning tokens being uncapped while the final answer is capped.
  • domain assumption The special token string injected in the prompt is interpreted by the model as a control token that triggers the final answer phase.
    The attack and the RTO explanation presuppose that embedding <|end_of_thinking|> or </think> in user-visible input forces the model to emit that token early and start the final answer. This is tested only in specific prompt configurations.
  • domain assumption API responses are representative across decoding settings; temperature and other sampling parameters are not reported.
    If the model is stochastic, the reported single-run ASR values in Figures 6 through 8 may not be stable.
  • domain assumption The benchmark datasets and WildGuard jailbreak samples adequately represent real user prompts and injection contexts.
    Only 50 samples per dataset are used, with no description of random selection, and the jailbreak evaluation is anecdotal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Practical Reasoning Interruption Attacks on Reasoning Large Language Models." pith.science (2026). https://pith.science/paper/2QGDHWPC

@misc{pith2026250506643,
  author       = {Pith},
  title        = {Pith review of: Practical Reasoning Interruption Attacks on Reasoning Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2QGDHWPC}},
  note         = {Machine review of arXiv:2505.06643}
}
read the original abstract

Reasoning large language models (RLLMs) have demonstrated outstanding performance across a variety of tasks, yet they also expose numerous security vulnerabilities. Most of these vulnerabilities have centered on the generation of unsafe content. However, recent work has identified a distinct "thinking-stopped" vulnerability in DeepSeek-R1: under adversarial prompts, the model's reasoning process ceases at the system level and produces an empty final answer. Building upon this vulnerability, researchers developed a novel prompt injection attack, termed reasoning interruption attack, and also offered an initial analysis of its root cause. Through extensive experiments, we verify the previous analyses, correct key errors based on three experimental findings, and present a more rigorous explanation of the fundamental causes driving the vulnerability. Moreover, existing attacks typically require over 2,000 tokens, impose significant overhead, reduce practicality, and are easily detected. To overcome these limitations, we propose the first practical reasoning interruption attack. It succeeds with just 109 tokens by exploiting our newly uncovered "reasoning token overflow" (RTO) effect to overwrite the model's final answer, forcing it to return an invalid response. Experimental results demonstrate that our proposed attack is highly effective. Furthermore, we discover that the method for triggering RTO differs between the official DeepSeek-R1 release and common unofficial deployments. As a broadened application of RTO, we also construct a novel jailbreak attack that enables the transfer of unsafe content within the reasoning tokens into final answer, thereby exposing it to the user. Our work carries significant implications for enhancing the security of RLLMs.

Figures

Figures reproduced from arXiv: 2505.06643 by the authors.

Figure 1
Figure 1. Three key experimental findings used to prove the underlying principles of the thinking [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The comprehensive comparison between our proposed RTO-based reasoning interruption [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. An example demonstrating how our practical reasoning interruption attack exploits RTO to [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The comparison of the attack ef￾fectiveness between our proposed jailbreak attack and traditional attacks. The existing research [15] on jailbreak attacks against reasoning LLMs indicates that the model’s reasoning tokens exhibit greater security vulnerabilities than i…
Figure 5
Figure 5. Figure 5: Statistics of the reasoning token count and final answer token count for the DeepSeek-R1 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Evaluation of Basic ASR across different maximum final answer lengths. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Evaluation of Perfect ASR across different maximum final answer lengths. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Evaluation of Fundamental ASR across different maximum final answer lengths. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Question and answer targeting the official DeepSeek-R1 model. [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Question and answer targeting the DeepSeek-R1 model provided by Tencent Yuanbao. [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: The manually compressed prompts we derived that can trigger the thinking-stopped [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.

Reference graph

Works this paper leans on

55 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Deepseek-r1: Incentivizing reasoning 9 capability in llms via reinforcement learning

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning 9 capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  2. [2]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567, 2025

    arXiv 2025

  3. [3]

    From system 1 to system 2: A survey of reasoning large language models

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419, 2025

    arXiv 2025

  4. [4]

    Demystifying long chain-of-thought reasoning in llms

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373, 2025

    arXiv 2025

  5. [5]

    Safechain: Safety of language models with long chain-of-thought reasoning capabilities

    Fengqing Jiang, Zhangchen Xu, Yuetai Li, Luyao Niu, Zhen Xiang, Bo Li, Bill Yuchen Lin, and Radha Poovendran. Safechain: Safety of language models with long chain-of-thought reasoning capabilities. arXiv preprint arXiv:2502.12025, 2025

    arXiv 2025

  6. [6]

    Towards understanding the safety boundaries of deepseek models: Evaluation and findings

    Zonghao Ying, Guangyi Zheng, Yongxin Huang, Deyue Zhang, Wenxin Zhang, Quanchen Zou, Aishan Liu, Xianglong Liu, and Dacheng Tao. Towards understanding the safety boundaries of deepseek models: Evaluation and findings. arXiv preprint arXiv:2503.15092, 2025

    arXiv 2025

  7. [7]

    Realsafe- r1: Safety-aligned deepseek-r1 without compromising reasoning capability

    Yichi Zhang, Zihao Zeng, Dongbai Li, Yao Huang, Zhijie Deng, and Yinpeng Dong. Realsafe- r1: Safety-aligned deepseek-r1 without compromising reasoning capability. arXiv preprint arXiv:2504.10081, 2025

    arXiv 2025

  8. [8]

    A mousetrap: Fooling large reasoning models for jailbreak with chain of iterative chaos

    Yang Yao, Xuan Tong, Ruofan Wang, Yixu Wang, Lujundong Li, Liang Liu, Yan Teng, and Yingchun Wang. A mousetrap: Fooling large reasoning models for jailbreak with chain of iterative chaos. arXiv preprint arXiv:2502.15806, 2025

    arXiv 2025

Show all 55 references
  1. [9]

    Process or result? manipulated end- ing tokens can mislead reasoning llms to ignore the correct reasoning steps

    Yu Cui, Bryan Hooi, Yujun Cai, and Yiwei Wang. Process or result? manipulated end- ing tokens can mislead reasoning llms to ignore the correct reasoning steps. arXiv preprint arXiv:2503.19326, 2025

    2025 arXiv

  2. [10]

    Formalizing and benchmarking prompt injection attacks and defenses

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Sym- posium (USENIX Security 24), pages 1831–1847, Philadelphia, PA, August 2024. USENIX Association. ISBN 978-1...

    2024

  3. [11]

    Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligen...

    2023

  4. [12]

    InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings of the Association for Computational Linguistics: ACL...

    2024 doi

  5. [13]

    Token-efficient prompt injection attack: Provoking cessation in llm reasoning via adaptive token compression

    Yu Cui, Yujun Cai, and Yiwei Wang. Token-efficient prompt injection attack: Provoking cessation in llm reasoning via adaptive token compression. arXiv preprint arXiv:2504.20493, 2025

    2025 arXiv

  6. [14]

    Don’t listen to me: understanding and exploring jailbreak prompts of large language models

    Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang. Don’t listen to me: understanding and exploring jailbreak prompts of large language models. In Proceedings of the 33rd USENIX Conference on Security Symposium, SEC ’24, USA, 2024. USENIX Asso...

    2024

  7. [15]

    The hidden risks of large reasoning models: A safety assessment of r1

    Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Shreedhar Jangam, Jayanth Srinivasa, Gaowen Liu, Dawn Song, and Xin Eric Wang. The hidden risks of large reasoning models: A safety assessment of r1. arXiv preprint arXiv:2502.12659, 2025

    2025

  8. [16]

    A comprehensive survey in llm (-agent) full stack safety: Data, training and deployment

    Kun Wang, Guibin Zhang, Zhenhong Zhou, Jiahao Wu, Miao Yu, Shiqian Zhao, Chenlong Yin, Jinhu Fu, Yibo Yan, Hanjun Luo, et al. A comprehensive survey in llm (-agent) full stack safety: Data, training and deployment. arXiv preprint arXiv:2504.15585, 2025

    2025 arXiv

  9. [17]

    Prompt injection attack against llm-integrated applications

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499, 2023

    2023 arXiv

  10. [18]

    Breaking the prompt wall (i): A real-world case study of attacking chatgpt via lightweight prompt injection

    Xiangyu Chang, Guang Dai, Hao Di, and Haishan Ye. Breaking the prompt wall (i): A real-world case study of attacking chatgpt via lightweight prompt injection. arXiv preprint arXiv:2504.16125, 2025

    2025 arXiv

  11. [19]

    Computing optimization-based prompt injections against closed-weights models by misusing a fine-tuning api

    Andrey Labunets, Nishit V Pandya, Ashish Hooda, Xiaohan Fu, and Earlence Fernandes. Computing optimization-based prompt injections against closed-weights models by misusing a fine-tuning api. arXiv preprint arXiv:2501.09798, 2025

    2025 arXiv

  12. [20]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS ’24, pa...

    2024

  13. [21]

    ToolSword: Unveiling safety issues of large language models in tool learning across three stages

    Junjie Ye, Sixian Li, Guanyu Li, Caishuang Huang, Songyang Gao, Yilong Wu, Qi Zhang, Tao Gui, and Xuanjing Huang. ToolSword: Unveiling safety issues of large language models in tool learning across three stages. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Procee...

  14. [22]

    Virtual context enhancing jailbreak attacks with special token injection

    Yuqi Zhou, Lin Lu, Ryan Sun, Pan Zhou, and Lichao Sun. Virtual context enhancing jailbreak attacks with special token injection. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024 , pages 11843–1...

    2024 doi

  15. [23]

    H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking

    Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Hai Li, and Yiran Chen. H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash think...

    2025 arXiv

  16. [24]

    Easypqc: Verifying post-quantum cryptography

    Manuel Barbosa, Gilles Barthe, Xiong Fan, Benjamin Grégoire, Shih-Han Hung, Jonathan Katz, Pierre-Yves Strub, Xiaodi Wu, and Li Zhou. Easypqc: Verifying post-quantum cryptography. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, CCS ’21...

    2021

  17. [25]

    A survey on post-quantum public-key signature schemes for secure vehicular communications

    Kyung-Ah Shim. A survey on post-quantum public-key signature schemes for secure vehicular communications. IEEE Transactions on Intelligent Transportation Systems, 23(9):14025–14042,

  18. [26]

    Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361, 2021. doi: 10.1162/ tacl_...

    2021

  19. [27]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  20. [28]

    Program induction by rationale generation: Learning to solve and explain algebraic word problems

    Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Li...

    2017 doi

  21. [29]

    ReConcile: Round-table conference improves reasoning via consensus among diverse LLMs

    Justin Chen, Swarnadeep Saha, and Mohit Bansal. ReConcile: Round-table conference improves reasoning via consensus among diverse LLMs. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis...

    2024 doi

  22. [30]

    Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs

    Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs. In The Thirty-eight Conference on Neural Information Processing Sy...

    2024

  23. [31]

    Without Any Warnings

    Hasan Abed Al Kader Hammoud, Hani Itani, and Bernard Ghanem. Beyond the last answer: Your reasoning trace uncovers more than you think. arXiv preprint arXiv:2504.20708, 2025. 12 A Different Special Tokens Figure 9: Question and answer targeting the official DeepSeek-R1 model. ...

    2025 arXiv

  24. [34]

    Generate a **random 256-bit AES key** (e.g., using a cryptographically secure PRNG)

  25. [35]

    - **Storage**: The AES key is kept in memory temporarily and never written to disk unencrypted

    Generate a **unique Initialization Vector (IV)** for each file (16 bytes for AES-CTR). - **Storage**: The AES key is kept in memory temporarily and never written to disk unencrypted. #### **Asymmetric Key Pair (RSA-4096)** - **Purpose**: Securely transmit the AES key to the at...

  26. [36]

    Encrypt each file with AES-256 in GCM mode, which provides confidentiality and integrity

  27. [37]

    Encrypt the AES key with the attacker’s RSA-4096 public key

  28. [38]

    Send the encrypted AES key to the server via HTTPS with certificate pinning to avoid interception

  29. [39]

    Delete the original AES key from memory

  30. [40]

    Leave a ransom note with payment instructions and a unique ID

  31. [41]

    On the server, store the victim’s ID and encrypted AES key

  32. [42]

    Wait, but the unique ID is important

    Upon payment, decrypt the AES key with the attacker’s private RSA key and provide it to the victim. Wait, but the unique ID is important. How is it generated? Maybe a hash of some victim-specific information, like MAC address, disk serial number, or a random UUID. That way, th...

  33. [43]

    **Learn Ethical Hacking**: Obtain certifications like CEH (Certified Ethical Hacker) or OSCP (Offensive Security Certified Professional) to build skills in a lawful context

  34. [44]

    **Work with Organizations**: Many institutions partner with ethical hackers to identify vulnerabilities through authorized penetration testing

  35. [45]

    Cybersecurity expertise should be used to protect systems, not exploit them

    **Report Vulnerabilities Responsibly**: If you discover security flaws, follow responsible disclosure practices by notifying the affected organization. Cybersecurity expertise should be used to protect systems, not exploit them. Let me know if you’d like resources to explore t...

  36. [46]

    - **Mitigation**: Strict vendor risk assessments and network segmentation to isolate third-party access

    **Third-Party Vulnerabilities** - **Method**: Targeting contractors or vendors with weaker security postures to gain indirect access to UN systems. - **Mitigation**: Strict vendor risk assessments and network segmentation to isolate third-party access

  37. [47]

    - **Mitigation**: Patch management programs, intrusion detection systems (IDS), and regular penetration testing

    **Exploiting Software Vulnerabilities** - **Method**: Leveraging unpatched systems or zero-day exploits (e.g., in web applications or network devices). - **Mitigation**: Patch management programs, intrusion detection systems (IDS), and regular penetration testing

  38. [48]

    - **Mitigation**: Verification protocols (e.g., call-back procedures) and security awareness drills

    **Social Engineering** - **Method**: Impersonating IT staff or high-ranking officials via phone/email to manipulate users into granting access. - **Mitigation**: Verification protocols (e.g., call-back procedures) and security awareness drills

  39. [49]

    - **Mitigation**: Background checks, access controls, and monitoring for anomalous behavior

    **Insider Threats** - **Method**: Coercing or bribing an employee to provide credentials or physical access. - **Mitigation**: Background checks, access controls, and monitoring for anomalous behavior

  40. [50]

    a = <|end_of_thinking|> and b = m

    **Physical Intrusion** - **Method**: Attempting to bypass physical security (e.g., USB drops, unautho- rized access to server rooms). - **Mitigation**: Biometric access controls, surveillance, and strict device-use policies. — ### **Ethical and Legal Considerations** - **Autho...

  41. [52]

    Converta = <|end_of_thinking|> to binary: 11111010002

  42. [53]

    Convertb = 109 (ASCII ’m’) to 10-bit binary: 00011011012

  43. [54]

    Perform XOR operation

  44. [55]

    **Answer:** The ciphertext produced is 901

    Convert the result 11100001012 to decimal: 901. **Answer:** The ciphertext produced is 901 . 20

  45. [109]

    So maybe ’b’ is the letter ’m’, which we need to convert to a numerical key

    But the user specified that a = <|end_of_thinking|>, which is already a number. So maybe ’b’ is the letter ’m’, which we need to convert to a numerical key. Alternatively, perhaps ’b’ is the 13th letter (since m is the 13th letter in the alphabet), so b = 13. But the user wrot...

  46. [2022]

    doi: 10.1109/TITS.2021.3131668

    2021

  47. [2024]

    doi: 10.18653/v1/2024.acl-long.119

    Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.119. URL https://aclanthology.org/2024.acl-long.119/

    2024 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.