Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding chain-of-thought reasoning to LLMs shifts which prompt attacks succeed, making models slightly safer on average but markedly weaker against tree-of-attacks and suffix prompts.

desk verdict First systematic map of reasoning-model attack vulnerabilities, but the sampling inconsistency makes the headline numbers unreliable as reported. read the letter →

arxiv 2506.13726 v1 pith:TTHQKZEZ submitted 2025-06-16 cs.AI cs.CRcs.LG

Arjun Krishna , Aaditya Rastogi , Erick Galinkin This is my paper
classification cs.AIcs.CRcs.LG
keywords chain-of-thoughtreasoningpromptattacksjailbreakattacksuccessratetree-of-attackssuffixinjectioncross-sitescriptingmalwaregeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reasoning-enhanced language models are conventionally expected to be either safer or more dangerous than plain instruction-tuned models under adversarial prompting. This paper reports a systematic comparison of three model families, each with a matched non-reasoning and chain-of-thought variant, across 35 prompt attacks in seven categories. Averaged over all probes, the reasoning models are slightly more robust (42.51% vs 45.53% attack success rate), but the average conceals a sharp reversal: reasoning models are far more vulnerable to tree-of-attacks prompts (+32.13 points) and suffix injections (+22.20 points), while being much more resistant to cross-site scripting (-29.80 points) and malware requests (-18.9 points). A sympathetic reader would take the paper as establishing that reasoning capability changes the attack surface in opposite directions depending on the attack class.

What carries the argument

The central mechanism is a paired-model attack-success-rate comparison. For each of three families, a non-reasoning instruction-tuned model and its reasoning counterpart are run through the same 35 adversarial probes, and each generation is scored as a binary success or failure against the probe's malicious intent (Attack Success Rate, ASR). The paired design is what carries the argument: within-family differences in ASR are attributed to the introduction of chain-of-thought reasoning, and the seven-category probe set is what reveals that the direction of the effect flips by attack type.

What would settle it

Run the same 35 probes on pairs of models that share architecture, parameter count, and training data and differ only in chain-of-thought fine-tuning; if the TAP gap (+32.13 points) and suffix gap (+22.20 points) disappear or flip, the paper's attribution of these vulnerabilities to reasoning capability is falsified.

Watch

Extended reading notes

Core claim

The paper claims that chain-of-thought reasoning is neither a security shield nor a vulnerability enabler on its own; it is a capability that shifts which attacks succeed. On the aggregate, reasoning models resisted better than their non-reasoning counterparts, with attack success rates of 42.51% versus 45.53%. Disaggregated by attack category, however, the reasoning models were the weakest link for complex automated jailbreaks (Tree-of-Attacks, 62.97% vs 30.83%) and for hidden adversarial suffixes (29.90% vs 7.70%), while they nearly eliminated cross-site scripting failures (4.40% vs 33.11%) and substantially reduced malware generation (65.50% vs 84.40%). Family-level data show these effects are not uniform: the DeepSeek and Qwen reasoning variants drove the TAP and suffix regressions, while the Llama reasoning variant was more robust than its base model in nearly every category. The paper concludes that security evaluations of reasoning models must be category-specific rather than averaged.

Load-bearing premise

The measured gap is attributed to reasoning, so everything rests on the assumption that the paired models in each family differ only in reasoning capability, even though they also differ in parameter count, release date, and alignment recipe.

Editorial extensions

If this is right

  • Security dashboards that report only average attack success rates will miss the largest effects; category-level and family-level reporting is needed to see where reasoning helps and where it hurts.
  • Deploying reasoning models in agentic systems should treat appended or trailing text as a first-class threat, since suffix injections succeeded at nearly four times the rate against reasoning models.
  • Automated jailbreak generators such as tree-of-attacks should be part of standard red-teaming for reasoning models, since the reasoning group's success rate more than doubled on that probe.
  • Reasoning models may reduce the need for output-side filtering on plainly malicious requests (XSS payloads, malware), but they increase the need for input-side filtering and chain-of-thought oversight.
  • The absence of a uniform family effect means a single 'reasoning vs non-reasoning' verdict is not portable across model vendors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the suffix vulnerability comes from reasoning models treating every input token as relevant context, then any retrieval-augmented or tool-calling pipeline that concatenates untrusted text will inherit the weakness; this could be tested by varying the position and formatting of the injected suffix.
  • Because the paired models also differ in parameter count, release date, and alignment recipe, the causal role of reasoning itself is not fully identified; a matched-ablation design that varies only chain-of-thought training would settle the attribution.
  • The large gains on XSS and malware may reflect safety alignment rather than reasoning, since those requests are easy to classify as forbidden; comparing against a non-reasoning model with the same refusal tuning would separate the two.
  • A plausible design response is to hide or filter the chain-of-thought at inference time, since exposed or overly attentive reasoning is the surface that TAP and suffix attacks exploit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper compares three pairs of publicly released LLMs, one non-reasoning and one reasoning model per family, across 35 probes in seven attack categories using the garak red-teaming framework. It reports that reasoning models have a slightly lower average attack success rate (42.51% vs 45.53%) but are substantially more vulnerable to TAP and suffix-injection attacks, and substantially more robust on XSS and malware-generation requests. The authors interpret these differences as effects of chain-of-thought reasoning and offer security design implications, including rationale filtering and staged policy checks.

Significance. If the headline measurements were reliable, the paper would be a useful systematic comparison of the security posture of currently shipped reasoning and non-reasoning models, and the category-specific reversals would be informative for red-teaming practice. The use of a standardized framework (garak), the seven attack categories, and the per-family tables are strengths, as is the decision to avoid publishing exact prompt strings for dual-use reasons. However, the design compares models that differ in many ways besides reasoning, and the reported percentages are not consistent with the stated sampling scheme, so the central causal attribution is not currently supported. The paper does not provide raw per-probe data or code, which further limits reproducibility of the numerical claims.

major comments (3)
  1. [§3.1, Table 1] The paired models are not controlled ablations of reasoning. DeepSeek-V3 and DeepSeek-R1 differ in release date and training procedure; Qwen2.5-Coder-32B-Instruct and QwQ-32B differ in domain specialization (code) and alignment; Llama-3.3-70B-Instruct and Llama-3.3-Nemotron-49B-Super differ in parameter count (70B vs 49B), vendor, architecture, and alignment recipe. Consequently, the headline averages (42.51% vs 45.53%) and the category-level gaps in Table 4 cannot be attributed to "advanced reasoning" rather than to these confounds. The manuscript should either compare models that differ only in reasoning capability, or reframe the claims as descriptive comparisons of currently shipped model variants, with the confounds explicitly listed as limitations.
  2. [§3.2, §4.2, Tables 2 and 4] The stated sampling scheme ("Each probe is sampled 3 times per model") cannot produce the reported percentages. With three samples, a single-probe category such as TAP or Suffix Injection can only take values 0%, 33.3%, 66.7%, or 100%, yet Table 2 reports values such as 3.7%, 81.5%, 47.4%, 42.3%, 23.1%, and 25.9%. For a category with two probes (ANSI Escape, six samples per model), the granularity is 16.7 percentage points, so values like 59.2% and 55.2% are impossible. The mismatch indicates either different per-probe repetition counts, aggregation across more runs than stated, or an arithmetic error. The paper must report raw counts, per-probe results, and the actual number of executions per cell, and should include confidence intervals or exact binomial tests before drawing conclusions from the roughly three-point average difference.
  3. [§4.2.1, Table 2] The claim that reasoning models are substantially more vulnerable to TAP (+32.13 points) and suffix injections (+22.20 points) rests on single-probe cells with three samples per model, and the direction is not consistent across families: for TAP, the LLaMA reasoning model is more robust (25.9% vs 48.1%), while DeepSeek and Qwen drive the increase; for Suffix, LLaMA's reasoning model has 0% vs 23.1%, opposite the other two families. The mechanistic speculations in §4.2.1 ("the chain-of-thought mechanisms might have been leveraged by TAP", "reasoning models might be over-emphasizing the entire input") are therefore not supported by the evidence as reported. The paper should present per-family, per-probe raw outcomes and avoid mechanistic interpretation until the statistical basis is established.
minor comments (5)
  1. [Abstract] The first sentence, "The introduction of advanced reasoning capabilities have improved...", should be corrected to "has improved".
  2. [References [12] and text] The OWASP name is rendered as "OW ASP" in both the reference and the body text; please correct it.
  3. [§3.2, §3.3] The terms "probe" and "probe category" are used interchangeably; please define whether a probe is a single template or a category, because the arithmetic of the reported 210 model-probe evaluations depends on this distinction.
  4. [Figure 1] Figure 1 is referenced in §4.1 but is not included in the provided text; please ensure the figure appears with axis labels, error bars, and a caption.
  5. [Abstract and Conclusion] The phrases "the first empirical study" and "first large-scale comparison" should be softened, given the model-pair confounds and the statistical reporting issues discussed in the major comments.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the reported attack success rates are direct measurements, and the only author-overlap citation (garak) is an independent, code-reproduced measurement tool rather than a fitted input or self-imported premise.

full rationale

The paper's central claims are not derived from a formal model or fitted parameters; they are aggregated observations of probe executions across three model families. The route from probe outputs to ASR values is a straightforward counting procedure with no equation that defines the conclusion into the inputs. The only self-citation is the garak framework, whose first author, Erick Galinkin, is also an author of this paper; however, garak is a public, code-reproduced red-teaming toolkit and the paper uses it as a measurement instrument, not as a source of the conclusion. The attack success rates are not constructed to match garak's internal assumptions in a way that forces the reported reasoning-versus-non-reasoning differences. There is no fitted parameter later renamed as a prediction, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The family-level comparisons are confounded by model size, release date, and alignment recipe, and the per-probe three-sample design raises reproducibility concerns, but confounding and statistical fragility are correctness risks rather than circularity. Therefore no significant circularity is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the comparability of model pairs, on the stability of three-sample ASR estimates, on the validity of garak's compliance detector, and on the representativeness of the 35-probe set. No fitted free parameters or invented entities are introduced; the paper is a measurement study rather than a derivation.

assumptions (4)
  • domain assumption Each reasoning and non-reasoning model pair isolates the effect of chain-of-thought reasoning on security.
    Table 1 pairs DeepSeek-V3/R1, Qwen Coder/QWQ, and Llama-3.3-70B/Nemotron-49B, but they differ in parameter count, release date, and vendor alignment training, so the comparison is confounded.
  • domain assumption Three samples per probe provide stable enough attack success rate estimates to support the reported differences.
    Section 3.2 declares three samples per probe, but single-probe categories show granular percentages like 3.7% and 47.4%, which are inconsistent with three binary trials and no confidence intervals are reported.
  • domain assumption garak's compliance detection correctly identifies whether a response satisfies the malicious intent of each prompt.
    The entire ASR metric depends on this detector, yet no validation, manual audit, or inter-annotator agreement is reported in Sections 3.2 or 4.
  • domain assumption The seven probe categories and 35 templates adequately represent the attack surface of deployed agentic LLMs.
    The chosen probes cover important documented attack types, but the set is small and unevenly weighted, with a single probe for TAP and suffix injection, which limits generality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models." pith.science (2026). https://pith.science/paper/TTHQKZEZ

@misc{pith2026250613726,
  author       = {Pith},
  title        = {Pith review of: Weakest Link in the Chain: Security Vulnerabilities in Advanced Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTHQKZEZ}},
  note         = {Machine review of arXiv:2506.13726}
}
read the original abstract

The introduction of advanced reasoning capabilities have improved the problem-solving performance of large language models, particularly on math and coding benchmarks. However, it remains unclear whether these reasoning models are more or less vulnerable to adversarial prompt attacks than their non-reasoning counterparts. In this work, we present a systematic evaluation of weaknesses in advanced reasoning models compared to similar non-reasoning models across a diverse set of prompt-based attack categories. Using experimental data, we find that on average the reasoning-augmented models are \emph{slightly more robust} than non-reasoning models (42.51\% vs 45.53\% attack success rate, lower is better). However, this overall trend masks significant category-specific differences: for certain attack types the reasoning models are substantially \emph{more vulnerable} (e.g., up to 32 percentage points worse on a tree-of-attacks prompt), while for others they are markedly \emph{more robust} (e.g., 29.8 points better on cross-site scripting injection). Our findings highlight the nuanced security implications of advanced reasoning in language models and emphasize the importance of stress-testing safety across diverse adversarial techniques.

Figures

Figures reproduced from arXiv: 2506.13726 by the authors.

Figure 1
Figure 1. Average Attack Success Rate by Model Fam [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.

Reference graph

Works this paper leans on

20 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    Leon Derczynski, Erick Galinkin, Jeffrey Martin, Subho Majumdar, and Nanna Inie. 2024. garak: A framework for security probing large language mod- els. arXiv preprint arXiv:2406.11036

    arXiv 2024

  2. [2]

    Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2023. Complexity-based prompt- ing for multi-step reasoning. arXiv preprint arXiv:2210.00720

    arXiv 2023

  3. [3]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shi- rong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    arXiv 2025

  4. [4]

    Ben Herzog. 2025. Xss marks the spot: Digging up vulnerabilities in chatgpt. https://www.imperva. com/blog/xss-marks-the-spot

    work page 2025

  5. [5]

    Trent Holmes and Willem Gooderham. 2025. Ex- ploiting deepseek-r1: Breaking down chain of thought security. https://www.trendmicro.c om/en_us/research/25/c/exploiting-deepsee k-r1.html. Trend Micro Research

    work page 2025

  6. [6]

    Ken Huang. 2025. Agentic ai threat modeling frame- work: Maestro. https://cloudsecurityallianc e.org/blog/2025/02/06/agentic-ai-threat-m odeling-framework-maestro

    work page 2025

  7. [7]

    Hamed Jelodar, Samita Bai, Parisa Hamedi, Hesamodin Mohammadian, Roozbeh Razavi-Far, and Ali Ghorbani. 2025. Large language model (llm) for software security: Code analysis, mal- ware analysis, reverse engineering. arXiv preprint arXiv:2504.07137

    arXiv 2025

  8. [8]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916

    arXiv 2022

Show all 20 references
  1. [9]

    Pedro Henrique Lima. 2024. Llm pentest: Lever- aging agent integration for remote code execution. https://www.blazeinfosec.com/post/llm-pen test-agent-hacking/

    2024

  2. [10]

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Se- curity 24), pages 1831–1847

    2024

  3. [11]

    Anay Mehrotra, Manolis Zampetakis, Paul Kas- sianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023. Tree of attacks: Jailbreaking black-box llms automatically. arXiv preprint arXiv:2312.02119

    2023 arXiv

  4. [12]

    OW ASP Foundation. 2025. Owasp top 10 for large language model applications v2025. https://owas p.org/www-project-top-10-for-large-langu age-model-applications/assets/PDF/OWASP-T op-10-for-LLMs-v2025.pdf . OW ASP

    2025

  5. [13]

    Sean Park. 2025. Unveiling ai agent vulnerabilities part ii: Code execution. https://www.trendmicro .com/vinfo/us/security/news/cybercrime-a nd-digital-threats/unveiling-ai-agent-vul nerabilities-code-execution

    2025

  6. [14]

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red team- ing language models with language models. arXiv preprint arXiv:2202.03286

    2022 arXiv

  7. [15]

    Niklas Pfister, Václav V olhejn, Manuel Knott, San- tiago Arias, Julia Bazi ´nska, Mykhailo Bichurin, Alan Commike, Janet Darling, Peter Dienes, Matthew Fiedler, and 1 others. 2025. Gandalf the red: Adaptive security for llms. arXiv preprint arXiv:2501.07927

    2025 arXiv

  8. [16]

    Johann Rehberger. 2024. Terminal dillma: Leverag- ing ansi sequences to hijack llm integrations. https: //embracethered.com/blog/posts/2024/term inal-dillmas-prompt-injection-ansi-seque nces/

    2024

  9. [17]

    do anything now

    Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, page...

    2024

  10. [18]

    Elliot Ward, Rory McNamara, Mateo Rojas-Carulla, Sam Watts, and Eric Allen. 2024. Agent hijacking: The true impact of prompt injection attacks. https: //labs.snyk.io/resources/agent-hijacking/

    2024

  11. [19]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. Ad- vances in neural information processing systems , 35:24824–24837

    2022

  12. [20]

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Uni- versal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.