Pith. sign in

REVIEW 4 major objections 4 minor 39 references

A survey of prompt-based attacks argues that LLM security requires redesigning the models themselves, not just filtering inputs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A survey categorizing prompt-based attacks on LLMs into four classes and proposing aspirational goals of un-distillable, un-finetunable, and un-editable models.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A readable but thin survey whose own scope claims don't hold up: it collects the right attacks, but the promised 'clear threat model' of prompt-based attacks is undermined by including non-prompt model-level attacks. the 4 major comments →

arxiv 2509.04615 v1 pith:P46LBUDM submitted 2025-09-04 cs.CL cs.CRcs.LG

Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

Brennen Hill , Surendra Parla , Venkata Abhijeeth Balabhadruni , Atharv Prajod Padmalayam , Sujay Chandra Shekara Sharma This is my paper
classification cs.CL cs.CRcs.LG
keywords prompt injectionjailbreakLLM securitythreat modeldata poisoningtrojan attacksred teamingadversarial prompts
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a literature survey that tries to turn the scattered catalog of prompt-based attacks on large language models into a structured threat model. It sorts attacks into four families—input manipulation and injection; semantic and knowledge-based manipulation; integration and model-level exploits; and output exploitation plus automated attacks—and argues that together they show surface-level defenses such as input filters and output guardrails are losing ground. The authors' central prescription is that the next generation of LLMs should be designed to be resistant to unauthorized distillation, fine-tuning, and editing, with security built into the model rather than bolted on. This matters because it gives defense researchers a single organizing map and a concrete set of design goals to aim at.

Core claim

The paper's central claim is that prompt-based attacks are not isolated tricks but a structured attack surface, and it offers a four-part categorization as the threat model: (1) input manipulation and injection, including direct and indirect injection and adversarial crafting; (2) semantic and knowledge-based manipulation, including chain-of-thought misuse, red teaming, and data poisoning; (3) integration and model-level exploits, including common jailbreak techniques and trojan/backdoor attacks that modify internal state; (4) output exploitation and automated attacks, including hallucination induction, data leaks, and RL-driven or universal-trigger attack generation. The intended payoff is

What carries the argument

The load-bearing object is the taxonomy itself, used as a threat model. Each category maps attack families to concrete mechanisms—repeated instructions and role-play override guardrails; poisoned inputs in retrieved or fine-tuning data alter behavior from within; bit-flipping and activation-steering trojans compromise the model without new prompts; and RL or universal-trigger search automates exploit discovery. The taxonomy's work is to convert a list of known attacks into design requirements for future LLM architectures.

Load-bearing premise

The paper's call for systemic defenses rests on assuming that attacks transfer across major LLMs because they share architectures, tokenizers, and training data; if transferability is low, per-model patching could remain viable and the urgency of built-in defenses drops.

What would settle it

A benchmark study that measures whether a set of prompt-injection and jailbreak prompts developed against one current LLM family succeed at similar rates on several other families, controlling for model size; high per-model variance and low cross-model success would falsify the paper's transferability premise.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Surface-level input filtering and output guardrails become necessary but not sufficient; the paper argues defenses must be moved into training and architecture.
  • Unauthorized distillation becomes a design constraint: watermarking and fingerprinting are framed as core mechanisms, not optional extras.
  • Poisoning and trojan attacks make 'un-finetunable' architectures a concrete research target, including secure fine-tuning and formal verification.
  • Automated attack generation means security evaluations must be continuous and benchmark-driven rather than one-time assessments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the paper's logic: run one attack from each of the four classes against the same defense method and measure whether defending one class weakens the others; the survey does not itself provide this experiment.
  • If attack transferability across shared model families holds, the same taxonomy could become a procurement checklist for regulated LLM deployments, with 'un-distillable' and 'un-finetunable' treated as contractual security properties.
  • The un-distillable goal sits in tension with open-weight model distribution; if taken seriously, it implies an explicit policy trade-off between transparency and protection from extraction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper is a short survey of attacks on large language models. Its stated goal is to provide a 'comprehensive' threat model of prompt-based attacks, organized into four categories: input manipulation/injection, semantic/knowledge-based manipulation, integration/model-level exploits, and output/automated attacks. The survey summarizes individual attacks and techniques (e.g., direct/indirect prompt injection, chain-of-thought misuse, red teaming, data poisoning, jailbreaking, trojan attacks, hallucination induction, automated attack generation) and concludes by arguing for foundational defenses that produce 'Un-Distillable, Un-Finetunable, and Un-Editable' models.

Significance. If the threat model were coherent, the paper could serve as a useful entry-level map of the LLM attack surface and as a compact catalog of attack families and representative references. Its main positive contribution is the organization of known attack techniques into a small number of themes and the emphasis on moving from reactive patching to architectural defenses. However, the central claim of a 'clear threat model' is undermined by an internal scope contradiction: the title and abstract restrict the survey to prompt-based attacks, while Section 4.2 explicitly includes model-internal, non-prompt manipulations such as bit-flipping and activation steering. The paper also makes an unsupported empirical assertion about attack transferability that is load-bearing for its motivation, and the 'comprehensive' characterization is not supported by the reference base or by an explicit inclusion methodology.

major comments (4)
  1. [§4.2, §3.2, Abstract] The scope of the survey is internally inconsistent. The abstract promises 'prompt-based attack methodologies,' but §4.2 states that trojan attacks, 'Unlike prompt-level attacks... involve manipulation of the model’s internal state or parameters,' then describes bit-flipping [25] and activation-vector steering [26]. The same TA² activation-steering attack also appears in §3.2 as [11]. Because the paper’s own text excludes these from the prompt-level class, the four-way taxonomy is not a threat model of prompt-based attacks; it is a mix of input-level and model-level threats. The authors must either remove the non-prompt attacks, explicitly redefine the scope to include model-manipulation attacks, or explain why these attacks belong in a prompt-based threat model and how the taxonomy boundary is drawn.
  2. [§1] The introduction asserts that 'common architectural foundations... across major LLMs like GPT, Claude, and Llama lead to a high degree of attack transferability.' This is a strong empirical claim with no citation or evidence. It is load-bearing because the paper uses it to argue that systemic, built-in defenses are essential. If transferability is low or limited to specific attack types, the motivation is weakened. Even if the claim is plausible, the manuscript should cite supporting studies (e.g., universal adversarial suffix transfer results) or qualify the claim to the specific attacks for which transferability has been demonstrated.
  3. [§2–§4] The 'comprehensive' characterization is overstated and the taxonomy is not clearly exclusive. The paper has only about 35 unique references and no stated inclusion criteria, so the claim of a comprehensive survey is not supported. In addition, the categories overlap: §4.1 repeats techniques already covered in §2.1 (jailbreaking, few-shot attack, obfuscation), and §4.2 overlaps with §3.2 (TA²). A survey that aims to provide a 'clear threat model' should define the categories so that each attack is assigned to exactly one class, or explicitly discuss overlaps and the rationale for the assignment.
  4. [§6, §7] The three design goals — 'Un-Distillable, Un-Finetunable, and Un-Editable' — are asserted in the conclusion and future work rather than derived from the survey. The paper does not explain how the categorized attacks map to these properties or why these three goals are necessary or sufficient. For example, data-leak attacks motivate 'Un-Distillable,' but no defense direction is connected to the specific attack mechanisms. The authors should either provide a mapping from attack classes to the design goals or explicitly frame these goals as a research agenda rather than a conclusion of the survey.
minor comments (4)
  1. [References] Several references are duplicated: [11], [26], and [27] are the same TA² paper; [3] and [21] are the same paper in different venues; [1] and [15] are the same paper; [24] and [35] are the same paper. These should be consolidated or cross-referenced.
  2. [§7] Typo: 'Theoretical F oundations' should be 'Theoretical Foundations'.
  3. [Throughout] The term 'Un-Finetunable' is inconsistently spaced/hyphenated; consider using a consistent form such as 'Un-Fine-Tunable.'
  4. [Reference [19]] Reference [19] is a Twitter post. If possible, replace with a peer-reviewed or archival source for typoglycemia-based jailbreaks.

Circularity Check

0 steps flagged

No circularity: the paper is a literature survey with no derivation chain, fitted parameters, or self-citations to reduce.

full rationale

This is a survey paper: it categorizes known prompt-based attack techniques from the literature and does not derive new results, fit parameters, or make quantitative predictions. There are no equations whose inputs equal outputs, no fitted values renamed as predictions, and no load-bearing self-citations (the authors cite no work by themselves). The assertion that common architectural foundations lead to attack transferability is an unstated assumption, not a circular derivation. The proposed threat model is a taxonomy of previously published attacks, and the design goals (Un-Distillable, Un-Finetunable, Un-Editable) are presented as motivations for future work, not as conclusions derived from the survey's own data. The scope inconsistency noted by skeptics (e.g., including model-internal Trojan attacks in a survey titled 'prompt-based attacks') is a coherence or completeness concern, not a circularity concern. Therefore, the paper does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

No free parameters are fit, and no new entities are invented. The survey rests on the accuracy of its cited sources and on an unproven assumption about cross-model transferability.

axioms (2)
  • domain assumption Common architectural foundations across major LLMs lead to high attack transferability
    Asserted in Section 1 without citation or experimental support; the survey's argument for systemic defenses relies on this.
  • domain assumption The cited 35 works are accurately summarized and representative of the field
    The survey builds its taxonomy entirely from cited literature; no systematic search or inclusion criteria is described, and some citations are duplicated.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs." pith.science (2026). https://pith.science/paper/P46LBUDM

@misc{pith2026250904615,
  author       = {Pith},
  title        = {Pith review of: Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P46LBUDM}},
  note         = {Machine review of arXiv:2509.04615}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The proliferation of Large Language Models (LLMs) has introduced critical security challenges, where adversarial actors can manipulate input prompts to cause significant harm and circumvent safety alignments. These prompt-based attacks exploit vulnerabilities in a model's design, training, and contextual understanding, leading to intellectual property theft, misinformation generation, and erosion of user trust. A systematic understanding of these attack vectors is the foundational step toward developing robust countermeasures. This paper presents a comprehensive literature survey of prompt-based attack methodologies, categorizing them to provide a clear threat model. By detailing the mechanisms and impacts of these exploits, this survey aims to inform the research community's efforts in building the next generation of secure LLMs that are inherently resistant to unauthorized distillation, fine-tuning, and editing.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 15 canonical work pages · 3 internal anchors

  1. [1]

    Jailbroken: How does llm safety training fail?, 2023

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail?, 2023. URL https://arxiv.org/abs/2307.02483

    Pith/arXiv arXiv 2023

  2. [2]

    Tricking llms into disobedience: Formalizing, analyzing, and detecting jailbreaks, 2024

    Abhinav Rao, Sachin Vashistha, Atharva Naik, Somak Aditya, and Monojit Choudhury. Tricking llms into disobedience: Formalizing, analyzing, and detecting jailbreaks, 2024. URL https: //arxiv.org/abs/2305.14965

    Pith/arXiv arXiv 2024

  3. [3]

    Exploiting programmatic behavior of llms: Dual-use through standard security attacks, 2023

    Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. Exploiting programmatic behavior of llms: Dual-use through standard security attacks, 2023. URL https://arxiv.org/abs/2302.05733

    Pith/arXiv arXiv 2023

  4. [4]

    Prompt injection attack against llm- integrated applications, 2024

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. Prompt injection attack against llm- integrated applications, 2024. URL https://arxiv.org/abs/2306.05499

    Pith/arXiv arXiv 2024

  5. [5]

    Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection, 2023

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection, 2023. URL https://arxiv.org/abs/2302.12173

    Pith/arXiv arXiv 2023

  6. [6]

    Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts, 2024

    Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Neil Zhenqiang Gong, and Xing Xie. Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts, 2024. URL https: //arxiv.org/abs/2306.04528

    Pith/arXiv arXiv 2024

  7. [7]

    Black box adversarial prompting for foundation models, 2023

    Natalie Maus, Patrick Chao, Eric Wong, and Jacob Gardner. Black box adversarial prompting for foundation models, 2023. URL https://arxiv.org/abs/2302.04237

    Pith/arXiv arXiv 2023

  8. [8]

    A prompting-based approach for adversarial example generation and robustness enhancement,

    Yuting Yang, Pei Huang, Juan Cao, Jintao Li, Yun Lin, Jin Song Dong, Feifei Ma, and Jian Zhang. A prompting-based approach for adversarial example generation and robustness enhancement,

    work page

  9. [9]

    Badchain: Backdoor chain-of-thought prompting for large language models, 2024

    Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and Bo Li. Badchain: Backdoor chain-of-thought prompting for large language models, 2024. URL https://arxiv.org/abs/2401.12242

    Pith/arXiv arXiv 2024

  10. [10]

    Preemptive answer "attacks" on chain-of-thought reasoning,

    Rongwu Xu, Zehan Qi, and Wei Xu. Preemptive answer "attacks" on chain-of-thought reasoning,

    work page

  11. [11]

    Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment, 2024

    Haoran Wang and Kai Shu. Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment, 2024. URL https://arxiv.org/abs/2311.09433

    Pith/arXiv arXiv 2024

  12. [12]

    Recent advancements in llm red-teaming: Techniques, defenses, and ethical considerations, 2024

    Tarun Raheja and Nilay Pochhi. Recent advancements in llm red-teaming: Techniques, defenses, and ethical considerations, 2024. URL https://arxiv.org/abs/2410.09097. 6

    Pith/arXiv arXiv 2024

  13. [13]

    Learning to poison large language models during instruction tuning, 2024

    Yao Qiang, Xiangyu Zhou, Saleh Zare Zade, Mohammad Amin Roshani, Prashant Khanduri, Douglas Zytko, and Dongxiao Zhu. Learning to poison large language models during instruction tuning, 2024. URL https://arxiv.org/abs/2402.13459

    Pith/arXiv arXiv 2024

  14. [14]

    Data poisoning in llms: Jailbreak-tuning and scaling laws, 2024

    Dillon Bowen, Brendan Murphy, Will Cai, David Khachaturov, Adam Gleave, and Kellin Pelrine. Data poisoning in llms: Jailbreak-tuning and scaling laws, 2024. URL https: //arxiv.org/abs/2408.02946

    Pith/arXiv arXiv 2024

  15. [15]

    Jailbroken: How does llm safety training fail? In Advances in Neural Information Processing Systems ,

    Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? In Advances in Neural Information Processing Systems ,

    work page

  16. [16]

    Universal and transferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. URL https://arxiv.org/abs/2307.15043

    Pith/arXiv arXiv 2023

  17. [17]

    0xk1h0 - github: Jailbreak prompts collection

    Kihong Lee. 0xk1h0 - github: Jailbreak prompts collection. https://github.com/0xk1h0/ ChatGPT_DAN, 2023. Accessed: 2024-12-13

    work page 2023

  18. [18]

    Llm jailbreak | mitre atlas ™

    MITRE. Llm jailbreak | mitre atlas ™. https://atlas.mitre.org/techniques/AML. T0054, 2024. Accessed: 2024-12-13

    work page 2024

  19. [19]

    Novel jailbreak technique via typoglycemia, 2023

    LaurieWired. Novel jailbreak technique via typoglycemia, 2023. URL https://twitter. com/lauriewired/status/1682825249203662848. Tweet

    arXiv 2023

  20. [20]

    Masterkey: Automated jailbreak across multiple large language model chatbots

    Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. Masterkey: Automated jailbreak across multiple large language model chatbots. In Proceedings of the Network and Distributed System Security Symposium (NDSS) ,

    work page

  21. [21]

    Exploiting programmatic behavior of llms: Dual-use through standard security attacks

    Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. In Proceedings of the 45th IEEE Symposium on Security and Privacy Workshops (SPW) , pages 132–143. IEEE, 2024. doi: 10.1109/SPW63631.2024.00018. URL https://arxiv.org/ abs/2302.05733

    arXiv 2024

  22. [22]

    Prompt injection attacks in various llms, 2023

    Murtuza Shergadwala. Prompt injection attacks in various llms, 2023. URL https://medium.com/@murtuza.shergadwala/ prompt-injection-attacks-in-various-llms-206f56cd6ee9 . Online; accessed 2024-12-13

    work page 2023

  23. [23]

    Prompt injection attack on gpt-4, March 2023

    William Zhang. Prompt injection attack on gpt-4, March 2023. URL https://www. robustintelligence.com/blog-posts/prompt-injection-attack-on-gpt-4 . Ac- cessed: 2024-12-13

    work page 2023

  24. [24]

    URL https://arxiv.org/abs/2307.08715

    Pith/arXiv arXiv

  25. [25]

    Trojtext: Test-time invisible textual trojan insertion

    Qian Lou, Yepeng Liu, and Bo Feng. Trojtext: Test-time invisible textual trojan insertion. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR) ,

    work page

  26. [26]

    Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment

    Haoran Wang and Kai Shu. Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM’24) , pages 2347–2357, 2024. doi: 10.1145/3627673.3679821. URL https://arxiv.org/abs/2311.09433. 7

    arXiv 2024

  27. [27]

    Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment

    Haoran Wang and Kai Shu. Trojan activation attack: Red-teaming large language models using activation steering for safety-alignment. arXiv preprint arXiv:2311.09433, 2023. doi: 10.48550/arXiv.2311.09433. URL https://arxiv.org/abs/2311.09433. Presented at the ACM International Conference on Information and Knowledge Management (CIKM’24)

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.09433 2023

  28. [29]

    Do llms have political correctness? analyzing ethical biases and jailbreak vulnerabilities in ai systems, 2024

    Isack Lee and Haebin Seong. Do llms have political correctness? analyzing ethical biases and jailbreak vulnerabilities in ai systems, 2024. URL https://arxiv.org/abs/2410.13334

    work page arXiv 2024

  29. [30]

    arXiv:2303.02242

    URL https://arxiv.org/abs/2303.02242. arXiv:2303.02242

    Pith/arXiv arXiv

  30. [31]

    Identifying and mitigating privacy risks stemming from language models: A survey, 2024

    Victoria Smith, Ali Shahin Shamsabadi, Carolyn Ashurst, and Adrian Weller. Identifying and mitigating privacy risks stemming from language models: A survey, 2024. URL https: //arxiv.org/abs/2310.01424

    Pith/arXiv arXiv 2024

  31. [32]

    Reinforcement learning- driven LLM agent for automated attacks on LLMs

    Xiangwen Wang, Jie Peng, Kaidi Xu, Huaxiu Yao, and Tianlong Chen. Reinforcement learning- driven LLM agent for automated attacks on LLMs. In Ivan Habernal, Sepideh Ghanavati, Abhi- lasha Ravichander, Vijayanta Jain, Patricia Thaine, Timour Igamberdiev, Niloofar Mireshghallah, and Oluwaseyi Feyisetan, editors, Proceedings of the Fifth Workshop on Privacy i...

    work page 2024

  32. [33]

    Llm lies: Hallucinations are not bugs, but features as adversarial examples, 2024

    Jia-Yu Yao, Kun-Peng Ning, Zhen-Hui Liu, Mu-Nan Ning, Yu-Yang Liu, and Li Yuan. Llm lies: Hallucinations are not bugs, but features as adversarial examples, 2024. URL https: //arxiv.org/abs/2310.01469

    Pith/arXiv arXiv 2024

  33. [34]

    Trojan Detection in Large Language Models: Insights from The Trojan Detection Challenge

    Narek Maloyan, Ekansh Verma, Bulat Nutfullin, and Bislan Ashinov. Trojan detection in large language models: Insights from the trojan detection challenge, 2024. URL https: //arxiv.org/abs/2404.13660

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [35]

    Privacy Risks of General-Purpose AI Systems: A Foundation for Investigating Practitioner Perspectives

    Stephen Meisenbacher, Alexandra Klymenko, Patrick Gage Kelley, Sai Teja Peddinti, Kurt Thomas, and Florian Matthes. Privacy risks of general-purpose ai systems: A foundation for investigating practitioner perspectives, 2024. URL https://arxiv.org/abs/2407.02027

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [38]

    Trojllm: A black-box trojan prompt attack on large language models, 2023

    Jiaqi Xue, Mengxin Zheng, Ting Hua, Yilin Shen, Yepeng Liu, Ladislau Boloni, and Qian Lou. Trojllm: A black-box trojan prompt attack on large language models, 2023. URL https://arxiv.org/abs/2306.06815

    Pith/arXiv arXiv 2023

  36. [40]

    Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global scale prompt hacking competition, 2024

    Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan Boyd-Graber. Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global scale prompt hacking competition, 2024. URL https://arxiv.org/abs/2311.16119. 8

    Pith/arXiv arXiv 2024

  37. [2022]

    URL https://arxiv.org/abs/2203.10714

    work page internal anchor Pith review Pith/arXiv arXiv

  38. [2023]

    URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ fd6613131889a4b656206c50a8bd7790-Paper-Conference.pdf

    work page 2023

  39. [2024]

    URL https://arxiv.org/abs/2405.20902

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.