Pith. sign in

REVIEW 4 major objections 7 minor 2 cited by

Attack and defense techniques in large language models: A survey and new perspectives

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey establishes an organizing taxonomy that sorts LLM attacks into adversarial prompt attacks, optimized attacks, application-level attacks, and model theft, then pairs each class with prevention-based or detection-based defenses.

desk verdict A loosely organized survey whose category boundaries overlap; useful as a reading list, but the 'systematic taxonomy' claim does not hold as stated. read the letter →

arxiv 2505.00976 v1 pith:QHBBFT4Y submitted 2025-05-02 cs.CR cs.AIcs.CLcs.LG

Zhiyu Liao , Kang Chen , Yuanguo Lin , Kangkang Li , Yunxuan Liu , Hefeng Chen , Xingwang Huang , Yuanhui Yu This is my paper · ORCID
classification cs.CRcs.AIcs.CLcs.LG
keywords Largelanguagemodels(LLMs)LLMattacksDefensetechniquesvulnerabilityPromptinjectionJailbreakleakageModeltheft
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to bring order to a fast-moving area: the security of large language models. It argues that the scattered attack literature can be organized into four families—adversarial prompt attacks, optimized attacks, attacks on LLM applications, and model theft—and that defenses fall naturally into prevention-based and detection-based approaches. If the taxonomy holds, newcomers can see which threats have known defenses and which cells remain underdeveloped, giving the field a shared vocabulary for comparing results. The paper's contribution is organizational rather than empirical: it does not run new attacks or defenses but consolidates reported successes and failures into a structured map.

What carries the argument

The organizing device is the taxonomy itself, anchored by the standard LLM workflow of tokenization, embedding, decoder computation, and output generation. Each attack family is described as exploiting a stage of that workflow, and each defense family is described by whether it intervenes before the model computes, after the model computes, or inside the model's parameters. Once an attack is placed in a cell, the paper can pair it with a corresponding defense category and flag cells that are underdeveloped; the taxonomy, not any new experiment, carries the argument.

What would settle it

A reader could take a large random sample of recent LLM security papers, classify each by the paper's four attack families and two defense families, and count how many fit without stretching. If a substantial fraction fall outside the categories, or if a re-collection of Tables 1 and 2 using a stated search protocol yields a materially different paper list, the claim of systematic coverage would fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that current LLM security research can be systematically summarized by where an attack enters an LLM's workflow. Attacks against the model itself and its applications divide into adversarial prompt attacks (jailbreak, prompt injection, prompt leakage), optimized attacks that self-improve their adversarial text, attacks on LLM-integrated applications, and model theft. Defenses divide into prevention-based strategies, which process inputs, outputs, or model internals to stop attacks, and detection-based strategies, which monitor inputs and outputs for malicious behavior. Within each cell the paper collects representative methods and reported metrics—for example, a genetic-algorithm jailbreaker reaching a 97.69 percent attack success rate on one model, a two-round prompt-leakage attack reaching up to 99 percent leakage, and a structured-query defense lowering manual prompt-injection success below 2 percent—and uses those results to identify open problems: adapting to evolving attacks, scaling defenses, explaining detection decisions, and standardizing evaluation.

Load-bearing premise

The survey's conclusions rest on the assumption that the papers collected in Tables 1 and 2 fairly represent the whole field, even though the survey does not state a search strategy, inclusion criteria, or coverage statistics.

Editorial extensions

If this is right

  • If the taxonomy is correct, jailbreak and prompt injection are distinct threats: jailbreak keeps the model's original safety prompts and tricks it through scenario framing, while prompt injection inserts new instructions that override the original context, so they require different defenses.
  • Treating prompt leakage as its own category elevates the protection of system prompts and prompt-market intellectual property to a first-class security problem, with reported leakage rates that make the risk concrete.
  • The prevention-versus-detection split gives defense research a shared frame: prevention methods act on inputs, outputs, or model internals, while detection methods monitor behavior, so a new defense can be located in the map and compared with neighbors.
  • The survey's list of open problems—adaptive scalable defenses, explainable security techniques, and standardized evaluation frameworks—identifies where the payoff for future research is likely to be.
  • Attacks on LLM-integrated applications are treated as potentially more damaging than attacks on the model alone, since a breached application can expose user data, vendor information, and system parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the taxonomy could double as a coverage map for benchmarks; a defense claiming generality should be tested in every cell of the map, not only in the cell where it was designed.
  • Editorial extension: because the paper reports subcategories of prompt injection but no unified metric for them, a natural next test is to score obfuscation, context ignoring, and indirect injection separately on one common benchmark to see whether they represent one weakness or three.
  • Editorial extension: the survey's selection premise implies a reproducibility check—rerun the classification on a larger or differently collected corpus of LLM security papers and see whether the same empty cells and open problems emerge.
  • Editorial extension: since the paper treats watermarking as the practical route against model theft, a robust-watermark scale measuring survival under watermark-removal attacks would be a direct test of that recommendation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This manuscript is a survey of security research on large language models. It proposes a taxonomy of attacks, dividing them into adversarial prompt attacks (jailbreak, prompt injection, and prompt leakage), optimized attacks, attacks on LLM applications, and model theft; it then organizes defenses into prevention-based and detection-based methods, and ends with a list of practical challenges and future research directions. The paper does not present new experiments or datasets; its contribution is an organizing framework and a review of recent work, with particular attention to LLM-integrated applications and watermark-based defenses.

Significance. At its best, the survey would provide a useful entry point to a fast-moving literature: it collects recent attack and defense papers, gives concrete quantitative results attributed to the original papers (e.g., AutoDAN ASR values, PRSA's 30% single-query success rate, Agarwal et al.'s 99% leakage rate), and highlights application-level risks and model watermarking as distinct areas. The value is conditional on the taxonomy being a genuine systematic classification; the manuscript does not yet meet that bar, so the significance is currently limited to that of a reference digest rather than a survey with a validated framework.

major comments (4)
  1. [§3, Table 1] The central claim in §6 that the survey "systematically" categorizes attacks and defenses is not supported because the taxonomy in §3 mixes overlapping and non-exclusive categories. Section 3.2 defines optimized attacks as attacks that "select or enhance attack algorithms using optimization techniques" and states that they "can be integrated with other attack techniques, such as jailbreak or prompt injection," yet Table 1 places AutoDAN [36] and GPTFuzzer [73] under Jailbreak while QROA [24] and JudgeDeceiver [54] are placed under Optimized Attacks. No criterion is given to separate a jailbreak that uses optimization from an optimized attack, so the assignment is arbitrary rather than systematic. Please add formal category definitions and inclusion rules, or explicitly treat these as orthogonal dimensions.
  2. [§3.1.3 and §3.3] Two further examples show that the taxonomy is not based on a single consistent criterion. Section 3.1.3 admits that prompt leakage "can be considered a form of prompt injection" but tables it as a separate category; Section 3.3 says attacks on LLM applications "are susceptible to the same types of attacks that target the LLM models themselves," so "attacks on application of LLMs" is a target surface rather than a strategy. The survey therefore mixes attack vector, goal, and target layers without defining their relationship. This needs to be resolved before the "systematic" classification in §6 can be accepted.
  3. [§1, Tables 1–2] The survey lacks any stated methodology. There is no description of search databases, date range, inclusion and exclusion criteria, screening process, or coverage statistics, so the completeness and representativeness of Tables 1 and 2 are not established. This is problematic for the contribution claims in §1 and the open-problem conclusions in §5, which depend on the selected literature being representative. Additionally, Tables 1 and 2 contain many blank cells (e.g., Table 1 rows for GPTFuzzer, prompt injection entries, and the application-attack row; Table 2 rows for the watermark defenses), so even the presented evidence is incomplete. Please add a methodology subsection and complete the tables, or soften the systematicity claims accordingly.
  4. [§3 introduction] The opening of Section 3 states that the section "presents a classification of three main categories of attack techniques," but the section and Table 1 actually enumerate four categories (adversarial prompt attack, optimized attacks, model theft, and attacks on application of LLMs). This internal inconsistency, together with the overlapping definitions above, weakens the paper's core organizational claim.
minor comments (7)
  1. [§3.1.1] The displayed equations for ABJ and DeepInception use corrupted Unicode tokens (e.g., sequences beginning with "/u1D43F") and are not readable as mathematical notation; please replace them with properly typeset formulas.
  2. [§3.1] The sentence "most adversarial attacks are based on the strategic manipulation of prompts. ." contains a stray period before the following sentence.
  3. [Table 1] The category label "Optimize attacks" should be "Optimized attacks" for consistency with the text.
  4. [§3.1.1] The heading "GPTFuzzer::" contains a double colon; this appears to be a typo.
  5. [§4.2.2] The heading "Detect-Based Defense" should be "Detection-Based Defense" for consistency with the rest of the paper.
  6. [§1] The phrase "LLM ontology and its applications" is never defined; "ontology" is not introduced or explained anywhere in the manuscript.
  7. [§5] The future-directions discussion is generic and not tied to the specific gaps visible in Tables 1 and 2; adding pointers to the identified missing categories would strengthen the section.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey organizes prior work and does not derive predictions from its own inputs.

full rationale

This is a literature survey rather than a derivation paper. Its central claims are that existing LLM attack and defense research can be organized into the categories shown in Fig. 1 and Tables 1–2, and that certain open problems follow from the surveyed literature. Neither claim is obtained by fitting a parameter to data and then reporting the same quantity as a prediction, nor by importing a conclusion from the authors' own prior results. The taxonomy is asserted from the selected prior work, and the only premises are the surveyed papers themselves; the paper does not compute a quantity and then report that same quantity as a derived result. The reference list contains no load-bearing self-citations invoked to license an ansatz or a uniqueness theorem. Potential weaknesses of the survey—absence of explicit inclusion criteria, overlapping category boundaries such as optimized attacks versus jailbreaks, or representativeness of the selected papers—are coverage and rigor concerns, not circularity. As the paper is self-contained as a summary and makes no independent benchmark claim, no circular step can be exhibited.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The survey introduces no fitted parameters and no new entities. Its conclusions rest on unverified assumptions about literature coverage and the exhaustiveness of its taxonomy, both treated as domain assumptions here.

assumptions (2)
  • domain assumption The literature selected for the survey is representative and complete enough to support the taxonomy and the claimed challenges (Sections 1 and 3).
    The survey presents no search strategy, inclusion criteria, or count of screened papers, so coverage is an unverified premise for its systematic claim.
  • domain assumption The categories used (adversarial prompt attack, optimized attacks, application-level attacks, model theft; prevention and detection defenses) are mutually exclusive and exhaustive.
    The paper states the classification without proving no overlap; e.g., 'optimized attacks' can combine with jailbreak or prompt injection (Section 3.2), so categories are not cleanly disjoint.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attack and defense techniques in large language models: A survey and new perspectives." pith.science (2026). https://pith.science/paper/QHBBFT4Y

@misc{pith2026250500976,
  author       = {Pith},
  title        = {Pith review of: Attack and defense techniques in large language models: A survey and new perspectives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QHBBFT4Y}},
  note         = {Machine review of arXiv:2505.00976}
}
read the original abstract

Large Language Models (LLMs) have become central to numerous natural language processing tasks, but their vulnerabilities present significant security and ethical challenges. This systematic survey explores the evolving landscape of attack and defense techniques in LLMs. We classify attacks into adversarial prompt attack, optimized attacks, model theft, as well as attacks on application of LLMs, detailing their mechanisms and implications. Consequently, we analyze defense strategies, including prevention-based and detection-based defense methods. Although advances have been made, challenges remain to adapt to the dynamic threat landscape, balance usability with robustness, and address resource constraints in defense implementation. We highlight open problems, including the need for adaptive scalable defenses, explainable security techniques, and standardized evaluation frameworks. This survey provides actionable insights and directions for developing secure and resilient LLMs, emphasizing the importance of interdisciplinary collaboration and ethical considerations to mitigate risks in real-world applications.

Figures

Figures reproduced from arXiv: 2505.00976 by the authors.

Figure 1
Figure 1. Taxonomy of attack and defense techniques related to LLMs in our survey However, despite ongoing advancements in research, the field of LLM security studies still faces some challenges. Some articles argue that the current research in certain direc￾tions is relatively scattered and not systematic. Additionally, some scholars point out that with the continuous exploration of LLM attack methods and the discovery of se… view at source ↗
Figure 2
Figure 2. The overall workflow of an LLM program tion 2, and elaborate on the main attack methods currently observed in the LLM ontology and its application in Sec￾tion 3. We summarize the existing defense mechanisms in Section 4, and provide an outlook on the future development of LLM in Section 5. Finally, Section 6 concludes this sur￾vey. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The objective of ABJ attack [34] is achieved by generating customized data based on malicious inputs and then analyzing the generated data by instructing the target model. AutoDAN [36] utilizes internal evaluation and genetic algorithms to determine the optimal template for completing the entire attack process. to obscure the attacker’s intentions, thereby evading detec￾tion by the model. Through a gradual process, … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The general workflow of SPML[52] involves compiling the input prompts using this method and then checking the compiled results based on specific methods to ensure the security of the input prompts. = 1 | | | | ∑ (,ℎ)∈ Attn,ℎ(), (4) where represents the set of important…
Figure 5
Figure 5. Figure 5: Watermark Formation: The imitation model is embedded with a similar API to that of the victim by incorporating the victim’s API into the output of the model. Watermark Detection: A specific algorithm is used to detect watermark signals from the imitation model. In the …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinations

    cs.AI 2025-06 conditional novelty 6.0 of 10

    SSP adds a learned, sample-specific noise prompt to an LLM input and scores hallucination by the cosine shift in intermediate representations, outperforming output-confidence baselines on QA benchmarks.

  2. On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models

    cs.CR 2026-08 conditional novelty 5.0 of 10

    A PRISMA-based survey of 85 papers shows agentic LLM security research is attack-heavy and perception-focused, leaving action-layer and code-execution risks understudied.

Reference graph

Works this paper leans on

92 extracted references · 45 canonical work pages · cited by 2 Pith papers

  1. [36]

    Privacy in large language models: Attacks, defen ses and future directions

    Li, H., Chen, Y., Luo, J., Kang, Y., Zhang, X., Hu, Q., Cha n, C., Song, Y., 2023a. Privacy in large language models: Attacks, defen ses and future directions. arXiv preprint arXiv:2310.10383

    arXiv

  2. [73]

    Gptfuzzer: Red teaming large lan- guage models with auto-generated jailbreak prompts

    Yu, J., Lin, X., Xing, X., 2023a. Gptfuzzer: Red teaming large lan- guage models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253

    arXiv

  3. [24]

    Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROA

    Jawad, H., BRUNEL, N.J.B., 2024. Qroa: A black-box query-response optimization attack on llms. arXiv preprin t arXiv:2406.02044

    work page Pith review arXiv 2024

  4. [54]

    Shi, J., Yuan, Z., Liu, Y., Huang, Y., Zhou, P., Sun, L., G ong, N.Z.,

    work page

  5. [1]

    , Wu, C.S.,

    Agarwal, D., Fabbri, A.R., Laban, P., Joty, S., Xiong, C. , Wu, C.S.,

  6. [2]

    Detecting language model a ttacks with perplexity

    Alon, G., Kamfonas, M., 2023. Detecting language model a ttacks with perplexity. arXiv preprint arXiv:2308.14132

    arXiv 2023

  7. [3]

    Model leeching: An extraction attack targeting llms

    Birch, L., Hackett, W., Trawicki, S., Suri, N., Garragha n, P., 2023. Model leeching: An extraction attack targeting llms. arXiv preprint arXiv:2309.10544

    arXiv 2023

  8. [4]

    Evaluating the su scepti- bility of pre-trained language models via handcrafted adve rsarial ex- amples

    Branch, H.J., Cefalu, J.R., McHugh, J., Hujer, L., Bahl, A., Iglesias, D.d.C., Heichman, R., Darwishi, R., 2022. Evaluating the su scepti- bility of pre-trained language models via handcrafted adve rsarial ex- amples. arXiv preprint arXiv:2209.02128

    arXiv 2022

Show all 92 references
  1. [5]

    Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Her bert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al.,

  2. [6]

    A survey on evaluatio n of large language models

    Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Ch en, H., Yi, X., Wang, C., Wang, Y., et al., 2024. A survey on evaluatio n of large language models. ACM Transactions on Intelligent Systems and Technology 15, 1–45

    2024

  3. [7]

    Jailbreaking black box large language models in tw enty queries

    Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G .J., Wong, E., 2023. Jailbreaking black box large language models in tw enty queries. arXiv preprint arXiv:2310.08419

    2023 arXiv

  4. [8]

    Stru q: Defend- ing against prompt injection with structured queries

    Chen, S., Piet, J., Sitawarin, C., Wagner, D., 2024. Stru q: Defend- ing against prompt injection with structured queries. arXi v preprint arXiv:2402.06363

    2024 arXiv

  5. [9]

    Denoising adversari al autoen- coders

    Creswell, A., Bharath, A.A., 2018. Denoising adversari al autoen- coders. IEEE transactions on neural networks and learning s ystems 30, 968–984

    2018

  6. [10]

    Risk taxonomy, mitigation, and assess- ment benchmarks of large language model systems

    Cui, T., Wang, Y., Fu, C., Xiao, Y., Li, S., Deng, X., Liu, Y., Zhang, Q., Qiu, Z., Li, P., et al., 2024. Risk taxonomy, mitigation, and assess- ment benchmarks of large language model systems. arXiv prep rint arXiv:2401.05778

    2024 arXiv

  7. [11]

    Security and priva cy chal- lenges of large language models: A survey

    Das, B.C., Amini, M.H., Wu, Y., 2025. Security and priva cy chal- lenges of large language models: A survey. ACM Computing Surveys 57, 1–39

    2025

  8. [12]

    At- tack prompt generation for red teaming and defending large l anguage models

    Deng, B., Wang, W., Feng, F., Deng, Y., Wang, Q., He, X., 2 023. At- tack prompt generation for red teaming and defending large l anguage models. arXiv preprint arXiv:2310.12505

    arXiv

  9. [13]

    Masterkey: Automated jailbreaking of large language model chatbots, in: Proc

    Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wan g, H., Zhang, T., Liu, Y., 2024. Masterkey: Automated jailbreaking of large language model chatbots, in: Proc. ISOC NDSS

    2024

  10. [14]

    A comprehensi ve sur- vey of attack techniques, implementation, and mitigation strategies in large language models, in: International Conference on Ubi quitous Security, Springer

    Esmradi, A., Yip, D.W., Chan, C.F., 2023. A comprehensi ve sur- vey of attack techniques, implementation, and mitigation strategies in large language models, in: International Conference on Ubi quitous Security, Springer. pp. 76–95

    2023

  11. [15]

    Recent advance s in robust optimization: An overview

    Gabrel, V ., Murat, C., Thiele, A., 2014. Recent advance s in robust optimization: An overview. European journal of operationa l research 235, 471–483. Y. Liao et al.: Preprint submitted to Elsevier Page 17 of 19 Attack and defense techniques in large language models: A su rve...

    2014

  12. [16]

    Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Ho lz, T., Fritz, M., 2023. Not what you’ve signed up for: Compromising real-w orld llm-integrated applications with indirect prompt injecti on, in: Pro- ceedings of the 16th ACM Workshop on Artificial Intelligence and Securi...

    2023

  13. [17]

    Gu, S., Yang, L., Du, Y., Chen, G., Walter, F., Wang, J., K noll, A.,

  14. [18]

    The ethics of c hatgpt in medicine and healthcare: a systematic review on large language mod- els (llms)

    Haltaufderheide, J., Ranisch, R., 2024. The ethics of c hatgpt in medicine and healthcare: a systematic review on large language mod- els (llms). NPJ Digital Medicine 7, 183

    2024

  15. [19]

    Cater: Intellectual property protection on text generation apis via conditional watermarks

    He, X., Xu, Q., Zeng, Y., Lyu, L., Wu, F., Li, J., Jia, R., 2 022. Cater: Intellectual property protection on text generation apis via conditional watermarks. Advances in Neural Information Processing Systems 35, 5431–5445

  16. [20]

    IEEE Transactions on Pattern Analysis and Ma chine Intelligence

    A review of safe reinforcement learning: Methods, theories and applications. IEEE Transactions on Pattern Analysis and Ma chine Intelligence

  17. [21]

    Are large pre- trained lan- guage models leaking your personal information? arXiv prep rint arXiv:2205.12628

    Huang, J., Shao, H., Chang, K.C.C., 2022. Are large pre- trained lan- guage models leaking your personal information? arXiv prep rint arXiv:2205.12628

    2022 arXiv

  18. [22]

    Attention tracker: Detecting prompt injectio n attacks in llms

    Hung, K.H., Ko, C.Y., Rawat, A., Chung, I., Hsu, W.H., Ch en, P.Y., et al., 2024. Attention tracker: Detecting prompt injectio n attacks in llms. arXiv preprint arXiv:2411.00348

    2024 arXiv

  19. [23]

    The use and misuse of pre-trained generative large language models in reli- ability engineering, in: 2024 Annual Reliability and Maint ainability Symposium (RAMS), IEEE

    Hu, Y., Goktas, Y., Yellamati, D.D., De Tassigny, C., 20 24. The use and misuse of pre-trained generative large language models in reli- ability engineering, in: 2024 Annual Reliability and Maint ainability Symposium (RAMS), IEEE. pp. 1–7

    2024

  20. [25]

    Feature selection and d imension- ality reduction: An extensive comparison in hand gesture cl assifica- tion by semg in eight channels armband approach

    Junior, J.J.A.M., Freitas, M.L., Siqueira, H.V ., Lazz aretti, A.E., Pi- chorim, S.F., Stevan Jr, S.L., 2020. Feature selection and d imension- ality reduction: An extensive comparison in hand gesture cl assifica- tion by semg in eight channels armband approach. Biomedical Sign...

    2020

  21. [26]

    Jain, N., Schwarzschild, A., Wen, Y., Somepalli, G., Ki rchenbauer, J., Chiang, P.y., Goldblum, M., Saha, A., Geiping, J., Golds tein, T.,

  22. [27]

    Common metadata framework: Int e- grated framework for trustworthy ai pipelines

    Koomthanam, A.J., Serebryakov, S., Tripathy, A., Naya k, G., Foltin, M., Bhattacharya, S., 2024. Common metadata framework: Int e- grated framework for trustworthy ai pipelines. IEEE Intern et Com- puting

    2024

  23. [28]

    Thieves on sesame street! model extraction of bert-based ap is

    Krishna, K., Tomar, G.S., Parikh, A.P., Papernot, N., I yyer, M., 2019. Thieves on sesame street! model extraction of bert-based ap is. arXiv preprint arXiv:1910.12366

    2019 arXiv

  24. [29]

    A syntactic analysis of the sentence structure in motivational quotes u sing tree diagram for english learning

    Kristianingsih, D.A., Ma’mun, A.H., Ahmad, Z., 2023. A syntactic analysis of the sentence structure in motivational quotes u sing tree diagram for english learning. Edunesia: Jurnal Ilmiah Pend idikan 4, 1447–1458

    2023

  25. [30]

    Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., Hashimoto, T.,

  26. [31]

    Exploiting programmatic behavior of llms: Dual-use t hrough standard security attacks, in: 2024 IEEE Security and Priva cy Work- shops (SPW), IEEE. pp. 132–143

    2024

  27. [32]

    Li, J., Liu, Y., Liu, C., Shi, L., Ren, X., Zheng, Y., Liu, Y., Xue, Y.,

  28. [33]

    Deep- inception: Hypnotize large language model to be jailbreake r

    Li, X., Zhou, Z., Zhu, J., Yao, J., Liu, T., Han, B., 2023b . Deep- inception: Hypnotize large language model to be jailbreake r. arXiv preprint arXiv:2311.03191

    arXiv

  29. [34]

    Figure it out: Analyzing-based jailbreak attack on large language models

    Lin, S., Li, R., Wang, X., Lin, C., Xing, W., Han, M., 2024 . Figure it out: Analyzing-based jailbreak attack on large language models. arXiv preprint arXiv:2407.16205

    2024 arXiv

  30. [35]

    Vocabulary attack to hij ack large lan- guage model applications

    Levi, P., Neumann, C.P., 2024. Vocabulary attack to hij ack large lan- guage model applications. arXiv preprint arXiv:2404.0263 7

    2024

  31. [37]

    Au tomatic and universal prompt injection attacks against large language models

    Liu, X., Yu, Z., Zhang, Y., Zhang, N., Xiao, C., 2024a. Au tomatic and universal prompt injection attacks against large language models. arXiv preprint arXiv:2403.04957

    arXiv

  32. [38]

    arXiv preprint arXiv:2401.16765

    A cross-language investigation into jailbreak attac ks in large language models. arXiv preprint arXiv:2401.16765

    arXiv

  33. [39]

    Jailbreaking chatgpt vi a prompt engineering: An empirical study

    Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., Zh ao, L., Zhang, T., Wang, K., Liu, Y., 2023c. Jailbreaking chatgpt vi a prompt engineering: An empirical study. arXiv preprint arXiv:230 5.13860

  34. [40]

    Summary of chatgpt-related re search and perspective towards the future of large language models

    Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He , H., Li, A., He, M., Liu, Z., et al., 2023d. Summary of chatgpt-related re search and perspective towards the future of large language models . Meta- Radiology , 100017

  35. [41]

    Liu, W., Gu, C., O’Neill, M., Qu, G., Montuschi, P., Lomb ardi, F.,

  36. [42]

    Dee p learning- based anomaly detection in cyber-physical systems: Progre ss and op- portunities

    Luo, Y., Xiao, Y., Cheng, L., Peng, G., Yao, D., 2021. Dee p learning- based anomaly detection in cyber-physical systems: Progre ss and op- portunities. ACM Computing Surveys (CSUR) 54, 1–36

    2021

  37. [43]

    Autodan: Gene rating stealthy jailbreak prompts on aligned large language model s

    Liu, X., Xu, N., Chen, M., Xiao, C., 2023a. Autodan: Gene rating stealthy jailbreak prompts on aligned large language model s. arXiv preprint arXiv:2310.04451

    arXiv

  38. [44]

    Robust ac- tive learning (roal): Countering dynamic adversaries in ac tive learn- ing with elastic weight consolidation

    Maulana Fajri, R., Pei, Y., Yin, L., Pechenizkiy, M., 20 24. Robust ac- tive learning (roal): Countering dynamic adversaries in ac tive learn- ing with elastic weight consolidation. arXiv e-prints , arX iv–2408

  39. [45]

    Prompt injection attack against llm-integrated applications

    Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zh ang, T., Liu, Y., Wang, H., Zheng, Y., et al., 2023b. Prompt injection attack against llm-integrated applications. arXiv preprint arXi v:2306.05499

    arXiv

  40. [46]

    Pasupuleti, R., Vadapalli, R., Mader, C., 2023. Cyber s ecurity issues and challenges related to generative ai and chatgpt, in: 202 3 Tenth International Conference on Social Networks Analysis, Man agement and Security (SNAMS), IEEE. pp. 1–5

    2023

  41. [47]

    , Xu, T., Sun, G., Xie, X., 2023

    Peng, W., Yi, J., Wu, F., Wu, S., Zhu, B., Lyu, L., Jiao, B. , Xu, T., Sun, G., Xie, X., 2023. Are you copying my model? protecting t he copyright of large language models for eaas via backdoor wat ermark. arXiv preprint arXiv:2305.10036

    2023 arXiv

  42. [48]

    Fo rmalizing and benchmarking prompt injection attacks and defenses, in : 33rd USENIX Security Symposium (USENIX Security 24), pp

    Liu, Y., Jia, Y., Geng, R., Jia, J., Gong, N.Z., 2024b. Fo rmalizing and benchmarking prompt injection attacks and defenses, in : 33rd USENIX Security Symposium (USENIX Security 24), pp. 1831– 1847

  43. [49]

    Sm oothllm: Defending large language models against jailbreaking atta cks

    Robey, A., Wong, E., Hassani, H., Pappas, G.J., 2023. Sm oothllm: Defending large language models against jailbreaking atta cks. arXiv preprint arXiv:2310.03684

    2023 arXiv

  44. [50]

    Maatouk, A., Piovesan, N., Ayed, F., De Domenico, A., De bbah, M.,

  45. [51]

    IEEE Communications Magazine

    Large language models for telecom: Forthcoming impac t on the industry. IEEE Communications Magazine

  46. [52]

    Spml: A dsl for defending language models against prompt attacks

    Sharma, R.K., Gupta, V ., Grossman, D., 2024. Spml: A dsl for defending language models against prompt attacks. arXiv pr eprint arXiv:2402.11755

    2024 arXiv

  47. [53]

    Adaptive and ro- bust watermark against model extraction attack

    Pang, K., Qi, T., Wu, C., Bai, M., 2024. Adaptive and ro- bust watermark against model extraction attack. arXiv prep rint arXiv:2405.02365

    2024 arXiv

  48. [55]

    Signed-prompt: A new approach to prevent prompt Y

    Suo, X., 2024. Signed-prompt: A new approach to prevent prompt Y. Liao et al.: Preprint submitted to Elsevier Page 18 of 19 Attack and defense techniques in large language models: A su rvey and new perspectives injection attacks against llm-integrated applications, i n: AIP Co...

    2024

  49. [56]

    Ignore previous prompt: A ttack tech- niques for language models

    Perez, F., Ribeiro, I., 2022. Ignore previous prompt: A ttack tech- niques for language models. arXiv preprint arXiv:2211.095 27

    2022

  50. [57]

    Universal adversarial triggers for attacking and analyzin g nlp

    Wallace, E., Feng, S., Kandpal, N., Gardner, M., Singh, S., 2019. Universal adversarial triggers for attacking and analyzin g nlp. arXiv preprint arXiv:1908.07125

    2019 arXiv

  51. [58]

    B., 2024

    Rossi, S., Michel, A.M., Mukkamala, R.R., Thatcher, J. B., 2024. An early categorization of prompt injection attacks on large l anguage models. arXiv preprint arXiv:2402.00898

    2024 arXiv

  52. [59]

    Shan, S., Ding, W., Passananti, J., Wu, S., Zheng, H., Zh ao, B.Y.,

  53. [60]

    Nightshade: Prompt-specific poisoning attacks on tex t-to- image generative models, in: 2024 IEEE Symposium on Securit y and Privacy (SP), IEEE. pp. 807–825

    2024

  54. [61]

    Selfdefend: Llms can defend themselves against jailbreaking in a practical manner

    Wang, X., Wu, D., Ji, Z., Li, Z., Ma, P., Wang, S., Li, Y., L iu, Y., Liu, N., Rahmel, J., 2024. Selfdefend: Llms can defend themselves against jailbreaking in a practical manner. arXiv preprint arXiv:2 406.05498

    2024

  55. [62]

    Survey of vulnerabilities in large lang uage mod- els revealed by adversarial attacks

    Shayegani, E., Mamun, M.A.A., Fu, Y., Zaree, P., Dong, Y ., Abu- Ghazaleh, N., 2023. Survey of vulnerabilities in large lang uage mod- els revealed by adversarial attacks. arXiv preprint arXiv: 2310.10844

    2023 arXiv

  56. [63]

    Jailbro ken: How does llm safety training fail? Advances in Neural Information Pr ocessing Systems 36, 80079–80110

    Wei, A., Haghtalab, N., Steinhardt, J., 2023a. Jailbro ken: How does llm safety training fail? Advances in Neural Information Pr ocessing Systems 36, 80079–80110

  57. [64]

    Optimization-based prompt injection attack to llm-a s-a-judge, in: Proceedings of the 2024 on ACM SIGSAC Conference on Com- puter and Communications Security, pp. 660–674

    2024

  58. [65]

    Jailbrea king gpt- 4v via self-adversarial attacks with system prompts

    Wu, Y., Li, X., Liu, Y., Zhou, P., Sun, L., 2023. Jailbrea king gpt- 4v via self-adversarial attacks with system prompts. arXiv preprint arXiv:2311.09127

    2023 arXiv

  59. [66]

    Meta-learning appr oaches for learning-to-learn in deep learning: A survey

    Tian, Y., Zhao, X., Huang, W., 2022. Meta-learning appr oaches for learning-to-learn in deep learning: A survey. Neurocomput ing 494, 203–223

    2022

  60. [67]

    Gradsafe: Dete cting unsafe prompts for llms via safety-critical gradient analysis

    Xie, Y., Fang, M., Pi, R., Gong, N., 2024. Gradsafe: Dete cting unsafe prompts for llms via safety-critical gradient analysis. ar Xiv preprint arXiv:2402.13494

    2024 arXiv

  61. [68]

    Imitation attac ks and de- fenses for black-box machine translation systems

    Wallace, E., Stern, M., Song, D., 2020. Imitation attac ks and de- fenses for black-box machine translation systems. arXiv pr eprint arXiv:2004.15015

    2020 arXiv

  62. [69]

    Adversarial demonstration attacks on large language models

    Wang, J., Liu, Z., Park, K.H., Jiang, Z., Zheng, Z., Wu, Z ., Chen, M., Xiao, C., 2023. Adversarial demonstration attacks on large language models. arXiv preprint arXiv:2305.14950

    2023 arXiv

  63. [70]

    A multiobje ctive learn- ing and ensembling approach to high-performance speech enh ance- ment with compact neural network architectures

    Wang, Q., Du, J., Dai, L.R., Lee, C.H., 2018. A multiobje ctive learn- ing and ensembling approach to high-performance speech enh ance- ment with compact neural network architectures. IEEE/ACM T rans- actions on Audio, Speech, and Language Processing 26, 1185– 1197

    2018

  64. [71]

    , Wang, Z., 2024b

    Yang, Y., Zhang, X., Jiang, Y., Chen, X., Wang, H., Ji, S. , Wang, Z., 2024b. Prsa: Prompt reverse stealing attacks against large language models. arXiv preprint arXiv:2402.19200

    arXiv

  65. [72]

    Wang, Z., Dong, X., Xue, H., Zhang, Z., Chiu, W., Wei, T., Ren, K.,

  66. [74]

    Assessi ng prompt injection risks in 200+ custom gpts

    Yu, J., Wu, Y., Shu, D., Jin, M., Xing, X., 2023b. Assessi ng prompt injection risks in 200+ custom gpts. arXiv preprint arXiv:2311.11538

    arXiv

  67. [75]

    Jailbreak and guard a ligned lan- guage models with only few in-context demonstrations

    Wei, Z., Wang, Y., Wang, Y., 2023b. Jailbreak and guard a ligned lan- guage models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387

    arXiv

  68. [76]

    Gener ate adversar- ial examples by adaptive moment iterative fast gradient sig n method

    Zhang, J., Qian, W., Nie, R., Cao, J., Xu, D., 2023. Gener ate adversar- ial examples by adaptive moment iterative fast gradient sig n method. Applied Intelligence 53, 1101–1114

    2023

  69. [77]

    Tastle: Dis tract large language models for automatic jailbreak attack

    Xiao, Z., Yang, Y., Chen, G., Chen, Y., 2024. Tastle: Dis tract large language models for automatic jailbreak attack. arXi v preprint arXiv:2403.08424

    2024 arXiv

  70. [78]

    Anomaly detection wit h robust deep autoencoders, in: Proceedings of the 23rd ACM SIGKDD in - ternational conference on knowledge discovery and data min ing, pp

    Zhou, C., Paffenroth, R.C., 2017. Anomaly detection wit h robust deep autoencoders, in: Proceedings of the 23rd ACM SIGKDD in - ternational conference on knowledge discovery and data min ing, pp. 665–674

    2017

  71. [79]

    Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie , X., Wu, F.,

  72. [80]

    Nature Machine Intelligence 5, 1486–1496

    Defending chatgpt against jailbreak attack via self- reminders. Nature Machine Intelligence 5, 1486–1496

  73. [81]

    Practical and ethical cha llenges of large language models in education: A systematic scoping review

    Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y., Gašević, D., 2024. Practical and ethical cha llenges of large language models in education: A systematic scoping review. British Journal of Educational Technology 55, 90–112

    2024

  74. [82]

    Harnessing the power of llms in practi ce: A survey on chatgpt and beyond

    Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., Hu, X., 2024a. Harnessing the power of llms in practi ce: A survey on chatgpt and beyond. ACM Transactions on Knowledg e Discovery from Data 18, 1–32

  75. [84]

    Poisonprompt: Backdoor attack on prompt-based large language models, in: ICASSP 2024-2024 I EEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE

    Yao, H., Lou, J., Qin, Z., 2024. Poisonprompt: Backdoor attack on prompt-based large language models, in: ICASSP 2024-2024 I EEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 7745–7749

    2024

  76. [87]

    Autod efense: Multi-agent llm defense against jailbreak attacks

    Zeng, Y., Wu, Y., Zhang, X., Wang, H., Wu, Q., 2024. Autod efense: Multi-agent llm defense against jailbreak attacks. arXiv p reprint arXiv:2403.04783

    2024 arXiv

  77. [89]

    Protecting language generation models via invisible watermarking, in: International Conf erence on Machine Learning, PMLR

    Zhao, X., Wang, Y.X., Li, L., 2023. Protecting language generation models via invisible watermarking, in: International Conf erence on Machine Learning, PMLR. pp. 42187–42199

    2023

  78. [91]

    Risks of discrimination violence and un lawful actions in llm-driven robots

    Zhou, R., 2024. Risks of discrimination violence and un lawful actions in llm-driven robots. Computer Life 12, 53–56

    2024

  79. [92]

    Universal and transferable adversarial attacks o n aligned language models

    Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J.Z., F redrikson, M., 2023. Universal and transferable adversarial attacks o n aligned language models. arXiv preprint arXiv:2307.15043 . Y. Liao et al.: Preprint submitted to Elsevier Page 19 of 19

    2023 arXiv

  80. [2020]

    Proceedin gs of the IEEE 108, 2214–2231

    Security in approximate computing and approximate co mput- ing for security: Challenges and opportunities. Proceedin gs of the IEEE 108, 2214–2231

  81. [2021]

    2633– 2650

    Extracting training data from large language models, in: 30th USENIX Security Symposium (USENIX Security 21), pp. 2633– 2650

  82. [2022]

    10379– 10388

    Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models, in: Proceedings of the IEEE/CVF co nfer- ence on computer vision and pattern recognition, pp. 10379– 10388

  83. [2023]

    arXiv preprint arXiv:2309.00614

    Baseline defenses for adversarial attacks against al igned lan- guage models. arXiv preprint arXiv:2309.00614

    arXiv

  84. [2024]

    arXiv preprint arXiv:240 4.16251

    Investigating the prompt leakage effect and black-box defenses for multi-turn llm interactions. arXiv preprint arXiv:240 4.16251

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.