Pith. sign in

REVIEW 4 major objections 6 minor 88 references

A Survey on Data Security in Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey argues that the data-security risks of large language models can be organized into five threat families — data poisoning, prompt injection, hallucination, prompt leakage, and bias — and that current defenses fall into three…

desk verdict A useful survey skeleton whose core tables and one formal definition have multiple demonstrable errors; correctable, but not citable in current form. read the letter →

arxiv 2508.02312 v1 pith:DUPFDLJA submitted 2025-08-04 cs.CR cs.AI

Kang Chen , Xiuze Zhou , Yuanguo Lin , Jinhe Su , Yuanhui Yu , Li Shen , Fan Lin This is my paper
classification cs.CRcs.AI
keywords largelanguagemodelsdatasecuritypoisoningpromptinjectionhallucinationleakagebiasdefensestrategies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a survey that tries to give researchers, practitioners, and policymakers a structured map of how data can compromise large language models. It claims that five threat families — data poisoning, prompt injection, hallucination, prompt leakage, and bias — cover the main data-centric risks, and that defenses cluster into adversarial training, reinforcement learning from human feedback, and data augmentation. It also categorizes evaluation datasets by domain and intended use, and it sketches future directions such as secure model updates, explainability-driven defenses, and governance frameworks. A sympathetic reader would take the paper as a practical orienting guide: a taxonomy to name a threat, a menu of defenses to counter it, and a dataset directory to test it.

What carries the argument

The organizing machinery is a lifecycle-stage taxonomy: the paper assigns each data-security risk to the phase of the LLM pipeline where it enters — collection, preprocessing, training, fine-tuning, or deployment — and maps defense strategies to those phases. Two formalized examples carry the technical detail: a virtual-prompt-injection definition that characterizes a backdoored model as responding to trigger inputs as if a virtual prompt had been appended, and a prompt-leakage reconstruction equation in which adversarial queries and a post-processing function combine to approximate the hidden system prompt. Together, these give the survey's taxonomy a concrete, checkable form rather than a purely descriptive list.

What would settle it

Check Table 3 against the cited sources: Jigsaw is a toxicity and online-comment dataset, while ECHR contains European Court of Human Rights legal cases, so a reader who finds these descriptions swapped, or who finds Burns et al. [7] filed under RLHF in Table 2, has concrete evidence that the survey's dataset and defense summaries cannot be trusted as a guide.

Watch

Extended reading notes

Core claim

The central claim is that LLM data security is best understood by locating threats along the model lifecycle, from training-data collection through preprocessing, fine-tuning, and user interaction. The paper's taxonomy treats data poisoning and prompt injection as deliberate manipulations, hallucination as a failure mode that can be triggered or amplified by bad data, prompt leakage as an information-exposure risk, and bias as a harm inherited from skewed corpora. For defenses, it groups methods into adversarial training (hardening the model against manipulated inputs), RLHF (aligning behavior with human preferences), and data augmentation (rebalancing or diversifying training data), and it evaluates each family's strengths and limitations. The paper also compiles a table of datasets used to attack or defend LLMs, organized by domain, and concludes that the field needs standardized evaluation metrics, traceable data provenance, and continual security verification.

Load-bearing premise

The survey's usefulness depends on its secondary summaries of the cited literature being faithful and representative, yet no systematic search or inclusion criteria are reported, and several specific entries are inaccurate.

Editorial extensions

If this is right

  • A practitioner facing a suspect LLM output can use the taxonomy to hypothesize which lifecycle stage was compromised — training data, fine-tuning data, or live prompt input.
  • Choosing a defense strategy becomes a triage decision: adversarial training for input-level manipulation, RLHF for alignment and hallucination reduction, and data augmentation for bias mitigation.
  • The dataset directory gives a starting point for benchmarking a new defense, with domain-appropriate attack and defense sets for movie, news, social, book, study, and research scenarios.
  • If the survey's framing is adopted, future work on LLM security will likely organize itself around provenance tracking and standardized robustness benchmarks rather than isolated attack reports.
  • The identified research gaps — secure model updates, explainability-driven monitoring, and governance frameworks — define a concrete agenda for the field.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The boundary of the taxonomy is contestable: hallucination is treated here as a data-security risk, but one could argue it is primarily a capability limitation that bad data merely exploits, and the paper does not defend that classification against alternatives.
  • The dataset table's reliability is load-bearing for the survey's practical value; if entries misattribute datasets to domains or swap descriptions, the guidance to researchers would actively mislead, which suggests an immediate verification pass on every table row.
  • The formal prompt-leakage reconstruction framework could be extended into a defense metric: measuring how closely a post-processing function can reconstruct a system prompt would quantify leakage risk across model families.
  • A natural testable extension is applying the same lifecycle taxonomy to multimodal LLMs, where poisoning and injection can enter through image or audio tokens as well as text.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This survey paper proposes to offer a comprehensive overview of data security risks in large language models, covering data poisoning, prompt injection, hallucination, prompt leakage, and bias; reviews defense strategies (adversarial training, RLHF, data augmentation); and provides a categorized catalog of datasets for evaluating robustness and security, together with future research directions. The paper is organized around a risk taxonomy (Section 3), a defense taxonomy (Section 4), a dataset table (Section 5), and a set of forward-looking recommendations (Section 6). Its central claim, stated in the abstract, is that it provides a reliable and comprehensive synthesis of these threats and defenses for researchers and practitioners.

Significance. If the factual content were verified and corrected, this survey would fill a useful niche: it explicitly organizes LLM-specific data-centric threats (poisoning, prompt injection, hallucination, prompt leakage, bias) alongside defenses and evaluation datasets, with tables that could serve as a quick reference. The paper makes no original derivations, so its value rests entirely on the fidelity of its secondary summaries and the accuracy of its citations. The taxonomy itself is reasonable and the coverage of threats is broad. However, the central artifacts—the defense table, the dataset table, and the formal definitions—currently contain demonstrable errors that undermine the reliability of the guidance. The paper does not claim or provide machine-checked proofs or reproducible code; its contribution is a structured literature synthesis, which is exactly where the identified factual errors are most damaging.

major comments (4)
  1. [Table 3, Section 5] Table 3 swaps the descriptions and URLs of Jigsaw and ECHR. The Jigsaw row is described as 'Dataset of cases published by the European Court of Human Rights' with a URL to a counterfactual-data-substitution repository, while the ECHR row is described as 'Dataset by Jigsaw containing online review data' with a URL to a poisoning-defense repository. The ECHR dataset is a legal-case corpus and Jigsaw is a toxic-comment dataset; both the descriptions and the URLs are inconsistent with the actual datasets. Since Table 3 is the primary artifact a reader would use to select a dataset, this error is load-bearing and requires a source-level correction of every row.
  2. [Table 2, Section 4.2] Table 2 lists Burns et al. [7] ('Discovering latent knowledge in language models without supervision') under the RLHF category, and Section 4.2 states 'As highlighted by Perez et al. [7], RLHF-trained models may retain latent unsafe behaviors...'. Reference [7] is Burns et al., not Perez et al., and it is a probing method for extracting latent knowledge, not an RLHF approach or a claim about RLHF-trained models per se. This is not a cosmetic citation slip: it misattributes a substantive claim about RLHF limitations and conflates a probing technique with a training paradigm, thereby misinforming the reader about the defense taxonomy.
  3. [Section 3.2, Eq. (1)] The formal definition of virtual prompt injection is incorrect as written. The equation states M(x) = response to x⊕p if x∈X, and response to x otherwise, where X is defined as the natural language instruction space. Since every input x is in X by definition, the condition is trivially true and the specification reduces to M(x) = response to x⊕p for all inputs, which is not what VPI describes. The condition should be membership in the trigger instruction subset X_t (e.g., x∈X_t), as described in the preceding paragraph. The current formulation garbles the core definition of the threat.
  4. [Section 5, Table 3] Several dataset-to-reference assignments in Section 5 are not supported by the cited papers. For example, 'English Gigaword [69]' is listed as a dataset for the defense paper [69] (Wen et al., 'Is adversarial training really a silver bullet...'), but that paper's own table (also reproduced in this survey's Table 2) reports CIFAR-10, CIFAR-100, and TinyImageNet, not English Gigaword. Similarly, 'AQuA [76]' is attributed to [76], a prompt-injection attack paper whose reported datasets in Table 1 are GSM8K, web-based QA, and SQuAD2.0, not AQuA. These mismatches indicate that the dataset-recommendation layer of the survey is unreliable and needs verification against the primary sources.
minor comments (6)
  1. [Section 3.3] The phrase 'commutable LLMs and a commutable groundtruth function' should likely be 'computable' rather than 'commutable'.
  2. [Section 3.2] The sentence 'Because it is not feasible to list all possible instructions, X_t can be used to define X_t' is circular and appears to be a typo; it should say that X_t is used to define the trigger scenario.
  3. [Table 1] Several entries in Table 1 are garbled, for example 'SuSuper-NaturalInstructions' and the concatenation of dataset names and metrics; the table should be reformatted so that each column is legible.
  4. [Section 4.2] The citation 'Perez et al. [7]' should be corrected to the actual author(s) of reference [7] (Burns et al.), or replaced with the intended source on latent unsafe behaviors; the reference list should be checked for year consistency ([7] is arXiv 2212.03827).
  5. [Section 5] The prose in Section 5 uses bracket references that are inconsistent with the dataset rows; for example, the text lists 'Jigsaw [67], [63], [45]' and 'ECHR [20]' while Table 3 assigns different references to these datasets. Please reconcile the text and table.
  6. [Section 1] The survey does not report a systematic search strategy or inclusion criteria for selecting the surveyed papers; adding a short methodology paragraph would help substantiate the 'comprehensive' claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the survey makes no original derivation, and its citations (including the authors' own prior background references) are not load-bearing for any claimed result.

full rationale

This paper is a literature survey. It proposes no new model, no fitted parameters, no prediction derived from a fitted value, and no formal derivation whose conclusion is presupposed by its inputs. The central claim is that the survey provides a comprehensive overview of LLM data security risks, defenses, and datasets; this claim is supported by secondary summaries of cited prior work, not by a derivation chain internal to the paper. The self-citations that appear in the reference list ([13], [16], [36]) are background references in the introduction and pipeline description and are not load-bearing for the survey's taxonomy or conclusions. Equation (1), which is meant to define virtual prompt injection behavior, is garbled: it writes 'if x in X' where the surrounding text defines a trigger subset X_t, making the condition trivially true. That is a formal correctness defect, not a circularity, because the paper is not using Eq. (1) to predict an independently defined outcome; it is an erroneous formalization of an external attack definition. Similarly, the reader-identified citation and dataset mismatches in Tables 2 and 3, and in Section 4.2, are fidelity/accuracy concerns that undermine the reliability of the survey, but they are not instances of the survey deriving its conclusions from itself or from a fitted input. No step in the paper reduces, by construction or by self-citation, to its own assumptions. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

The paper is a literature survey and introduces no free parameters, axioms, or invented entities. Equations reproduced from cited works, such as Eq. (4) for adversarial training and Eq. (5) for FairFlow, are not new derivations. The main risk lies in the accuracy of secondary summaries rather than in formal assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Survey on Data Security in Large Language Models." pith.science (2026). https://pith.science/paper/DUPFDLJA

@misc{pith2026250802312,
  author       = {Pith},
  title        = {Pith review of: A Survey on Data Security in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DUPFDLJA}},
  note         = {Machine review of arXiv:2508.02312}
}
read the original abstract

Large Language Models (LLMs), now a foundation in advancing natural language processing, power applications such as text generation, machine translation, and conversational systems. Despite their transformative potential, these models inherently rely on massive amounts of training data, often collected from diverse and uncurated sources, which exposes them to serious data security risks. Harmful or malicious data can compromise model behavior, leading to issues such as toxic output, hallucinations, and vulnerabilities to threats such as prompt injection or data poisoning. As LLMs continue to be integrated into critical real-world systems, understanding and addressing these data-centric security risks is imperative to safeguard user trust and system reliability. This survey offers a comprehensive overview of the main data security risks facing LLMs and reviews current defense strategies, including adversarial training, RLHF, and data augmentation. Additionally, we categorize and analyze relevant datasets used for assessing robustness and security across different domains, providing guidance for future research. Finally, we highlight key research directions that focus on secure model updates, explainability-driven defenses, and effective governance frameworks, aiming to promote the safe and responsible development of LLM technology. This work aims to inform researchers, practitioners, and policymakers, driving progress toward data security in LLMs.

Figures

Figures reproduced from arXiv: 2508.02312 by the authors.

Figure 1
Figure 1. Overview of the Survey Structure on LLMs Data Security, beginning with background and LLM vulnerabilities, then addressing data security risks, mitigation techniques, datasets, and concluding with future directions in LLM security and gover￾nance. approaches contribute to secure and stable model outputs by improving model robustness, incorporating human-aligned reinforcement signals, and enhancing dataset diversity … view at source ↗
Figure 2
Figure 2. Data training with an LLM platform. The workflow highlights critical machine learning development phases vulnerable to data security risks: training data collection, input processing, model pre-training, fine-tuning, and deployment. Each stage presents unique threat surfaces requiring specific protection measures. 2. Background 2.1. LLM Architecture Like deep learning-based NLP systems, LLMs follow a data-centric pi… view at source ↗
Figure 3
Figure 3. An overview of the data poisoning scenario. Attackers inject triggers (e.g., "Mars") into training data to create poisoned samples. A model trained on this data produces harmful outputs when triggered. This process shows both accessible (trigger insertion) and hidden (model tuning) attack phases [29] [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: LLM-based application shown in typical usage (top) versus under a prompt injection scenario (bottom). The figure contrasts normal and malicious user interactions with an LLM. A kind user asks neutral questions (e.g., "Should I do a Ph.D?"), receiving typical responses.…
Figure 5
Figure 5. Figure 5: The overall architecture of Mix-Debias. A three-stage framework combines counterfactual augmentation, semantic expansion via PLMs, and mixup-based fine-tuning using 𝜆-weighted sentence embeddings to enhance model robustness [74]. Even with these promising advancements,…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

88 extracted references · 49 canonical work pages

  1. [69]

    Is adversarial training really a silver bullet for mitigating data poi- soning?,in: ProceedingsoftheInternationalConferenceonLearning Representations

    Wen, R., Zhao, Z., Liu, Z., Backes, M., Wang, T., Zhang, Y., 2023. Is adversarial training really a silver bullet for mitigating data poi- soning?,in: ProceedingsoftheInternationalConferenceonLearning Representations. URL:https://openreview.net/forum?id=zKvm1ETDOq

    work page 2023

  2. [7]

    Discovering latent knowledge in language models without supervision

    Burns, C., Ye, H., Klein, D., Steinhardt, J., 2024. Discovering latent knowledge in language models without supervision. URL:https:// arxiv.org/abs/2212.03827, arXiv:2212.03827

    arXiv 2024

  3. [76]

    Goal- guided generative prompt injection attack on large language models

    Zhang, C., Jin, M., Yu, Q., Liu, C., Xue, H., Jin, X., 2024a. Goal- guided generative prompt injection attack on large language models. arXiv preprint arXiv:2404.07234

    arXiv

  4. [1]

    Agarwal, D., Fabbri, A.R., Laban, P., Joty, S., Xiong, C., Wu, C.S.,

  5. [2]

    On the dangers of stochastic parrots: Can language models be too big?, in: Proceedings of the 2021 ACM conference on fairness, ac- countability, and transparency, pp

    Bender, E.M., Gebru, T., McMillan-Major, A., Shmitchell, S., 2021. On the dangers of stochastic parrots: Can language models be too big?, in: Proceedings of the 2021 ACM conference on fairness, ac- countability, and transparency, pp. 610–623

    2021

  6. [3]

    Pythia: A suite for analyzing large language models across training and scaling, in: International Conference on Machine Learning, PMLR

    Biderman, S., Schoelkopf, H., Anthony, Q.G., Bradley, H., O’Brien, K., Hallahan, E., Khan, M.A., Purohit, S., Prashanth, U.S., Raff, E., et al., 2023. Pythia: A suite for analyzing large language models across training and scaling, in: International Conference on Machine Learning, PMLR. pp. 2397–2430

    2023

  7. [4]

    Lan- guage(technology)ispower: Acriticalsurveyof"bias"innlp

    Blodgett, S.L., Barocas, S., Daumé III, H., Wallach, H., 2020. Lan- guage(technology)ispower: Acriticalsurveyof"bias"innlp. arXiv preprint arXiv:2005.14050

    arXiv 2020

  8. [5]

    Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., et al.,

Show all 88 references
  1. [6]

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhari- wal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.,

  2. [8]

    Autohall: Automated halluci- nation dataset generation for large language models

    Cao, Z., Yang, Y., Zhao, H., 2023. Autohall: Automated halluci- nation dataset generation for large language models. arXiv preprint arXiv:2310.00259

    2023

  3. [9]

    Extracting training data from diffusion models, in: 32nd USENIX Security Symposium (USENIX Security 23), pp

    Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., Wallace, E., 2023. Extracting training data from diffusion models, in: 32nd USENIX Security Symposium (USENIX Security 23), pp. 5253–5270

    2023

  4. [10]

    The se- cretsharer: Evaluatingandtestingunintendedmemorizationinneural networks, in: 28th USENIX security symposium (USENIX security 19), pp

    Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., Song, D., 2019. The se- cretsharer: Evaluatingandtestingunintendedmemorizationinneural networks, in: 28th USENIX security symposium (USENIX security 19), pp. 267–284

    2019

  5. [11]

    Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al.,

  6. [12]

    Killing one bird with two stones: Model extraction and attribute inference attacks against bert- based apis

    Chen, C., He, X., Lyu, L., Wu, F., 2021. Killing one bird with two stones: Model extraction and attribute inference attacks against bert- based apis. arXiv preprint arXiv:2105.10909

    2021 arXiv

  7. [13]

    Transformer network for re- maining useful life prediction of lithium-ion batteries

    Chen, D., Hong, W., Zhou, X., 2022. Transformer network for re- maining useful life prediction of lithium-ion batteries. IEEE Access 10, 19621–19628. K. Chen et al.:Preprint submitted to Elsevier Page 13 of 15 A Survey on Data Security in Large Language Models

    2022

  8. [14]

    Deepreinforcementlearningfromhumanpreferences

    Christiano, P.F., Leike, J., Brown, T., Martic, M., Legg, S., Amodei, D.,2017. Deepreinforcementlearningfromhumanpreferences. Ad- vances in neural information processing systems 30

    2017

  9. [15]

    2633– 2650

    Extracting training data from large language models, in: 30th USENIX security symposium (USENIX Security 21), pp. 2633– 2650

  10. [16]

    Ding, Y., Jia, S., Ma, T., Mao, B., Zhou, X., Li, L., Han, D.,

  11. [17]

    Documenting large web- textcorpora: Acasestudyonthecolossalcleancrawledcorpus.arXiv preprint arXiv:2104.08758

    Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groen- eveld, D., Mitchell, M., Gardner, M., 2021. Documenting large web- textcorpora: Acasestudyonthecolossalcleancrawledcorpus.arXiv preprint arXiv:2104.08758

    2021 arXiv

  12. [18]

    Flocks of stochasticparrots: Differentiallyprivatepromptlearningforlargelan- guage models

    Duan, H., Dziedzic, A., Papernot, N., Boenisch, F., 2023. Flocks of stochasticparrots: Differentiallyprivatepromptlearningforlargelan- guage models. Advances in Neural Information Processing Systems 36, 76852–76871

    2023

  13. [19]

    Security and privacy chal- lengesoflargelanguagemodels: Asurvey

    Das, B.C., Amini, M.H., Wu, Y., 2025. Security and privacy chal- lengesoflargelanguagemodels: Asurvey. ACMComputingSurveys 57, 1–39

    2025

  14. [20]

    Biasandfairness in large language models: A survey

    Gallegos, I.O., Rossi, R.A., Barrow, J., Tanjim, M.M., Kim, S., Der- noncourt,F.,Yu,T.,Zhang,R.,Ahmed,N.K.,2024. Biasandfairness in large language models: A survey. Computational Linguistics , 1– 79

    2024

  15. [21]

    What doesn’t kill you makes you robust (er): How to adversarially train against data poisoning

    Geiping, J., Fowl, L., Somepalli, G., Goldblum, M., Moeller, M., Goldstein, T., 2021. What doesn’t kill you makes you robust (er): How to adversarially train against data poisoning. arXiv preprint arXiv:2102.13624

    2021 arXiv

  16. [22]

    A survey of adversarial defenses and robustness in nlp

    Goyal, S., Doddapaneni, S., Khapra, M.M., Ravindran, B., 2023. A survey of adversarial defenses and robustness in nlp. ACM Comput- ing Surveys 55, 1–39

    2023

  17. [23]

    Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., Fritz, M., 2023. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection, in: Pro- ceedings of the 16th ACM Workshop on Artificial Intelligence and Security...

    2023

  18. [24]

    Differentially private continual learning

    Farquhar, S., Gal, Y., 2019. Differentially private continual learning. arXiv preprint arXiv:1902.06497

    2019 arXiv

  19. [25]

    Asurveyonhallucinationin large language models: Principles, taxonomy, challenges, and open questions

    Huang,L.,Yu,W.,Ma,W.,Zhong,W.,Feng,Z.,Wang,H.,Chen,Q., Peng,W.,Feng,X.,Qin,B.,etal.,2025. Asurveyonhallucinationin large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems 43, 1–55

    2025

  20. [26]

    3600–3614

    Hui,B.,Yuan,H.,Gong,N.,Burlina,P.,Cao,Y.,2024.Pleak: Prompt leaking attacks against large language model applications, in: Pro- ceedingsofthe2024onACMSIGSACConferenceonComputerand Communications Security, pp. 3600–3614

    2024

  21. [27]

    Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., Li, B.,

  22. [28]

    On large language models’ hallucination with regard to known facts

    Jiang, C., Qi, B., Hong, X., Fu, D., Cheng, Y., Meng, F., Yu, M., Zhou, B., Zhou, J., 2024a. On large language models’ hallucination with regard to known facts. arXiv preprint arXiv:2403.20009

    arXiv

  23. [29]

    An evaluation of the safety of chatgpt with malicious prompt injection

    Han, J., Guo, M., 2024. An evaluation of the safety of chatgpt with malicious prompt injection. URL:https://www.researchsquare. com/article/rs-4487194/v1, doi: 10.21203/rs.3.rs-4487194/v1, arXiv:rs-4487194

    2024 doi

  24. [30]

    Robust pre-training by adversarial contrastive learning

    Jiang, Z., Chen, T., Chen, T., Wang, Z., 2020. Robust pre-training by adversarial contrastive learning. Advances in neural information processing systems 33, 16199–16210

    2020

  25. [31]

    Provenance documentation to enable explainable and trustworthy ai: A literature review

    Kale, A., Nguyen, T., Harris Jr, F.C., Li, C., Zhang, J., Ma, X., 2023. Provenance documentation to enable explainable and trustworthy ai: A literature review. Data Intelligence 5, 139–162

    2023

  26. [32]

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav), in: International con- ference on machine learning, PMLR

    Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al., 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav), in: International con- ference on machine learning, PMLR. pp. 2668–2677

    2018

  27. [33]

    Cybercrime and privacy threats of large language models

    Kshetri, N., 2023. Cybercrime and privacy threats of large language models. IT Professional 25, 9–13

    2023

  28. [34]

    Weight poisoning attacks on pre-trained models

    Kurita, K., Michel, P., Neubig, G., 2020. Weight poisoning attacks on pre-trained models. arXiv preprint arXiv:2004.06660

    2020 arXiv

  29. [35]

    Turning generative models degenerate: The power of data poisoning attacks

    Jiang, S., Kadhe, S.R., Zhou, Y., Ahmed, F., Cai, L., Baracaldo, N., 2024b. Turning generative models degenerate: The power of data poisoning attacks. arXiv preprint arXiv:2407.12281

    arXiv

  30. [36]

    De- coding knowledge attribution in mixture-of-experts: A framework of basic-refinementcollaborationandefficiencyanalysis

    Li, J., Wang, B., Zhou, X., Jiang, P., Liu, J., Hu, X., 2025. De- coding knowledge attribution in mixture-of-experts: A framework of basic-refinementcollaborationandefficiencyanalysis. arXivpreprint arXiv:2505.24593

    2025 arXiv

  31. [37]

    Sok: Certified robustness for deep neural networks, in: 2023 IEEE symposium on security and privacy (SP), IEEE

    Li, L., Xie, T., Li, B., 2023b. Sok: Certified robustness for deep neural networks, in: 2023 IEEE symposium on security and privacy (SP), IEEE. pp. 1289–1310

    2023

  32. [38]

    Backdoorlearning: Asurvey

    Li,Y.,Jiang,Y.,Li,Z.,Xia,S.T.,2022. Backdoorlearning: Asurvey. IEEEtransactionsonneuralnetworksandlearningsystems35,5–22

    2022

  33. [39]

    Badedit: Backdooring large language models by model editing

    Li, Y., Li, T., Chen, K., Zhang, J., Liu, S., Wang, W., Zhang, T., Liu, Y., 2024. Badedit: Backdooring large language models by model editing. arXiv preprint arXiv:2403.13355

    2024 arXiv

  34. [40]

    Learning without forgetting

    Li, Z., Hoiem, D., 2017. Learning without forgetting. IEEE transac- tions on pattern analysis and machine intelligence 40, 2935–2947

    2017

  35. [41]

    Trustworthy ai: From principles to practices

    Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., Zhou, B., 2023a. Trustworthy ai: From principles to practices. ACM Computing Sur- veys 55, 1–46

  36. [42]

    Prompt injection attack againstllm-integratedapplications

    Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al., 2023. Prompt injection attack againstllm-integratedapplications. arXivpreprintarXiv:2306.05499

    2023 arXiv

  37. [43]

    Backdoor attacks and de- fenses in feature-partitioned collaborative learning

    Liu, Y., Yi, Z., Chen, T., 2020. Backdoor attacks and de- fenses in feature-partitioned collaborative learning. arXiv preprint arXiv:2007.03608

    2020 arXiv

  38. [44]

    Towards deep learning models resistant to adversarial attacks

    Mądry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A., 2017. Towards deep learning models resistant to adversarial attacks. stat 1050

    2017

  39. [45]

    It’s all in the name: Mitigating gender bias with name-based counterfactual data substitution

    Maudslay, R.H., Gonen, H., Cotterell, R., Teufel, S., 2019. It’s all in the name: Mitigating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871

    2019 arXiv

  40. [46]

    McKenzie, I.R., Lyzhov, A., Pieler, M., Parrish, A., Mueller, A., Prabhu, A., McLean, E., Kirtland, A., Ross, A., Liu, A., et al.,

  41. [47]

    Towards understandingandmitigatingsocialbiasesinlanguagemodels,in: In- ternationalConferenceonMachineLearning,PMLR.pp.6565–6576

    Liang,P.P.,Wu,C.,Morency,L.P.,Salakhutdinov,R.,2021. Towards understandingandmitigatingsocialbiasesinlanguagemodels,in: In- ternationalConferenceonMachineLearning,PMLR.pp.6565–6576

    2021

  42. [48]

    Biases in large language mod- els: origins, inventory, and discussion

    Navigli, R., Conia, S., Ross, B., 2023. Biases in large language mod- els: origins, inventory, and discussion. ACM Journal of Data and Information Quality 15, 1–21

    2023

  43. [49]

    Training language models to follow instructions with human feedback

    Ouyang,L.,Wu,J.,Jiang,X.,Almeida,D.,Wainwright,C.,Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al., 2022. Training language models to follow instructions with human feedback. Ad- vances in neural information processing systems 35, 27730–27744

    2022

  44. [50]

    On the risk of misinformation pollution with large language models

    Pan, Y., Pan, L., Chen, W., Nakov, P., Kan, M.Y., Wang, W.Y., 2023. On the risk of misinformation pollution with large language models. arXiv preprint arXiv:2305.13661

    2023 arXiv

  45. [51]

    De- tection of adversarial training examples in poisoning attacks through anomaly detection

    Paudice,A.,Muñoz-González,L.,Gyorgy,A.,Lupu,E.C.,2018. De- tection of adversarial training examples in poisoning attacks through anomaly detection. arXiv preprint arXiv:1802.03041

    2018 arXiv

  46. [52]

    Ignore previous prompt: Attack tech- niques for language models

    Perez, F., Ribeiro, I., 2022. Ignore previous prompt: Attack tech- niques for language models. arXiv preprint arXiv:2211.09527

    2022 arXiv

  47. [53]

    arXiv preprint arXiv:2306.09479

    Inverse scaling: When bigger isn’t better. arXiv preprint arXiv:2306.09479

    arXiv

  48. [54]

    Traceabilityfortrustworthyai: Areviewofmod- els and tools

    Mora-Cantallops, M., Sánchez-Alonso, S., García-Barriocanal, E., Sicilia,M.A.,2021. Traceabilityfortrustworthyai: Areviewofmod- els and tools. Big Data and Cognitive Computing 5, 20

    2021

  49. [55]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J., 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine K. Chen et al.:Preprint submitted to Elsevier Page 14 of 15 A Survey ...

    2020

  50. [56]

    Raji, I.D., Smart, A., White, R.N., Mitchell, M., Gebru, T., Hutchin- son, B., Smith-Loud, J., Theron, D., Barnes, P., 2020. Closing the ai accountability gap: Defining an end-to-end framework for inter- nal algorithmic auditing, in: Proceedings of the 2020 conference on fairn...

    2020

  51. [57]

    Towards understanding sycophancy in language models

    Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bow- man, S.R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S.R., et al., 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548

    2023 arXiv

  52. [58]

    Societal bi- ases in language generation: Progress and challenges

    Sheng, E., Chang, K.W., Natarajan, P., Peng, N., 2021. Societal bi- ases in language generation: Progress and challenges. arXiv preprint arXiv:2105.04054

    2021 arXiv

  53. [59]

    Deep inside con- volutional networks: Visualising image classification models and saliency maps

    Simonyan, K., Vedaldi, A., Zisserman, A., 2013. Deep inside con- volutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034

    2013 arXiv

  54. [60]

    You are what you write: Preservingprivacyintheeraoflargelanguagemodels

    Plant, R., Giuffrida, V., Gkatzia, D., 2022. You are what you write: Preservingprivacyintheeraoflargelanguagemodels. arXivpreprint arXiv:2204.09391

    2022 arXiv

  55. [61]

    A survey of ro- bustadversarialtraininginpatternrecognition: Fundamental,theory, and methodologies

    Qian, Z., Huang, K., Wang, Q.F., Zhang, X.Y., 2022. A survey of ro- bustadversarialtraininginpatternrecognition: Fundamental,theory, and methodologies. Pattern Recognition 131, 108889

    2022

  56. [62]

    Certified defenses for data poisoning attacks

    Steinhardt, J., Koh, P.W.W., Liang, P.S., 2017. Certified defenses for data poisoning attacks. Advances in neural information processing systems 30

    2017

  57. [63]

    Tokpo, E.K., Calders, T., 2024. Fairflow: An automated approach to model-based counterfactual data augmentation for nlp, in: Joint Eu- ropean Conference on Machine Learning and Knowledge Discovery in Databases, Springer. pp. 160–176

    2024

  58. [64]

    On adaptive attackstoadversarialexampledefenses

    Tramer, F., Carlini, N., Brendel, W., Madry, A., 2020. On adaptive attackstoadversarialexampledefenses. Advancesinneuralinforma- tion processing systems 33, 1633–1645

    2020

  59. [65]

    A multiscale visualization of attention in the trans- former model

    Vig, J., 2019. A multiscale visualization of attention in the trans- former model. arXiv preprint arXiv:1906.05714

    2019 arXiv

  60. [66]

    Concealed data poisoning attacks on nlp models

    Wallace, E., Zhao, T.Z., Feng, S., Singh, S., 2020. Concealed data poisoning attacks on nlp models. arXiv preprint arXiv:2010.12563

    2020 arXiv

  61. [67]

    Say the rightthingright: Ethicsissuesinnaturallanguagegenerationsystems, in: ProceedingsoftheFirstACLWorkshoponEthicsinNaturalLan- guage Processing, pp

    Smiley, C., Schilder, F., Plachouras, V., Leidner, J.L., 2017. Say the rightthingright: Ethicsissuesinnaturallanguagegenerationsystems, in: ProceedingsoftheFirstACLWorkshoponEthicsinNaturalLan- guage Processing, pp. 103–108

    2017

  62. [68]

    arXiv preprint arXiv:2310.07298

    Staab,R.,Vero,M.,Balunović,M.,Vechev,M.,2023.Beyondmemo- rization: Violatingprivacy viainferencewith largelanguage models. arXiv preprint arXiv:2310.07298

    2023 arXiv

  63. [70]

    Establishing data prove- nance for responsible artificial intelligence systems

    Werder, K., Ramesh, B., Zhang, R., 2022. Establishing data prove- nance for responsible artificial intelligence systems. ACM Transac- tions on Management Information Systems (TMIS) 13, 1–23

    2022

  64. [71]

    Hallucination is inevitable: An innate limitation of large language models

    Xu, Z., Jain, S., Kankanhalli, M., 2024. Hallucination is inevitable: An innate limitation of large language models. arXiv preprint arXiv:2401.11817

    2024 arXiv

  65. [72]

    On protecting the data privacy of large language models (llms): A survey

    Yan, B., Li, K., Xu, M., Dong, Y., Zhang, Y., Ren, Z., Cheng, X., 2024a. On protecting the data privacy of large language models (llms): A survey. arXiv preprint arXiv:2403.05156

    arXiv

  66. [73]

    Yan, J., Yadav, V., Li, S., Chen, L., Tang, Z., Wang, H., Srinivasan, V., Ren, X., Jin, H., 2024b. Backdooring instruction-tuned large lan- guage models with virtual prompt injection, in: Proceedings of the 2024 Conference of the North American Chapter of the Association forCo...

    2024

  67. [74]

    Poisoning lan- guagemodelsduringinstructiontuning, in: InternationalConference on Machine Learning, PMLR

    Wan, A., Wallace, E., Shen, S., Klein, D., 2023. Poisoning lan- guagemodelsduringinstructiontuning, in: InternationalConference on Machine Learning, PMLR. pp. 35413–35425

    2023

  68. [75]

    Ethicalandsocialrisksofharmfromlanguagemodels

    Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al., 2021. Ethicalandsocialrisksofharmfromlanguagemodels. arXivpreprint arXiv:2112.04359

    2021 arXiv

  69. [77]

    Zhang, C., Zhou, X., Wan, Y., Zheng, X., Chang, K.W., Hsieh, C.J.,

  70. [78]

    Theoretically principled trade-off between robustness and accuracy, in: International conference on machine learning, PMLR

    Zhang, H., Yu, Y., Jiao, J., Xing, E., ElGhaoui, L., Jordan, M., 2019. Theoretically principled trade-off between robustness and accuracy, in: International conference on machine learning, PMLR. pp. 7472– 7482

    2019

  71. [79]

    Adversarial attacks on deep-learning models in natural language processing: A survey

    Zhang, W.E., Sheng, Q.Z., Alhazmi, A., Li, C., 2020. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Transactions on Intelligent Systems and Technology (TIST) 11, 1–41

    2020

  72. [80]

    Siren’s song in the ai ocean: A survey on hallucination in large language models, 2023

    Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., Chen, Y., et al., 2024b. Siren’s song in the ai ocean: A survey on hallucination in large language models, 2023. URL https://arxiv. org/abs/2309.01219 . K. Chen et al.:Preprint submitted to E...

    2023 arXiv

  73. [81]

    Mixup-based unified frame- worktoovercomegenderbiasresurgence,in: Proceedingsofthe46th InternationalACMSIGIRConferenceonResearchandDevelopment in Information Retrieval, pp

    Yu, L., Mao, Y., Wu, J., Zhou, F., 2023. Mixup-based unified frame- worktoovercomegenderbiasresurgence,in: Proceedingsofthe46th InternationalACMSIGIRConferenceonResearchandDevelopment in Information Retrieval, pp. 1755–1759

    2023

  74. [82]

    Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.T., Sun, M., et al., 2024. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback,in: ProceedingsoftheIEEE/CVFConferenceonComputer Vision and Pattern...

    2024

  75. [2018]

    Manipulating machine learning: Poisoning attacks and coun- termeasures for regression learning, in: 2018 IEEE symposium on security and privacy (SP), IEEE. pp. 19–35

    2018

  76. [2020]

    Advances in neural information processing systems 33, 1877–1901

    Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901

    1901

  77. [2021]

    arXiv preprint arXiv:2108.07258

    On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

    arXiv

  78. [2022]

    arXiv preprint arXiv:2206.05511

    Improving the adversarial robustness of nlp models by infor- mation bottleneck. arXiv preprint arXiv:2206.05511

    arXiv

  79. [2023]

    arXiv preprint arXiv:2310.05627

    Integrating stock features and globalinformation via large lan- guage models for enhanced stock return prediction. arXiv preprint arXiv:2310.05627

    arXiv

  80. [2024]

    arXiv e-prints , arXiv–2404

    Investigatingthepromptleakageeffectandblack-boxdefenses for multi-turn llm interactions. arXiv e-prints , arXiv–2404

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.