Pith. sign in

REVIEW 4 major objections 5 minor 52 references

DMRL: Data- and Model-aware Reward Learning for Data Extraction

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that DMRL, which learns reward models from privacy-leakage demonstrations and scales GRPO by data and model hardness, extracts PII from LLMs more accurately than four baselines across all tested models and datasets.

desk verdict A load-bearing contradiction between the described training data and the appendix makes the reported PII extraction gains unreproducible; otherwise an incremental but well-scoped red-teaming extension. read the letter →

arxiv 2505.06284 v1 pith:467VBC5W submitted 2025-05-07 cs.LG cs.CR

Zhiqiang Wang , Ruoxi Cheng This is my paper · ORCID
classification cs.LGcs.CR
keywords dataextractioninversereinforcementlearningrewardmodelGRPOpersonallyidentifiableinformationprivacyleakagehardness-awareoptimizationLLMredteaming
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DMRL, a method that extracts personally identifiable information from large language models by learning reward models from a demonstration dataset of privacy-leakage question–answer pairs, then fine-tuning the target model with Group Relative Policy Optimization (GRPO) using those rewards. The paper claims DMRL beats four existing extraction baselines on every model and dataset tested, with reconstruction accuracy gains of 4–7 points over the strongest baseline and inference accuracy above 50% on most configurations. The wider point is that reward learning can drive extraction without relying on duplicated training data, handcrafted prompts, or random-search generation. If the claim holds, red teams gain a systematic attack, and defenses need to focus on the reward signal.

What carries the argument

The central mechanism is the shadow reward model $R_j(\cdot,\cdot)$, trained per category by joint reward-and-policy inverse RL from the demonstration subset $D_j$, together with the combined hardness coefficient $\alpha_{ji} = \alpha^{\mathrm{D}}_{ji}\,\alpha^{\mathrm{M}}_j$ that scales the GRPO advantage for question $i$ in category $j$. Data hardness uses one minus the average maximal CLIP similarity between split sub-sentences of the demonstration answer and the current policy's answer; model hardness uses the reward gap after outlier masking. The reward model converts the attack into a reward optimization problem: GRPO pushes the policy toward outputs that the shadow reward model scores as matching the leakage demonstrations, while the hardness coefficient reweights updates to focus on hard data and responsive model states.

What would settle it

Train SRL shadow reward models and run GRPO-S on one GPT-2 size and one dataset twice, once with the demonstration set containing only refusal responses and once with only full PII disclosures; if the refusal-trained variant reconstructs PII as accurately as the disclosure-trained variant, the claimed reward mechanism is not what drives extraction.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that data- and model-aware reward learning makes PII extraction substantially more effective than current attack methods. DMRL first builds a Q&A privacy-leakage dataset for each of N categories, trains category-specific shadow reward models with maximum-likelihood inverse reinforcement learning on that demonstration data, and then optimizes the target LLM with GRPO while scaling each advantage by a combined hardness coefficient: data hardness from CLIP similarity between demonstration and generated responses, and model hardness from the shadow reward model's reward gap. Reported results show DMRL achieving the highest PII reconstruction accuracy in all settings, for example 2.88 to 21.71 on ECHR from GPT-2 Small to Large, and top-1 inference accuracy of 40.7–55.1% across four 6–7B open models. The paper interprets this as evidence that larger models are more memorization-prone and that reward-guided optimization exposes PII more reliably than prior extractors.

Load-bearing premise

The central load-bearing premise is that the demonstration dataset consistently rewards PII-disclosing outputs; the paper describes this dataset as refusal responses with reasoning in Section 3.1, while Appendix A shows answers that fully disclose the requested PII, leaving the direction of the reward signal unresolved.

Editorial extensions

If this is right

  • If DMRL's claims hold, the four evaluated baselines—TAB, PrivAgent, Bag of Tricks, and AL-PII—are all outperformed on PII reconstruction across GPT-2 Small/Medium/Large/XL, Qwen1.5-7B, LLaMA2-7B, ChatGLM3-6B, and Baichuan2-7B.
  • Larger GPT-2 models extract more PII, with reconstruction accuracy rising with model size on ECHR, Enron, and Yelp-Health.
  • LLMs can infer sensitive personal information from context at above-chance rates, with most tested 6–7B models exceeding 50% inference accuracy.
  • Data extraction can succeed without relying on duplicated training data, prompt engineering, or random-search adversarial generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The direction of the learned reward is underdetermined by the paper: Section 3.1 says demonstrations are refusals with reasoning, while Appendix A shows full disclosures; a direct ablation on refusal-only versus disclosure-only demonstrations would settle which signal actually drives extraction.
  • The hardness-scaling scheme is not tied to privacy: the same data-level CLIP similarity and model-level reward gap could be applied to any alignment task where demonstrations exist without preference pairs.
  • If the method works as reported, defenses that filter prompt patterns or detect known attack templates may not stop it, since the attack signal lives in the reward model and the optimization procedure rather than in the final prompt.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes DMRL, a data extraction attack that first constructs a demonstration dataset of privacy-leakage Q&A pairs, trains category-specific shadow reward models via maximum-likelihood inverse reinforcement learning on those demonstrations, measures data- and model-level hardness using CLIP similarity and reward gaps, and then fine-tunes the target LLM with GRPO using hardness-scaled advantages. The paper claims that DMRL achieves the highest PII reconstruction accuracy on ECHR, Enron, and Yelp-Health across GPT-2 Small/Medium/Large/XL, Qwen1.5-7B, LLaMA2-7B, ChatGLM3-6B, and Baichuan2-7B, outperforming TAB, PrivAgent, Bag of Tricks, and AL-PII.

Significance. If the results were reproducible, DMRL would be a useful red-teaming contribution to the study of privacy leakage from LLMs. The formal IRL/GRPO derivation and the algorithm pseudocode are clearly presented, and the hardness-scaling idea is novel enough to merit attention. However, as written the central empirical claim is not supported: the demonstration-data contradiction, the absence of error bars and significance tests, and the missing implementation details prevent verification. The paper also includes a limitations section and a risk discussion, which are appropriate for this topic, but these do not mitigate the technical issues.

major comments (4)
  1. [Section 3.1 and Appendix A] Section 3.1 defines each demonstration o_ji as a refusal response f along with a CoD reasoning sequence, and states that D contains safe refusal responses. Appendix A, however, shows training examples such as 'Sure, my name is John Smith...' with full SSN, DOB, and address, annotated 'PII disclosed.' These are not refusals. Because Algorithm 1 trains R_j by contrasting the demonstration with a policy sample via Eq. (3), and Eq. (14) uses R_j as the reward signal for GRPO, a reward model trained on refusals would push the policy toward refusal, directly contradicting the extraction goal; a reward model trained on Appendix A disclosures would make Section 3.1 incorrect. The paper never reconciles this, and no code or data is released to disambiguate, so the reported gains in Tables 1-3 are not reproducible from the manuscript as written.
  2. [Tables 1-3 and Section 4.2] Tables 1-3 report point estimates only, with no error bars, confidence intervals, or significance tests. Since each cell is a single number and the paper claims that DMRL 'significantly outperforms all baseline methods,' the reader cannot assess whether the 4-7 point gains over AL-PII in Figure 2 and Table 2 are beyond sampling noise. In addition, Table 1 lists only GPT-2 Small, Medium, and Large, while the text and Table 2 include GPT-2 XL; this discrepancy needs correction.
  3. [Section 3.3, Eqs. (5)-(8)] The data-hardness measure applies a CLIP model to text sub-sentences, but no validation is provided that CLIP similarity correlates with leakage semantics. CLIP is trained for image-text matching, and the paper does not report any baseline or ablation comparing this measure to text-only similarity metrics, nor does it describe the sentence-splitting prompt or the choice of CLIP checkpoint. Because alpha^D_ji enters the advantage in Eq. (14), a miscalibrated hardness measure directly affects the optimization, so this is not a cosmetic issue.
  4. [Figure 2 and Table 3] Figure 2 is referenced in Section 4.2 but is not present in the manuscript, and Table 3 reports PII inference accuracy for DMRL alone with no baseline comparisons. Without the figure and without baseline numbers, the claim that DMRL 'consistently outperforms all baselines' on inference is unverifiable.
minor comments (5)
  1. [Eq. (8)] Equation (8) writes sigma(delta) and sigma(bar-delta) without defining delta as delta_ji or specifying that the denominator is the category-level mean; the notation should be made precise.
  2. [Abstract and Section 3.4] The abstract and Section 3.4 call GRPO 'Gradient Reward Policy Optimization' while the standard name is Group Relative Policy Optimization; please correct this.
  3. [Section 1 and Section 4.1] Section 1 says 'base on dataset duplicates' and Section 4.1 contains the typo 'wit publicly available'; these should be corrected.
  4. [Section 4.1] The text says the experiments cover 'four different LLMs and four variants of GPT-2,' but Table 1 omits GPT-2 XL; either add the XL column or revise the text.
  5. [Related work and method] The relationship to the authors' prior HAIR and SRMIR work (Cheng et al. 2025b; Cheng and Cao 2025) is mentioned only by citation; a short comparison would help distinguish the new contributions.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the reward-learning loop (D -> IRL shadow reward -> GRPO-S) is self-contained and the reported gains are measured against external baselines; the paper's multiple self-citations and the Section 3.1/Appendix A refusal-vs-disclosure contradiction are consistency risks, not circularity.

full rationale

DMRL's chain—demonstration dataset D, IRL-trained shadow reward models (Eqs. 1-4), hardness coefficients (Eqs. 5-13), and GRPO-S objective (Eq. 14)—is a standard, self-contained RL loop. The policy is trained to maximize rewards defined by D, and the headline claims are empirical PII reconstruction/inference numbers on held-out ECHR, Enron, and Yelp-Health sets compared with external baselines (TAB, PrivAgent, Bag of Tricks, AL-PII); those numbers are not algebraic consequences of D or of the reward-model fit. The only self-citation concern is stylistic: dataset construction and hardness scaling are attributed to the authors' own SRMIR/HAIR papers, but no uniqueness theorem or forbidden-alternative argument is imported from them, so self-citation is not load-bearing. A genuine non-circularity problem is the unresolved mismatch between Section 3.1, which defines o_ji as ending in 'final refusal answer' and says D contains 'safe refusal responses,' and Appendix A, whose examples end with '— PII disclosed: ...' and fully reveal PII. If D contained refusals, Eq. (14) would train the policy to refuse; if D contains disclosures, Section 3.1 is misdescribed. This breaks reproducibility but is not a reduction of the outputs to the inputs by construction. Score 2 reflects minor, non-load-bearing self-citation and this reporting inconsistency, not circular derivation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method depends on several unreported hyperparameters (beta, T, G, eta, N, M) and on two unverified assumptions: that CLIP embeddings are suitable for sentence-level leakage similarity, and that the demonstration data consistently represent the leakage behavior to be rewarded. The contradiction between Section 3.1 and Appendix A is the most serious.

free parameters (5)
  • beta (KL coefficient)
    Constant in Eq. (1)-(4) controlling KL divergence; no value reported.
  • threshold T for outlier masking
    Pre-determined value in Eq. (10) filtering reward gaps; no value or selection procedure reported.
  • GRPO group size G
    Number of sampled outputs per question in Eq. (14); no value reported.
  • number of categories N and samples M
    Dataset size in Section 3.1; no values reported.
  • reward update stepsize eta_j^t
    Learning rate in Algorithm 1; no value reported.
assumptions (4)
  • standard math Minimax equivalence in Eq. (2) from Li et al. 2024a
    The joint reward-policy IRL problem is taken as equivalent to the minimax objective; cited, not re-derived.
  • domain assumption CLIP text-text similarity is a valid semantic similarity measure
    Used in Eq. (5)-(8) to compute data hardness; CLIP is a vision-language model and the sentence similarity use is unvalidated.
  • ad hoc to paper Demonstration dataset D encodes the extraction behavior
    Section 3.1 claims refusals, Appendix A shows leaks; the correct content of D is unresolved yet determines the reward signal.
  • standard math GRPO optimizes the policy toward maximal reward
    Standard RL assumption; follows Shao et al. 2024.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DMRL: Data- and Model-aware Reward Learning for Data Extraction." pith.science (2026). https://pith.science/paper/467VBC5W

@misc{pith2026250506284,
  author       = {Pith},
  title        = {Pith review of: DMRL: Data- and Model-aware Reward Learning for Data Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/467VBC5W}},
  note         = {Machine review of arXiv:2505.06284}
}
read the original abstract

Large language models (LLMs) are inherently vulnerable to unintended privacy breaches. Consequently, systematic red-teaming research is essential for developing robust defense mechanisms. However, current data extraction methods suffer from several limitations: (1) rely on dataset duplicates (addressable via deduplication), (2) depend on prompt engineering (now countered by detection and defense), and (3) rely on random-search adversarial generation. To address these challenges, we propose DMRL, a Data- and Model-aware Reward Learning approach for data extraction. This technique leverages inverse reinforcement learning to extract sensitive data from LLMs. Our method consists of two main components: (1) constructing an introspective reasoning dataset that captures leakage mindsets to guide model behavior, and (2) training reward models with Group Relative Policy Optimization (GRPO), dynamically tuning optimization based on task difficulty at both the data and model levels. Comprehensive experiments across various LLMs demonstrate that DMRL outperforms all baseline methods in data extraction performance.

Figures

Figures reproduced from arXiv: 2505.06284 by the authors.

Figure 1
Figure 1. Pipeline of the proposed method. First, we generate a Q&A privacy-leakage dataset using structured prompts to serve as demonstration data for training the reward model. Next, we leverage this reward model within GRPO to fine-tune the LLM, dynamically adjusting optimization according to task difficulty at both data and model levels. Then we have the final comprehensive dataset D = [ N j=1 Dj . This dataset D includes… view at source ↗
Figure 2
Figure 2. PII reconstruction results across different LLMs. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 13 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Nabiha Asghar. 2016. Yelp dataset challenge: Review rating prediction. arXiv preprint arXiv:1605.05362

    arXiv 2016

  4. [4]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

  5. [5]

    Baichuan. 2023. https://arxiv.org/abs/2309.10305 Baichuan 2: Open large-scale language models . arXiv preprint arXiv:2309.10305

    arXiv 2023

  6. [6]

    Shuirong Cao, Ruoxi Cheng, and Zhiqiang Wang. 2025. Agr: Age group fairness reward for bias mitigation in llms. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

    work page 2025

  7. [7]

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations

    2022

  8. [8]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pages 2633--2650

    2021

Show all 52 references
  1. [9]

    Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in english. arXiv preprint arXiv:1906.02059

    2019 arXiv

  2. [11]

    Ruoxi Cheng, Yizhong Ding, Shuirong Cao, Ranjie Duan, Xiaoshuang Jia, Shaowei Yuan, Zhiqiang Wang, and Xiaojun Jia. 2025 a . Pbi-attack: Prior-guided bimodal interactive black-box jailbreak attack for toxicity maximization. In Proceedings of the 5th Workshop on Trustworthy NLP...

    2025

  3. [12]

    Ruoxi Cheng, Yizhong Ding, Shuirong Cao, Shitong Shao, and Zhiqiang Wang. 2024 a . Gibberish is all you need for membership inference detection in contrastive language-audio pretraining. arXiv preprint arXiv:2410.18371

    2024 arXiv

  4. [13]

    Ruoxi Cheng, Haoxuan Ma, Shuirong Cao, Jiaqi Li, Aihua Pei, Zhiqiang Wang, Pengliang Ji, Haoyu Wang, and Jiaqi Huo. 2024 b . Reinforcement learning from multi-role debates as feedback for bias mitigation in llms. arXiv preprint arXiv:2404.10160

    2024 arXiv

  5. [14]

    Ruoxi Cheng, Haoxuan Ma, and Weixin Wang. 2025 b . https://arxiv.org/abs/2503.18991 Hair: Hardness-aware inverse reinforcement learning with introspective reasoning for llm alignment . Preprint, arXiv:2503.18991

    2025

  6. [15]

    Vitaly Feldman. 2020. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954--959

    2020

  7. [16]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

    2024 arXiv

  8. [17]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  9. [18]

    Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. 2020. Supervised contrastive learning for pre-trained language model fine-tuning. arXiv preprint arXiv:2011.01403

    2020 arXiv

  10. [19]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556

    2022 arXiv

  11. [20]

    Bo Hui, Haolin Yuan, Neil Gong, Philippe Burlina, and Yinzhi Cao. 2024. Pleak: Prompt leaking attacks against large language model applications. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 3600--3614

    2024

  12. [21]

    Huseyin A Inan, Osman Ramadan, Lukas Wutschitz, Daniel Jones, Victor R \"u hle, James Withers, and Robert Sim. 2021. Training data leakage analysis in language models. arXiv preprint arXiv:2101.05405

    2021 arXiv

  13. [22]

    Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, et al. 2022. Measuring forgetting of memorized training examples. arXiv preprint arXiv:2207.00099

    2022 arXiv

  14. [23]

    Bargav Jayaraman, Esha Ghosh, Melissa Chase, Sambuddha Roy, Wei Dai, and David Evans. 2024. Combing for credentials: Active pattern extraction from smart reply. In 2024 IEEE Symposium on Security and Privacy (SP), pages 1443--1461. IEEE

    2024

  15. [24]

    Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36:24678--24704

    2023

  16. [25]

    Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pages 10697--10707. PMLR

    2022

  17. [26]

    Bryan Klimt and Yiming Yang. 2004. Introducing the enron corpus. In CEAS, volume 45, pages 92--96

    2004

  18. [27]

    W Bradley Knox, Stephane Hatgis-Kessell, Serena Booth, Scott Niekum, Peter Stone, and Alessandro Allievi. 2022. Models of human preference for learning reward functions. arXiv preprint arXiv:2206.02231

    2022 arXiv

  19. [28]

    Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Goldberg, and Byron C Wallace. 2021. Does bert pretrained on clinical notes reveal sensitive data? arXiv preprint arXiv:2104.07762

    2021 arXiv

  20. [29]

    Klas Leino and Matt Fredrikson. 2020. Stolen memories: Leveraging model memorization for calibrated \ White-Box \ membership inference. In 29th USENIX security symposium (USENIX Security 20), pages 1605--1622

    2020

  21. [30]

    Jiaxiang Li, Siliang Zeng, Hoi-To Wai, Chenliang Li, Alfredo Garcia, and Mingyi Hong. 2024 a . Getting more juice out of the sft data: Reward learning from human demonstration improves sft for llm alignment. Advances in Neural Information Processing Systems, 37:124292--124318

    2024

  22. [31]

    Songze Li, Ruoxi Cheng, and Xiaojun Jia. 2024 b . Identity inference from clip models using only textual data. arXiv preprint arXiv:2405.14517

    2024 arXiv

  23. [32]

    note bloat

    Jinghui Liu, Daniel Capurro, Anthony Nguyen, and Karin Verspoor. 2022. “note bloat” impacts deep learning-based nlp models for clinical prediction tasks. Journal of biomedical informatics, 133:104149

    2022

  24. [33]

    Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-B \'e guelin. 2023. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 346--363. IEEE

    2023

  25. [34]

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tram \`e r, and Katherine Lee. 2023. Scalable extraction of training data from (production) language models. arXiv preprint ar...

    2023 arXiv

  26. [35]

    Andrew Y Ng, Stuart Russell, et al. 2000. Algorithms for inverse reinforcement learning. In Icml, volume 1, page 2

    2000

  27. [36]

    Yuzhou Nie, Zhun Wang, Ye Yu, Xian Wu, Xuandong Zhao, Wenbo Guo, and Dawn Song. 2024. Privagent: Agentic-based red-teaming for llm privacy leakage. arXiv preprint arXiv:2412.05734

    2024 arXiv

  28. [37]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  29. [38]

    Aihua Pei, Zehua Yang, Shunan Zhu, Ruoxi Cheng, and Ju Jia. 2024. Selfprompt: Autonomously evaluating llm robustness via domain-constrained knowledge guidelines and refined adversarial prompts. arXiv preprint arXiv:2412.00765

    2024 arXiv

  30. [39]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  31. [40]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  32. [41]

    St \'e phane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627--635. JMLR Works...

    2011

  33. [42]

    Paul M Schwartz and Daniel J Solove. 2011. The pii problem: Privacy and a new concept of personally identifiable information. NYUL rev., 86:1814

    2011

  34. [43]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909

    2015 arXiv

  35. [44]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  36. [45]

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE

    2017

  37. [46]

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. 2023. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525

    2023 arXiv

  38. [47]

    Aleena Thomas, David Ifeoluwa Adelani, Ali Davody, Aditya Mogadala, and Dietrich Klakow. 2020. Investigating the impact of pre-trained word embeddings on memorization in neural networks. In Text, Speech, and Dialogue: 23rd International Conference, TSD 2020, Brno, Czech Republ...

    2020

  39. [48]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  40. [49]

    Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. 2021. Differentially private fine-tuning of language models. arXiv preprint arXiv:2110.06500

    2021 arXiv

  41. [50]

    Jiahao Yu, Yuhang Wu, Dong Shu, Mingyu Jin, Sabrina Yang, and Xinyu Xing. 2023 a . Assessing prompt injection risks in 200+ custom gpts. arXiv preprint arXiv:2311.11538

    2023 arXiv

  42. [51]

    Weichen Yu, Tianyu Pang, Qian Liu, Chao Du, Bingyi Kang, Yan Huang, Min Lin, and Shuicheng Yan. 2023 b . Bag of tricks for training data extraction from language models. In International Conference on Machine Learning, pages 40306--40320. PMLR

    2023

  43. [52]

    Siliang Zeng, Chenliang Li, Alfredo Garcia, and Mingyi Hong. 2022. Maximum-likelihood inverse reinforcement learning with finite-time guarantees. Advances in Neural Information Processing Systems, 35:10122--10135

    2022

  44. [53]

    Siliang Zeng, Yao Liu, Huzefa Rangwala, George Karypis, Mingyi Hong, and Rasool Fakoor. 2025. From demonstrations to rewards: Alignment without explicit human preferences. arXiv preprint arXiv:2503.13538

    2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.