Pith. sign in

REVIEW 3 cited by

Training Data Extraction From Pre-trained Language Models: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.16157 v1 pith:JV7CC5KX submitted 2023-05-25 cs.CL

Shotaro Ishihara This is my paper · ORCID
classification cs.CL
keywords dataextractionlanguagetrainingfirstmodelsplmspre-trained
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

As the deployment of pre-trained language models (PLMs) expands, pressing security concerns have arisen regarding the potential for malicious extraction of training data, posing a threat to data privacy. This study is the first to provide a comprehensive survey of training data extraction from PLMs. Our review covers more than 100 key papers in fields such as natural language processing and security. First, preliminary knowledge is recapped and a taxonomy of various definitions of memorization is presented. The approaches for attack and defense are then systemized. Furthermore, the empirical findings of several quantitative studies are highlighted. Finally, future research directions based on this review are suggested.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Automated Privacy Information Annotation in Large Language Model Interactions

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A 249K-query English/Chinese dataset with 154K privacy phrases and a benchmark showing fine-tuned 1B-7B local models can detect privacy leaks, with 87.6% leakage accuracy but only 44.7% information-level F1.

  2. Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks

    cs.LG 2025-04 conditional novelty 4.0 of 10

    The paper organizes large-small model collaboration into downward, upward, and inference-time transfer, and advocates multi-objective benchmarks for private-domain tasks.

  3. Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions

    eess.SY 2025-01 conditional novelty 2.0 of 10

    The paper surveys recent work, models, applications, and challenges of using LLMs in intelligent transportation systems, without presenting new experimental results.

Pith tools