Pith. sign in

REVIEW 3 major objections 6 minor 56 references

Uncovering Gradient Inversion Risks in Practical Language Model Training

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Gradient inversion can recover up to 92.9% of private text data during federated fine-tuning of masked language models, even when embeddings are frozen and dropout is active.

desk verdict Grab is a credible extension of gradient inversion to practical masked-LM settings, but the headline margins need equal-compute evidence before the claim is taken at face value. read the letter →

arxiv 2507.21198 v1 pith:SWL6C2YJ submitted 2025-07-28 cs.LG cs.AI

Xinguo Feng , Zhongkui Ma , Zihan Wang , Eu Joe Chegne , Mengyao Ma , Alsharif Abuadbba , Guangdong Bai This is my paper · ORCID
classification cs.LGcs.AI
keywords gradientinversionfederatedlearninglanguagemodelsprivacyattackdropoutmaskbeamsearchtextrecoveryBERT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that gradient inversion attacks—long considered ineffective against language models because text is discrete—are a practical privacy threat in federated fine-tuning of masked language models. Its attack, Grab, alternates continuous optimization that recovers tokens through gradient matching with discrete beam search that reorders them, adding a learned dropout mask so the attack works when dropout is active. The paper reports recovering up to 92.9% of private training text, beating the best prior attack by up to 28.9% in benchmark settings and 48.5% in practical settings with frozen embeddings, activated dropout, and unknown sequence lengths. If correct, this means sharing gradients in federated language-model training leaks substantial training text under realistic defenses.

What carries the argument

The central mechanism is the hybrid optimization loop of Algorithm 1: a continuous step and a discrete step alternate over several rounds. In the continuous step, gradient descent minimizes a combined $L_2+L_1$ recovery loss over dummy embeddings, dummy labels, and a per-layer dropout mask (clamped to $[0,1]$), which is meant to compensate for the randomness of the victim's activated dropout. In the discrete step, beam search reorders the recovered tokens by substituting each position with candidates from the token set, keeping the $n_b$ lowest-loss beams, while including the padding token to signal the end of shorter sequences. The output of one step seeds the other in the next round, and the best result from either is returned.

What would settle it

Run Grab against a victim whose dropout mask is re-sampled on every forward pass during the gradient computation (the standard training regime), and compare the recovery rate with and without dropout mask learning; if the gap is negligible or recovery collapses, the claim that mask learning offsets dropout-induced noise in practical training is falsified. A complementary check: freeze the learned mask after optimization and verify that it matches the victim's realized mask statistics for the same input.

Watch

Extended reading notes

Core claim

Grab is a gradient inversion attack that recovers private text from the shared gradients of a federated fine-tuning round without assuming known labels or exact sequence lengths. The attack alternates a continuous optimization, which matches dummy gradients to the victim's gradients while jointly updating dummy embeddings, dummy labels, and a clamped continuous dropout mask, with a discrete optimization that uses beam search to reorder the tokens recovered by the continuous step, using the padding token so sequence lengths can float. The paper claims up to 92.9% ROUGE recovery on the BERT base model, with consistent superiority over DLG, TAG, and LAMP across three datasets and batch sizes up to 32, and reports that the attack remains effective at batch sizes up to 128 and against gradient-noise and gradient-pruning defenses.

Load-bearing premise

The attack depends on the assumption that a single continuous dummy dropout mask, learned by gradient descent and clamped to $[0,1]$, can effectively stand in for the victim's randomly sampled per-forward-pass dropout masks, so that gradient matching still recovers tokens; the paper offers no guarantee that this holds when dropout masks are re-sampled every step.

Editorial extensions

If this is right

  • Federated fine-tuning of masked language models leaks training text through shared gradients even with frozen embedding layers, activated dropout, and no known labels or sequence lengths.
  • Freezing embeddings and activating dropout—individually recommended lightweight defenses—do not stop Grab; both together are the practical setting where the attack still outperforms baselines.
  • The attack's effectiveness degrades as batch size grows but remains substantial (about 40% ROUGE-1 at batch size 128), so larger batches reduce but do not eliminate the risk.
  • Gradient noise and gradient pruning defenses reduce recovery but only become effective at noise or pruning levels that destroy model utility measured by MCC.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if these recovery rates persist for other masked language models, gradient inversion should be treated as a baseline privacy threat in any federated fine-tuning service that exposes raw gradients, and defenses should assume text can be reconstructed even without label or length knowledge.
  • A testable extension: recording the optimized dropout mask across iterations and comparing it with the victim's actual mask statistics would isolate whether mask learning genuinely tracks the victim's dropout, or whether the beam search alone drives the gains.
  • The padding-token length discovery trick suggests inversion can be adapted to other structured discrete data (code, SQL, graphs), where an 'end' token could play the same role as padding in finding variable-length sequences.
  • The paper's stronger results on BERT than RoBERTa hint that the discrete prior interacts with model architecture; probing this difference could tell whether a better reordering prior would close the gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies gradient inversion attacks against masked language models in federated learning and proposes Grab, a hybrid optimization attack. Grab alternates continuous optimization (with a learned dummy dropout mask) and discrete optimization (beam search over tokens), and is evaluated on three datasets with BERT and RoBERTa models in benchmark and practical settings. The authors report up to 92.9% ROUGE recovery and improvements over LAMP, DLG, and TAG of up to 28.9% in benchmark settings and 48.5% in practical settings, plus robustness to gradient noise and gradient pruning. Code is released.

Significance. If the results hold, the paper makes a strong contribution: it demonstrates that gradient inversion can recover substantial text from gradient updates in realistic federated-learning configurations with frozen embeddings and activated dropout, correcting the prevailing view that the discrete nature of tokens makes such attacks impractical against language models. The evaluation is unusually broad, covering multiple datasets, batch sizes up to 128, several BERT and RoBERTa sizes, varying dropout rates, relaxed label and sequence-length assumptions, and two defense families. The release of code and the clear algorithm descriptions support reproducibility. The attack is evaluated against ground-truth data rather than derived from it, and hyperparameter tuning on the same evaluation datasets is standard practice for attack papers, so I do not see a circularity problem. The main caveats are the compute-budget confound in the baseline comparison and the gap between the threat model and the conditions under which the headline numbers are produced; both are addressable in revision.

major comments (3)
  1. [Section 5.1 and Figures 2-3] The comparative evaluation gives Grab n_h=5 and n_c=2,000, totaling 10,000 continuous optimization steps plus 25 discrete steps, while DLG, TAG, and LAMP run at their default budgets of 2,000-2,500 continuous steps. The text mentions a trial that standardizes all approaches to 10,000 steps and asserts that baselines 'early fall into a local optimum,' but the results of that trial are not reported. Because the abstract's headline margins (up to 28.9% and 48.5%) are comparative, this unequal-compute setup is load-bearing: without the standardized-trial numbers or equal-compute learning curves, the claimed superiority cannot be attributed to Grab's hybrid optimization or dropout-mask learning rather than to the larger compute budget. Please report the standardized trial as a table or figure, including baseline recovery rates at intermediate step counts, or rerun all baselines at 10,000 steps and use those results for the comparisons.
  2. [Sections 5.1-5.3 and Section 4.1] The main comparative experiments in Sections 5.2 and 5.3 adopt the known-label and known-sequence-length assumptions, as stated in Section 5.1, while the threat model in Section 4.1 explicitly does not grant the adversary such knowledge. This directly limits the headline claims: the 'up to 92.9%' recovery rate and the 'up to 48.5%' practical improvement are not achieved under the paper's own practical threat model. The ablation in Section 5.4.3 indicates that Grab tolerates relaxing these assumptions, but those results are not part of the headline comparison and are presented only later. Please either move the assumption-free results into the main evaluation or qualify the abstract and introductory claims to state clearly that the headline numbers use known labels and known sequence lengths.
  3. [Section 4.3, Algorithm 2 line 6] The dropout-mask update is the component credited with 'resolving activated dropout,' but the paper does not check whether the learned continuous mask actually tracks the victim's random binary dropout mask, or whether it merely provides extra degrees of freedom that overfit the recovery loss. Because the mask is optimized on the same recovery objective as the dummy inputs, the Grab-versus-Grab* comparisons in Figure 3 and Table 3 do not uniquely support the proposed mechanism. Please add a diagnostic such as the correlation between the learned mask and the true mask, or an upper-bound ablation that feeds the true mask to the attacker, and report the sensitivity of the attack to the Bernoulli initialization of the mask.
minor comments (6)
  1. [Section 3.4] The heading 'Droupout Mask Learning' should read 'Dropout Mask Learning'.
  2. [Section 6] In the concluding section, 'gradient decent' should be 'gradient descent'.
  3. [Table 5] The entries 'Grab 1' through 'Grab 4' are only explained in footnotes below the table; please define them in the table caption for readability.
  4. [Figure 3] The meaning of 'Grab*' is given only in the body text; please define it in the figure caption.
  5. [Section 5.4.1] The described configuration for BERT_tiny (6 layers, 12 heads, 67 million parameters) does not match the common TinyBERT checkpoint; please clarify the exact model variant used.
  6. [Appendix B.3] The ROUGE calculation uses a maximum over recovered sequences for each reference sequence, which is more lenient than LAMP's original index-based matching; since all approaches are scored this way the comparison is fair, but this choice is important enough to state explicitly in the main evaluation section.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Grab is an empirical attack whose success is scored against external ground-truth text, and no load-bearing claim reduces by construction to a fitted input or self-citation.

full rationale

The paper's central claim is an empirical one: Grab recovers private text from shared gradients, and its recovery rate is measured against the victim's actual training samples using ROUGE scores against ground truth. The attack pipeline (Algorithm 1) alternates continuous optimization over dummy embeddings, dummy labels, and a learned dropout mask with discrete beam-search reordering, all driven by a gradient-matching loss. The target tokens are not used to set the algorithm choices; hyperparameters such as n_h=5, n_c=2,000, n_e=2,000, and learning rates are fixed and applied consistently to the attack. Dropout mask learning is adopted from an external prior work (Scheliga et al.), not from the authors' own results, so it is independent support rather than a self-citation loop. The only overlapping author citation is reference [31] (Ma et al., including two present authors), which appears in the related-work discussion of membership inference attacks and is not load-bearing for any Grab claim. The skeptical concern about unequal baseline iteration budgets (10,000 steps for Grab versus 2,000-2,500 defaults for baselines, with the standardized trial not numerically reported) is a legitimate experimental-fairness issue, but it is not circularity under the given taxonomy: the absolute recovery numbers are still anchored to external ground truth, and no prediction in the paper is obtained by fitting a parameter to the quantity it later claims to predict. Accordingly, no specific circular step can be exhibited from the paper's equations or citations, and the score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rest on several domain assumptions about the attacker's knowledge and the effectiveness of dropout mask learning, plus a set of hand-chosen hyperparameters. There are no invented entities.

free parameters (4)
  • alpha_L1 = 0.01
    Weight for the L1 norm term in the recovery loss (Eq. 1). Chosen empirically and aligned with LAMP_L2+L1.
  • learning rate lambda = 0.01
    Gradient descent step size for dummy embedding, labels, and dropout mask; decay factor 0.89 every 50 steps (Section 5.1).
  • iteration and beam counts = n_e=2000, n_p=2000, n_h=5, n_c=2000, n_d=5, n_b=4
    Set empirically to trade efficiency and efficacy (Section 5.1). These directly affect recovery quality.
  • AdamW optimizer hyperparameters = default
    Optimizer chosen for gradient descent; hyperparameters not further tuned.
assumptions (5)
  • domain assumption Attacker knows the victim model architecture, including dropout rate and dropout layer positions (Section 4.1).
    Needed to initialize and optimize the dummy dropout mask and to run the victim's model for gradient matching.
  • ad hoc to paper A single continuous dummy dropout mask, optimized via gradient descent and clamped to [0,1], can represent the victim's stochastic per-forward-pass dropout masks closely enough for token recovery (Section 4.3, Algorithm 2).
    Load-bearing for practical settings; no theoretical guarantee is given, and empirical evidence is indirect.
  • domain assumption The combined L2 + L1 distance on gradients is a suitable surrogate for text recovery (Section 4.2, Eq. 1).
    Adopted from prior attacks; the choice is validated only empirically.
  • domain assumption A single-client FL setup with batch size b is functionally equivalent to m clients each with batch size b/m (Section 5.1).
    Used to simplify experiments; the paper reports consistency in Table 8 but not a formal proof.
  • domain assumption ROUGE score with max-over-order is a fair recovery metric (Appendix B.3).
    Used to compare recovered and reference batches; differs from LAMP's index-aligned comparison and could favor reordered outputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uncovering Gradient Inversion Risks in Practical Language Model Training." pith.science (2026). https://pith.science/paper/SWL6C2YJ

@misc{pith2026250721198,
  author       = {Pith},
  title        = {Pith review of: Uncovering Gradient Inversion Risks in Practical Language Model Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SWL6C2YJ}},
  note         = {Machine review of arXiv:2507.21198}
}
read the original abstract

The gradient inversion attack has been demonstrated as a significant privacy threat to federated learning (FL), particularly in continuous domains such as vision models. In contrast, it is often considered less effective or highly dependent on impractical training settings when applied to language models, due to the challenges posed by the discrete nature of tokens in text data. As a result, its potential privacy threats remain largely underestimated, despite FL being an emerging training method for language models. In this work, we propose a domain-specific gradient inversion attack named Grab (gradient inversion with hybrid optimization). Grab features two alternating optimization processes to address the challenges caused by practical training settings, including a simultaneous optimization on dropout masks between layers for improved token recovery and a discrete optimization for effective token sequencing. Grab can recover a significant portion (up to 92.9% recovery rate) of the private training data, outperforming the attack strategy of utilizing discrete optimization with an auxiliary model by notable improvements of up to 28.9% recovery rate in benchmark settings and 48.5% recovery rate in practical settings. Grab provides a valuable step forward in understanding this privacy threat in the emerging FL training mode of language models.

Figures

Figures reproduced from arXiv: 2507.21198 by the authors.

Figure 1
Figure 1. Overview of Grab. gradient information in FL. It is assumed that the adversary is able to eavesdrop on the communication between the server and the victim client, and has full knowledge of the model architecture and parameters. To recover the input data batch 𝑩 ∗ 𝑖 , an alternative data batch 𝑩𝑖 is randomly initialized and optimized by solving the following optimization problem, arg min 𝑩𝑖 D (∇𝜽 𝑡 L (𝑩𝑖), ∇𝜽 𝑡 L (𝑩 … view at source ↗
Figure 1
Figure 1. By iteratively alternating the two optimization processes, [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Performance in benchmark settings on the BERT [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Performance in practical settings on the BERT [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 4
Figure 4. Figure 4: Performance in practical settings with different dropout rates. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Performance in practical settings against defense by different gradient noise levels. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 43 canonical work pages

  1. [1]

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security (CCS). 308–318

    work page 2016

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  3. [3]

    Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, et al. 2017. Privacy- preserving deep learning via additively homomorphic encryption.IEEE transac- tions on information forensics and security13, 5 (2017), 1333–1345

    work page 2017

  4. [4]

    Mislav Balunovic, Dimitar Dimitrov, Nikola Jovanović, and Martin Vechev. 2022. Lamp: Extracting text from gradients with language model priors. Inthe 2022 Advances in Neural Information Processing Systems (NeurIPS). 7641–7654

    work page 2022

  5. [5]

    Dongqi Cai, Shangguang Wang, Yaozong Wu, Felix Xiaozhu Lin, and Mengwei Xu. 2023. Federated few-shot learning for mobile NLP. InProceedings of the 2023 Annual International Conference on Mobile Computing and Networking (MobiCom). 1–17

    work page 2023

  6. [6]

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022. Membership inference attacks from first principles. In Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP). 1897–1914

    work page 2022

  7. [7]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. InProceedings of the 2021 USENIX Security Symposium (USENIX Security). 2633–2650

    work page 2021

  8. [8]

    Jieren Deng, Yijue Wang, Ji Li, Chao Shang, Hang Liu, Sanguthevar Rajasekaran, and Caiwen Ding. 2021. Tag: Gradient attack on transformer-based language models.arXiv preprint arXiv:2103.06819(2021)

    arXiv 2021

Show all 56 references
  1. [9]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805(2018)

    2018 arXiv

  2. [10]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recogn...

    2021

  3. [11]

    FedML. 2023. Releasing FedLLM: Build Your Own Large Language Models on Proprietary Data using the FedML Platform. https://blog.fedml.ai/releasing- fedllm-build-your-own-large-language-models-on-proprietary-data-using- the-fedml-platform/. Accessed: January, 2024

    2023

  4. [12]

    Liam Fowl, Jonas Geiping, Steven Reich, Yuxin Wen, Wojtek Czaja, Micah Gold- blum, and Tom Goldstein. 2022. Decepticons: Corrupted transformers breach privacy in federated learning for language models. InProceedings of the 2022 International Conference on Learning Representati...

    2022

  5. [13]

    Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. 2020. Inverting gradients-how easy is it to break privacy in federated learning?. Inthe 2020 Advances in Neural Information Processing Systems (NeurIPS). 16937–16947

    2020

  6. [14]

    Jonas Geiping, Liam Fowl, and Yuxin Wen. 2023. breaching. https://github.com/ JonasGeiping/breaching. Assessed: August 2023

    2023

  7. [15]

    Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, and Danqi Chen. 2022. Recovering private text in federated learning of language models. In the 2022 Advances in Neural Information Processing Systems (NeurIPS). 8130–8143

    2022

  8. [16]

    Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co- adaptation of feature detectors.arXiv preprint arXiv:1207.0580(2012)

    2012 arXiv

  9. [17]

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. InProceedings of the 2020 International Conference on Learning Representations (ICLR)

    2020

  10. [18]

    Pingyi Hu, Zihan Wang, Ruoxi Sun, Hu Wang, and Minhui Xue. 2022. M 4 I: Multi-modal Models Membership Inference. Inthe 2022 Advances in Neural Information Processing Systems (NeurIPS). 1867–1882

    2022

  11. [19]

    Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora. 2021. Evaluating gradient inversion attacks and defenses in federated learning. Inthe 2021 Advances in Neural Information Processing Systems (NeurIPS). 7232–7241

    2021

  12. [20]

    Yangsibo Huang, Zhao Song, Kai Li, and Sanjeev Arora. 2020. Instahide: Instance- hiding schemes for private distributed learning. InProceedings of the 2020 Inter- national Conference on Machine Learning (ICML). 4507–4518

    2020

  13. [21]

    Huggingface. 2022. Tweet Sentiment Extraction. https://huggingface.co/datasets/ SetFit/tweet_sentiment_extraction. Accessed: January, 2024

    2022

  14. [22]

    Huggingface. 2024. Yahoo Answers Topics. https://huggingface.co/datasets/ community-datasets/yahoo_answers_topics. Accessed: January, 2024

    2024

  15. [23]

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks.The Journal of the Acoustical Society of America62, S1 (1977), S63–S63

    1977

  16. [24]

    Jinwoo Jeon, Kangwook Lee, Sewoong Oh, Jungseul Ok, et al . 2021. Gradient inversion with generative image prior. Inthe 2021 Advances in Neural Information Processing Systems (NeurIPS). 29898–29908

    2021

  17. [25]

    Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2019. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351(2019)

    2019 arXiv

  18. [26]

    Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2020. A survey on deep learning for named entity recognition.IEEE transactions on knowledge and data engineering34, 1 (2020), 50–70

    2020

  19. [27]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. InText Summarization Branches Out. 74–81

    2004

  20. [28]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 (2019)

    2019 arXiv

  21. [29]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. InProceedings of the 2019 International Conference on Learning Representations (ICLR)

    2019

  22. [30]

    Jiahao Lu, Xi Sheryl Zhang, Tianli Zhao, Xiangyu He, and Jian Cheng. 2022. APRIL: Finding the Achilles’ Heel on Privacy for Vision Transformers. InProceed- ings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10051–10060

    2022

  23. [31]

    Mengyao Ma, Yanjun Zhang, Pathum Chamikara Mahawaga Arachchige, Leo Yu Zhang, Mohan Baruwal Chhetri, and Guangdong Bai. 2023. Loden: Making every client in federated learning a defender against the poisoning membership inference attacks. InProceedings of the 2023 ACM Asia Conf...

    2023

  24. [32]

    Brian W Matthews. 1975. Comparison of the predicted and observed secondary structure of T4 phage lysozyme.Biochimica et Biophysica Acta (BBA)-Protein Structure405, 2 (1975), 442–451

    1975

  25. [33]

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep net- works from decentralized data. InProceedings of the 2017 International Conference on Artificial Intelligence and Statistics (AISTATS). 1273–1282

    2017

  26. [34]

    Walaa Medhat, Ahmed Hassan, and Hoda Korashy. 2014. Sentiment analysis algorithms and applications: A survey.Ain Shams engineering journal5, 4 (2014), 1093–1113

    2014

  27. [35]

    Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019. Exploiting unintended feature leakage in collaborative learning. InProceedings of the 2019 IEEE Symposium on Security and Privacy (SP). 691–706

    2019

  28. [36]

    Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg- Kirkpatrick, and Reza Shokri. 2022. Quantifying Privacy Risks of Masked Lan- guage Models Using Membership Inference Attacks. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Pro...

    2022

  29. [37]

    Nikita Munot and Sharvari S Govilkar. 2014. Comparative study of text sum- marization methods.International Journal of Computer Applications102, 12 (2014)

    2014

  30. [38]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Inthe 2022 Advances in Neural Information Processing...

    2022

  31. [39]

    Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales.arXiv preprint cs/0506075 (2005)

    2005 arXiv

  32. [40]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners.OpenAI blog 1, 8 (2019), 9

    2019

  33. [41]

    Google Research. 2020. TensorFlow code and pre-trained models for BERT. https://github.com/google-research/bert. Accessed: January 2024

    2020

  34. [42]

    2010.Artificial intelligence a modern approach

    Stuart J Russell. 2010.Artificial intelligence a modern approach. Pearson Education, Inc

    2010

  35. [43]

    Daniel Scheliga, Patrick Mäder, and Marco Seeland. 2023. Dropout is NOT All You Need to Prevent Gradient Leakage. InProceedings of the 2023 Association for the Advancement of Artificial Intelligence (AAAI). 9733–9741

    2023

  36. [44]

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Mem- bership inference attacks against machine learning models. InProceedings of the 2017 IEEE Symposium on Security and Privacy (SP). 3–18

    2017

  37. [45]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. InProceedings of the 2013 conference on Empirical Methods in Natural Language ...

    2013

  38. [46]

    Yuanyishu Tian, Yao Wan, Lingjuan Lyu, Dezhong Yao, Hai Jin, and Lichao Sun

  39. [47]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Inthe 2017 Advances in Neural Information Processing Systems (NeurIPS)

    2017

  40. [48]

    Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2019. Neural net- work acceptability judgments.Transactions of the Association for Computational Uncovering Gradient Inversion Risks in Practical Language Model Training CCS ’24, October 14–18, 2024, Salt Lake City, UT, USA ...

    2019

  41. [49]

    Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Mehmet Emre Gursoy, Stacey Truex, and Yanzhao Wu. 2020. A framework for evaluating gradient leakage attacks in federated learning.arXiv preprint arXiv:2004.10397(2020)

    2020 arXiv

  42. [50]

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy risk in machine learning: Analyzing the connection to overfitting. InProceedings of the 2018 IEEE Computer Security Foundations Symposium (CSF). 268–282

    2018

  43. [51]

    Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M Alvarez, Jan Kautz, and Pavlo Molchanov. 2021. See through gradients: Image batch recovery via gradinversion. InProceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16337–16346

    2021

  44. [52]

    Haisong Zhang. 2020. roberta-tiny-cased. https://github.com/haisongzhang/ roberta-tiny-cased. Accessed: January 2024

    2020

  45. [53]

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2018. mixup: Beyond empirical risk minimization. InProceedings of the 2018 Interna- tional Conference on Learning Representations (ICLR)

    2018

  46. [54]

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. idlg: Improved deep leakage from gradients.arXiv preprint arXiv:2001.02610(2020)

    2020 arXiv

  47. [55]

    Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. In the 2019 Advances in Neural Information Processing Systems (NeurIPS). A Supplementary Tables A.1 FL Setup Table 8 shows the result of GRAB in benchmark settings on the CoLA dataset with batch sizes fr...

    2019

  48. [2022]

    Fedbert: When federated learning meets pre-training.ACM Transactions on Intelligent Systems and Technology (TIST)13, 4 (2022), 1–26

    2022

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.