Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SecFPP is a federated prompt-personalization protocol that claims to provide strong privacy against honest-but-curious servers and colluding users while matching non-private personalization accuracy, by revealing only secret-shared…

desk verdict A genuinely new secure clustering protocol; the privacy proof only covers a single distance to one center, not the full distance matrix the server sees every round. read the letter →

arxiv 2505.22447 v1 pith:ZAO6OFDM submitted 2025-05-28 cs.CR

Sizai Hou , Songze Li , Baturalp Buyukates This is my paper
classification cs.CR
keywords federatedlearningpromptpersonalizationmultimodallargelanguagemodelsprivacypreservationsecretsharingLagrangecodedcomputationadaptiveclusteringdifferential
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SecFPP is a protocol for federated prompt personalization in multimodal language models that aims to protect a user's personalized prompt from an honest-but-curious server and colluding users while preserving personalization accuracy. The paper argues that a hierarchical prompt decomposition—a global component adapted to domain-level heterogeneity and a local component kept private on-device—can handle multi-granular data imbalance without the accuracy collapse caused by differential privacy. Its central move is to reveal only secret-shared distances from reduced prompts to cluster centers, so the server can cluster users by domain without ever seeing the prompts themselves. The paper claims this yields strong information-theoretic protection, with mutual information between a prompt and the observed distance exponentially smaller than the prompt's entropy, while matching or exceeding non-private baselines on several heterogeneous image classification benchmarks.

What carries the argument

The central object is SecPC, a privacy-preserving adaptive clustering algorithm that combines Lagrange coded computation (LCC) with a one-shot adaptive k-means update. Each user slices its reduced prompt, generated by truncated SVD/PCA, and secret-shares it among the other users; every user computes coded squared distances from each prompt to each cluster center, and the server reconstructs the distances by Reed-Solomon decoding and recovers the true distances by rescaling. These distances are the only prompt-dependent information the server sees; from them it updates cluster assignments, and group-wise secure aggregation trains the domain-level global prompts. The protocol's privacy analysis centers on Theorem 1, which gives an expression for the mutual information between a single reduced prompt and the squared distance to its cluster center under Gaussian assumptions, supported empirically by KSG estimates showing $\log_{10}(h(P_i)/MI(P_i;D^2)) > 4$.

What would settle it

A concrete test would be to run SecFPP's distance-revelation step on real learned soft prompts from a heterogeneous federation and estimate $MI(P_i;D^2)$ with KSG or a neural estimator; if the ratio $h(P_i)/MI(P_i;D^2)$ drops below the claimed exponential gap (log10 ratio below 4) for non-Gaussian prompts or after several rounds, the information-theoretic privacy claim would be falsified for practical use.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that privacy and personalization performance are not in fundamental conflict in federated prompt learning. SecFPP decomposes each user's prompt into a cluster-level global prompt and a private local prompt, clustering users with a secret-sharing-based adaptive k-means algorithm, SecPC, built on Lagrange coded computation. Because only coded distances are reconstructed by the server, the global prompt is learned group-wise without exposing the reduced prompt vectors, and the local prompt never leaves the device. The paper reports that this matches or exceeds non-private FPP methods such as FedOTP and FedGPG while greatly outperforming DP-FPL, whose accuracy drops as the privacy budget tightens; on a multi-domain split like Caltech-101+Food-101, SecFPP reaches 87.6% versus 65.7% for DP-FPL with strict privacy.

Load-bearing premise

The privacy guarantee assumes real user prompts are well modeled by zero-mean Gaussian random vectors, so the revealed distance to a cluster center carries negligible information about the prompt, and that repeated rounds of distance observations do not accumulate extra leakage.

Editorial extensions

If this is right

  • If SecFPP works as claimed, federated prompt personalization can provide meaningful privacy without the personalization-performance collapse that DP noise causes in lightweight prompts.
  • Domain-level adaptation can be made private through clustering on distances rather than on prompt vectors, so a server can group users by underlying data domain without seeing user-specific representations.
  • The protocol's overhead is small enough for practice—user communication per round is $O(nd/\ell + kn)$ and measured user computation stays under 0.4 seconds even at dimension 4000—so privacy need not be a bottleneck.
  • The mutual-information analysis suggests that scalar distance observations leak exponentially less than a cluster center vector, supporting the design choice of revealing only distances.
  • On heterogeneous multi-domain benchmarks, SecFPP is reported to surpass privacy-preserving baselines and match non-private ones, making it the first FPP method with strong privacy and no accuracy penalty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper validates privacy empirically only for synthetic i.i.d. Gaussian prompts in a single round; extending the mutual-information analysis to multi-round, correlated real prompts and to the clustering assignment itself would test whether the information-theoretic guarantee survives practice.
  • Because Theorem 1 gives an expression rather than a bound, a concrete next step is to compute or bound mutual information for non-Gaussian prompt distributions, such as learned soft prompts after gradient updates, and under repeated distance observations.
  • The protocol's accuracy evaluation covers CLIP-based image classification; transferring SecFPP to text-only MLLMs or generation tasks would test whether the hierarchical prompt decomposition and distance-based clustering generalize beyond vision.
  • The cluster assignment itself is revealed to the server and could leak domain-level membership information, such as which dataset a user's data resembles; this channel is not explicitly quantified in the current analysis.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes SecFPP, a federated prompt personalization (FPP) protocol for multimodal LLMs that splits each user's prompt into a cluster-level global component and a local component. Clustering of dimension-reduced personalized prompts is performed with a secret-sharing-based adaptive k-means protocol (SecPC) built on Lagrange coded computation (LCC), so the server observes only reconstructed squared distances to cluster centers rather than raw prompts. The authors claim that this gives strong information-theoretic privacy without the accuracy loss incurred by differential privacy, and they report accuracy on single- and multi-domain image classification tasks with CLIP. The paper also presents a mutual-information analysis (Theorem 1), empirical MI estimates for synthetic Gaussian prompts, and complexity measurements.

Significance. If the privacy claim were sound, SecFPP would be a valuable addition to the FPP literature: it would offer a privacy-preserving alternative to DP-based FPP without sacrificing personalization accuracy, and the protocol design is concrete and implementable. The paper's strengths include a detailed protocol description (Algorithms 1-2), a formal-looking MI derivation for a single distance observation, a complexity analysis in Table 3, and a broad empirical comparison across seven datasets and several multi-domain splits. The LCC-based secure aggregation and clustering machinery is well motivated, and the overhead measurements in Section 5.4 suggest the privacy cost is modest. However, the central privacy guarantee is not established for the actual protocol, because the analysis covers only a single distance under a Gaussian prompt model, while the server receives a full distance matrix every round; and the accuracy claims in the abstract are stronger than Table 2 supports. The paper would be acceptable only after these load-bearing gaps are addressed.

major comments (5)
  1. [Section 4.3, Theorem 1, and Section 5.3] The privacy analysis analyzes MI(P_i; D^2) for a single squared distance under the assumption that prompt entries are i.i.d. Gaussian. The actual protocol (Algorithm 1, Phase 3) reveals the full n×k distance matrix {d_{i,s}^{(t)}} in every round t, and users appear across T=100 rounds. The theorem is silent about MI(P_i; {d_{i,s}^{(t)}}_{t,s}), and the empirical MI study in Section 5.3 also considers only a single distance. This is load-bearing because if k ≥ d+1 and the cluster centers affinely span R^d, classical multidimensional scaling can reconstruct all reduced prompts from one distance matrix up to a global isometry, and repeated rounds can align that isometry. The claim in Section 5.3 that SecFPP offers 'strong information-theoretic privacy guarantees' is therefore unsupported; the authors need either a multi-round joint-leakage analysis or an attack-based evaluation that accounts for the full revealed distance matrices.
  2. [Section 4.3, proof of Theorem 1] The theorem states that prompts are distributed as N_d(μ_i, σ_i) with general μ_i, but the proof treats h(D^2) as the entropy of a central chi-squared distribution when P_i is inside the cluster. For a general nonzero μ_i, P_i - P_avg has mean μ_i - μ_avg, which is not zero, so D^2 is non-central chi-squared. Similarly, in the outside-cluster case the proof sets the non-centrality parameter to τ_i = d·μ_i^2, ignoring μ_avg. Unless the authors intend to assume μ_i = 0 and μ_avg = 0, the derived expression in Eq. (3) is not valid under the theorem's stated assumptions. This matters because Theorem 1 is the only formal privacy argument in the paper.
  3. [Section 4.2, Algorithm 2, line 7] The text states that the local prompt component P_L,i is 'always privately kept by users' and that only P_i and P_G,s join the secret-sharing algorithms. However, Algorithm 2 line 7 applies PCA to the full personalized prompt P_i^(t+1) = P_G,s + P_L,i and the resulting reduced vector is used in SecPC. Therefore information derived from P_L,i is revealed to the server via the reconstructed distances. This contradicts the claim of local-prompt privacy and needs to be clarified or corrected, since the confidentiality of the local component is part of the paper's contribution.
  4. [Table 2 and Section 5.2] The paper claims that SecFPP 'significantly outperforms both non-private and privacy-preserving baselines,' but Table 2 does not report error bars, standard deviations, or significance tests, and the point estimates do not consistently favor SecFPP. For example, on CIFAR-10 SecFPP achieves 89.4 versus FedOTP's 89.6; on Oxford-Pet it achieves 86.3 versus FedOTP's 86.7; and on Caltech-101+Oxford-Pet it achieves 91.2 versus FedGPG's 91.4. Only a subset of multi-domain splits shows a clear advantage. The accuracy claim should be tempered to 'competitive or better on some multi-domain splits,' or the experiments need repeated-seed reporting with confidence intervals to support the stronger statement.
  5. [Section 5.3] The empirical MI evaluation is performed on synthetic i.i.d. Gaussian prompt vectors, not on the actual reduced prompts produced by the learned prompts in the image-classification experiments. Real learned prompts, after PCA, need not be zero-mean Gaussians, and the protocol's revealed quantities are observed over many rounds. The reported 10^4 gap between h(P_i) and MI(P_i; D^2) therefore does not establish that the actual revealed distance matrices leak negligible information. The authors should either measure MI on their real trained prompts or provide a reconstruction attack on the actual protocol outputs.
minor comments (6)
  1. [Table 1 and Table 2] The headline numbers 91.6 (single dataset) and 91.2 (multi-domain) in Table 1 do not directly match any single row in Table 2; the paper should explain how these aggregates are computed.
  2. [Algorithm 1, line 10] The computation of the squared Euclidean norm is performed inside a finite field, but Appendix A only describes element-wise quantization and overflow avoidance; the paper should specify how the ℓ2 norm is computed modulo q and how quantization error affects the revealed distances.
  3. [Theorem 1 and Appendix B] The notation 'Nodd' and 'Neven' is used in Eq. (3) without a definition; the paper should write N_odd and N_even or define the sets explicitly.
  4. [Figure 2] The figure caption says MI is plotted in log-scale, but the axis labels are not visible in the text; the authors should provide readable axis labels and, if possible, error bars for the KSG estimates.
  5. [Section 5.4] The communication-time calculation assumes a specific 98 Mbps bandwidth; the paper should state the assumed network model more precisely and include the per-round byte counts.
  6. [Front matter and references] The author block and copyright line contain '2018' and 'Anonymous Conference' placeholders; these should be updated before submission.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the accuracy results are direct empirical benchmarks against external baselines, the privacy analysis is a conditional derivation from an explicit Gaussian model, and the few self-citations are to off-the-shelf secure-computation primitives that do not carry the paper's conclusions.

full rationale

The paper's claim that SecFPP outperforms non-private and privacy-preserving baselines is supported by directly measured accuracy in Table 2 against external systems (PromptFL, FedOTP, FedGPG, DP-FPL); no accuracy number is obtained by fitting a parameter and then reporting a closely related quantity as a prediction. The privacy analysis in Theorem 1 is an independent information-theoretic calculation under the paper's explicitly stated Gaussian model, and the empirical MI study in Section 5.3 samples prompts from exactly that model and estimates MI with a standard KSG estimator; the resulting gap is a conditional demonstration, not a fitted input renamed as a prediction. The paper itself limits the claim in Remark 1 ('deriving a tight upper bound on this mutual information is intractable'), and Section 5.3's 'rigorous information-theoretic constraint' extends the synthetic-Gaussian result to practical settings without a proof; that is a scope/validity limitation, not circularity. The protocol invokes LCC (Yu et al.) and prior secure aggregation works [5, 23, 24, 54], some co-authored by the present authors, but these are externally published primitives used off-the-shelf; neither the performance comparison nor the MI derivation reduces to a self-citation chain. No equation in the paper is equivalent to its input by construction, and no uniqueness theorem is imported from the authors' prior work to force the design. The only self-citations are non-load-bearing references to established primitives, so they do not raise the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central privacy claim rests on the Gaussian prompt model and the LCC threshold assumption. The performance claim rests on the PCA rank and experimental hyperparameters. No new physical or mathematical entities are introduced.

free parameters (4)
  • PCA rank k = 8
    Dimensionality reduction rank for prompt clustering; hand-chosen, affects clustering quality and privacy.
  • LCC privacy threshold alpha = 1/3
    Fraction of colluding users tolerated; chosen for experiments, affects security.
  • Quantization scale lambda = 10^3
    Scaling factor for finite field quantization; implementation parameter.
  • Dirichlet beta = 0.3
    Heterogeneity parameter for data partition; experimental setting.
assumptions (4)
  • domain assumption Honest-but-curious adversary with no server-client collusion
    Section 3.2 states the threat model; security claims depend on this assumption.
  • ad hoc to paper Prompt entries are i.i.d. Gaussian for the MI analysis
    Section 4.3 assumes N(mu_i, sigma_i) i.i.d. entries; real prompts are not shown to satisfy this.
  • standard math LCC provides perfect secrecy within threshold
    Relies on Lagrange coded computing security properties from [65].
  • standard math KSG mutual information estimator is accurate at sample size 1000
    Section 5.3 relies on KSG estimator accuracy based on [19].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/ZAO6OFDM

@misc{pith2026250522447,
  author       = {Pith},
  title        = {Pith review of: Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZAO6OFDM}},
  note         = {Machine review of arXiv:2505.22447}
}
read the original abstract

Prompt learning is a crucial technique for adapting pre-trained multimodal language models (MLLMs) to user tasks. Federated prompt personalization (FPP) is further developed to address data heterogeneity and local overfitting, however, it exposes personalized prompts - valuable intellectual assets - to privacy risks like prompt stealing or membership inference attacks. Widely-adopted techniques like differential privacy add noise to prompts, whereas degrading personalization performance. We propose SecFPP, a secure FPP protocol harmonizing generalization, personalization, and privacy guarantees. SecFPP employs hierarchical prompt adaptation with domain-level and class-level components to handle multi-granular data imbalance. For privacy, it uses a novel secret-sharing-based adaptive clustering algorithm for domain-level adaptation while keeping class-level components private. While theoretically and empirically secure, SecFPP achieves state-of-the-art accuracy under severe heterogeneity in data distribution. Extensive experiments show it significantly outperforms both non-private and privacy-preserving baselines, offering a superior privacy-performance trade-off.

Figures

Figures reproduced from arXiv: 2505.22447 by the authors.

Figure 1
Figure 1. Workflow of SecFPP. On the right, users decompose prompts into global and local components; the global prompt [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The mutual information comparisons: distance [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The computational cost in seconds per round. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Privacy-Preserving Federated Prompt Tuning under Data Heterogeneity: A Subspace-Decomposed Expert Approach

    cs.CV 2026-07 conditional novelty 6.0 of 10

    FedSEPT uses low-rank shared expert prompts with on-device instance-aware fusion to improve the privacy–utility trade-off in federated prompt tuning.

Reference graph

Works this paper leans on

75 extracted references · 47 canonical work pages · cited by 1 Pith paper

  1. [1]

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 308–318

    2016

  2. [2]

    Sanjiv K Bhatia et al. 2004. Adaptive K-Means Clustering.. In FLAIRS. 695–699

    work page 2004

  3. [3]

    Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Prac- tical secure aggregation for privacy-preserving machine learning. In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . 1175–1191

    2017

  4. [4]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101 – Mining Discriminative Components with Random Forests. In European Conference on Computer Vision

    work page 2014

  5. [5]

    Baturalp Buyukates, Jinhyun So, Hessam Mahdavifar, and Salman Avestimehr

    work page

  6. [6]

    Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. 2023. Unleashing the potential of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735 (2023)

    arXiv 2023

  7. [7]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , and A. Vedaldi. 2014. Describing Textures in the Wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)

    work page 2014

  8. [8]

    Tianyu Cui, Hongxia Li, Jingya Wang, and Ye Shi. 2024. Harmonizing Gener- alization and Personalization in Federated Prompt Learning. In International Conference on Machine Learning . PMLR, 9646–9661

    work page 2024

Show all 75 references
  1. [9]

    Christian Darken and John Moody. 1990. Fast adaptive k-means clustering: some empirical results. In 1990 IJCNN international joint conference on neural networks . IEEE, 233–238

    1990

  2. [10]

    Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey. Comput. Surveys 57, 6 (2025), 1–39

    2025

  3. [11]

    Lea Demelius, Roman Kern, and Andreas Trügler. 2025. Recent advances of differential privacy in centralized deep learning: A systematic survey. Comput. Surveys 57, 6 (2025), 1–28

    2025

  4. [12]

    Wenlong Deng, Christos Thrampoulidis, and Xiaoxiao Li. 2024. Unlocking the potential of prompt-tuning in bridging generalized and personalized federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6087–6097

    2024

  5. [13]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv prepri...

    2020 arXiv

  6. [14]

    Jiacheng Du, Jiahui Hu, Zhibo Wang, Peng Sun, Neil Zhenqiang Gong, Kui Ren, and Chun Chen. 2024. SoK: On Gradient Leakage in Federated Learning. arXiv preprint arXiv:2404.05403 (2024)

    2024 arXiv

  7. [15]

    Minxin Du, Xiang Yue, Sherman SM Chow, Tianhao Wang, Chenyu Huang, and Huan Sun. 2023. Dp-forward: Fine-tuning and inference on language models with differential privacy in forward pass. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security ....

    2023

  8. [16]

    Cynthia Dwork. 2006. Differential privacy. In International colloquium on au- tomata, languages, and programming . Springer, 1–12

    2006

  9. [17]

    Kennedy Edemacu and Xintao Wu. 2024. Privacy preserving prompt engineering: A survey. Comput. Surveys (2024)

    2024

  10. [18]

    Xinguo Feng, Zhongkui Ma, Zihan Wang, Eu Joe Chegne, Mengyao Ma, Alsharif Abuadbba, and Guangdong Bai. 2024. Uncovering Gradient Inversion Risks in Practical Language Model Training. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security . ...

    2024

  11. [19]

    Weihao Gao, Sewoong Oh, and Pramod Viswanath. 2018. Demystifying fixed 𝑘-nearest neighbor information estimators. IEEE Transactions on Information Theory 64, 8 (2018), 5629–5661

    2018

  12. [20]

    Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller

  13. [21]

    Tao Guo, Song Guo, and Junxiao Wang. 2023. Pfedprompt: Learning personalized prompt for vision-language models in federated learning. In Proceedings of the ACM Web Conference 2023. 1364–1374

    2023

  14. [22]

    Tao Guo, Song Guo, Junxiao Wang, Xueyang Tang, and Wenchao Xu. 2023. Promptfl: Let federated participants cooperatively learn prompts instead of models–federated learning in age of foundation model. IEEE Transactions on Mobile Computing 23, 5 (2023), 5179–5194

    2023

  15. [23]

    Sizai Hou, Songze Li, Tayyebeh Jahani-Nezhad, and Giuseppe Caire. 2024. PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning. arXiv preprint arXiv:2407.08954 (2024)

    2024 arXiv

  16. [24]

    Tayyebeh Jahani-Nezhad, Mohammad Ali Maddah-Ali, Songze Li, and Giuseppe Caire. 2023. SwiftAgg+: Achieving asymptotically optimal communication loads in secure aggregation for federated learning. IEEE Journal on Selected Areas in Communications 41, 4 (2023), 977–989

    2023

  17. [25]

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual prompt tuning. In Euro- pean conference on computer vision . Springer, 709–727

    2022

  18. [26]

    Kedlaya and Christopher Umans

    Kiran S. Kedlaya and Christopher Umans. 2011. Fast polynomial factorization and modular composition. SIAM J. Comput. 40, 6 (2011), 1767–1802

    2011

  19. [27]

    Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. 2004. Estimating mutual information. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics 69, 6 (2004), 066138

    2004

  20. [28]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  21. [29]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)

    2021 arXiv

  22. [30]

    Fei-Fei Li, Marco Andreeto, Marc’Aurelio Ranzato, and Pietro Perona. 2022. Cal- tech 101. doi:10.22002/D1.20086

    2022 doi

  23. [31]

    Guanghao Li, Wansen Wu, Yan Sun, Li Shen, Baoyuan Wu, and Dacheng Tao

  24. [32]

    Hongxia Li, Wei Huang, Jingya Wang, and Ye Shi. 2024. Global and local prompts cooperation via optimal transport for federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 12151–12161

    2024

  25. [33]

    Zhuohang Li, Jiaxin Zhang, Luyang Liu, and Jian Liu. 2022. Auditing privacy defenses in federated learning via generative gradient leakage. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 10132– 10142

    2022

  26. [34]

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602 (2021)

    2021 arXiv

  27. [35]

    Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian. 2022. Prompt distribution learning. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition . 5206–5215

    2022

  28. [36]

    James MacQueen. 1967. Some methods for classification and analysis of multivari- ate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, Vol. 5. University of California press, 281–298

    1967

  29. [37]

    Stefan M Moser. 2020. Expected logarithm and negative integer moments of a noncentral𝜒 2-distributed random variable. Entropy 22, 9 (2020), 1048

    2020

  30. [38]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classifica- tion over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing . IEEE, 722–729

    2008

  31. [39]

    OpenAI. [n. d.]. OpenAI GPT Store. https://openai.com/index/introducing-the- gpt-store/

  32. [40]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition . IEEE, 3498–3505

    2012

  33. [41]

    Ivo Petrov, Dimitar I Dimitrov, Maximilian Baader, Mark Müller, and Martin Vechev. 2024. Dager: Exact gradient inversion for large language models. Ad- vances in Neural Information Processing Systems 37 (2024), 87801–87830

    2024

  34. [42]

    PromptBase. [n. d.]. PromptBase. https://promptbase.com

  35. [43]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  36. [44]

    Brian C Ross. 2014. Mutual information between discrete and continuous data sets. PloS one 9, 2 (2014), e87357. Anonymous Conference, , Sizai Hou, Songze Li, and Baturalp Buyukates

    2014

  37. [45]

    Reent Schlegel, Siddhartha Kumar, Eirik Rosnes, and Alexandre Graell i Amat

  38. [46]

    Adi Shamir. 1979. How to share a secret. Commun. ACM 22, 11 (1979), 612–613

    1979

  39. [47]

    Jiawei Shao, Yuchang Sun, Songze Li, and Jun Zhang. 2022. Dres-fl: Dropout- resilient secure federated learning for non-iid clients via secret data sharing. Advances in Neural Information Processing Systems 35 (2022), 10533–10545

    2022

  40. [48]

    Xinyue Shen, Yiting Qu, Michael Backes, and Yang Zhang. 2024. Prompt Stealing Attacks Against{Text-to-Image} Generation Models. In 33rd USENIX Security Symposium (USENIX Security 24) . 5823–5840

    2024

  41. [49]

    IEEE Transactions on Communications (2023)

    CodedPaddedFL and CodedSecAgg: Straggler mitigation and secure aggre- gation in federated learning. IEEE Transactions on Communications (2023)

    2023

  42. [50]

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anand- kumar, and Chaowei Xiao. 2022. Test-time prompt tuning for zero-shot gener- alization in vision-language models. Advances in Neural Information Processing Systems 35 (2022), 14274–14289

    2022

  43. [51]

    snackprompt. [n. d.]. snackprompt. https://snackprompt.com

  44. [52]

    Jinhyun So, Başak Güler, and A Salman Avestimehr. 2020. Byzantine-resilient secure federated learning. IEEE Journal on Selected Areas in Communications 39, 7 (2020), 2168–2181

    2020

  45. [53]

    Weiyan Shi, Ryan Shea, Si Chen, Chiyuan Zhang, Ruoxi Jia, and Zhou Yu. 2022. Just fine-tune twice: Selective differential privacy for large language models. arXiv preprint arXiv:2204.07667 (2022)

    2022 arXiv

  46. [54]

    Jinhyun So, Chaoyang He, Chien-Sheng Yang, Songze Li, Qian Yu, Ramy E Ali, Basak Guler, and Salman Avestimehr. 2022. Lightsecagg: a lightweight and versatile design for secure aggregation in federated learning. Proceedings of Machine Learning and Systems 4 (2022), 694–720

    2022

  47. [55]

    Linh Tran, Wei Sun, Stacy Patterson, and Ana Milanova. 2025. Privacy-Preserving Personalized Federated Prompt Learning for Multimodal Large Language Models. arXiv preprint arXiv:2501.13904 (2025)

    2025 arXiv

  48. [56]

    Minh Vu, Truc Nguyen, My T Thai, et al . 2024. Analysis of privacy leakage in federated large language models. In International Conference on Artificial Intelligence and Statistics. PMLR, 1423–1431

    2024

  49. [57]

    Jinhyun So, Basak Guler, and Salman Avestimehr. 2020. A scalable approach for privacy-preserving collaborative machine learning. Advances in Neural Informa- tion Processing Systems 33 (2020), 8054–8066

    2020

  50. [58]

    Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor. 2020. Federated learning with differential privacy: Algorithms and performance analysis. IEEE transactions on information forensics and security 15 (2020), 3454–3469

    2020

  51. [59]

    Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems 2, 1-3 (1987), 37–52

    1987

  52. [60]

    Yixin Wu, Rui Wen, Michael Backes, Pascal Berrang, Mathias Humbert, Yun Shen, and Yang Zhang. 2024. Quantifying privacy risks of prompts in visual prompt learning. In 33rd USENIX Security Symposium (USENIX Security 24) . 5841–5858

    2024

  53. [61]

    Binghui Wang, Jiayi Guo, Ang Li, Yiran Chen, and Hai Li. 2021. Privacy- preserving representation learning on graphs: A mutual information perspective. In Proceedings of the 27th acm sigkdd conference on knowledge discovery & data mining. 1667–1676

    2021

  54. [62]

    Xingpeng Xiao, Yaomin Zhang, Heyao Chen, Wenkun Ren, Junyi Zhang, and Jian Xu. 2025. A Differential Privacy-Based Mechanism for Preventing Data Leakage in Large Language Model Training. Academic Journal of Sociology and Management 3, 2 (2025), 33–42

    2025

  55. [63]

    Fu-En Yang, Chien-Yi Wang, and Yu-Chiang Frank Wang. 2023. Efficient model personalization in federated learning via client-specific prompt generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 19159– 19168

    2023

  56. [64]

    Haomiao Yang, Mengyu Ge, Dongyun Xue, Kunlan Xiang, Hongwei Li, and Rongxing Lu. 2023. Gradient leakage attacks in federated learning: Research frontiers, taxonomy and future directions. IEEE Network (2023)

    2023

  57. [65]

    Shuyin Xia, Daowan Peng, Deyu Meng, Changqing Zhang, Guoyin Wang, Elisa- beth Giem, Wei Wei, and Zizhong Chen. 2020. A fast adaptive k-means with no bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020)

    2020

  58. [66]

    Xi Sheryl Zhang, Weifan Guan, Jiahao Lu, Zhaopeng Qiu, Jian Cheng, Xian Wu, and Yefeng Zheng. 2024. GraphLeak: Patient Record Leakage through Gradients with Knowledge Graph. In Proceedings of the ACM Web Conference 2024 . 4706– 4716

    2024

  59. [67]

    Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610 (2020)

    2020 arXiv

  60. [68]

    Haodong Zhao, Wei Du, Fangqi Li, Peixuan Li, and Gongshen Liu. 2023. Fed- prompt: Communication-efficient and privacy-preserving prompt tuning in fed- erated learning. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 1–5

    2023

  61. [69]

    Avestimehr

    Qian Yu, Songze Li, Netanel Raviv, Seyed Mohammadreza Mousavi Kalan, Mahdi Soltanolkotabi, and Salman A. Avestimehr. 2019. Lagrange coded computing: Optimal design for resiliency, security, and privacy. In The 22nd International Conference on Artificial Intelligence and Statis...

    2019

  62. [70]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision 130, 9 (2022), 2337–2348

    2022

  63. [71]

    Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Advances in neural information processing systems 32 (2019). Privacy-preserving Prompt Personalization in Federated Learning for Multimodal Large Language Models Anonymous Conference, , Appendix A Additi...

    2019

  64. [73]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16816–16825

    2022

  65. [2020]

    Inverting gradients-how easy is it to break privacy in federated learning? Advances in neural information processing systems 33 (2020), 16937–16947

    2020

  66. [2023]

    arXiv preprint arXiv:2303.08678 (2023)

    Visual prompt based personalized federated learning. arXiv preprint arXiv:2303.08678 (2023)

    2023 arXiv

  67. [2024]

    IEEE Journal on Selected Areas in Information Theory (2024)

    LightVeriFL: A lightweight and verifiable secure aggregation for federated learning. IEEE Journal on Selected Areas in Information Theory (2024)

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.