Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation

Chanhee Park; Heuiseok Lim; Hyeonseok Moon; Jeongho Yoon; Yongchan Chun

REVIEW 2 major objections 1 minor 7 references

Privacy-Preserving Fine-Tuning lets large language models adapt and infer from sensitive data using only k-pooled embeddings instead of raw text.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-10 17:24 UTC

load-bearing objection PPFT's two-stage embedding pipeline is a practical attempt at text-free LLM inference but the privacy protection stops at training and does not cover the embeddings sent at inference time. the 2 major comments →

arxiv 2604.06831 v1 submitted 2026-04-08 cs.CR cs.AI

Towards Privacy-Preserving Large Language Model: Text-free Inference Through Alignment and Adaptation

Jeongho Yoon , Chanhee Park , Yongchan Chun , Hyeonseok Moon , Heuiseok Lim This is my paper

classification cs.CR cs.AI

keywords privacy-preserving fine-tuninglarge language modelstext-free inferenceprompt embeddingsnoise injectiondomain adaptationalignmentprivacy utility balance

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents PPFT as a two-stage process to remove the need for users to send raw prompts to LLM services. In the first stage a client encoder is trained with a server projection module so the LLM can condition on pooled embeddings. The second stage adds noise to those embeddings while fine-tuning the projection and LLM on private domain data. This setup aims to protect text from reconstruction while keeping model performance close to standard fine-tuning. A reader would care because current LLM use requires transmitting personal or sensitive text, creating exposure risks that this method seeks to avoid.

Core claim

PPFT first trains a client-side encoder together with a server-side projection module and LLM so the server conditions on k-pooled prompt embeddings rather than raw text. It then fine-tunes the projection module and LLM on private domain-specific data by injecting noise into the embeddings, enabling adaptation without exposing plain-text prompts and without requiring access to the decoder's internal parameters.

What carries the argument

PPFT's two-stage pipeline that replaces raw text with k-pooled prompt embeddings during both alignment and noise-injected adaptation.

Load-bearing premise

The k-pooled embeddings from the client encoder must retain enough semantic information for effective LLM conditioning even after noise injection during fine-tuning.

What would settle it

An experiment in which an adversary reconstructs original prompt text from the k-pooled embeddings at high accuracy, or in which downstream task performance falls substantially below the noise-free baseline, would show the approach fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Clients can submit sensitive prompts for LLM use without ever transmitting the plain text.
Service providers can adapt the model to private domains using only noisy embeddings from that data.
Performance on domain-specific and general benchmarks stays close to non-private upper bounds.
The method avoids the heavy computational costs of earlier privacy techniques that still require text transmission.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same embedding-plus-noise pattern could be tested on multimodal models that handle images or audio alongside text.
If the noise level can be tuned per task, the method might allow different privacy-utility points for different users.
Wider adoption would reduce the amount of raw user text that must be stored or logged by LLM providers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

PPFT's two-stage embedding pipeline is a practical attempt at text-free LLM inference but the privacy protection stops at training and does not cover the embeddings sent at inference time.

read the letter

The paper lays out PPFT as a two-stage process: first align a client encoder and server projection module so the LLM can condition on k-pooled embeddings rather than raw text, then adapt the projection and model on private data by injecting noise into those embeddings. The setup avoids sending plain prompts and keeps the decoder internals off-limits during adaptation, which is a reasonable engineering choice compared with some heavier prior defenses. The reported experiments on domain benchmarks claim the performance drop stays small relative to a noise-free baseline, which would be the useful outcome if it holds up in the full results. That part of the work is concrete and targets a real deployment barrier for sensitive data. The soft spot is the inference stage. Noise appears only during the adaptation fine-tuning; once the system is live, the k-pooled embeddings are transmitted without added noise or other safeguards. If those embeddings carry enough semantic content to keep utility high, the same content could support reconstruction attacks on actual user prompts. The abstract gives no sign that inversion resistance was tested or that differential privacy bounds were measured for the deployed embeddings, so the headline privacy-utility balance rests on an unverified assumption. This is the kind of paper that belongs in a reading group focused on applied privacy for LLMs. Readers working on regulated services would get value from the pipeline description and the benchmark numbers, even if they need to dig into the inversion analysis themselves. It deserves peer review so the experimental details, choice of k, noise scale, and privacy metrics can be checked properly.

Referee Report

2 major / 1 minor

Summary. The paper proposes Privacy-Preserving Fine-Tuning (PPFT), a two-stage pipeline for text-free LLM inference. Stage 1 trains a client encoder with a server-side projection and LLM to condition on k-pooled prompt embeddings rather than raw text. Stage 2 fine-tunes the projection and LLM on private domain data using noise-injected embeddings. The central claim is that this yields a favorable privacy-utility tradeoff, with experiments on domain-specific and general benchmarks showing competitive performance and only minimal degradation relative to noise-free baselines.

Significance. If the privacy guarantees hold at inference time and the empirical results are reproducible with clear metrics, PPFT would address a practical gap in deploying LLMs for sensitive applications by avoiding raw-text transmission while preserving utility. The approach is grounded in standard embedding alignment and noise mechanisms rather than novel cryptographic primitives, so its value lies in the empirical demonstration of the balance rather than theoretical novelty.

major comments (2)

Abstract (two-stage description): Noise injection is described only for Stage 2 fine-tuning on private data; the deployed inference path transmits k-pooled embeddings with no indicated protection. Because these embeddings are required to retain sufficient semantics for competitive LLM conditioning, the same information may enable prompt reconstruction by an adversary observing the transmitted embeddings, undermining the privacy claim for actual user queries. This is load-bearing for the headline privacy-utility balance asserted for the deployed system.
Abstract (experimental claims): The abstract asserts 'minimal degradation' and 'striking balance' on domain-specific and general benchmarks but supplies no concrete metrics, baselines, statistical significance tests, or data-exclusion rules. Without these details the central empirical claim cannot be evaluated, which is load-bearing because the paper's contribution is presented as an empirical training procedure rather than a theoretical derivation.

minor comments (1)

Notation: The parameter k (number of pooled embeddings) and the noise scale are introduced without explicit definitions or ranges in the abstract; these should be stated clearly when first used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the insightful comments, which help clarify the presentation of our privacy claims and empirical results. We address each major comment below and indicate the planned revisions.

read point-by-point responses

Referee: Abstract (two-stage description): Noise injection is described only for Stage 2 fine-tuning on private data; the deployed inference path transmits k-pooled embeddings with no indicated protection. Because these embeddings are required to retain sufficient semantics for competitive LLM conditioning, the same information may enable prompt reconstruction by an adversary observing the transmitted embeddings, undermining the privacy claim for actual user queries. This is load-bearing for the headline privacy-utility balance asserted for the deployed system.

Authors: We agree that the distinction between training and inference protection requires explicit clarification. PPFT's privacy guarantee centers on never transmitting raw text: Stage 1 alignment enables the server to operate on k-pooled embeddings, and Stage 2 uses noise only to adapt to private data without exposing that data. At inference, embeddings are sent without noise to maintain utility. We acknowledge that semantically rich embeddings could in principle be subject to inversion attacks. In the revision we will (1) state the threat model precisely (honest-but-curious server receives only aligned embeddings, not raw text), (2) add a limitations paragraph discussing reconstruction risk and why the learned projection may increase inversion difficulty, and (3) note that stronger inference-time protections (e.g., additional noise or cryptographic layers) are left for future work. This does not alter the reported utility results but makes the privacy claim accurate. revision: partial
Referee: Abstract (experimental claims): The abstract asserts 'minimal degradation' and 'striking balance' on domain-specific and general benchmarks but supplies no concrete metrics, baselines, statistical significance tests, or data-exclusion rules. Without these details the central empirical claim cannot be evaluated, which is load-bearing because the paper's contribution is presented as an empirical training procedure rather than a theoretical derivation.

Authors: We concur that the abstract should contain quantitative anchors. The body of the paper reports concrete numbers (performance within a few percentage points of the noise-free upper bound on both domain-specific and general benchmarks, with standard deviations from multiple runs and explicit baselines), but these were omitted from the abstract for brevity. In the revised abstract we will insert the key figures (e.g., “within 3 % of the full-text baseline on medical and legal tasks”) together with a brief statement of the evaluation protocol. This change directly addresses the referee’s concern without affecting any technical content. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical two-stage training pipeline

full rationale

The paper describes PPFT as an empirical training procedure consisting of an initial alignment stage (encoder + projection + LLM trained on non-private data to enable k-pooled embedding conditioning) followed by a noise-injected fine-tuning stage on private data. No equations, first-principles derivations, or predictions are presented that reduce claimed performance or privacy properties to fitted parameters by construction. All utility and privacy assertions rest on experimental benchmark results rather than any self-referential loop. Any self-citations that may exist are not load-bearing for the core method, which remains self-contained against external evaluation.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The approach rests on standard assumptions about embedding semantics and differential privacy via noise, with no new entities postulated; free parameters such as pooling count k and noise scale are implicit but not quantified in the abstract.

free parameters (2)

k (number of pooled embeddings)
Controls information compression from prompt to server input; value chosen to balance privacy and utility but not reported.
noise injection scale
Determines privacy strength versus performance drop during adaptation; tuned empirically.

axioms (2)

domain assumption Pooled embeddings preserve sufficient task-relevant semantics for LLM conditioning
Invoked in the first training stage to justify replacing raw text.
domain assumption Noise-injected embeddings allow effective domain adaptation without raw text exposure
Core premise of the second stage.

pith-pipeline@v0.9.0 · 5515 in / 1196 out tokens · 63598 ms · 2026-05-10T17:24:00.203288+00:00 · methodology

0 comments

read the original abstract

Current LLM-based services typically require users to submit raw text regardless of its sensitivity. While intuitive, such practice introduces substantial privacy risks, as unauthorized access may expose personal, medical, or legal information. Although prior defenses strived to mitigate these risks, they often incur substantial computational overhead and degrade model performance. To overcome this privacy-efficiency trade-off, we introduce Privacy-Preserving Fine-Tuning (PPFT), a novel training pipeline that eliminates the need for transmitting raw prompt text while maintaining a favorable balance between privacy preservation and model utility for both clients and service providers. Our approach operates in two stages: first, we train a client-side encoder together with a server-side projection module and LLM, enabling the server to condition on k-pooled prompt embeddings instead of raw text; second, we fine-tune the projection module and LLM on private, domain-specific data using noise-injected embeddings, allowing effective adaptation without exposing plain text prompts and requiring access to the decoder's internal parameters. Extensive experiments on domain-specific and general benchmarks demonstrate that PPFT achieves a striking balance between privacy and utility, maintaining competitive performance with minimal degradation compared to noise-free upper bounds.

Figures

Figures reproduced from arXiv: 2604.06831 by Chanhee Park, Heuiseok Lim, Hyeonseok Moon, Jeongho Yoon, Yongchan Chun.

**Figure 2.** Figure 2: Overview of PPFT. Stage 1 aligns pooled client-side embeddings with the decoder to enable text-free [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: Results of embedding inversion attacks and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

**Figure 5.** Figure 5: Reconstruction performance with and without noise injection. Overall, the pooling size k acts as a key control knob that jointly regulates communication efficiency and the performance–privacy balance. 5.2 Effect of Noise Types [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

**Figure 6.** Figure 6: Inversion attacks on PPFT using a Stage-1 [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗

**Figure 7.** Figure 7: Reconstruction quality across iterations for [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

7 extracted references · 7 canonical work pages

[1]

Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro

Differentially private low-rank adaptation of large language model using federated learning.ACM Transactions on Management Information Systems, 16(2):1–24. Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro

work page
[2]

Split-and-denoise: Protect large language model inference with local differential privacy

Chatqa: Surpassing gpt-4 on conversational qa and rag.Advances in Neural Information Processing Systems, 37:15416–15459. Peihua Mai, Ran Yan, Zhe Huang, Youjia Yang, and Yan Pang. 2023. Split-and-denoise: Protect large lan- guage model inference with local differential privacy. arXiv preprint arXiv:2310.09130. Abhijit Mishra, Mingda Li, and Soham Deo. 202...

work page arXiv 2023
[3]

InFindings of the Association for Computational Linguistics: ACL 2025, pages 26196– 26220

Protecting users from themselves: Safeguard- ing contextual privacy in interactions with conver- sational agents. InFindings of the Association for Computational Linguistics: ACL 2025, pages 26196– 26220. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi- subject multi-choice dataset for medical domain ques...

work page arXiv 2025
[4]

Advances in Neural Information Processing Systems, 37:90629–90660

Mammoth2: Scaling instructions from the web. Advances in Neural Information Processing Systems, 37:90629–90660. Ziqian Zeng, Jianwei Wang, Junyao Yang, Zhengdong Lu, Haoran Li, Huiping Zhuang, and Cen Chen. 2025. Privacyrestore: Privacy-preserving inference in large language models via privacy removal and restoration. InProceedings of the 63rd Annual Meet...

work page 2025
[5]

Universal Zero-shot Embedding Inversion,

Universal zero-shot embedding inversion. arXiv preprint arXiv:2504.00147. Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, and Xuan-Jing Huang. 2023. Textobfuscator: Making pre-trained language model a privacy protector via obfuscating word representations. InFindings of the Associa- tion for Computational Linguistics: AC...

work page arXiv 2023
[6]

instruction:

(SFT split) •rajpurkar/squad(Rajpurkar et al., 2016) •tau/commonsense_qa(Talmor et al., 2019) B.3 Stage 2: Domain Adaptation Corpora Stage 2 adapts the aligned model to sensitive do- mains (medical and legal) while preserving the text- free training interface. To strengthen MCQA be- havior for both decoders, we additionally include pszemraj/unified-mcqa. ...

work page 2016
[7]

has a history of

The attacker is trained on the corresponding training split and evaluated on the test split. Attack on PPFT (ours).For PPFT, the attacker operates on the same noisy, pooled embedding rep- resentation that is exposed to the server. Specifi- cally, we reuse the encoder and k-pooling module from the Stage 1-aligned LLaMA-1B PPFT model Pooling sizeϵ=0.01ϵ=0.4...

work page arXiv 2025

[1] [1]

Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro

Differentially private low-rank adaptation of large language model using federated learning.ACM Transactions on Management Information Systems, 16(2):1–24. Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro

work page

[2] [2]

Split-and-denoise: Protect large language model inference with local differential privacy

Chatqa: Surpassing gpt-4 on conversational qa and rag.Advances in Neural Information Processing Systems, 37:15416–15459. Peihua Mai, Ran Yan, Zhe Huang, Youjia Yang, and Yan Pang. 2023. Split-and-denoise: Protect large lan- guage model inference with local differential privacy. arXiv preprint arXiv:2310.09130. Abhijit Mishra, Mingda Li, and Soham Deo. 202...

work page arXiv 2023

[3] [3]

InFindings of the Association for Computational Linguistics: ACL 2025, pages 26196– 26220

Protecting users from themselves: Safeguard- ing contextual privacy in interactions with conver- sational agents. InFindings of the Association for Computational Linguistics: ACL 2025, pages 26196– 26220. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi- subject multi-choice dataset for medical domain ques...

work page arXiv 2025

[4] [4]

Advances in Neural Information Processing Systems, 37:90629–90660

Mammoth2: Scaling instructions from the web. Advances in Neural Information Processing Systems, 37:90629–90660. Ziqian Zeng, Jianwei Wang, Junyao Yang, Zhengdong Lu, Haoran Li, Huiping Zhuang, and Cen Chen. 2025. Privacyrestore: Privacy-preserving inference in large language models via privacy removal and restoration. InProceedings of the 63rd Annual Meet...

work page 2025

[5] [5]

Universal Zero-shot Embedding Inversion,

Universal zero-shot embedding inversion. arXiv preprint arXiv:2504.00147. Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, and Xuan-Jing Huang. 2023. Textobfuscator: Making pre-trained language model a privacy protector via obfuscating word representations. InFindings of the Associa- tion for Computational Linguistics: AC...

work page arXiv 2023

[6] [6]

instruction:

(SFT split) •rajpurkar/squad(Rajpurkar et al., 2016) •tau/commonsense_qa(Talmor et al., 2019) B.3 Stage 2: Domain Adaptation Corpora Stage 2 adapts the aligned model to sensitive do- mains (medical and legal) while preserving the text- free training interface. To strengthen MCQA be- havior for both decoders, we additionally include pszemraj/unified-mcqa. ...

work page 2016

[7] [7]

has a history of

The attacker is trained on the corresponding training split and evaluated on the test split. Attack on PPFT (ours).For PPFT, the attacker operates on the same noisy, pooled embedding rep- resentation that is exposed to the server. Specifi- cally, we reuse the encoder and k-pooling module from the Stage 1-aligned LLaMA-1B PPFT model Pooling sizeϵ=0.01ϵ=0.4...

work page arXiv 2025