{"id":"de0dd1bd-560c-4192-b36d-73c08862cdd3","arxiv_id":"2506.16196","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"POST tunes a soft prompt privately on a small distilled model and transfers it to the large model using a two-term public-data loss, with optional differential privacy.","lead":"This paper shows how to adapt a large language model to a private task without sending the private data to the model's host: tune a soft prompt on a small distilled copy, then transfer the prompt to the large model using public data. If it holds up, users get large-model quality with local data and much lower computing cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq 5's KL divergence over prompt-induced output differences is mathematically undefined; the central transfer loss cannot be evaluated as printed.","rationale":"The paper's central claim is that a soft prompt tuned on a small distilled model can be transferred to a larger LLM using only public data, with accuracy approaching full prompt tuning. The transfer step in Section 4.3 is the mechanism that must make this work, and Eq (5) is the formal statement of half of that mechanism. An undefined KL divergence over output differences is not merely a stylistic issue: a reader cannot implement the described method, and the empirical results cannot be tied to the stated objective without an unstated normalization or alternative loss. This is the most load-bearing concern I found because it sits at the core of the novel contribution. I note in the paper's favor that the authors provide code, extensive ablations, and an honest limitations section; in particular, the ablation in Table 6 independently supports the claim that distillation-induced feature alignment is important, and the runtime analysis is concrete. The reader's conditional verdict is appropriate: the paper deserves acceptance only if the transfer loss is either corrected in the text or shown to match a well-defined implemented objective. My concern is partially aligned with the reader's weakest assumption, since the reader also flags the transfer step, but the reader focuses on feature-space alignment and alpha selection rather than the mathematical validity of Eq (5). The proposed code inspection settles whether this concern is a presentational error or a genuine flaw in the stated method.","tokens_in":26909,"tokens_out":11272,"duration_ms":145584,"concrete_test":"Clone the released POST repository and inspect the implementation of the transfer loss corresponding to Eq (5). Run one reported configuration (e.g., Llama2-7b with sst2 as private data and tweet as public data from Table 1) with a breakpoint on the L₂ computation and check the arguments passed to the KL/divergence function. If the code applies softmax to the difference vectors, uses MSE/cosine on the differences, or otherwise converts the arguments into valid distributions, then Eq (5) as printed is a misstatement that should be corrected but the central method may still be sound. If the code passes raw output differences to kl_div, the run should raise an error or produce NaN, showing that the stated objective cannot be what produced the empirical results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 defines the transfer objective as L=(1−α)L₁+αL₂, with L₂ in Eq (5) written as Σ KLDiv((Φt(pt+x)−Φt(x)), (Φs(ps+x)−Φs(x))). As stated, this term is mathematically ill-posed: the two arguments are differences of model outputs, not probability distributions, so the Kullback–Leibler divergence does not apply. If Φ returns probability distributions, the difference vectors contain negative entries and do not sum to one; if Φ returns logits, KL divergence is not defined on raw logits without a softmax. No normalization or alternative divergence is specified in the paper. Because L₂ is described as the component that matters when the large model has strong zero-shot performance or the compressed model is weak, the claimed transfer mechanism cannot be reproduced from the paper alone. The released code may implement a valid surrogate (e.g., softmax over the differences, MSE, or cosine distance); in that case Eq (5) misdescribes the actual method. If no valid surrogate is used, the reported results are not grounded in the stated objective. This is a formal correctness risk distinct from the reader's alpha-selection concern, though both affect the transfer step.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes POST, a three-stage framework for privately adapting a hosted LLM: the provider distills the large model into a small student; the user tunes a soft prompt on the student with private data, optionally under differential privacy; and the provider transfers the prompt to the large model using only a small public dataset. The transfer step optimizes a weighted combination of two KL-based losses, one matching prompted outputs and one matching prompt-induced output changes. Experiments on Roberta-base, GPT2-XL, and Llama2-7b across five classification and two generation tasks report accuracies close to full prompt tuning, large gains over direct transfer and zero-shot, DP via PromptDPSGD, LiRA leakage reduction, runtime savings, and comparisons to DP-OPT and zero-shot transfer baselines.","tokens_in":27106,"tokens_out":5891,"duration_ms":67122,"significance":"If the headline results hold, POST is a useful contribution: it gives a concrete protocol for avoiding private-data exposure to LLM providers while retaining most of the utility of full prompt tuning, with substantial compute savings, and it is one of the first demonstrations that soft prompts can be transferred effectively using public data only. The evaluation is unusually broad: three model families, multiple private/public dataset pairs, DP and non-DP settings, generation tasks, membership-inference analysis, runtime accounting, and a large ablation suite. The authors also release code, which materially supports reproducibility. However, the two concerns below affect the exact formulation of the transfer objective and the validity of the reported selection protocol, so the contribution needs revision before the claims can be accepted as stated.","major_comments":[{"comment":"The loss L2 as printed is mathematically undefined: the arguments of KLDiv are differences of model outputs. If Phi returns probability vectors, each difference can contain negative entries and does not sum to one; if Phi returns logits, KL divergence requires probabilities. No normalization or alternative divergence is specified. Because L2 is the term that drives transfer when the target model has strong zero-shot performance or the student is weak, the reported transfer results cannot be reproduced from the paper alone. Please state the actual operation used in the code (e.g., softmax followed by KL, MSE, or cosine distance) and correct Eq. (5) accordingly.","section":"Section 4.3, Eq. (5)"},{"comment":"The reported headline accuracies appear to be selected using private test labels. Section 5.1 defines Full ZS as the teacher's zero-shot accuracy on the private test sets, and Eq. (6) uses this ZS and the compressed model's accuracy C to set alpha. Table 20 then reports the 'best-performing alpha' by transferred accuracy on the same private test sets, and the alpha values in Table 12 do not always match the heuristic row of Table 20 (e.g., Llama2-7b sst2: Table 12 uses 0.9 while the heuristic gives 1.0). Additionally, Table 1 reports two public datasets per private task and the summary/Table 5 use the better of the two. As a result, the numbers in the main tables do not demonstrate a deployable privacy-preserving protocol, because the transfer step in practice would not know private test labels. Please specify how alpha and the public dataset are chosen from information available to the parties (e.g., a private validation split held out by the user, or a purely public criterion) and re-run the main comparisons under that protocol.","section":"Section 5.2, Tables 1/12/20, Eq. (6)"},{"comment":"Even after fixing Eq. (5), Eq. (4) needs the same clarification: KLDiv is only defined if both arguments are probability distributions, so the paper should state whether Phi_t and Phi_s outputs are softmax distributions or logits, and whether a temperature or smoothing is applied. The current notation leaves this ambiguous, which is a reproducibility blocker for the central transfer objective.","section":"Section 4.3 and Section 5.6, Eq. (4)"}],"minor_comments":[{"comment":"The title contains a typo: 'Trasnfer' should be 'Transfer'.","section":"Table 12"},{"comment":"The phrase 'pertaining data leakage' should be 'pretraining data leakage'.","section":"Section D.7"},{"comment":"The word 'analze' should be 'analyze' in the caption.","section":"Table 18"},{"comment":"The entry '59,54' uses a decimal comma instead of a decimal point; please standardize numeric formatting.","section":"Table 5"},{"comment":"The tweet ground-truth label list for label 1 has a stray extra closing bracket: '[ \" moderate\", \" neutral\", \" balanced\"]]'.","section":"Table 9"},{"comment":"The heuristic in Eq. (6) can produce negative values when ZS < RG; please state whether values are clipped to a valid range and what the user should do in that case.","section":"Equation (6)"}],"recommendation":"major_revision","confidential_remarks":"The empirical contribution is substantial, but the Eq. (5) issue and the selection-on-test issue are both fixable; I would not reject. I recommend asking the authors to correct the transfer-loss definition, provide a private-data-free selection protocol for alpha and public data, and re-check all headline numbers under that protocol. The code release is a strong asset; the authors should be asked to point to the exact implementation of Eq. (5)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely useful: tune a soft prompt on a small distilled model, then transfer it to the large model using only public data. The two-term loss (output mimicry plus direction-change alignment) is new, and Table 6's demonstration that KD feature alignment is what makes transfer work is a real insight. The experiments are extensive — three model families, DP and non-DP, classification and generation, honest limitations — and the central effect (transfer beats direct transfer and zero-shot) reproduces consistently.\n\nThat said, there are two soft spots worth flagging, one of them serious. The serious one: Eq 5 writes L2 as a KL divergence between differences of model outputs. KL divergence is defined between probability distributions, not between difference vectors. If the model outputs are probabilities, the differences contain negative entries and don't sum to one; if they're logits, KL isn't defined without a softmax. As printed, the central objective cannot be evaluated. The released code may use a valid surrogate, but the paper must state what it actually does; right now the method is unreproducible from the text.\n\nThe second issue is evaluation-side. The alpha heuristic in Eq 6 uses zero-shot and compressed-model accuracies on the private test set, and the public dataset choices in Tables 12 and 16 are informed by private test performance. The transfer optimization itself never touches private test data, so the result isn't circular in the strict sense, but the hyperparameter selection leaks test information. That weakens the generalization claims.\n\nMinor: the main tables lack error bars (Table 20 has them, so the authors know how). Also, the runtime comparison includes KD amortization caveats, which is handled fairly.\n\nOverall, the empirical contribution is real and the paper is worth engaging with. The Eq 5 problem is likely fixable, but it needs to be fixed — a referee should not have to guess the actual transfer objective. This deserves peer review, not a desk reject.","headline":"Solid empirical work on private soft prompt transfer, but Eq 5's KL divergence over output differences is mathematically undefined as printed, and the alpha selection leaks private test information.","tokens_in":27666,"tokens_out":1628,"would_cite":false,"duration_ms":21726,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"POST shows that soft prompts tuned privately on a small distilled model can be transferred to a larger LLM using only public data, recovering most of the accuracy of full prompt tuning.","keywords":["soft prompt transfer","prompt tuning","knowledge distillation","differential privacy","large language models","privacy-preserving adaptation","parameter-efficient fine-tuning","membership inference"],"falsifier":"Run POST with a same-size student that has equal task accuracy but was produced by ordinary fine-tuning on the distillation corpus instead of knowledge distillation. The paper's Table 6 predicts a clear drop in transferred accuracy (up to roughly 15 points on tweet); finding no drop would falsify the feature-alignment premise.","tokens_in":26709,"feed_emoji":"🔒","tokens_out":7197,"duration_ms":73073,"temperature":0.7,"pith_summary":"Soft prompts that adapt large language models are normally tuned directly on the big model, which forces users either to pay for expensive full-model backpropagation or to send private data to the model provider. This paper proposes POST, a framework that instead tunes the soft prompt locally on a small model distilled from the large one, then transfers that prompt back to the large LLM using only a small public dataset. The authors set out to show that this pipeline keeps the private data confidential, can add differential privacy guarantees, and still reaches accuracy close to full prompt tuning on the large model. On classification and open-ended generation tasks across three model families, POST beats direct prompt transfer and prior transfer baselines by large margins, reduces compute, and loses little utility when differential privacy is turned on.","feed_headline":"Private prompts tuned on small models transfer to big LLMs","feed_subtitle":"A public-data refinement step recovers near full prompt-tuning accuracy, cutting cost and keeping private data local.","key_machinery":"The central machinery is knowledge-distilled feature alignment plus a two-term public-data transfer loss. The provider distills the target LLM into a small student (for example, Llama2-7b to a 2-layer model) using a combination of logit KL, language-model, and embedding-cosine losses, so the student matches the teacher's predictive behavior enough to make prompts portable. The transfer step then optimizes the target prompt $p_t$ to minimize $L=(1-\\alpha)L_1+\\alpha L_2$, with $L_1$ the KL divergence between the prompted outputs of the two models and $L_2$ the KL divergence between the prompt-induced output changes, both evaluated on public data. Local tuning can run under PromptDPSGD for $(\\epsilon,\\delta)$-differential privacy, and DP post-processing carries the guarantee through the transfer.","core_discovery":"The paper's central claim is that a soft prompt's task knowledge is not permanently locked to the model it was tuned on. If the small source model is produced by knowledge distillation from the target LLM, the tuned prompt can be re-encoded for the target by optimizing the target prompt on public data to match two quantities: the source model's prompted output logits, and the direction of the output change that the prompt induces relative to the un-prompted model. The claim is supported experimentally: transferred prompts on Llama2-7b, GPT2-XL, and Roberta-base approach the Full PT upper bound, improve over direct transfer by a large amount, and outperform previous zero-shot transfer and discrete-prompt baselines, while differentially private tuning reduces membership-inference leakage.","pith_inferences":["An implication the authors leave implicit is that the same two-term transfer loss could be applied to other parameter-efficient adapters, such as prefix or LoRA-style modules, whenever a distilled student is available.","The observed regularizing effect of DP noise suggests a testable extension: adding small noise during local tuning might improve transfer generally, even for users who do not require formal privacy.","Because public-dataset choice matters for transfer quality, a systematic or synthetic public-data selection method would remove the remaining heuristic; the paper's task-similarity finding is a starting point for such a rule."],"forward_implications":["Users can adapt a hosted LLM to private tasks without ever sending training data to the provider, since local tuning and public-data transfer are separate from the private data.","Differentially private local tuning transfers to the large model with little utility loss, and membership-inference AUC drops relative to non-private prompts.","Small public datasets of under roughly 100 samples and a few thousand transfer steps are enough to reach convergence, so the transfer step is cheap.","Prompt tuning on a small student model cuts runtime substantially, with a sixfold speedup on the largest tested dataset before accounting for the one-time distillation cost.","Soft prompt transfer now works across model families and for open-ended generation, not only classification, giving providers a reason to offer soft-prompt adaptation without collecting user data."],"supporting_citations":[{"why":"Establishes knowledge distillation, the compression method POST uses to align the small model with the target LLM.","marker":"Hinton et al. (2015)"},{"why":"Supplies the distillation objective (logit, language-model, and embedding-cosine losses) used to build the student.","marker":"Sanh et al. (2019)"},{"why":"Defines soft prompt tuning, the adaptation paradigm and upper-bound baseline (Full PT) that POST transfers.","marker":"Lester et al. (2021)"},{"why":"Previous soft prompt transfer that needs private data; also the source of the 100-token prompt initialization.","marker":"Su et al. (2022)"},{"why":"Zero-shot soft prompt transfer baseline that POST compares against and outperforms.","marker":"Wu et al. (2023)"},{"why":"Provides PromptDPSGD, the algorithm POST uses to tune the local prompt with differential privacy.","marker":"Duan et al. (2023a)"},{"why":"Discrete-prompt DP transfer baseline (DP-OPT) that POST's empirical comparison must beat.","marker":"Hong et al. (2023)"},{"why":"Formal differential privacy definition that justifies the DP guarantees carried through post-processing.","marker":"Dwork et al. (2006)"}],"fun_headline_variants":["Private prompt tuning on small models transfers to big LLMs","Keep data local: tune prompts on small models, transfer to LLMs","POST: private soft prompt transfer cuts cost, preserves privacy","Distill, tune locally, transfer: private soft prompts for LLMs","Small-model tuned prompts gain big-LLM accuracy via public refinement"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The transfer step assumes the distilled student's feature space is aligned with the large model's, so matching the student's prompted logits and output-change direction on public data is enough to re-encode the prompt's task knowledge in the large model.","fun_headline_variants_meta":{"raw":{"variants":["Private prompt tuning on small models transfers to big LLMs","Keep data local: tune prompts on small models, transfer to LLMs","POST: private soft prompt transfer cuts cost, preserves privacy","Distill, tune locally, transfer: private soft prompts for LLMs","Small-model tuned prompts gain big-LLM accuracy via public refinement"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1292,"prompt_tokens":949,"completion_tokens":343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":254}},"tokens_in":565,"tokens_out":343,"duration_ms":4434,"temperature":1.0,"reasoning_tokens":254,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:44:33.792234+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run POST with a same-size student that has equal task accuracy but was produced by ordinary fine-tuning on the distillation corpus instead of knowledge distillation. The paper's Table 6 predicts a clear drop in transferred accuracy (up to roughly 15 points on tweet); finding no drop would falsify the feature-alignment premise.","supporting_citations":[],"review_version":1}