{"id":"7ee2f4db-58ad-474e-a405-3fcde48e5504","arxiv_id":"2504.18147","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"NoEsis couples DP-trained shared prompt tokens with non-private per-domain LoRA experts, achieving private knowledge transfer across Python, Java, and Go code completion.","lead":"This paper introduces NoEsis, a two-stage training method that keeps domain-specific code models separate while sharing a small set of privacy-protected prompt tokens between them. It reports that on code completion, the approach preserves most of the accuracy of a non-private shared model while reducing cross-domain membership inference.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Privacy claim is broader than the algorithm proves: Stage 2 releases non-private domain experts, so the stated full-document-level DP does not hold for the full output; a per-client cross-domain guarantee is implied but never stated or proved.","rationale":"The utility side of the central claim is in reasonably good shape: the reported Table 2 and Figure 2 numbers support the 'at least 77% gap bridged' statement, the code is available, and five-seed variability is reported. My concern is exclusively with the privacy half of the headline claim. Section 3 defines document-level DP with adjacency over the full multi-domain dataset, and the abstract and conclusion claim 'provable privacy guarantees.' But Algorithm 1's second stage updates E_k with un-noised, unclipped SGD; those experts are then distributed. Under the stated adjacency, the complete released object is not (ε,δ)-DP: if a Python document changes, E_Python changes without any privacy mechanism. The paper's escape is that E_k is never shared across domains, so a client in Java receives only E_Java plus P_T. For Python/Go documents, E_Java is plausibly independent, and P_T is DP; post-processing then gives a genuine cross-domain guarantee. That is the right theorem to state, and it would make the Table 1 'Private' claim meaningful. But it is not stated, and the informal assertion in Section 3 does not itself establish it. Because the paper's novelty claim is precisely that it is the only modular method that is also private, this gap is load-bearing. A conditional acceptance with a required per-client privacy theorem, or a scoped rewording of the privacy claim, is appropriate. This matches the reader's weakest assumption; I see no reason to move the verdict.","tokens_in":18942,"tokens_out":11337,"duration_ms":116464,"concrete_test":"Formally instantiate the per-client release: for a Java client, define output O=(P_T, E_Java). Prove that for any two full datasets D and D' that differ in a single Python or Go document, the distributions of O are (ε,δ)-indistinguishable. The proof must show (i) Stage 1 makes P_T (ε,δ)-DP with respect to that adjacency, and (ii) E_Java depends on Python/Go documents only through P_T—i.e., the Stage 2 update for E_Java (Algorithm 1, Stage 2) uses gradients from Java documents alone. A direct numerical check: fix P_T, the random seed, and D_Java, then rerun Stage 2 after changing one Python document. If E_Java differs beyond floating-point rounding, the post-processing argument fails; if the proof goes through, revise the paper's wording to 'cross-domain DP per client' and keep the rest of the claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central privacy assertion—'provable privacy guarantees' at (ε=1, δ=1e-6), and Table 1's 'Private ✓'—is not matched by a stated privacy theorem for the actual released artifacts. Algorithm 1 runs DP-SGD only on the prompt tokens P_T in Stage 1, then trains per-domain LoRA experts E_k with plain SGD in Stage 2. Under the paper's own adjacency definition (Section 3: two datasets are adjacent when one document in any domain is removed), the full tuple (P_T, E_1,...,E_K) is not differentially private: each E_k changes when a document in its own domain changes, and E_k is part of deployment ('the LoRA parameters are sent to each respective domain and merged with the backbone'). The only reading that makes the privacy claim true is a per-client cross-domain guarantee: a client in domain c receives (P_T, E_c), and for any adjacent pair differing outside domain c, E_c is independent of the changed document while P_T is DP, so post-processing yields (ε,δ)-DP for that client. This reading is plausible and likely provable, but the paper neither states it nor proves it; it asserts it informally ('domain experts are never shared'). The unqualified 'provable privacy guarantees' therefore overstates what is established, and this is load-bearing because the paper's claimed uniqueness (the only method that is modular, private, and transferable) rests on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces NoEsis, a two-stage parameter-efficient fine-tuning framework for multi-domain LLM adaptation that combines DP-trained shared prompt tokens (Stage 1) with non-private per-domain LoRA experts (Stage 2). On CodeXGLUE code completion across Python, Java, and Go, it reports higher accuracy than share-nothing and prompt-tuning-only baselines, closes at least 77% of the accuracy gap to the non-private baseline, and lowers cross-domain membership-inference attack AUC relative to a Mix-LoRA baseline. The authors claim document-level DP at ε=1, δ=1e-6 and position NoEsis as the only approach that is simultaneously modular, private, and able to transfer knowledge across domains.","tokens_in":19245,"tokens_out":5925,"duration_ms":64618,"significance":"If the intended privacy guarantee is stated precisely and proved, this is a useful and timely contribution: it offers a concrete architecture for serving per-domain adapters with a shared DP-trained prompt backbone, and it ships with reproducible code, detailed hyperparameter reporting, multi-seed variability analysis, and ablations. The empirical utility results are internally consistent with the plotted numbers, and the code-release and experimental documentation are strengths. The main gap is formal: as written, the DP claim is broader than the released artifacts actually satisfy, and the missing theorem is load-bearing because the paper's claimed uniqueness rests on the privacy property. The central claim is defensible and likely fixable by restating the guarantee as a per-client cross-domain guarantee and proving it.","major_comments":[{"comment":"The paper's privacy claim is not supported for the artifacts that are actually released. Under the adjacency definition in Section 3, two datasets are adjacent when one document in any domain is removed. Stage 1 makes the prompt tokens P_T differentially private with respect to the full dataset, but Stage 2 trains each domain expert E_k non-privately on its own domain. For an adjacent pair differing in a document of domain c, the released expert E_c changes arbitrarily, so the full tuple (P_T, E_1, ..., E_K) is not (1, 1e-6)-DP. The statement that is both true and likely intended is a per-client cross-domain guarantee: a client in domain c receives (P_T, E_c), and for adjacent datasets differing only outside domain c, E_c is a post-processing function of P_T with D_c fixed, so the pair is (ε,δ)-DP. This theorem is neither stated nor proved, and the unqualified phrases 'provable privacy guarantees' and Table 1's 'Private ✓' therefore overstate what is established.","section":"Section 3, Algorithm 1, Table 1, Abstract"},{"comment":"The pseudocode for Stage 2 is ambiguous in a way that matters for privacy. It says to 'Randomly draw batch B_t of size N_b from D' and update the full expert parameter set E_t with SGD. Read literally, this would update every domain expert on batches that may contain documents from other domains, which would destroy even the per-client cross-domain guarantee analyzed above. The surrounding text says experts are per-domain, so the intended behavior is presumably that only the expert matching each document's domain receives the corresponding gradient (as in routed Mix-LoRA training), but Algorithm 1 should state this explicitly. The revision should specify that each expert E_k is trained only on D_k, or else the 'domain experts are never shared' argument is not well-defined.","section":"Algorithm 1, Stage 2"},{"comment":"The Limitations section acknowledges the small model size, the single benchmark, and the limited set of attacks, but it does not acknowledge the mismatch between the claimed full-document DP and the actually provided per-client cross-domain guarantee. Since the formal privacy scope is the main correctness issue in the paper, the limitations discussion should explicitly state that the guarantee is per receiving domain and does not protect a domain's own documents from a client holding that domain's expert.","section":"Limitations, Appendix A"}],"minor_comments":[{"comment":"The caption of Figure 5 refers to 'results in Table 5,' but the displayed MIA results are in Table 4; Table 5 is the ablation study.","section":"Appendix C.4, Figure 5 caption"},{"comment":"Equation (1) uses W on both sides of the decomposition W = W + α Σ B^(k)A^(k); renaming the adapted weight (e.g., W') would avoid notational collision.","section":"Equation (1), Section 3"},{"comment":"Even if the intended per-domain update is clear from context, the pseudocode line Et ← Et−1 − SGD(θ0, P_T, E_{t−1}, B_t) should be rewritten to show that for a batch containing documents from domain k, only E_k is updated and other experts receive zero gradient.","section":"Algorithm 1, Stage 2"},{"comment":"The column header 'PEFTPython' appears to be a formatting error; the headers for 'PEFT' and the per-domain accuracies should be separated.","section":"Table 2 header"},{"comment":"The text should state explicitly which non-private model is used as the 'non-private result' in Table 4; Appendix C.4 mentions that the studied Mix-LoRA model is trained with SGD, but Table 4 itself does not name the baseline.","section":"Table 4 and Appendix C.4"},{"comment":"The bibliography entry 'Chen et al. Evaluating large language models trained on code' should use the full author list rather than 'Chen et al.' as the author name.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The empirical contribution is solid and the code release is a plus. The main fix is formal: state and prove the per-client cross-domain privacy theorem, and remove or qualify the full-document DP phrasing. I believe this is achievable within a revision and do not see a fundamental mismatch between the method and the intended threat model. One editorial note: the paper is an ICLR workshop paper, and the revision should also adjust the contribution framing so that the uniqueness claim in Table 1 is tied to the precise threat model rather than to unqualified document-level DP."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the empirical story is in good shape, but the privacy headline is broader than what Algorithm 1 proves. The paper combines DP-trained shared prompt tokens with non-private Mix-LoRA domain experts and evaluates on CodeXGLUE for Python/Java/Go. That combination is new as far as I know, and the utility numbers are credible: NoEsis beats Share Nothing on all three languages and the 77% gap-bridged claim matches the table. The cross-domain MIA results are a nice extra.\n\nThe soft spot is the one the stress-test note identified. The DP guarantee is computed for the prompt tokens only; Stage 2 trains the LoRA experts with plain SGD and those experts are part of the deployment artifact. Under the paper's own adjacency definition (any document in any domain), the full released tuple (P, E_1..E_K) is not DP. The claim that survives is a per-client cross-domain guarantee: a client in domain c gets P and E_c, and for adjacent datasets that differ only outside domain c, E_c is unaffected and P is DP, so post-processing gives epsilon-DP. That's a real and useful guarantee, and the text gestures at it with 'domain experts are never shared,' but it is never stated formally or proved. The abstract's 'provable privacy guarantees' and Table 1's 'Private ✓' therefore overstate what is established. This is a load-bearing fix, not cosmetic.\n\nMinor issues: hyperparameters are selected by downstream accuracy without an explicit held-out validation split (Table 7, C.3); there is an inconsistency about the common LoRA rank between Table 2 (rc=512) and the text (rc=4); and 'Share Nothing' with ε=0.0 is odd notation. The limitations appendix does acknowledge the small model, single benchmark, and single attack type, which is honest.\n\nOverall: the central engineering contribution is solid and the error analysis is reproducible — code is available. The privacy framing needs to be rewritten to state and prove the per-client guarantee, or to present the result as cross-domain leakage reduction without claiming full-DP for the whole system. That's a revise-and-resubmit in my book, not a rejection. I'd send it to a serious referee; a skeptical reviewer with a DP background will find the fix straightforward.","headline":"Empirically solid hybrid of DP prompt tuning and Mix-LoRA, but the privacy claim covers only the shared prompt tokens; the per-client guarantee is plausible, unstated, and needs to be proved.","tokens_in":19824,"tokens_out":2786,"would_cite":false,"duration_ms":28129,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that NoEsis, a hybrid of DP-trained prompt tokens and non-private Mix-LoRA experts, can deliver document-level differential privacy, knowledge transfer, and modularity at the same time, backed by code-completion…","keywords":["Differential privacy","Knowledge transfer","Modular LLMs","LoRA experts","Prompt tuning","Membership inference attacks","Code completion","Multi-domain adaptation"],"falsifier":"Train NoEsis twice on the same data with the same hyperparameters and random seed, once with the full training set and once with a single document removed from one domain, then measure how much the final shared prompt-token matrix changes. If the change exceeds what the chosen epsilon equal to 1 and delta equal to 1e-6 accountant permits for a single document, then the training procedure is not actually implementing the claimed document-level differential privacy.","tokens_in":18713,"feed_emoji":"🔐","tokens_out":7828,"duration_ms":77716,"temperature":0.7,"pith_summary":"The paper argues that modular LLMs can have privacy, modularity, and knowledge transfer at once, defusing a trade-off that earlier approaches left unresolved. It proposes NoEsis, where a small number of trainable prompt tokens are trained with differential privacy and shared across domains, while each domain gets its own non-private LoRA expert. On multilingual code completion, the method keeps document-level privacy at epsilon equal to 1 and delta equal to 1e-6 and still bridges at least 77% of the accuracy gap between having no sharing and having no privacy. If true, this means organizations with separate private codebases can train one modular model and serve per-domain copies without leaking each other's training data.","feed_headline":"Private shared prompts close 77% of the code-completion accuracy gap","feed_subtitle":"A small set of DP-trained prompt tokens lets each domain keep its own private expert while sharing knowledge.","key_machinery":"The machinery is a two-stage training loop built from two parameter groups. Stage 1 trains a small matrix of prompt embeddings P, prepended to every domain's input, using DP-SGD with per-document gradient clipping and Gaussian noise, so P satisfies document-level (epsilon, delta)-DP. Stage 2 keeps P fixed and trains a Mix-LoRA per domain: low-rank matrices A(k) and B(k) update the frozen backbone through W = W0 + alpha times the sum of B(k)A(k), applied to the feed-forward layers. Deployment merges each expert into the backbone and distributes the shared prompt tokens to every client. The separation between shared-but-private parameters and private-but-non-shared experts is what carries the argument for both utility and privacy.","core_discovery":"The central claim is that the apparent conflict between differential privacy and multi-domain knowledge transfer can be resolved by splitting parameters by role: shared knowledge lives in a small set of DP-trained prompt tokens, while domain-specific skills live in non-private LoRA adapters that are never shared. The paper demonstrates this split on CodeXGLUE code completion: 32 shared prompt tokens outperform a shared LoRA with comparable parameter counts, the domain experts supply the bulk of accuracy, and the DP guarantee on the prompts empirically pushes cross-domain membership inference toward random chance. The paper presents NoEsis as the first framework satisfying modularity, privacy, and knowledge transfer simultaneously, with the strongest quantitative claim being that it closes at least 77% of the accuracy gap between a non-shared baseline and a non-private baseline on code completion.","pith_inferences":["Inference beyond the paper: the two-stage split suggests a general recipe, namely train any small shared interface (prompts, prefixes, or a low-rank common adapter) with DP and keep domain experts non-private; the paper tests prompts, but the same logic could apply to other shared parameter sets if the privacy accountant is run on them.","Inference beyond the paper: the reported membership inference protection is for likelihood-threshold attacks; the paper's own limitations section names adversarial and reconstruction attacks as open, so empirical privacy should be read narrowly until those attack surfaces are also tested.","Inference beyond the paper: because the DP guarantee is not proven end-to-end over the non-private experts, a stronger privacy claim would require either private expert gradients or an explicit composition argument; the experiments support utility plus empirical MIA protection more strongly than a formal guarantee on the whole served model.","A testable extension: apply the same hybrid to natural-language domains or to alternative adapters such as DoRA or VeRA; if the 77% gap-bridging pattern persists, the parameter-splitting mechanism is general, not a quirk of code completion."],"forward_implications":["If NoEsis is right, modular LLMs can be fine-tuned per domain without giving up knowledge transfer, because the shared knowledge can live in a tiny DP-trained prompt set while each domain's adaptation stays in its own private expert.","On code completion, the shared prompt tokens are a parameter-efficient transfer mechanism: 32 tokens outperform a shared LoRA at four times the parameter count, and larger prompt sets do not help further.","Scarce domains benefit the most from transfer: Go, which has roughly 50 times fewer training documents than Python, gains 2.41 accuracy points over a share-nothing modular model.","Standard Mix-LoRA leaks across domains, with a cross-domain membership inference attack reaching AUC 65.4% when attacking Java from Go, while NoEsis reduces that attack to 55.7%, closer to random guessing.","Deployment is modular: each domain's LoRA is merged into the shared backbone and the same DP-trained prompt tokens are sent to all clients, so training once can serve every domain."],"supporting_citations":[{"why":"Supplies the DP-SGD algorithm (gradient clipping plus Gaussian noise) used in Stage 1 to train the shared prompt tokens.","marker":"Abadi et al. 2016"},{"why":"Shows that differentially private prompt tuning can retain accuracy even at small epsilon, motivating the choice of prompt tokens as the shared private parameters.","marker":"Duan et al. 2023"},{"why":"Introduces LoRA, which the paper uses as the basis of the domain-specific Mix-LoRA experts.","marker":"Hu et al. 2021"},{"why":"Introduces Mix-LoRA for routing-based domain adaptation, which NoEsis extends with differential privacy.","marker":"Li et al. 2024"},{"why":"Provides the document-level differential privacy definition used to state the privacy guarantee.","marker":"Dwork & Roth 2014"},{"why":"Supplies the privacy accountant used to compute the noise multiplier from the target epsilon and delta.","marker":"Yousefpour et al. 2021"},{"why":"Provides the pretrained CodeT5+ backbone and the deduplication procedure that makes the private-domain simulation valid.","marker":"Wang et al. 2023"},{"why":"Serves as the non-private modular baseline that motivates the multi-domain code completion setting.","marker":"Gong et al. 2022"}],"fun_headline_variants":["DP-trained prompt tokens close 77% of code-completion gap","Differential privacy and LoRA experts: 77% gap closed","NoEsis: private prompts, shared knowledge, 77% less gap","77% of accuracy gap closed with DP prompt tokens","Modular LLM: DP prompts + private experts bridge 77%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole-system privacy claim rests on the assumption that the domain-specific LoRA experts, trained without any privacy protection and never shared, cannot be combined with the shared prompt tokens to leak information beyond the differential privacy guarantee on those tokens.","fun_headline_variants_meta":{"raw":{"variants":["DP-trained prompt tokens close 77% of code-completion gap","Differential privacy and LoRA experts: 77% gap closed","NoEsis: private prompts, shared knowledge, 77% less gap","77% of accuracy gap closed with DP prompt tokens","Modular LLM: DP prompts + private experts bridge 77%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000212,"raw_usage":{"total_tokens":1380,"prompt_tokens":866,"completion_tokens":514,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":423}},"tokens_in":482,"tokens_out":514,"duration_ms":5219,"temperature":1.0,"reasoning_tokens":423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:23:57.582795+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train NoEsis twice on the same data with the same hyperparameters and random seed, once with the full training set and once with a single document removed from one domain, then measure how much the final shared prompt-token matrix changes. If the change exceeds what the chosen epsilon equal to 1 and delta equal to 1e-6 accountant permits for a single document, then the training procedure is not actually implementing the claimed document-level differential privacy.","supporting_citations":[{"cited_title":"The dataset is augmented with domain labels, which are used for deterministic routing","cited_arxiv_id":null,"evidence_quote":"Supplies the privacy accountant used to compute the noise multiplier from the target epsilon and delta."},{"cited_title":"document","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained CodeT5+ backbone and the deduplication procedure that makes the private-domain simulation valid."}],"review_version":1}