{"id":"34f4532b-cb84-4af5-b7ad-0d83f4b07817","arxiv_id":"2505.14959","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Batch-level aggregated gradients, LoRA adapters, and de-biased label differential privacy let advertisers and platforms train conversion models in a clean room with modest AUC loss and much lower communication cost.","lead":"An ad platform and an advertiser can train a conversion model together when the advertiser keeps purchase data inside a data clean room, sharing only one big averaged gradient instead of per-user gradients. The authors report that fine-tuning small LoRA adapters plus a privacy correction keeps most of the model's accuracy while cutting training communication from days to under an hour.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Batch-level aggregated gradients are exactly invertible: the platform knows p_i and v_i, so a single update reveals all labels via a linear solve; Section 2.1's privacy premise is false rather than merely informal.","rationale":"The reader's weakest_assumption pointed at Section 2.1's informal privacy claim; my check sharpens it into a concrete inversion attack. The attack does not require many gradients, auxiliary information, or collusion with the clean room: a single aggregated gradient is affine in the unknown label vector y, and the platform has all the coefficients. Because d ≥ b in all reported LoRA configurations, exact recovery is expected in the absence of label DP. This makes the privacy premise not merely unproven but false in the no-DP setting. The paper's label DP mechanism is still formally valid, so the central claim could be repaired by (i) deleting the claim that batch aggregation prevents label inference, (ii) stating that privacy rests solely on label DP, and (iii) reporting the combined LoRA+DP accuracy. Since Table 1 is presented without DP, the headline '95% of full fine-tuning gain' is not a privacy-preserving result as written. For ε = 3–5, the DP guarantee is weak in practice, but that is a parameter choice rather than a logical flaw. The reader's CONDITIONAL verdict already requested a formal privacy analysis; this attack tells the authors exactly what such an analysis will find. I therefore do not move the verdict to reject, but the revision conditions must include removing the false batch-aggregation privacy claim and evaluating LoRA+DP jointly.","tokens_in":7223,"tokens_out":14789,"duration_ms":138859,"concrete_test":"Re-run the Table 1 setup without label DP for one training step on a real batch (b ≈ 10^4) and have the platform compute s = Σ_i p_i v_i − g, then solve min_{y∈{0,1}^b} ||V y − s||_2 (or solve the exact linear system with a left inverse and round). If label recovery accuracy is at or near 100%, the Section 2.1 claim that large-batch aggregated gradients resist label inference is refuted. A lighter reproducibility check is to run the same inversion on a small MLP with d=100, b=10, where exact recovery can be verified by hand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 2.1 the clean room returns g = Σ_i (p_i − y_i) v_i, where v_i = ∂z_i/∂f and z_i and v_i are computed by, and therefore known to, the platform. The platform also knows p_i = σ(z_i). It can compute s = Σ_i p_i v_i − g = Σ_i y_i v_i = V y, where V is the d×b matrix whose columns are v_i and y is the binary label vector. In every LoRA configuration of Table 1, d = ∥f∥ exceeds the batch size b (e.g., 2 million trainable parameters versus tens of thousands of samples), so V is generically full column rank and the platform can recover y exactly as (V^T V)^{-1} V^T s. Thus large-batch aggregation does not merely resist DLG; it is linearly invertible in the no-DP regime. This makes the Section 2.1 claim that labels 'shall not be easily attacked' false, and the stated condition for adding label DP (∥f∥ > b) is exactly the regime where this inversion is possible. Table 1 appears to report LoRA accuracy without label DP, so the headline '95% of fine-tuning gain' is not a privacy-preserving result as written. Label DP is the only effective protection, but its interaction with LoRA is never jointly evaluated, and ε = 3–5 corresponds to a weak practical guarantee (the randomized label is correct with probability roughly 95–99%), which may not meet the advertiser's privacy requirement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a split-learning-style framework for privacy-preserving conversion rate (CVR) prediction in a data clean room setting. The advertising platform (feature party) computes logits and per-sample partial derivatives, the clean room (label party) computes a batch-level aggregated gradient, and the model is updated without sharing sample-level gradients. To reduce communication, the paper introduces gated LoRA adapters that fine-tune only about 1% of the model parameters. To protect labels, it applies label differential privacy with a debiased loss that accounts for the label-flipping probability q_epsilon = e^epsilon/(e^epsilon+1). Experiments on internal industrial datasets report that fine-tuning 2 million parameters captures about 95% of the ROC-AUC gain of full fine-tuning, and that with debiasing the calibration ratio becomes 1.0 while ROC-AUC drops only 0.2%-0.5% for epsilon in {3,5}. The paper also reports up to 4x gradient compression with QSGD/BF16 at a small ROC-AUC cost.","tokens_in":7405,"tokens_out":8384,"duration_ms":71786,"significance":"If the privacy claims held, this framework would be a practical contribution to CVR modeling in data clean rooms, with a reported reduction in training communication from 2.5 days to under 1 hour for large advertisers. The debiased-loss derivation in Section 2.3 and Appendix A.3 is mathematically correct and is a useful, reproducible element: the debiased loss l_deb(p)=l(p q+(1-p)(1-q)) follows directly from the label-flip mechanism and is not fit to the reported AUC numbers. The industrial-scale evaluation is another strength. However, the central privacy-preserving claim is undermined by a load-bearing flaw: Section 2.1's assertion that large-batch aggregated gradients resist label recovery is false in the exact parameter regime used in the paper, and the paper never jointly evaluates the proposed LoRA adapters with label DP. The claimed privacy-preserving accuracy is therefore not established as written.","major_comments":[{"comment":"The claim that batch-level aggregated gradients 'shall not be easily attacked' is incorrect. The platform knows p_i = sigma(z_i) and v_i = dz_i/df for every sample, because it computes z_i and v_i and sends them to the clean room. From the returned gradient g = sum_i (p_i - y_i) v_i, the platform can form s = sum_i p_i v_i - g = sum_i y_i v_i = V y, where V is the d x b matrix with columns v_i and y is the binary label vector. In every LoRA configuration in Table 1, d = ||f|| (e.g., 2 million for rank 32) exceeds the batch size b (tens of thousands), so V generically has full column rank and y = (V^T V)^{-1} V^T s is recovered exactly. Thus the no-DP LoRA results in Table 1 do not protect label privacy, and the paper's stated condition for adding label DP (||f|| > b) is precisely the condition under which the linear inversion succeeds. The paper never reports LoRA plus label DP jointly, so the headline 'about 95% of the gain from fine tuning all parameters' (Section 3) is not a privacy-preserving result as written.","section":"Section 2.1, aggregated gradient formula"},{"comment":"The experimental support for the privacy-preserving accuracy claims is incomplete. Table 2 reports a single baseline and single-point ROC-AUC differences with no error bars, significance tests, or absolute AUC values, and the model configuration (full fine-tuning vs. LoRA, rank, batch size) underlying Table 2 is not stated. Because Table 1 (LoRA) and Table 2 (label DP and compression) are presented as separate settings, the reader cannot determine the ROC-AUC or calibration of the proposed framework when LoRA and label DP are combined. The statement that with debiasing the ROC-AUC drop is 0.2%-0.5% 'which is acceptable in production' is not supported without variance estimates and without specifying the base model the drop is measured against.","section":"Section 3, Tables 1 and 2"},{"comment":"The privacy guarantee is underspecified. The paper does not define the threat model (who the adversary is, what auxiliary information it has), does not provide a privacy accounting for label DP (per-epoch epsilon, composition over multiple training epochs, or how batch aggregation affects the DP guarantee), and does not analyze the d < b regime where the linear system for labels is underdetermined. The claim that label DP 'effectively defends' against the equation-solving attack is an assertion, not a demonstrated property. A concrete evaluation against the linear-recovery attack described above, with and without label DP, is needed to support the central privacy claim. Additionally, the label-keep probabilities implied by epsilon in {3,5} are approximately 0.95 and 0.99, so the practical privacy protection for advertisers is weak; the paper does not discuss whether this satisfies advertiser privacy requirements.","section":"Sections 2.1 and 2.3"}],"minor_comments":[{"comment":"The 'Baseline' row lists 200 million trainable parameters, which appears inconsistent with a no-fine-tuning baseline; please clarify whether these parameters are trainable and whether the baseline is updated at all.","section":"Table 1"},{"comment":"The description of gradient compression does not state whether QSGD/top-k/powerSGD is applied to the per-sample partial derivatives before they are sent to the clean room or to the aggregated gradient, and how compression interacts with the aggregated-gradient computation.","section":"Section 2.2"},{"comment":"The calibration ratio formula is typeset ambiguously; please write it with explicit parentheses, e.g., E[p_i] / (n_+/(n_+ + n_-)).","section":"Appendix A.3"},{"comment":"The paper should either cite a concrete reference for the claim that large-batch aggregation resists DLG-style attacks or remove the claim; as written it is unsupported and contradicted by the linear-recovery argument.","section":"Section 2.1"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be a published RecSys 2024 short paper; if the journal submission is intended as an extended version, it should substantially expand the experimental section and address the privacy flaw rather than present the same 5-page content. The authors' own references [14], [18], and [24] on label leakage and label protection in split learning are directly relevant to the inversion attack and should be engaged with. The conclusion's language about 'a new standard' and 'benchmark' is overstated given the open privacy issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has a real industrial systems story, but the privacy argument that carries it is not just informal—it's wrong. In Section 2.1 the clean room returns g = Σ(p_i − y_i)v_i. The platform computed p_i and v_i itself, so it can form s = Σp_i v_i − g = V y. Whenever the parameter dimension exceeds the batch size—which is exactly the condition the paper gives for adding label DP—V is generically full column rank and y is recovered exactly by a linear solve. Large-batch aggregation does not resist this; it enables it. The DLG discussion is beside the point.\n\nWhat's genuinely useful: the gated LoRA adapter design is a clean way to fine-tune a shared CVR model per advertiser, and the de-biased loss l_deb(p) = l(pq + (1−p)(1−q)) is correct and handles the class imbalance in conversion labels. The reported LoRA accuracy (95% of full fine-tuning gain at 1% of parameters) is plausible and the communication savings are real. Those are contributions worth keeping.\n\nThe soft spots beyond the privacy flaw: Table 1 reports LoRA without label DP, Table 2 reports label DP (presumably on full fine-tuning), and the paper never evaluates LoRA + label DP jointly. So the headline accuracy numbers are not the privacy-preserving result. There are no error bars or significance tests, and the dataset is internal, which is common for this genre but still limits verification.\n\nThe bottom line: the framework could work as a communication-efficient fine-tuning system with label DP as the only real privacy protection, but the paper doesn't present it that way. It claims large-batch aggregation is itself a privacy defense, and that claim is false. The abstract's 'new standard' language is also overblown for an internal evaluation.\n\nI'd send this to a serious referee because the flaw is substantive and the de-biasing and LoRA pieces are worth discussing, but my recommendation would be major revision or reject unless the authors re-frame the contribution and add a joint LoRA+DP evaluation. For someone building a split-learning system, read it for the adapter design and the de-biasing trick, not for the privacy guarantee.\n\nBest.","headline":"The privacy premise is false: the platform can solve for all labels from a single batch update, so the paper's central claim collapses.","tokens_in":8130,"tokens_out":4005,"would_cite":false,"duration_ms":33325,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that conversion models can be trained collaboratively on advertiser labels without sharing sample-level gradients, using batch-level aggregated gradients, gated LoRA adapters, and label differential privacy with a…","keywords":["privacy-preserving machine learning","CVR prediction","data clean room","split learning","label differential privacy","LoRA fine-tuning","gradient compression","conversion modeling"],"falsifier":"Run a label-inference attack, such as a DLG-style optimization or a linear-system solver, against the aggregated gradients the paper's protocol would produce; if the attack recovers individual labels or substantially raises the accuracy of a label-guessing adversary beyond random chance using only the exchanged messages, the central privacy claim is falsified.","tokens_in":6869,"feed_emoji":"🔐","tokens_out":5448,"duration_ms":46984,"temperature":0.7,"pith_summary":"The paper proposes a way to train conversion-rate (CVR) models when the advertiser holding the conversion labels does not want to share sample-level data, and the advertising platform holding the features is too large to train inside a data clean room. It claims that by exchanging only batch-level aggregated gradients between the clean room and the platform, and by fine-tuning only a small set of gated LoRA adapters, the model can capture about 95% of the ROC-AUC gain of full fine-tuning while cutting training time per day of data from 2.5 days to under an hour. Adding label differential privacy with a debiased loss reportedly keeps predictions fully calibrated and limits the ROC-AUC drop to 0.2-0.5% for epsilon between 5 and 3. If these numbers hold, advertisers can contribute conversion signals without exposing raw conversions, and platforms can serve many advertisers by adding one small adapter per advertiser. The paper's central claim is that privacy protection, communication efficiency, and accuracy can be had together in this setup.","feed_headline":"Train private CVR models with 1% of parameters and 95% of the gain","feed_subtitle":"Batch-level gradients and label DP keep ROC-AUC loss under 0.5% while cutting training time from days to under an hour.","key_machinery":"The load-bearing pieces are (1) batch-level aggregated gradients in place of sample-level gradients, computed in the clean room as sum_i (p_i - y_i) * dz_i/df; (2) gated LoRA adapter layers, a parameter-efficient fine-tuning mechanism that adds small trainable low-rank matrices to selected layers (QKV in sequence transformers, linear layers in feature interaction and multi-task towers) and gates them per advertiser; and (3) label differential privacy with a debiasing loss, where labels are flipped with probability determined by epsilon and the loss is evaluated on the transformed prediction p' = p q_epsilon + (1-p)(1-q_epsilon) so the model output remains calibrated. These work together to reduce communication, hide labels, and preserve accuracy.","core_discovery":"The central discovery is that sample-level gradients are unnecessary for collaborative CVR training: the clean room can compute the aggregated gradient sum_i (p_i - y_i) * dz_i/df from the platform's per-sample partial derivatives and the labels, and send only that sum back. With tens of thousands of samples per batch, the paper argues this aggregation hides individual labels from gradient-matching attacks such as DLG. To make the scheme practical, the platform fine-tunes only gated LoRA adapters (1-2% of parameters), which yields 95% of the full fine-tuning ROC-AUC gain, and further compresses gradients with QSGD or BF16. For label privacy, random label flipping with epsilon in {3,5} is applied, and a debiased loss l_deb(p)=l(p q_epsilon + (1-p)(1-q_epsilon)) recovers full calibration (1.0) with only a 0.2-0.5% ROC-AUC drop. The paper reports these results on internal datasets with over 10 billion entries and roughly 300 features.","pith_inferences":["The privacy argument is informal: an adversary who sees many aggregated gradients across training steps might reconstruct label statistics or even individual labels, especially with known model weights; a formal end-to-end DP analysis over the full training run would be needed to make the guarantee rigorous.","The debiasing formula assumes a fixed, known flip probability; if label flipping probabilities vary across samples or adapt over time, the calibration recovery may need re-derivation.","The same three-part recipe (aggregated gradients, adapters, label DP with debiasing) could transfer to other feature-party/label-party settings outside advertising, such as medical or financial collaboration, wherever raw labels cannot be shared."],"forward_implications":["Advertisers who currently withhold conversion data can safely contribute it, unlocking CVR improvements for their campaigns and lowering cost per action.","Training a large CVR model in a clean-room setting becomes feasible on ordinary network links: a day of data that took 2.5 days to train now takes under an hour.","Each advertiser can be served by its own LoRA adapter on a shared base model, enabling parallel, independent, multi-tenant fine-tuning without cross-advertiser interference.","Label DP with the debiased loss gives calibrated probability estimates at epsilon as low as 3, so downstream ranking systems can trust the predicted conversion scores."],"supporting_citations":[{"why":"Provides the low-rank adapter method that the gated LoRA fine-tuning is based on.","marker":"[11]"},{"why":"Supplies the label differential privacy mechanism used to flip labels with probability tied to epsilon.","marker":"[7]"},{"why":"The label-leakage result for split learning that motivates the switch to aggregated gradients.","marker":"[14]"},{"why":"Provides differentially private label protection in split learning, which the paper extends with debiasing.","marker":"[24]"},{"why":"The QSGD gradient compression algorithm tested for reducing communication overhead.","marker":"[3]"},{"why":"The DLG gradient-matching attack that defines the threat model for label privacy.","marker":"[28]"}],"fun_headline_variants":["Batch gradients replace sample-level for private CVR models","Aggregated gradients hide labels, cut comms, keep AUC","Label DP and batch sums make CVR training privacy-safe","Clean-room CVR: batch gradients, LoRA adapters, label DP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole privacy promise rests on an unproven assumption that an adversary cannot extract meaningful label information from batch-level aggregated gradients, especially when the batch is large and the fine-tuned parameters are few; the paper offers no formal guarantee for this.","fun_headline_variants_meta":{"raw":{"variants":["Batch gradients replace sample-level for private CVR models","Aggregated gradients hide labels, cut comms, keep AUC","Label DP and batch sums make CVR training privacy-safe","Clean-room CVR: batch gradients, LoRA adapters, label DP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00063,"raw_usage":{"total_tokens":2923,"prompt_tokens":973,"completion_tokens":1950,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":1878}},"tokens_in":589,"tokens_out":1950,"duration_ms":16695,"temperature":1.0,"reasoning_tokens":1878,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:26:39.485538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a label-inference attack, such as a DLG-style optimization or a linear-system solver, against the aggregated gradients the paper's protocol would produce; if the attack recovers individual labels or substantially raises the accuracy of a label-guessing adversary beyond random chance using only the exchanged messages, the central privacy claim is falsified.","supporting_citations":[],"review_version":1}