REVIEW 3 major objections 4 minor 29 references
Privacy Preserving Conversion Modeling in Data Clean Room
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that conversion models can be trained collaboratively on advertiser labels without sharing sample-level gradients, using batch-level aggregated gradients, gated LoRA adapters, and label differential privacy with a…
desk verdict The privacy premise is false: the platform can solve for all labels from a single batch update, so the paper's central claim collapses. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing pieces are (1) batch-level aggregated gradients in place of sample-level gradients, computed in the clean room as sum_i (p_i - y_i) * dz_i/df; (2) gated LoRA adapter layers, a parameter-efficient fine-tuning mechanism that adds small trainable low-rank matrices to selected layers (QKV in sequence transformers, linear layers in feature interaction and multi-task towers) and gates them per advertiser; and (3) label differential privacy with a debiasing loss, where labels are flipped with probability determined by epsilon and the loss is evaluated on the transformed prediction p' = p q_epsilon + (1-p)(1-q_epsilon) so the model output remains calibrated. These work together to reduce communication, hide labels, and preserve accuracy.
What would settle it
Run a label-inference attack, such as a DLG-style optimization or a linear-system solver, against the aggregated gradients the paper's protocol would produce; if the attack recovers individual labels or substantially raises the accuracy of a label-guessing adversary beyond random chance using only the exchanged messages, the central privacy claim is falsified.
Extended reading notes
Core claim
The central discovery is that sample-level gradients are unnecessary for collaborative CVR training: the clean room can compute the aggregated gradient sum_i (p_i - y_i) * dz_i/df from the platform's per-sample partial derivatives and the labels, and send only that sum back. With tens of thousands of samples per batch, the paper argues this aggregation hides individual labels from gradient-matching attacks such as DLG. To make the scheme practical, the platform fine-tunes only gated LoRA adapters (1-2% of parameters), which yields 95% of the full fine-tuning ROC-AUC gain, and further compresses gradients with QSGD or BF16. For label privacy, random label flipping with epsilon in {3,5} is applied, and a debiased loss l_deb(p)=l(p q_epsilon + (1-p)(1-q_epsilon)) recovers full calibration (1.0) with only a 0.2-0.5% ROC-AUC drop. The paper reports these results on internal datasets with over 10 billion entries and roughly 300 features.
Load-bearing premise
The whole privacy promise rests on an unproven assumption that an adversary cannot extract meaningful label information from batch-level aggregated gradients, especially when the batch is large and the fine-tuned parameters are few; the paper offers no formal guarantee for this.
Editorial extensions
If this is right
- Advertisers who currently withhold conversion data can safely contribute it, unlocking CVR improvements for their campaigns and lowering cost per action.
- Training a large CVR model in a clean-room setting becomes feasible on ordinary network links: a day of data that took 2.5 days to train now takes under an hour.
- Each advertiser can be served by its own LoRA adapter on a shared base model, enabling parallel, independent, multi-tenant fine-tuning without cross-advertiser interference.
- Label DP with the debiased loss gives calibrated probability estimates at epsilon as low as 3, so downstream ranking systems can trust the predicted conversion scores.
Reading between the lines
- The privacy argument is informal: an adversary who sees many aggregated gradients across training steps might reconstruct label statistics or even individual labels, especially with known model weights; a formal end-to-end DP analysis over the full training run would be needed to make the guarantee rigorous.
- The debiasing formula assumes a fixed, known flip probability; if label flipping probabilities vary across samples or adapt over time, the calibration recovery may need re-derivation.
- The same three-part recipe (aggregated gradients, adapters, label DP with debiasing) could transfer to other feature-party/label-party settings outside advertising, such as medical or financial collaboration, wherever raw labels cannot be shared.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a split-learning-style framework for privacy-preserving conversion rate (CVR) prediction in a data clean room setting. The advertising platform (feature party) computes logits and per-sample partial derivatives, the clean room (label party) computes a batch-level aggregated gradient, and the model is updated without sharing sample-level gradients. To reduce communication, the paper introduces gated LoRA adapters that fine-tune only about 1% of the model parameters. To protect labels, it applies label differential privacy with a debiased loss that accounts for the label-flipping probability q_epsilon = e^epsilon/(e^epsilon+1). Experiments on internal industrial datasets report that fine-tuning 2 million parameters captures about 95% of the ROC-AUC gain of full fine-tuning, and that with debiasing the calibration ratio becomes 1.0 while ROC-AUC drops only 0.2%-0.5% for epsilon in {3,5}. The paper also reports up to 4x gradient compression with QSGD/BF16 at a small ROC-AUC cost.
Significance. If the privacy claims held, this framework would be a practical contribution to CVR modeling in data clean rooms, with a reported reduction in training communication from 2.5 days to under 1 hour for large advertisers. The debiased-loss derivation in Section 2.3 and Appendix A.3 is mathematically correct and is a useful, reproducible element: the debiased loss l_deb(p)=l(p q+(1-p)(1-q)) follows directly from the label-flip mechanism and is not fit to the reported AUC numbers. The industrial-scale evaluation is another strength. However, the central privacy-preserving claim is undermined by a load-bearing flaw: Section 2.1's assertion that large-batch aggregated gradients resist label recovery is false in the exact parameter regime used in the paper, and the paper never jointly evaluates the proposed LoRA adapters with label DP. The claimed privacy-preserving accuracy is therefore not established as written.
major comments (3)
- [Section 2.1, aggregated gradient formula] The claim that batch-level aggregated gradients 'shall not be easily attacked' is incorrect. The platform knows p_i = sigma(z_i) and v_i = dz_i/df for every sample, because it computes z_i and v_i and sends them to the clean room. From the returned gradient g = sum_i (p_i - y_i) v_i, the platform can form s = sum_i p_i v_i - g = sum_i y_i v_i = V y, where V is the d x b matrix with columns v_i and y is the binary label vector. In every LoRA configuration in Table 1, d = ||f|| (e.g., 2 million for rank 32) exceeds the batch size b (tens of thousands), so V generically has full column rank and y = (V^T V)^{-1} V^T s is recovered exactly. Thus the no-DP LoRA results in Table 1 do not protect label privacy, and the paper's stated condition for adding label DP (||f|| > b) is precisely the condition under which the linear inversion succeeds. The paper never reports LoRA plus label DP jointly, so the headline 'about 95% of the gain from fine tuning all parameters' (Section 3) is not a privacy-preserving result as written.
- [Section 3, Tables 1 and 2] The experimental support for the privacy-preserving accuracy claims is incomplete. Table 2 reports a single baseline and single-point ROC-AUC differences with no error bars, significance tests, or absolute AUC values, and the model configuration (full fine-tuning vs. LoRA, rank, batch size) underlying Table 2 is not stated. Because Table 1 (LoRA) and Table 2 (label DP and compression) are presented as separate settings, the reader cannot determine the ROC-AUC or calibration of the proposed framework when LoRA and label DP are combined. The statement that with debiasing the ROC-AUC drop is 0.2%-0.5% 'which is acceptable in production' is not supported without variance estimates and without specifying the base model the drop is measured against.
- [Sections 2.1 and 2.3] The privacy guarantee is underspecified. The paper does not define the threat model (who the adversary is, what auxiliary information it has), does not provide a privacy accounting for label DP (per-epoch epsilon, composition over multiple training epochs, or how batch aggregation affects the DP guarantee), and does not analyze the d < b regime where the linear system for labels is underdetermined. The claim that label DP 'effectively defends' against the equation-solving attack is an assertion, not a demonstrated property. A concrete evaluation against the linear-recovery attack described above, with and without label DP, is needed to support the central privacy claim. Additionally, the label-keep probabilities implied by epsilon in {3,5} are approximately 0.95 and 0.99, so the practical privacy protection for advertisers is weak; the paper does not discuss whether this satisfies advertiser privacy requirements.
minor comments (4)
- [Table 1] The 'Baseline' row lists 200 million trainable parameters, which appears inconsistent with a no-fine-tuning baseline; please clarify whether these parameters are trainable and whether the baseline is updated at all.
- [Section 2.2] The description of gradient compression does not state whether QSGD/top-k/powerSGD is applied to the per-sample partial derivatives before they are sent to the clean room or to the aggregated gradient, and how compression interacts with the aggregated-gradient computation.
- [Appendix A.3] The calibration ratio formula is typeset ambiguously; please write it with explicit parentheses, e.g., E[p_i] / (n_+/(n_+ + n_-)).
- [Section 2.1] The paper should either cite a concrete reference for the claim that large-batch aggregation resists DLG-style attacks or remove the claim; as written it is unsupported and contradicted by the linear-recovery argument.
Circularity Check
No material circularity: the LoRA gains and label-DP ROC-AUC results are empirical, and the debiasing loss is derived from the label-flip probability rather than fitted to the reported metrics.
full rationale
The paper's derivation chain is not circular. The headline LoRA result (+3.48 vs +3.63 ROC-AUC with 4M vs 200M trainable parameters) is a direct measurement against a baseline, not a quantity reconstructed from the model's own inputs. The label-DP debiasing loss l_deb(p)=l(p q_e+(1-p)(1-q_e)) is derived algebraically from the flip probability q_e=e^eps/(e^eps+1), with no constant fitted to the reported ROC-AUC or calibration values; the reported calibration of 1.0 is a designed consequence of that transformation, but it is presented as a consistency check rather than as a predicted output used to validate the method. The paper cites the authors' prior label-leakage work [14,18,24] for background on gradient-based label inference, but the framework's own privacy argument is an informal assertion in Section 2.1 and the experimental accuracy measurements do not depend on those citations as premises. Whether the batch-aggregated gradient is actually hard to invert is a correctness or privacy question, not a circularity question; even if the premise were false, the ROC-AUC numbers would remain empirical. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no known result is merely renamed. Hence no material circularity.
Assumptions & free parameters
free parameters (3)
- Label DP privacy budget epsilon =
5 and 3 (tested)
- LoRA rank =
64, 32, 1
- Aggregation batch size =
tens of thousands (exact value not reported)
assumptions (3)
- domain assumption The data clean room is a trusted third party that does not collude with the advertising platform and correctly performs the join and loss computation.
- domain assumption With a very large batch (tens of thousands), aggregated gradients of the form sum (p_i - y_i) dz_i/df do not allow label recovery by gradient-matching attacks.
- domain assumption The CVR model can be pretrained without conversion data, and LoRA adapters can recover most of the benefit of full fine-tuning.
invented entities (1)
-
Gated per-advertiser LoRA adapter
independent evidence
Cite this review
Pith. "Pith review of Privacy Preserving Conversion Modeling in Data Clean Room." pith.science (2026). https://pith.science/paper/N5O6B4G6
@misc{pith2026250514959,
author = {Pith},
title = {Pith review of: Privacy Preserving Conversion Modeling in Data Clean Room},
year = {2026},
howpublished = {\url{https://pith.science/paper/N5O6B4G6}},
note = {Machine review of arXiv:2505.14959}
}
read the original abstract
In the realm of online advertising, accurately predicting the conversion rate (CVR) is crucial for enhancing advertising efficiency and user satisfaction. This paper addresses the challenge of CVR prediction while adhering to user privacy preferences and advertiser requirements. Traditional methods face obstacles such as the reluctance of advertisers to share sensitive conversion data and the limitations of model training in secure environments like data clean rooms. We propose a novel model training framework that enables collaborative model training without sharing sample-level gradients with the advertising platform. Our approach introduces several innovative components: (1) utilizing batch-level aggregated gradients instead of sample-level gradients to minimize privacy risks; (2) applying adapter-based parameter-efficient fine-tuning and gradient compression to reduce communication costs; and (3) employing de-biasing techniques to train the model under label differential privacy, thereby maintaining accuracy despite privacy-enhanced label perturbations. Our experimental results, conducted on industrial datasets, demonstrate that our method achieves competitive ROCAUC performance while significantly decreasing communication overhead and complying with both advertiser privacy requirements and user privacy choices. This framework establishes a new standard for privacy-preserving, high-performance CVR prediction in the digital advertising landscape.
Figures
Reference graph
Works this paper leans on
-
[1]
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 308–318
2016
-
[2]
Alham Fikri Aji and Kenneth Heafield. 2017. Sparse communication for dis- tributed gradient descent.arXiv preprint arXiv:1704.05021(2017)
arXiv 2017
-
[3]
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2017. QSGD: Communication-efficient SGD via gradient quantization and encoding. Advances in neural information processing systems30 (2017)
2017
-
[4]
Olivier Chapelle. 2014. Modeling delayed feedback in display advertising. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 1097–1105
2014
-
[5]
Cynthia Dwork. 2006. Differential privacy. InInternational colloquium on au- tomata, languages, and programming. Springer, 1–12
2006
-
[6]
Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang. 2022. Label inference attacks against vertical federated learning. In31st USENIX security symposium (USENIX Security 22). 1397–1414
2022
-
[7]
Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, and Chiyuan Zhang. 2021. Deep learning with label differential privacy.Advances in neural information processing systems34 (2021), 27131–27145
2021
-
[8]
Yuyao Guo, Haoming Li, Xiang Ao, Min Lu, Dapeng Liu, Lei Xiao, Jie Jiang, and Qing He. 2022. Calibrated Conversion Rate Prediction via Knowledge Distillation under Delayed Feedback in Online Advertising. InProceedings of the 31st ACM International Conference on Information & Knowledge Management. 3983–3987. RecSys ’24, October 14–18, 2024, Bari, Italy Kun...
work page 2022
Show all 29 references
-
[9]
Elad Haramaty, Zohar Karnin, Arnon Lazerson, Liane Lewin-Eytan, and Yoelle Maarek. 2023. Extended conversion: Capturing successful interactions in voice shopping. InProceedings of the 17th ACM Conference on Recommender Systems. 826–832
2023
-
[10]
Tilman Herbrich. 2022. Data Clean Rooms.Computer Law Review International 23, 4 (2022), 109–120
2022
-
[11]
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685(2021)
2021 arXiv
-
[12]
Garrett Johnson, Julian Runge, and Eric Seufert. 2022. Privacy-centric digital advertising: Implications for research.Customer Needs and Solutions9, 1 (2022), 49–54
2022
-
[13]
Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. InProceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 768–776
2012
-
[14]
Oscar Li, Jiankai Sun, Xin Yang, Weihao Gao, Hongyi Zhang, Junyuan Xie, Vir- ginia Smith, and Chong Wang. 2021. Label leakage and protection in two-party split learning.arXiv preprint arXiv:2102.08504(2021)
2021 arXiv
-
[15]
Yang Liu, Yan Kang, Tianyuan Zou, Yanhong Pu, Yuanqin He, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang, and Qiang Yang. 2024. Vertical Federated Learning: Concepts, Advances, and Challenges.IEEE Transactions on Knowledge and Data Engineering(2024)
2024
-
[16]
Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. InThe 41st International ACM SIGIR Conference on Research & Development in Information Retriev...
2018
-
[17]
Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, and Aaron Flores. 2019. Predicting different types of conversions with multi-task learning in online advertising. InProceedings of the 25th acm sigkdd international conference on knowledge discovery & data mining. 2689–2697
2019
-
[18]
Jiankai Sun, Xin Yang, Yuanshun Yao, and Chong Wang. 2022. Label Leakage and Protection from Forward Embedding in Vertical Federated Learning. (2022). arXiv:2203.01451 [cs.LG]
2022 arXiv
-
[19]
Nam-Phuong Tran, Nhu-Ngoc Dao, The-Vi Nguyen, and Sungrae Cho. 2022. Privacy-preserving learning models for communication: A tutorial on advanced split learning. In2022 13th International Conference on Information and Commu- nication Technology Convergence (ICTC). IEEE, 1059–1064
2022
-
[20]
Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi. 2019. PowerSGD: Practical low-rank gradient compression for distributed optimization.Advances in Neural Information Processing Systems32 (2019)
2019
-
[21]
Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and An- drew Zhai. 2023. Transact: Transformer-based realtime user action model for recommendation at pinterest. InProceedings of the ...
2023
-
[22]
Hang Xu, Chen-Yu Ho, Ahmed M Abdelmoniem, Aritra Dutta, El Houcine Bergou, Konstantinos Karatsenidis, Marco Canini, and Panos Kalnis. 2021. Grace: A compressed communication framework for distributed machine learning. In2021 IEEE 41st international conference on distributed co...
2021
-
[23]
Zixuan Xu, Penghui Wei, Weimin Zhang, Shaoguo Liu, Liang Wang, and Bo Zheng
-
[24]
Xin Yang, Jiankai Sun, Yuanshun Yao, Junyuan Xie, and Chong Wang. 2022. Differentially Private Label Protection in Split Learning. arXiv:2203.02073 [cs.LG] https://arxiv.org/abs/2203.02073
2022 arXiv
-
[25]
Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Michael He, et al. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. arXiv preprint arXiv:2402.17152(2024)
2024 arXiv
-
[26]
Buyun Zhang, Liang Luo, Yuxin Chen, Jade Nie, Xi Liu, Daifeng Guo, Yanli Zhao, Shen Li, Yuchen Hao, Yantao Yao, et al. 2024. Wukong: Towards a Scaling Law for Large-Scale Recommendation.arXiv preprint arXiv:2403.02545(2024)
2024 arXiv
-
[27]
Buyun Zhang, Liang Luo, Xi Liu, Jay Li, Zeliang Chen, Weilin Zhang, Xiaohan Wei, Yuchen Hao, Michael Tsang, Wenjun Wang, et al. 2022. DHEN: A deep and hierarchical ensemble network for large-scale click-through rate prediction. arXiv preprint arXiv:2203.11014(2022)
2022 arXiv
-
[28]
Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep Leakage from Gradients. arXiv:1906.08935 [cs.LG] Privacy Preserving Conversion Modeling in Data Clean Room RecSys ’24, October 14–18, 2024, Bari, Italy A Appendix A.1 CVR model architecture A1 shows our CVR model architecture. ...
2019 arXiv
-
[2022]
InProceedings of the ACM Web Conference 2022
Ukd: Debiasing conversion rate estimation via uncertainty-regularized knowledge distillation. InProceedings of the ACM Web Conference 2022. 2078– 2087
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.