Pith. sign in

REVIEW 3 major objections 4 minor 29 references

Privacy Preserving Conversion Modeling in Data Clean Room

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read The paper claims that conversion models can be trained collaboratively on advertiser labels without sharing sample-level gradients, using batch-level aggregated gradients, gated LoRA adapters, and label differential privacy with a…

desk verdict The privacy premise is false: the platform can solve for all labels from a single batch update, so the paper's central claim collapses. read the letter →

arxiv 2505.14959 v1 pith:N5O6B4G6 submitted 2025-05-20 cs.LG cs.IR

classification cs.LGcs.IR
keywords privacy-preservingmachinelearningCVRpredictiondatacleanroomsplitlabeldifferentialprivacyLoRAfine-tuninggradientcompressionconversionmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to train conversion-rate (CVR) models when the advertiser holding the conversion labels does not want to share sample-level data, and the advertising platform holding the features is too large to train inside a data clean room. It claims that by exchanging only batch-level aggregated gradients between the clean room and the platform, and by fine-tuning only a small set of gated LoRA adapters, the model can capture about 95% of the ROC-AUC gain of full fine-tuning while cutting training time per day of data from 2.5 days to under an hour. Adding label differential privacy with a debiased loss reportedly keeps predictions fully calibrated and limits the ROC-AUC drop to 0.2-0.5% for epsilon between 5 and 3. If these numbers hold, advertisers can contribute conversion signals without exposing raw conversions, and platforms can serve many advertisers by adding one small adapter per advertiser. The paper's central claim is that privacy protection, communication efficiency, and accuracy can be had together in this setup.

What carries the argument

The load-bearing pieces are (1) batch-level aggregated gradients in place of sample-level gradients, computed in the clean room as sum_i (p_i - y_i) * dz_i/df; (2) gated LoRA adapter layers, a parameter-efficient fine-tuning mechanism that adds small trainable low-rank matrices to selected layers (QKV in sequence transformers, linear layers in feature interaction and multi-task towers) and gates them per advertiser; and (3) label differential privacy with a debiasing loss, where labels are flipped with probability determined by epsilon and the loss is evaluated on the transformed prediction p' = p q_epsilon + (1-p)(1-q_epsilon) so the model output remains calibrated. These work together to reduce communication, hide labels, and preserve accuracy.

What would settle it

Run a label-inference attack, such as a DLG-style optimization or a linear-system solver, against the aggregated gradients the paper's protocol would produce; if the attack recovers individual labels or substantially raises the accuracy of a label-guessing adversary beyond random chance using only the exchanged messages, the central privacy claim is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that sample-level gradients are unnecessary for collaborative CVR training: the clean room can compute the aggregated gradient sum_i (p_i - y_i) * dz_i/df from the platform's per-sample partial derivatives and the labels, and send only that sum back. With tens of thousands of samples per batch, the paper argues this aggregation hides individual labels from gradient-matching attacks such as DLG. To make the scheme practical, the platform fine-tunes only gated LoRA adapters (1-2% of parameters), which yields 95% of the full fine-tuning ROC-AUC gain, and further compresses gradients with QSGD or BF16. For label privacy, random label flipping with epsilon in {3,5} is applied, and a debiased loss l_deb(p)=l(p q_epsilon + (1-p)(1-q_epsilon)) recovers full calibration (1.0) with only a 0.2-0.5% ROC-AUC drop. The paper reports these results on internal datasets with over 10 billion entries and roughly 300 features.

Load-bearing premise

The whole privacy promise rests on an unproven assumption that an adversary cannot extract meaningful label information from batch-level aggregated gradients, especially when the batch is large and the fine-tuned parameters are few; the paper offers no formal guarantee for this.

Editorial extensions

If this is right

  • Advertisers who currently withhold conversion data can safely contribute it, unlocking CVR improvements for their campaigns and lowering cost per action.
  • Training a large CVR model in a clean-room setting becomes feasible on ordinary network links: a day of data that took 2.5 days to train now takes under an hour.
  • Each advertiser can be served by its own LoRA adapter on a shared base model, enabling parallel, independent, multi-tenant fine-tuning without cross-advertiser interference.
  • Label DP with the debiased loss gives calibrated probability estimates at epsilon as low as 3, so downstream ranking systems can trust the predicted conversion scores.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The privacy argument is informal: an adversary who sees many aggregated gradients across training steps might reconstruct label statistics or even individual labels, especially with known model weights; a formal end-to-end DP analysis over the full training run would be needed to make the guarantee rigorous.
  • The debiasing formula assumes a fixed, known flip probability; if label flipping probabilities vary across samples or adapt over time, the calibration recovery may need re-derivation.
  • The same three-part recipe (aggregated gradients, adapters, label DP with debiasing) could transfer to other feature-party/label-party settings outside advertising, such as medical or financial collaboration, wherever raw labels cannot be shared.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a split-learning-style framework for privacy-preserving conversion rate (CVR) prediction in a data clean room setting. The advertising platform (feature party) computes logits and per-sample partial derivatives, the clean room (label party) computes a batch-level aggregated gradient, and the model is updated without sharing sample-level gradients. To reduce communication, the paper introduces gated LoRA adapters that fine-tune only about 1% of the model parameters. To protect labels, it applies label differential privacy with a debiased loss that accounts for the label-flipping probability q_epsilon = e^epsilon/(e^epsilon+1). Experiments on internal industrial datasets report that fine-tuning 2 million parameters captures about 95% of the ROC-AUC gain of full fine-tuning, and that with debiasing the calibration ratio becomes 1.0 while ROC-AUC drops only 0.2%-0.5% for epsilon in {3,5}. The paper also reports up to 4x gradient compression with QSGD/BF16 at a small ROC-AUC cost.

Significance. If the privacy claims held, this framework would be a practical contribution to CVR modeling in data clean rooms, with a reported reduction in training communication from 2.5 days to under 1 hour for large advertisers. The debiased-loss derivation in Section 2.3 and Appendix A.3 is mathematically correct and is a useful, reproducible element: the debiased loss l_deb(p)=l(p q+(1-p)(1-q)) follows directly from the label-flip mechanism and is not fit to the reported AUC numbers. The industrial-scale evaluation is another strength. However, the central privacy-preserving claim is undermined by a load-bearing flaw: Section 2.1's assertion that large-batch aggregated gradients resist label recovery is false in the exact parameter regime used in the paper, and the paper never jointly evaluates the proposed LoRA adapters with label DP. The claimed privacy-preserving accuracy is therefore not established as written.

major comments (3)
  1. [Section 2.1, aggregated gradient formula] The claim that batch-level aggregated gradients 'shall not be easily attacked' is incorrect. The platform knows p_i = sigma(z_i) and v_i = dz_i/df for every sample, because it computes z_i and v_i and sends them to the clean room. From the returned gradient g = sum_i (p_i - y_i) v_i, the platform can form s = sum_i p_i v_i - g = sum_i y_i v_i = V y, where V is the d x b matrix with columns v_i and y is the binary label vector. In every LoRA configuration in Table 1, d = ||f|| (e.g., 2 million for rank 32) exceeds the batch size b (tens of thousands), so V generically has full column rank and y = (V^T V)^{-1} V^T s is recovered exactly. Thus the no-DP LoRA results in Table 1 do not protect label privacy, and the paper's stated condition for adding label DP (||f|| > b) is precisely the condition under which the linear inversion succeeds. The paper never reports LoRA plus label DP jointly, so the headline 'about 95% of the gain from fine tuning all parameters' (Section 3) is not a privacy-preserving result as written.
  2. [Section 3, Tables 1 and 2] The experimental support for the privacy-preserving accuracy claims is incomplete. Table 2 reports a single baseline and single-point ROC-AUC differences with no error bars, significance tests, or absolute AUC values, and the model configuration (full fine-tuning vs. LoRA, rank, batch size) underlying Table 2 is not stated. Because Table 1 (LoRA) and Table 2 (label DP and compression) are presented as separate settings, the reader cannot determine the ROC-AUC or calibration of the proposed framework when LoRA and label DP are combined. The statement that with debiasing the ROC-AUC drop is 0.2%-0.5% 'which is acceptable in production' is not supported without variance estimates and without specifying the base model the drop is measured against.
  3. [Sections 2.1 and 2.3] The privacy guarantee is underspecified. The paper does not define the threat model (who the adversary is, what auxiliary information it has), does not provide a privacy accounting for label DP (per-epoch epsilon, composition over multiple training epochs, or how batch aggregation affects the DP guarantee), and does not analyze the d < b regime where the linear system for labels is underdetermined. The claim that label DP 'effectively defends' against the equation-solving attack is an assertion, not a demonstrated property. A concrete evaluation against the linear-recovery attack described above, with and without label DP, is needed to support the central privacy claim. Additionally, the label-keep probabilities implied by epsilon in {3,5} are approximately 0.95 and 0.99, so the practical privacy protection for advertisers is weak; the paper does not discuss whether this satisfies advertiser privacy requirements.
minor comments (4)
  1. [Table 1] The 'Baseline' row lists 200 million trainable parameters, which appears inconsistent with a no-fine-tuning baseline; please clarify whether these parameters are trainable and whether the baseline is updated at all.
  2. [Section 2.2] The description of gradient compression does not state whether QSGD/top-k/powerSGD is applied to the per-sample partial derivatives before they are sent to the clean room or to the aggregated gradient, and how compression interacts with the aggregated-gradient computation.
  3. [Appendix A.3] The calibration ratio formula is typeset ambiguously; please write it with explicit parentheses, e.g., E[p_i] / (n_+/(n_+ + n_-)).
  4. [Section 2.1] The paper should either cite a concrete reference for the claim that large-batch aggregation resists DLG-style attacks or remove the claim; as written it is unsupported and contradicted by the linear-recovery argument.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the LoRA gains and label-DP ROC-AUC results are empirical, and the debiasing loss is derived from the label-flip probability rather than fitted to the reported metrics.

full rationale

The paper's derivation chain is not circular. The headline LoRA result (+3.48 vs +3.63 ROC-AUC with 4M vs 200M trainable parameters) is a direct measurement against a baseline, not a quantity reconstructed from the model's own inputs. The label-DP debiasing loss l_deb(p)=l(p q_e+(1-p)(1-q_e)) is derived algebraically from the flip probability q_e=e^eps/(e^eps+1), with no constant fitted to the reported ROC-AUC or calibration values; the reported calibration of 1.0 is a designed consequence of that transformation, but it is presented as a consistency check rather than as a predicted output used to validate the method. The paper cites the authors' prior label-leakage work [14,18,24] for background on gradient-based label inference, but the framework's own privacy argument is an informal assertion in Section 2.1 and the experimental accuracy measurements do not depend on those citations as premises. Whether the batch-aggregated gradient is actually hard to invert is a correctness or privacy question, not a circularity question; even if the premise were false, the ROC-AUC numbers would remain empirical. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no known result is merely renamed. Hence no material circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The reportable 'free parameters' are experimental settings (privacy budget, LoRA rank, batch size) rather than constants fitted to force a target result. The load-bearing assumptions are the trusted clean room, the informal large-batch privacy argument, and the availability of a pretrained base model.

free parameters (3)
  • Label DP privacy budget epsilon = 5 and 3 (tested)
    Controls the label flip probability q_eps = e^eps/(e^eps+1); Table 2 reports AUC and calibration for eps=5 and eps=3. The de-biasing formula and reported privacy-utility tradeoff depend on it.
  • LoRA rank = 64, 32, 1
    Table 1 reports ROC-AUC gains for ranks 64, 32, 1, corresponding to 4M, 2M, and 0.06M trainable parameters. Rank is chosen by hand; the central '95% of full fine-tuning gain' result depends on rank 32.
  • Aggregation batch size = tens of thousands (exact value not reported)
    The privacy argument that aggregated gradients do not leak labels relies on batch size b being large relative to parameter count ||f||; no exact value is given.
assumptions (3)
  • domain assumption The data clean room is a trusted third party that does not collude with the advertising platform and correctly performs the join and loss computation.
    Section 2.1 and the intro describe the clean room as 'a trusted third party for both parties'; the entire protocol assumes this trust model.
  • domain assumption With a very large batch (tens of thousands), aggregated gradients of the form sum (p_i - y_i) dz_i/df do not allow label recovery by gradient-matching attacks.
    Section 2.1 states this informally and says DLG-style attacks 'shall not be easily' applied; no formal analysis of residual leakage through iterative aggregates, logits, or auxiliary information is given.
  • domain assumption The CVR model can be pretrained without conversion data, and LoRA adapters can recover most of the benefit of full fine-tuning.
    Table 1 assumes a pretrained base model and shows LoRA gains; the pretraining procedure and base model are not described.
invented entities (1)
  • Gated per-advertiser LoRA adapter independent evidence
    purpose: Parameter-efficient fine-tuning of a shared CVR model for individual advertisers while serving non-fine-tuned advertisers with the original weights.
    The adapter's effectiveness is measured in Table 1 across ranks 64, 32, and 1, giving a falsifiable handle on the internal dataset; the component itself is a design choice rather than a physically postulated entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Privacy Preserving Conversion Modeling in Data Clean Room." pith.science (2026). https://pith.science/paper/N5O6B4G6

@misc{pith2026250514959,
  author       = {Pith},
  title        = {Pith review of: Privacy Preserving Conversion Modeling in Data Clean Room},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N5O6B4G6}},
  note         = {Machine review of arXiv:2505.14959}
}
read the original abstract

In the realm of online advertising, accurately predicting the conversion rate (CVR) is crucial for enhancing advertising efficiency and user satisfaction. This paper addresses the challenge of CVR prediction while adhering to user privacy preferences and advertiser requirements. Traditional methods face obstacles such as the reluctance of advertisers to share sensitive conversion data and the limitations of model training in secure environments like data clean rooms. We propose a novel model training framework that enables collaborative model training without sharing sample-level gradients with the advertising platform. Our approach introduces several innovative components: (1) utilizing batch-level aggregated gradients instead of sample-level gradients to minimize privacy risks; (2) applying adapter-based parameter-efficient fine-tuning and gradient compression to reduce communication costs; and (3) employing de-biasing techniques to train the model under label differential privacy, thereby maintaining accuracy despite privacy-enhanced label perturbations. Our experimental results, conducted on industrial datasets, demonstrate that our method achieves competitive ROCAUC performance while significantly decreasing communication overhead and complying with both advertiser privacy requirements and user privacy choices. This framework establishes a new standard for privacy-preserving, high-performance CVR prediction in the digital advertising landscape.

Figures

Figures reproduced from arXiv: 2505.14959 by the authors.

Figure 1
Figure 1. Comparison of (a) standard split learning frame [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Adapter based CVR model. In modern ads delivery systems, the CVR models used in late stage ads ranking are usually very large, with model sizes from a few GBs to TBs [21, 25–27]. This creates a significant challenge due to the high communication costs associated with transmitting sample￾level partial derivatives from the advertising platform to the clean room. In our use case, it would take about 2.5 days to complet… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 11 canonical work pages

  1. [1]

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 308–318

  2. [2]

    Alham Fikri Aji and Kenneth Heafield. 2017. Sparse communication for dis- tributed gradient descent.arXiv preprint arXiv:1704.05021(2017)

  3. [3]

    Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2017. QSGD: Communication-efficient SGD via gradient quantization and encoding. Advances in neural information processing systems30 (2017)

  4. [4]

    Olivier Chapelle. 2014. Modeling delayed feedback in display advertising. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 1097–1105

  5. [5]

    Cynthia Dwork. 2006. Differential privacy. InInternational colloquium on au- tomata, languages, and programming. Springer, 1–12

  6. [6]

    Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang. 2022. Label inference attacks against vertical federated learning. In31st USENIX security symposium (USENIX Security 22). 1397–1414

  7. [7]

    Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, and Chiyuan Zhang. 2021. Deep learning with label differential privacy.Advances in neural information processing systems34 (2021), 27131–27145

  8. [8]

    Yuyao Guo, Haoming Li, Xiang Ao, Min Lu, Dapeng Liu, Lei Xiao, Jie Jiang, and Qing He. 2022. Calibrated Conversion Rate Prediction via Knowledge Distillation under Delayed Feedback in Online Advertising. InProceedings of the 31st ACM International Conference on Information & Knowledge Management. 3983–3987. RecSys ’24, October 14–18, 2024, Bari, Italy Kun...

Show all 29 references
  1. [9]

    Elad Haramaty, Zohar Karnin, Arnon Lazerson, Liane Lewin-Eytan, and Yoelle Maarek. 2023. Extended conversion: Capturing successful interactions in voice shopping. InProceedings of the 17th ACM Conference on Recommender Systems. 826–832

  2. [10]

    Tilman Herbrich. 2022. Data Clean Rooms.Computer Law Review International 23, 4 (2022), 109–120

  3. [11]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685(2021)

  4. [12]

    Garrett Johnson, Julian Runge, and Eric Seufert. 2022. Privacy-centric digital advertising: Implications for research.Customer Needs and Solutions9, 1 (2022), 49–54

  5. [13]

    Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating conversion rate in display advertising from past erformance data. InProceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining. 768–776

  6. [14]

    Oscar Li, Jiankai Sun, Xin Yang, Weihao Gao, Hongyi Zhang, Junyuan Xie, Vir- ginia Smith, and Chong Wang. 2021. Label leakage and protection in two-party split learning.arXiv preprint arXiv:2102.08504(2021)

  7. [15]

    Yang Liu, Yan Kang, Tianyuan Zou, Yanhong Pu, Yuanqin He, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang, and Qiang Yang. 2024. Vertical Federated Learning: Concepts, Advances, and Challenges.IEEE Transactions on Knowledge and Data Engineering(2024)

  8. [16]

    Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. InThe 41st International ACM SIGIR Conference on Research & Development in Information Retriev...

  9. [17]

    Junwei Pan, Yizhi Mao, Alfonso Lobos Ruiz, Yu Sun, and Aaron Flores. 2019. Predicting different types of conversions with multi-task learning in online advertising. InProceedings of the 25th acm sigkdd international conference on knowledge discovery & data mining. 2689–2697

  10. [18]

    Jiankai Sun, Xin Yang, Yuanshun Yao, and Chong Wang. 2022. Label Leakage and Protection from Forward Embedding in Vertical Federated Learning. (2022). arXiv:2203.01451 [cs.LG]

  11. [19]

    Nam-Phuong Tran, Nhu-Ngoc Dao, The-Vi Nguyen, and Sungrae Cho. 2022. Privacy-preserving learning models for communication: A tutorial on advanced split learning. In2022 13th International Conference on Information and Commu- nication Technology Convergence (ICTC). IEEE, 1059–1064

  12. [20]

    Thijs Vogels, Sai Praneeth Karimireddy, and Martin Jaggi. 2019. PowerSGD: Practical low-rank gradient compression for distributed optimization.Advances in Neural Information Processing Systems32 (2019)

  13. [21]

    Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and An- drew Zhai. 2023. Transact: Transformer-based realtime user action model for recommendation at pinterest. InProceedings of the ...

  14. [22]

    Hang Xu, Chen-Yu Ho, Ahmed M Abdelmoniem, Aritra Dutta, El Houcine Bergou, Konstantinos Karatsenidis, Marco Canini, and Panos Kalnis. 2021. Grace: A compressed communication framework for distributed machine learning. In2021 IEEE 41st international conference on distributed co...

  15. [23]

    Zixuan Xu, Penghui Wei, Weimin Zhang, Shaoguo Liu, Liang Wang, and Bo Zheng

  16. [24]

    Xin Yang, Jiankai Sun, Yuanshun Yao, Junyuan Xie, and Chong Wang. 2022. Differentially Private Label Protection in Split Learning. arXiv:2203.02073 [cs.LG] https://arxiv.org/abs/2203.02073

  17. [25]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhao- jie Gong, Fangda Gu, Michael He, et al. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. arXiv preprint arXiv:2402.17152(2024)

  18. [26]

    Buyun Zhang, Liang Luo, Yuxin Chen, Jade Nie, Xi Liu, Daifeng Guo, Yanli Zhao, Shen Li, Yuchen Hao, Yantao Yao, et al. 2024. Wukong: Towards a Scaling Law for Large-Scale Recommendation.arXiv preprint arXiv:2403.02545(2024)

  19. [27]

    Buyun Zhang, Liang Luo, Xi Liu, Jay Li, Zeliang Chen, Weilin Zhang, Xiaohan Wei, Yuchen Hao, Michael Tsang, Wenjun Wang, et al. 2022. DHEN: A deep and hierarchical ensemble network for large-scale click-through rate prediction. arXiv preprint arXiv:2203.11014(2022)

  20. [28]

    Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep Leakage from Gradients. arXiv:1906.08935 [cs.LG] Privacy Preserving Conversion Modeling in Data Clean Room RecSys ’24, October 14–18, 2024, Bari, Italy A Appendix A.1 CVR model architecture A1 shows our CVR model architecture. ...

  21. [2022]

    InProceedings of the ACM Web Conference 2022

    Ukd: Debiasing conversion rate estimation via uncertainty-regularized knowledge distillation. InProceedings of the ACM Web Conference 2022. 2078– 2087

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.