Pith. sign in

REVIEW 4 major objections 6 minor 18 references

EGEAN: An Exposure-Guided Embedding Alignment Network for Post-Click Conversion Estimation

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that aligning embeddings to the exposure space, together with a parameter-varying doubly robust estimator, removes covariate-shift bias in post-click conversion prediction and improves real-world advertising revenue.

desk verdict Solid engineering with a broken theoretical claim: the PVDR estimator as written is not doubly robust, but the paper's empirical contribution deserves a referee. read the letter →

arxiv 2412.06852 v1 pith:DBU7Z4RD submitted 2024-12-08 cs.LG cs.AI

Huajian Feng , Guoxiao Zhang , Yadong Zhang , Yi We , Qiang Liu This is my paper · ORCID
classification cs.LGcs.AI
keywords post-clickconversionratecovariateshiftsampleselectionbiasdoublyrobustestimatorembeddingalignmentmetriclearningmulti-taskonlineadvertising
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the covariate shift between clicked and unclicked ad impressions—not just the missing labels that cause sample selection bias—is a correctable source of error in post-click conversion rate (CVR) estimation. Its proposed network, EGEAN, aligns user-item embeddings toward the exposure space using an exposure prediction task, task-personalized modules, and an MMD metric loss, so that knowledge from non-clicked impressions transfers to clicked ones. The paper also proposes PVDR, a parameter-varying doubly robust estimator with a steady-state control condition, which it claims is unbiased when that condition holds and which is designed to handle small propensity scores better than the StableDR estimator it extends. On the Ali-CCP public dataset and a three-month Meituan industrial dataset, EGEAN+PVDR reports the highest CVR and CTCVR AUC, and a seven-day online A/B test reports 5.94% higher CVR and 6.29% higher GMV against the deployed DDPO baseline.

What carries the argument

The load-bearing objects are the EGEAN architecture and the PVDR estimator. The architecture's exposure prediction task defines positive samples as exposed impressions and uses in-batch negative sampling, giving the shared embedding a target that exists in both click and non-click spaces; the task-personalized network (LoRA adaptors plus EPNet and PPNet gates) then lets the CTR and CVR towers modify those embeddings and parameters without forgetting the shared structure, while the metric-learning loss, measured with MMD in an RKHS, pulls CVR-task embeddings back toward the shared embedding. The PVDR estimator is the ratio of a propensity-weighted prediction-error sum to λ|D|+(1−λ) times the sum of inverse propensities, with the steady-state condition λ+(1−λ)A=B asserting unbiasedness. Together these mechanisms are what the paper claims remove both sample selection bias and covariate shift.

What would settle it

Recompute PVDR's expectation under the missing-not-at-random model and check whether the steady-state condition forces the first-order bias to zero; a two-point or synthetic dataset with known true CVRs and very small propensities would expose any residual bias directly. Simpler: run EGEAN+PVDR with λ fixed at a value violating Eq. (11) and observe whether AUC or calibration degrades in the predicted way.

Watch

Extended reading notes

Core claim

The paper's central claim is that an exposure-guided alignment mechanism can correct the covariate shift between click and non-click spaces, and that this correction, combined with a parameter-varying doubly robust estimator, yields unbiased CVR estimation over the full impression space. Concretely, EGEAN trains an exposure probability prediction task on exposed samples, fine-tunes shared embeddings with LoRA, personalizes both embeddings and MLP layers through task-specific gates, and minimizes the Maximum Mean Discrepancy between CVR-task embeddings and shared embeddings. The PVDR estimator generalizes StableDR by introducing a parameter λ that interpolates between IPS (λ=0) and StableDR (λ=1); when the steady-state condition λ+(1−λ)A=B is met, the paper asserts the estimator is doubly robust and unbiased. Empirically, EGEAN+PVDR reaches 0.6604 CVR AUC on Ali-CCP and 0.7093 on Meituan, outperforming all listed baselines, and the deployed system reportedly serves main Meituan traffic after the A/B test.

Load-bearing premise

The estimator's unbiasedness collapses if the steady-state condition λ+(1−λ)A=B (Eq. 11) is not a genuine sufficient condition for unbiasedness, since the paper asserts this identity without proof and chooses λ so that it holds.

Editorial extensions

If this is right

  • If PVDR is indeed unbiased under the steady-state condition, CVR models can be trained and evaluated directly on the entire impression space without relying on the click space alone.
  • The exposure-guided alignment gives a way to transfer information from non-clicked impressions to clicked ones without adversarial training, which the paper argues is unstable on small datasets.
  • Because λ interpolates between IPS and StableDR, PVDR offers a tunable bias-variance trade-off for small propensity scores.
  • The reported online gains (5.94% CVR, 6.29% GMV) imply that correcting covariate shift can produce revenue-relevant improvements in a large-scale advertising system.
  • Ablations show that each of the three components—exposure network, task personalization, and metric learning—contributes to the final AUC, meaning the gains are not from a single trick.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The unbiasedness claim of PVDR is only as strong as Eq. (11), which the paper states without proof; a natural next step is to derive the condition from the expectation of Eq. (12) and test it on synthetic data with known ground truth.
  • Exposure-guided alignment should transfer to other label-sparse prediction tasks that share an intermediate event between two spaces, such as post-view conversion or subscription prediction.
  • The paper reports point AUC values without confidence intervals; comparing EGEAN+PVDR against EGEAN+DR on repeated runs would quantify how much of the gain comes from the architecture versus the estimator.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes EGEAN, a multi-task network for post-click conversion estimation, with three components: an exposure probability prediction task, a task-personalized network (LoRA, EPNet, PPNet), and an MMD-based metric learning module. It also introduces PVDR, a 'Parameter Varying Doubly Robust Estimator' with a steady-state control condition intended to reduce variance under small propensities. The paper reports offline AUC results on Ali-CCP and Meituan, an ablation study, and a seven-day online A/B test showing 5.94% CVR and 6.29% GMV gains. The central theoretical claim is in Section 2.2.2: 'When the steady-state condition is met, PVDR is a doubly robust estimator, and the CVR estimation is unbiased.' The manuscript provides no derivation of this claim, and the estimator as written does not match the standard doubly robust form.

Significance. If the architecture results are reproducible, the exposure-guided alignment idea and the large-scale online deployment are notable strengths of the paper; the released code is also a positive element. However, the paper's main contribution is the PVDR estimator and its claimed unbiasedness/double-robustness property. That property is not merely under-derived; the estimator in Eq. (12) contains no imputation term, so the claim cannot hold in the standard doubly robust sense. Because the title, abstract, and contribution list all rest on this theoretical assertion, the significance of the paper as submitted is substantially undermined.

major comments (4)
  1. [Section 2.2.2, Eq. (12)] The central claim that PVDR is doubly robust and unbiased is not supported by the estimator as written. The loss in Eq. (12), L_PVDR = sum o e / p / [lambda |D| + (1-lambda) sum o/p], contains no imputation term; it is a self-normalized IPS estimator. A doubly robust loss must combine the inverse-propensity term o e/p with an imputation term such as (1 - o/p) e_hat, so that a correct imputation model can compensate for a misspecified propensity model. Here e_hat appears only as a scalar through the steady-state equation (11)/(13) used to choose lambda. Choosing lambda with e_hat does not add the missing correction to the numerator. Under a misspecified propensity model, E[o e/p] is not E[e], and no term in Eq. (12) can compensate unless e_hat is exactly the true error e, which is not argued. Thus the double-robustness/unbiasedness claim is at best unproven and, in the standard sense, false.
  2. [Section 2.2.2, Eqs. (11) and (13)] The steady-state condition lambda + (1-lambda) A = B is asserted without derivation. No proof is given that this condition is sufficient for unbiasedness, and no assumptions are stated under which it would be. Moreover, lambda is data-dependent because it is solved from Eq. (13) using the fitted imputation model and propensity estimates; an unbiasedness proof would require taking expectations over this data-dependent lambda. If Eq. (11) can always be satisfied by choosing lambda, then the condition is a normalization identity enforced by construction, not a substantive modeling assumption, and invoking it as a precondition for unbiasedness is circular. This is load-bearing because the contribution list and abstract claim the PVDR estimator's theoretical property.
  3. [Section 2.2.2, Eq. (10)] The imputation model is not actually defined as a supervised model. Eq. (10) writes Lhat = (1/|D|) sum e_hat, which is just an average of predicted values; the text does not specify the target labels or the training objective used to fit phi_theta to the 'CVR prediction error' e_u,i. Without a concrete training objective, the role of e_hat in the steady-state condition is not implementable or reproducible from the manuscript. This matters because e_hat is the only mechanism by which the imputation model influences the estimator.
  4. [Section 3.1, Table 2] The offline results report AUC values without standard deviations, numbers of repeated runs, or significance tests. The text states that EGEAN 'significantly outperforms' the baselines, but the table alone does not support a claim of statistical significance. This is secondary to the theoretical problems but affects the empirical credibility of the paper, especially because the gains over EGEAN+DR are small in absolute AUC terms.
minor comments (6)
  1. [Section 3.3] The text refers to the 'TSN module' but the described module is the task personalized network, abbreviated TPN elsewhere; the terminology should be made consistent.
  2. [Section 3.4] The online A/B section says 'As shown in Table 3' but Table 3 is the ablation table; no table reports the online A/B results, traffic allocation, or confidence intervals.
  3. [References] References [7] and [8] are the same StableDR paper, and reference [15] is a duplicate of reference [14]; these should be merged.
  4. [General] The ACM template metadata still contains placeholder conference information ('Conference acronym ’XX', 2018, Woodstock, NY, placeholder DOI) and needs to be updated for submission.
  5. [Section 2.2.2, Eq. (13)] Eq. (13) simply restates Eq. (11) with A and B expanded; consider defining A and B once and numbering a single condition to avoid redundancy.
  6. [Section 3.4] The paper reports '5.94% CVR and 6.29% GMV improvement' but does not state the baseline value, the number of users in each arm, or the significance of the differences; these details are needed for an industrial validation claim.

Circularity Check

1 steps flagged · score 7.0 of 10

The PVDR unbiasedness/double-robustness claim is asserted for a 'steady-state condition' that the paper itself imposes by solving for λ, so the theoretical result reduces to its own definition; the empirical AUC/A-B comparisons are external and not circular.

  1. self definitional [Section 2.2.2, Eqs. (11)-(13)]
    "We propose an ingenious formula to determine that PVDR is unbiased, which we refer to as the steady-state control condition. Specifically, λ+(1−λ)A = B (11) ... When the steady-state condition is met, PVDR is a doubly robust estimator, and the CVR estimation is unbiased. λ+(1−λ) 1/|D| Σ_{(u,i)∈D} o_{u,i}/p̂_{u,i} = Σ_{(u,i)∈D} o_{u,i}ê_{u,i}/p̂_{u,i} / Σ_{(u,i)∈D} ê_{u,i} (13)"

    Eq. (13) determines λ from the fitted propensity p̂ and imputation model ê, so Eq. (11) is enforced by construction rather than assumed. The unbiasedness/double-robustness statement is then conditioned on this self-imposed equation, with no derivation showing that Eq. (11) implies E[L_PVDR] = E[L_ideal]. Since ê appears in Eq. (12) only through the scalar λ (there is no imputation term (1−o/p̂)ê in the loss), the 'doubly robust' property cannot be inherited from any imputation correction; it is attached to the estimator by definition, and the 'when the condition is met' caveat is satisfied by the authors' own choice of λ.

full rationale

The circularity is concentrated in the theoretical claim of Section 2.2.2. The steady-state condition is not an external, testable hypothesis; it is an equation solved to pick the free parameter λ using the models under evaluation, and the estimator's unbiasedness is asserted for that condition rather than proved. The empirical parts of the paper are self-contained: Table 2 compares EGEAN variants against external baselines on public and industrial data, and the online A/B test reports deployment outcomes, so those comparisons are independent evidence. There are no load-bearing self-citations: StableDR, AECM, DDPO, and other cited works are not by the present authors, and no uniqueness theorem is imported from prior work. The score is 7 rather than 8 because the central theoretical result reduces by definition but the main empirical contribution (embedding alignment + AUC gains) is externally benchmarked; however, the PVDR unbiasedness claim itself is the paper's stated basis for the loss, and it is definitionally tied to the fitted steady-state condition.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The ledger contains the three fitted components the estimator depends on. No new physical or conceptual entities are introduced beyond architectural modules.

free parameters (3)
  • lambda (PVDR steady-state parameter) = not reported
    Solved from the steady-state condition lambda + (1 minus lambda) times A equals B, which depends on the fitted imputation model and propensity estimates. It is central to the unbiasedness claim.
  • imputation model parameters theta = not reported
    The imputation model estimates CVR prediction errors and enters the steady-state condition; the paper gives no training details.
  • propensity model parameters = not reported
    The estimator requires propensity scores p_hat, but the paper never describes how they are estimated; the debiasing behavior depends entirely on them.
assumptions (3)
  • domain assumption The exposure event is the shared part of click and non-click events, indicating an intrinsic connection between the covariate distributions in the click and non-click spaces.
    This motivates the exposure task and embedding alignment. If false, the alignment may not transfer useful information (Section 1, Section 2.1.1).
  • ad hoc to paper PVDR is unbiased when the steady-state condition lambda + (1 minus lambda) times A equals B holds.
    Stated without proof; the condition is used to set lambda, so the unbiasedness claim rests on an unverified identity (Section 2.2.2, Eq. 11-13).
  • standard math MMD in a reproducing kernel Hilbert space measures the distributional distance between CVR and shared embeddings.
    Standard result from the metric learning literature, used without proof (Section 2.1.3, Eq. 6).

how reviews work

0 comments
Cite this review

Pith. "Pith review of EGEAN: An Exposure-Guided Embedding Alignment Network for Post-Click Conversion Estimation." pith.science (2026). https://pith.science/paper/DBU7Z4RD

@misc{pith2026241206852,
  author       = {Pith},
  title        = {Pith review of: EGEAN: An Exposure-Guided Embedding Alignment Network for Post-Click Conversion Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DBU7Z4RD}},
  note         = {Machine review of arXiv:2412.06852}
}
read the original abstract

Accurate post-click conversion rate (CVR) estimation is crucial for online advertising systems. Despite significant advances in causal approaches designed to address the Sample Selection Bias problem, CVR estimation still faces challenges due to Covariate Shift. Given the intrinsic connection between the distribution of covariates in the click and non-click spaces, this study proposes an Exposure-Guided Embedding Alignment Network (EGEAN) to address estimation bias caused by covariate shift. Additionally, we propose a Parameter Varying Doubly Robust Estimator with steady-state control to handle small propensities better. Online A/B tests conducted on the Meituan advertising system demonstrate that our method significantly outperforms baseline models with respect to CVR and GMV, validating its effectiveness. Code is available: https://github.com/hydrogen-maker/EGEAN.

Figures

Figures reproduced from arXiv: 2412.06852 by the authors.

Figure 1
Figure 1. Illustration of user behaviors in Meituan online advertising. Currently, some causal approaches are proposed to adapt for the biased estimates from NMAR data[5, 9, 12–14, 16, 18]. The proof of the unbiasedness of these IPS and DR estimators is based on the assumption that the distribution of covariates is consistent, which has been pointed out to be false in AECM[16]. AECM[16] first introduces an adversarial method … view at source ↗
Figure 2
Figure 2. and PVDR estimator. User features Item features Concat MLP Show Exposure Task Share Embedding Lookup Table Gate MLP MLP MLP Weight 1 Weight 0 Weight 2 MLP MLP MLP MLP MLP MLP PCTR PCVR PVDR Loss Imputation Error CVR Embedding Metric Learning CVR Embedding Shared Embedding Maximum Density Divergence Metric Loss CTR Embedding [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) and [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 7 canonical work pages

  1. [1]

    Jianxin Chang, Chenbin Zhang, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. 2023. Pepnet: Parameter and embedding personalized network for infusing with personalized prior information. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 3795–3804

    2023

  2. [2]

    Bo Geng, Dacheng Tao, and Chao Xu. 2011. DAML: Domain adaptation metric learning. IEEE Transactions on Image Processing 20, 10 (2011), 2980–2989

    work page 2011

  3. [3]

    Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 9), Yee Whye Teh and Mike Titterington (Eds.). PMLR, Chia Laguna Resort, Sardinia, Italy, 249–2...

    work page 2010

  4. [4]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    arXiv 2021

  5. [5]

    Jiahui Huang, Lan Zhang, Junhao Wang, Shanyang Jiang, Dongbo Huang, Cheng Ding, and Lan Xu. 2024. Utilizing Non-click Samples via Semi-supervised Learn- ing for Conversion Rate Prediction. In Proceedings of the 18th ACM Conference on Recommender Systems. 350–359

    2024

  6. [6]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimiza- tion. In 3rd International Conference on Learning Representations, ICLR 2015,San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings . arXiv:1412.6980 https://arxiv.org/abs/1412.6980

    arXiv 2015

  7. [7]

    Haoxuan Li, Chunyuan Zheng, and Peng Wu. 2023. StableDR: Stabilized Dou- bly Robust Learning for Recommendation on Data Missing Not at Random. arXiv:2205.04701 [cs.LG] https://arxiv.org/abs/2205.04701

    arXiv 2023

  8. [8]

    Haoxuan Li, Chunyuan Zheng, and Peng Wu. 2023. StableDR: Stabilized Doubly Robust Learning for Recommendation on Data Missing Not at Random. In The Eleventh International Conference on Learning Representations . 163–175

    work page 2023

Show all 18 references
  1. [9]

    Yu Liu, Qinglin Jia, Shuting Shi, Chuhan Wu, Zhaocheng Du, Zheng Xie, Ruiming Tang, Muyu Zhang, and Ming Li. 2024. Ranking-Aware Unbiased Post-Click Conversion Rate Estimation via AUC Optimization on Entire Exposure Space. In Proceedings of the 18th ACM Conference on Recommend...

    2024

  2. [10]

    Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018. Entire space multi-task model: An effective approach for estimating post-click conversion rate. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrie...

    2018

  3. [11]

    Andrew L Maas, Awni Y Hannun, Andrew Y Ng, et al. 2013. Rectifier nonlineari- ties improve neural network acoustic models. In Proc. icml, Vol. 30. Atlanta, GA, 3

    2013

  4. [12]

    Hongzu Su, Lichao Meng, Lei Zhu, Ke Lu, and Jingjing Li. 2024. DDPO: Direct Dual Propensity Optimization for Post-Click Conversion Rate Estimation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1179–1188

    2024

  5. [13]

    Hao Wang, Tai-Wei Chang, Tianqiao Liu, Jianmin Huang, Zhichao Chen, Chao Yu, Ruopeng Li, and Wei Chu. 2022. ESCM2: entire space counterfactual multi- task model for post-click conversion rate estimation. In Proceedings of the 45th International ACM SIGIR Conference on Research...

    2022

  6. [14]

    Wenhao Zhang, Wentian Bao, Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. 2020. Large-scale causal approaches to debiasing post-click conversion rate estimation with multi-task learning. In Proceedings of The Web Conference 2020. 2775–2781

    2020

  7. [15]

    Wenhao Zhang, Wentian Bao, Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. 2020. Large-scale Causal Approaches to Debiasing Post- click Conversion Rate Estimation with Multi-task Learning. In Proceedings of The Web Conference 2020 (Taipei, Taiwan) (WWW ’20)...

    2020 doi

  8. [16]

    Xinyue Zhang, Cong Huang, Kun Zheng, Hongzu Su, Tianxu Ji, Wei Wang, Hongkai Qi, and Jingjing Li. 2024. Adversarial-Enhanced Causal Multi-Task Framework for Debiasing Post-Click Conversion Rate Estimation. In Proceedings of the ACM on Web Conference 2024 . 3287–3296

    2024

  9. [17]

    Sicheng Zhao, Xiangyu Yue, Shanghang Zhang, Bo Li, Han Zhao, Bichen Wu, Ravi Krishna, Joseph E Gonzalez, Alberto L Sangiovanni-Vincentelli, Sanjit A Seshia, et al. 2020. A review of single-source deep unsupervised visual domain adaptation. IEEE Transactions on Neural Networks ...

    2020

  10. [18]

    Feng Zhu, Mingjie Zhong, Xinxing Yang, Longfei Li, Lu Yu, Tiehua Zhang, Jun Zhou, Chaochao Chen, Fei Wu, Guanfeng Liu, et al. 2023. DCMT: A Direct Entire- Space Causal Multi-Task Framework for Post-Click Conversion Estimation. In 2023 IEEE 39th International Conference on Data...

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.