{"id":"4d30112c-b9b8-4ad8-9960-3c3c862a2c2d","arxiv_id":"2411.16095","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"LDACP, a model combining smoothed bucket classification and proxy-label regression, improves the accuracy of long-delayed conversion counts and raises the bidding system's compliance rate by 2.29% in an online A/B test.","lead":"This paper predicts how many conversions an ad campaign will eventually produce, despite most conversions occurring hours or days after impressions. The prediction feeds the automated bidding system, which improves cost compliance for advertisers and revenue for the platform.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The offline evaluation and the VRMP proxy label both mishandle y=0 samples (Eqs. 12, 17, 19; Table 2 min=0), so the reported CR/MAPE gains are not well-defined for zero-conversion campaigns that the method is meant to help.","rationale":"The reader's weakest-assumption concern was that the VRMP module relies on the empirical observation that PCOC has a narrow range and no long tail. My concern is more specific and more damaging: the observed narrow range is partly an artifact of the paper's Eq. (12), which assigns PCOC=1 to every y=0 sample, and the evaluation metrics themselves divide by y without specifying a convention for y=0. Since zero-conversion campaigns are a core part of the long-delayed-conversion bidding problem, the offline evidence as presented is not fully well-defined. I do not see a reason to reject the method outright: the architecture is reasonable, the ablation gains are consistent with the proposed mechanism, and an online A/B test was run. The right outcome is a conditional accept, with the condition that the authors disclose the fraction of y=0 samples, specify how they are handled in every metric and loss, and verify that the reported offline gains are robust to a reasonable alternative handling. This is a clarification and reproducibility issue rather than a demonstrated failure, so the reader's conditional verdict stands, but the stated conditions should be expanded to include this explicit check.","tokens_in":15012,"tokens_out":7682,"duration_ms":76898,"concrete_test":"Ask the authors to (1) report the proportion of y=0 samples in the Kuai-AD test set, and (2) recompute Table 3 MAPE/CR under three explicit protocols: exclude y=0; count y=0 as non-compliant; and define the ratio as 0 when y=0. If LDACP's advantage over CREAD-P (MAPE 0.2228 vs 0.2914; CR 62.32% vs 53.80%) changes by more than roughly 0.02 MAPE or 1 CR percentage point under any reasonable protocol, the offline comparison is not robust. Also re-examine the PCOC histogram separately for y>0 and y=0 to see whether the narrow-range claim in Section 4.3 survives without the arbitrary PCOC=1 assignment.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's strongest empirical claims rest on Kuai-AD metrics that are undefined for y=0. Section 3 defines labels y_j in N (including 0), and Table 2 reports min conversions = 0. However, CR in Eq. (19) is (1/N) * sum eta(0.8 <= y_hat/y <= 1.2), the MoE loss in Eq. (17) contains y_hat/y, and MAPE has the same denominator issue. No exclusion or special rule for zero-label samples is stated. If y=0 samples are dropped, the reported CR/MAPE describe only campaigns with at least one conversion, which is not the population the bidding system cares about; inactive campaigns are exactly the ones with no or very few tracked conversions. If they are kept with an implicit convention, the reported numbers depend on an undisclosed choice and are not reproducible. The VRMP proxy definition in Eq. (12) is also problematic: for y=0 it sets PCOC=1, whereas z/y is undefined or infinite, and z can be positive. The Section 4.3 claim that PCOC has a narrow range and no long tail (Fig. 2b) therefore depends on this arbitrary censoring; it cannot be read as a distributional property that will transfer to a new platform or time period. This is load-bearing because VRMP is the component credited with the largest ablation gain (CR -9.42% when removed, Table 4) and because the online A/B uses the predicted count for campaigns whose real-time tracked count is a severe undercount.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the problem of predicting the number of long-delayed ad conversions at the campaign level for use in automated bidding. The proposed LDACP model combines two modules: BCMS, which converts regression into a tree-structured bucket classification with non-normalized soft labels and label smoothing, and VRMP, which regresses the ratio PCOC = z/y (aggregated pCTCVR over actual conversions) and converts the prediction back to a conversion count. The two predictions are fused by a learned Mixture-of-Experts gate. The authors report offline experiments on a 2.7M-sample Kuai-AD dataset showing consistent improvements over six baselines (MAPE 0.2228 vs. 0.2914 for the best baseline; CR 62.32% vs. 53.80%), an ablation study attributing large gains to both modules, and an online A/B test on Kuaishou reporting a 2.29% compliance-rate improvement, 11.06% revenue increase, and 8.68% conversion increase.","tokens_in":15427,"tokens_out":4266,"duration_ms":40673,"significance":"If the reported results hold, the paper makes a useful industrial contribution: it formulates a practically important problem, provides a clean decomposition of the long-tail and label-discontinuity difficulties, and validates the method on a large real dataset and in an online deployment. The offline evaluation is substantial and the ablations are informative. I agree with the reader that the VRMP proxy is not circular: PCOC is a reparameterization of the target y, not a self-referential fitted constant. That said, the central offline claims rest on metrics that are not well-defined for y=0 samples, and the online test is reported without any statistical uncertainty. Both issues need to be resolved before the headline numbers can be taken at face value.","major_comments":[{"comment":"The label space includes y=0 (Table 2 reports minimum conversions = 0), but the evaluation metrics CR in Eq. (19), MAPE in Section 5.1.2, and the MoE loss in Eq. (17) all divide by y. The paper does not state whether y=0 samples are excluded, assigned a special value, or handled by clipping. If they are dropped, the reported MAPE and CR describe only campaigns with at least one conversion, which is not the population most relevant to the bidding system: inactive campaigns with zero or very few tracked conversions are exactly where the predicted count is needed. If they are kept, the numerical results depend on an undisclosed convention and are not reproducible. This is load-bearing because Tables 3 and 4, including the headline MAPE 0.2228 and CR 62.32%, are computed from these metrics.","section":"Section 5.1.2, Eq. (19); Section 3, Table 2; Section 4.3, Eqs. (12) and (17)"},{"comment":"The core premise of VRMP is that PCOC has a narrow value range and no long tail (Section 4.3, Figure 2b). This property is in part manufactured by the definition in Eq. (12), which sets PCOC=1 for all y=0 samples even though z/y is undefined there. Without reporting the fraction of y=0 samples and the PCOC distribution conditional on y>0, Figure 2b cannot be read as evidence of a transferable distributional property. Since VRMP is credited with the largest ablation gain (CR drops by 9.42% when removed, Table 4), the authors should provide a more careful distributional analysis, including the zero-inflation rate, and state explicitly how the proxy is defined for zero-label samples in both training and evaluation.","section":"Section 4.3, Eq. (12); Figure 2b; Table 4"},{"comment":"The online A/B test is reported only as point estimates: the experimental group improves CR by 2.29% overall, with daily improvements ranging from 0.59 to 4.66 percentage points. No confidence intervals, p-values, number of campaigns, or traffic volumes are given, so the reader cannot assess whether the observed differences are statistically significant. The same applies to the revenue increase of 11.06% and the conversion increase of 8.68%. Given the commercial importance of the claim, the authors should report at least campaign-level variance and a significance test, or clearly label the result as a directional observation.","section":"Section 5.3, Table 5"},{"comment":"The label window of three days is chosen without sensitivity analysis, and the conversion-delay statistics in Appendix A show that the p90 delay exceeds 2,770 minutes for several campaign types, so a three-day window still censors a non-negligible tail. Since all offline metrics and the downstream bidding signal depend on this label definition, the authors should either justify the window with data or provide results for alternative windows (e.g., one, two, four, and seven days).","section":"Section 5.1.1; Appendix A"}],"minor_comments":[{"comment":"There is a typographical issue in Eq. (19): the summation is written as \"Y∑︁ y\", which should presumably be a standard sum over all labels y in the test set.","section":"Section 5.1.2, Eq. (19)"},{"comment":"The conversion from predicted PCOC to predicted conversion count divides by \\hat{PCOC}, but the paper does not describe how zero or negative predictions of PCOC are handled; MAE regression can in principle produce such values, and the resulting division would be undefined or nonsensical.","section":"Section 4.3, Eq. (15)"},{"comment":"The symbol e_i is used for the expectation value of a leaf node, but e_{i->j} is already used for an edge in Table 1; using two different meanings for e in the same paper is confusing and should be resolved by renaming one of them.","section":"Section 4.2.2, Eq. (9) and Table 1"},{"comment":"The paper does not report the fraction of y=0 samples in Kuai-AD; this fraction is important both for interpreting the PCOC distribution in Figure 2b and for assessing how much of the reported CR/MAPE is determined by the convention for zero labels.","section":"Section 5.1.1 and Section 4.3"},{"comment":"The online deployment clips the predicted conversion count to be not less than the real-time tracked count, but the paper does not state whether this same clipping is applied to the offline predictions and to the MoE loss in Eq. (17); the offline and online settings should be aligned.","section":"Section 5.3 and Section 4.4"},{"comment":"The hyperparameters alpha and beta in Eq. (18), as well as the smoothing temperature 10 in Eq. (21), are fixed empirically to 1 and 10, respectively, but no sensitivity study is reported; a short sensitivity analysis would increase confidence in the robustness of the method.","section":"Section 5.1.3 and Eq. (18)"}],"recommendation":"major_revision","confidential_remarks":"The paper is an industry-oriented WWW Companion submission with a strong empirical component. The main concern is not novelty or circularity but the undefined handling of y=0 in the central evaluation metrics and in the VRMP proxy definition; this is fixable with additional analysis and clarification. The online A/B section would also benefit from standard statistical reporting. I would not reject the paper on these grounds, but the reported numbers should be made precise before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, know this: the paper reports a real, believable win on Kuaishou's bidding system, but its headline offline numbers are computed on a metric that is undefined for y=0, and the paper never says what it does with those samples. That is the main issue to chase if you referee it.\n\nThe new thing here is the specific combination for campaign-level delayed conversion count: a tree-based progressive bucketing classifier (TPM) with soft label smoothing, a proxy-label regression on PCOC (the ratio of aggregated pCTCVR to actual conversions), and a MoE to fuse the two. Each ingredient is known, but the application to long-delayed conversion counts is new and the engineering is sensible. The offline experiment is large (2.7M samples, four industries) and shows consistent MAPE/CR gains over six baselines. Ablations attribute gains to both smoothing and VRMP. The online A/B test reports +2.29% compliance rate, +11% revenue, +8.68% conversions over five days. That is a meaningful result for the ad bidding subfield, even if it is not a scientific breakthrough.\n\nThe soft spots, in order of severity. The y=0 problem is load-bearing, not cosmetic. Labels are defined in N (including 0) and Table 2 confirms min conversions = 0, but CR (Eq. 19), MAPE, and the MoE loss (Eq. 17) all divide by y. No exclusion or special rule is stated. If zero-conversion campaigns are dropped, the reported CR/MAPE describe only campaigns with at least one conversion—which is not the population the bidding system struggles with, since inactive campaigns are exactly the ones with no tracked conversions. If they are kept with an implicit convention, the numbers are not reproducible. The VRMP proxy label in Eq. (12) is similarly ad hoc: for y=0 it sets PCOC to 1, whereas z/y is undefined, and z can be positive. The claim that PCOC has a narrow range and no long tail (Fig. 2b) therefore rests on this arbitrary censoring; it is not a transferable distributional property. Since VRMP is the component with the largest ablation gain (CR -9.42% when removed), this matters.\n\nMinor issues: the online A/B test has no confidence intervals, the 3-day conversion window is arbitrary, and hyperparameters (alpha, beta, smoothing temperature) are hand-picked. The paper also ships no code or data, which limits reproducibility but is standard for industry papers.\n\nBottom line: this is a solid industrial paper with a fixable evaluation gap. The y=0 handling needs to be stated explicitly and the metrics redefined or the analysis restricted. If that survives, it deserves a serious referee. I'd send it to review but with a clear request for clarification on the zero-label handling.","headline":"The paper's offline metrics break for the zero-conversion campaigns it is meant to fix; fix that and the industrial story holds.","tokens_in":15987,"tokens_out":2544,"would_cite":false,"duration_ms":21485,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"For long-delayed ad conversions, the paper proposes LDACP, a two-module model that predicts campaign-level conversion counts so automated bidding can use the true expected CPA instead of the stale real-time count.","keywords":["ad conversions prediction","label smoothing","long-tail distribution","automated bidding strategy","online advertising","cost compliance rate","proxy labels","mixture of experts"],"falsifier":"Compute the PCOC distribution on a different advertising platform or a different time window from the one used here. If many campaigns show PCOC values far from 1 or a heavy right tail, then VRMP's proxy-label regression loses its advantage; an offline comparison where LDACP fails to beat CREAD-P or TPM-N on such data would falsify the general claim.","tokens_in":14856,"feed_emoji":"📈","tokens_out":4722,"duration_ms":40321,"temperature":0.7,"pith_summary":"For online ads whose conversions arrive hours or days after the impression, the bidding system's real-time conversion count lags behind, so it overestimates the current cost per action and bids too timidly. This paper proposes predicting the final conversion count for each ad campaign at bidding-adjustment time, then feeding that predicted number into the bidding loop. The proposed model, LDACP, combines a bucket-classification module with smoothed labels and a value-regression module trained on proxy labels derived from the ranking model's prediction bias. In offline tests it beats the best baseline by 0.0686 in MAPE and 8.52 percentage points in cost-compliance rate; an online A/B test on the platform reports a 2.29% higher compliance rate, 11.06% higher platform revenue, and 8.68% more conversions.","feed_headline":"Predicted conversions lift ad-bid compliance by 2.29%","feed_subtitle":"A two-module model forecasts long-delayed conversions so bidding stops overestimating costs, raising revenue 11.06%.","key_machinery":"Two sub-modules carried by two distributional observations. BCMS (Bucket Classification Module with label Smoothing) builds on a tree-based bucket classifier but assigns each edge a soft target that varies continuously with distance to the bucket boundary, trained by cross-entropy for hard labels and MSE for soft labels. VRMP (Value Regression Module with Proxy labels) learns $\\mathrm{PCOC}=z/y$, the ratio of aggregated pCTCVR to actual conversions, then recovers the predicted count as $\\hat{y}_g = z/\\hat{\\mathrm{PCOC}}$. The MoE gate $\\lambda$ is trained with a MAPE loss and shifts weight from the bucket module to the proxy-regression module as the label grows.","core_discovery":"The paper's central claim is that campaign-level conversion counting for long-delayed ads is best done by splitting the task: a bucket classifier whose one-hot labels near bucket boundaries are replaced by smoothly varying soft labels, and a regression head that predicts the ranking model's bias (aggregated predicted conversion rate divided by actual conversions) rather than the raw long-tailed count. Because that bias ratio concentrates in a narrow range with no long tail on the authors' data, ordinary regression works for it, and converting back with the aggregated pCTCVR yields a tail-accurate count. A mixture-of-experts gate then blends the two predictions, weighting the bucket output for small counts and the proxy-regression output for large counts. The result is a single predicted conversion number that the automated bidding system can use in place of the incomplete real-time count.","pith_inferences":["If the narrow-range PCOC property turns out to be universal across platforms, the proxy-label trick could be reused for any delayed-feedback count where a biased online estimator exists.","The soft-label smoothing scheme could be tested on other integer-valued long-tail regression tasks, such as watch-time or purchase-count prediction, where boundary discontinuities hurt hard-label classifiers.","A natural stress test is to hold out product categories with unusually high or low PCOC spread; if VRMP degrades there, the MoE gate could be extended to use PCOC spread as an input feature.","The paper's five-day online test leaves open whether the gains persist over a longer horizon; measuring compliance rate and revenue drift over several weeks would be a direct follow-up."],"forward_implications":["Bidding strategies can react to predicted final conversion counts rather than the small real-time count, avoiding the conservative bids that starve young campaigns of traffic.","Campaigns with conversion delays of hours to days (e.g., app installs, live-stream actions) get closer to their target CPA, raising the platform's compliance rate.","The platform's revenue and total conversions rise when inactive campaigns resume winning auctions.","The label-smoothing recipe is not tied to the specific tree classifier; the paper states it applies to any bucket-classification method with one-hot labels.","Tail prediction improves because the proxy-regression side handles large counts that the wide final bucket would otherwise over- or under-estimate."],"supporting_citations":[{"why":"Provides the tree-based progressive bucket classification method (TPM) that BCMS extends with its label-smoothing technique.","marker":"[14]"},{"why":"CREAD baseline, a classification-restoration framework for watch-time prediction that the paper compares against.","marker":"[13]"},{"why":"MDME baseline, a divide-and-conquer bucket classification approach for lifetime value prediction used as a comparison.","marker":"[12]"},{"why":"ZILN baseline, a parametric log-normal distribution model for customer lifetime value that the paper contrasts with its proxy-label regression.","marker":"[9]"},{"why":"Supplies the leaf-node expectation formula used during inference to convert bucket probabilities into a predicted count.","marker":"[20]"},{"why":"Defines the mixture-of-experts structure that integrates the BCMS and VRMP predictions.","marker":"[17]"}],"fun_headline_variants":["LDACP: Predict delayed ad conversions for better bidding","Two-module model forecasts long-delayed ad conversions","Mixture of experts handles long-tail ad conversion counts","Predicting delayed ad conversions to fix CPA overestimate","LDACP: Two-module model for long-delayed ad conversions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole proxy-regression side rests on an empirical claim: the ranking model's bias ratio PCOC stays in a narrow range with no long tail (Figure 2b on the offline dataset). If that ratio is heavy-tailed or has large outliers on another platform or period, converting back through it will be unstable and the long-tail problem returns.","fun_headline_variants_meta":{"raw":{"variants":["LDACP: Predict delayed ad conversions for better bidding","Two-module model forecasts long-delayed ad conversions","Mixture of experts handles long-tail ad conversion counts","Predicting delayed ad conversions to fix CPA overestimate","LDACP: Two-module model for long-delayed ad conversions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000864,"raw_usage":{"total_tokens":3781,"prompt_tokens":1013,"completion_tokens":2768,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":2689}},"tokens_in":629,"tokens_out":2768,"duration_ms":19479,"temperature":1.0,"reasoning_tokens":2689,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:33:33.306441+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the PCOC distribution on a different advertising platform or a different time window from the one used here. If many campaigns show PCOC values far from 1 or a heavy right tail, then VRMP's proxy-label regression loses its advantage; an offline comparison where LDACP fails to beat CREAD-P or TPM-N on such data would falsify the general claim.","supporting_citations":[{"cited_title":"Tree based progressive regression model for watch-time prediction in short- video recommendation","cited_arxiv_id":null,"evidence_quote":"Provides the tree-based progressive bucket classification method (TPM) that BCMS extends with its label-smoothing technique."},{"cited_title":"Cread: A classification-restoration framework with error adaptive discretization for watch time prediction in video recommender systems","cited_arxiv_id":null,"evidence_quote":"CREAD baseline, a classification-restoration framework for watch-time prediction that the paper compares against."},{"cited_title":"Billion-user customer lifetime value prediction: an industrial-scale solution from kuaishou","cited_arxiv_id":null,"evidence_quote":"MDME baseline, a divide-and-conquer bucket classification approach for lifetime value prediction used as a comparison."},{"cited_title":"Deep ordinal regression network for monocular depth estimation","cited_arxiv_id":null,"evidence_quote":"Supplies the leaf-node expectation formula used during inference to convert bucket probabilities into a predicted count."}],"review_version":1}