{"id":"2ebe7404-6ac8-4c3d-9b62-0ab515ed530c","arxiv_id":"2506.00959","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Budget allocation by clustering users in a learned hidden representation space and optimizing per cluster improves order volume and gross merchandise volume by up to 0.65% relative to individual-level baselines in Meituan A/B tests.","lead":"Meituan has developed a new discount allocation method that groups similar users into clusters and decides each cluster's discount rather than predicting each individual user's response. Online experiments show small but steady gains in orders and spending compared with older methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core approximation—cluster-level stochastic programming over historical cluster statistics approximating the individual-level optimum—is unvalidated; heterogeneous clusters would make HRC systematically wrong.","rationale":"The reader's weakest assumption correctly identifies the load-bearing point. The method's entire premise is that optimizing over a few clusters is a good proxy for optimizing over millions of individuals. This fails if clusters contain users with opposite treatment responses, because the cluster mean can point to the wrong treatment for a large fraction of the cluster. The paper's empirical evidence (offline EOM and online A/B) is suggestive but does not address this premise; a cluster could look good on average while being suboptimal for most individuals. The proposed test uses RCT holdout data to directly measure the gap between cluster-level and individual-level allocation, which settles whether the approximation is acceptable. A secondary internal issue--the omega_i * sigma scaling in Eq. (7)--is also worth checking, as it indicates the risk-adjusted objective is not a proper portfolio variance. The theoretical proof in Appendix A is also flawed (Eq. 13's inequality is an upper bound, not a lower bound), but it only motivates the choice of hidden representations; the cluster homogeneity concern is more direct. Therefore the verdict remains CONDITIONAL pending this diagnostic.","tokens_in":11260,"tokens_out":9100,"duration_ms":89336,"concrete_test":"On the RCT holdout Meituan-w2 (or w5), compute the oracle individual-level allocation from ground-truth outcomes, then evaluate its EOM against the EOM of HRC's cluster-level allocation from (5)-(7). If the cluster-level EOM falls more than a small tolerance (e.g., 1%) below the oracle, the central approximation fails. Also re-solve (7) with sqrt(omega_i) * sigma in the variance penalty and check whether the allocation and EOM change materially.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3-4.4 replaces the individual-level problem (1) with cluster-level stochastic program (7) using historical cluster statistics (6). This is valid only if each cluster is approximately homogeneous in treatment response and the statistics remain representative at serving time. K-means on the representation z (Eq. 5) provides no such guarantee: the representation is trained to predict revenue and propensity, not to make treatment effects constant within clusters. The paper offers no diagnostic for within-cluster heterogeneity, cluster stability over time, or distribution shift. Additionally, Eq. (7) penalizes variance with omega_i * sigma rather than sqrt(omega_i) * sigma, so the risk term is not the variance of the cluster total and can distort allocation for large clusters. If clusters mix users with opposite responses, the cluster-level 'optimal' strategy can be systematically worse than individual-level baselines, directly undermining the claimed online gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HRC (Hidden Representation Clustering), an industrial pipeline for online budget allocation. A multi-task representation network is trained to predict revenue and propensity, its hidden representations are clustered with K-means, cluster-level mean/variance statistics are used to solve a variance-penalized stochastic integer program, and the representation plus clustering is distilled into a K-class classification model for online serving. The method is evaluated on five weeks of Meituan data with EOM-based offline comparisons against six baselines and with online A/B tests against two deployed algorithms, reporting OV/GMV gains.","tokens_in":11500,"tokens_out":4724,"duration_ms":47794,"significance":"If the empirical claims hold, the paper offers a practically useful and deployable alternative to individual-level predict-then-optimize and decision-focused learning: it reduces the optimization scale from tens of millions of individuals to K clusters, avoids large-scale counterfactual prediction, and replaces a heavy online pipeline with a distilled classifier. The manuscript's strengths are the large-scale industrial dataset (44M samples), the inclusion of two online A/B deployments, and the explicit treatment of observation-data training via a monotonic network. However, the central quantitative claims are currently undermined by an internal inconsistency in the reported online gains, an invalid proof in the theoretical appendix, and missing uncertainty and sensitivity analysis. These issues are fixable within the manuscript's scope, so the contribution is potentially strong but not yet ready in its present form.","major_comments":[{"comment":"The text states that 'HRC outperforms Slearner-HEU by 11% and 22%, respectively' and 'surpasses DFL-PL by 22% and 33%', while Table 2 reports 0.13%/0.23% and 0.53%/0.65%, and the abstract reports 0.53%/0.65% for the comparison with DFL-PL. These numbers differ by two orders of magnitude. Since the online A/B result is the central claim, please reconcile the text with the table and report test statistics or confidence intervals for the daily gaps.","section":"Section 5.3, Table 2"},{"comment":"The proof of Theorem 1 is invalid as written. The 'low sensitivity condition' is stated as ||∂y/∂g||²_F >> 1, which is a high-sensitivity condition on the output map, not a low-sensitivity condition on g. From Eq. (13), E||y~ - y||²_F ≤ σ²_Δ ||∂y/∂g||²_F ||J_g||²_F; comparing with Eq. (11) gives E||z~ - z||²_F << E||y~ - y||²_F only if ||∂y/∂g||_F ≥ 1, which is not implied by low sensitivity of g. The theorem and its proof should be corrected, or the theoretical claim should be removed or substantially weakened.","section":"Appendix A.1, Eq. (14)"},{"comment":"The replacement of the individual-level problem (1) with the cluster-level stochastic program (7) assumes that each cluster is approximately homogeneous in treatment response and that the historical cluster statistics remain representative at serving time. K-means on the representation trained via (2) provides no such guarantee, and the paper offers no diagnostic for within-cluster heterogeneity, cluster stability across time, or distribution shift. Please provide such diagnostics or a formal bound on the approximation error. In addition, the variance penalty in (7) uses ω_i σ_r and ω_i σ_c; for cluster totals, the standard deviation under within-cluster independence would scale as sqrt(ω_i)σ, so the current penalty over-penalizes large clusters and can distort the allocation.","section":"Sections 4.3-4.4, Eqs. (6)-(7)"},{"comment":"Several key hyperparameters are not reported anywhere: K in Eq. (5), λ and κ in Eq. (7), and α in Eq. (2). Table 3 lists network and training hyperparameters but omits these, even though the clustering granularity and the risk-aversion terms directly determine the solved allocation. Without these values the experiments are not reproducible; please report them and, ideally, a sensitivity analysis over K, λ, and κ.","section":"Section 5.1 and Table 3"},{"comment":"The offline EOM results are reported as single curves without error bars, confidence intervals, or the number of repeated runs. Given the high variance visible in the data statistics in Table 1 and the small apparent differences in some budget ranges, the claimed offline superiority of HRC is not statistically supported. Please add pointwise confidence intervals or at least report standard errors over seeds or bootstrap resamples for the EOM estimates.","section":"Section 5.2, Figures 3-5"}],"minor_comments":[{"comment":"The sentence '?? indicates the EOM results under five budget points, where HRC outperforms the second-best approach for {0.1, 0.1, 0.1, 0.1, 0.1} under budgets of {1, 2, 3, 4, 5}' contains a missing reference and an unclear tuple; please rewrite it and supply the missing caption or table reference.","section":"Section 5.2, Generalization paragraph"},{"comment":"The label on the right panel reads 'GMV label', which appears to be a typo for 'GMV'.","section":"Figure 3"},{"comment":"The columns ΔOV/OV_B, ΔGMV/GMV_B, and ΔCost/Cost_B mix percentage changes and percentage-point changes; please clarify the units and define the baseline subscript B explicitly.","section":"Table 2"},{"comment":"The abstract states that HRC 'outperforms the online algorithm by 0.53% and 0.65%' but does not identify the baseline; the corresponding Table 2 comparison is with DFL-PL, so please name the baseline in the abstract.","section":"Abstract"},{"comment":"The reference 'Mandi and Guns [2010]' is cited in the text as 2020, and the URL field in the bibliography entry appears malformed; please correct the citation year and reference entry.","section":"References"},{"comment":"The proof states I(Z; T | X) → 0 for RCT data, but Z is a deterministic function of X, so the conditional independence of T and Z given X should be justified rather than assumed; as written the step is not immediate.","section":"Appendix A.2, Theorem 2"}],"recommendation":"major_revision","confidential_remarks":"This is a potentially valuable industrial case study, and the deployment-oriented framing is a real strength. The most urgent issue for the editor is the internal inconsistency in the online gains (Section 5.3 vs. Table 2 vs. abstract), which must be resolved before any further consideration. The theoretical appendix is currently more of a liability than a support, and the missing hyperparameters (K, λ, κ, α) and absent error bars make the empirical claims difficult to verify. I would encourage the editor to request the raw daily A/B data or at least a detailed reconciliation, and to have the authors either fix Theorem 1 or explicitly label it as a heuristic motivation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an industrial application paper, not a methods paper. The cluster-based reformulation of online budget allocation is new as a combination, and the offline/online evidence is real, but the current write-up has an internal numerical inconsistency, a broken proof in Appendix A, and an unvalidated homogeneity assumption. I'd send it to referees with major-revision expectations.\n\nThe genuinely new part is replacing individual-level predict-then-optimize with K-means over hidden representations plus a cluster-level stochastic program, then distilling to a K-class model for online serving. The multi-task network with a monotonic hypernetwork for observational data is also a sensible engineering choice. The offline comparison against six baselines, including DFL variants, is thorough, and the EOM results on three Meituan datasets show HRC leading or competitive. The online A/B results, once you read the table, show small but positive gains (0.13-0.65% OV/GMV), meaningful in e-commerce.\n\nThe soft spots, in order:\n\n1. Section 5.3 prose says 11%/22% and 22%/33% while Table 2 says 0.13%/0.23% and 0.53%/0.65%. That's a large internal inconsistency. Probably the text is wrong, but it needs fixing before anyone can trust the reported gains.\n\n2. Theorem 1 in Appendix A is not a proof. The step from the low-sensitivity condition on g to ||∂y/∂g|| >> 1 doesn't follow, and even if it did, an upper bound on E||y~-y|| doesn't give the stated inequality. The claim about hidden representations being more robust than outputs may be true in practice, but the proof doesn't show it.\n\n3. The stochastic program in Eq. (7) penalizes variance with omega_i * sigma, but the standard deviation of the cluster total is sqrt(omega_i)*sigma. As written, the risk term overpenalizes large clusters and can distort allocation. This is a modeling error that should be corrected or justified as a heuristic.\n\n4. The core assumption that clusters are homogeneous in treatment response is unvalidated. K-means on representations trained to predict revenue and propensity doesn't guarantee homogeneous effects, and the paper gives no diagnostic. This doesn't kill the method—the offline EOM results are suggestive—but it's the load-bearing premise and needs at least a sensitivity analysis.\n\n5. No confidence intervals in the EOM plots, and K, lambda, kappa are not reported. That hurts reproducibility.\n\nThe references are relevant and the citation pattern is normal. Overall, the central idea is plausible and the experiments are extensive; the flaws are fixable. I would engage with this paper as a referee, and the authors should get a chance to revise.","headline":"Plausible industrial method with a real bug in the reported gains and a flawed robustness proof; the core cluster-based idea deserves a revision, not a desk reject.","tokens_in":11945,"tokens_out":4234,"would_cite":false,"duration_ms":42293,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that online budget allocation is more robust and faster when optimized over clusters of similar users rather than over individuals, and supports the claim with Meituan offline experiments and live A/B tests.","keywords":["online budget allocation","marketing optimization","representation learning","treatment effect estimation","clustering","stochastic programming","decision-focused learning","uplift modeling"],"falsifier":"On a held-out randomized dataset, compute the cluster-level optimal treatment assignment from HRC's clusters and compare its achieved revenue with the individual-level optimal assignment under the same budget; if within-cluster response variance is high, for example each cluster contains both users who do best under the smallest discount and users who do best under the largest discount, the cluster-level assignment will perform measurably worse, contradicting the claim that cluster statistics preserve near-optimality.","tokens_in":11076,"feed_emoji":"📊","tokens_out":3904,"duration_ms":36254,"temperature":0.7,"pith_summary":"This paper sets out to make online marketing budget allocation robust to noisy, large-scale data by switching the unit of optimization from individual users to clusters of users with similar hidden representations. It claims that predicting and optimizing over clusters, whose revenue and cost statistics are fed into a stochastic program, avoids the compounding errors of individual-level counterfactual prediction. The authors support this with offline comparisons on five weeks of Meituan data and online A/B tests reporting that the cluster-based approach outperforms a decision-focused baseline by 0.53% in order volume and 0.65% in gross merchandise volume, and an S-learner heuristic baseline by 0.13% and 0.23%.","feed_headline":"Clusters beat individuals for noisy budget allocation","feed_subtitle":"Grouping users by hidden representations adds 0.53% order volume and 0.65% GMV in live Meituan A/B tests.","key_machinery":"Hidden representation clustering (HRC) is the pipeline that carries the argument: it trains a multi-task network with a Dragonnet-style representation module and separate revenue and propensity heads, projects original features into hidden representations from the first two layers, clusters these representations with K-means, and solves a K-cluster integer stochastic program whose objective uses cluster mean and variance statistics with risk-aversion coefficients. A monotonic revenue head built with a hypernetwork is used when training on observational data, enforcing that predicted revenue increases with treatment value. The paper justifies clustering in representation space rather than output space with a proof that hidden representations are less sensitive to input noise when the Jacobian of the representation mapping satisfies a low-sensitivity condition, so cluster assignments are more stable than individual predictions.","core_discovery":"The central claim is that the online budget allocation problem can be reformulated from the perspective of clusters rather than individuals without losing solution quality, while gaining robustness to noise and drastically cutting solve time. Individuals are mapped by a multi-task representation network to hidden representations, clustered with K-means, and cluster-level revenue and cost means and variances are used as the parameters of an integer stochastic program. In this program the objective subtracts risk-aversion terms from each cluster's expected revenue, so uncertainty in cluster statistics is explicitly penalized. The paper further claims that this cluster-level formulation outperforms six individual-level marketing optimization algorithms in offline EOM evaluations and beats deployed baselines in online A/B tests.","pith_inferences":["The cluster-then-optimize idea could transfer to other resource allocation problems where individual-level counterfactuals are unstable, such as ad bidding, coupon targeting, or dynamic pricing, as long as cluster statistics can be estimated credibly from historical data.","A testable extension is to measure within-cluster homogeneity: on held-out RCT data, compare the cluster-level optimal assignment against the individual-level optimal assignment under the same budget; the revenue gap quantifies the information lost by clustering.","The robustness claim depends on cluster composition remaining stable over time, so a practical improvement would be periodically re-estimating cluster statistics or monitoring cluster drift rather than assuming the historical clusters persist indefinitely.","Because the objective is a mean-variance trade-off, the risk coefficients could be tuned per budget segment using online feedback, turning the current static setting into a closed-loop allocation policy."],"forward_implications":["If cluster-level statistics remain representative, allocation strategies can be recomputed in seconds for any budget, enabling a library of strategies stored as mapping tables for online requests.","Distilling the representation module and clustering model into a K-classification model makes online serving fast enough for millions of requests per day.","With RCT training data, the paper argues that hidden representations are independent of the treatment assignment, so clusters capture inherent user attributes rather than treatment-conditioned responses.","The variance-penalized objective gives a direct robustness knob: adjusting the risk-aversion coefficients trades expected revenue against uncertainty in the cluster statistics.","The approach extends to multi-valued treatments, demonstrated with six discount rates, and the monotonic revenue head adapts it to biased observational data."],"supporting_citations":[{"why":"Supplies the Dragonnet architecture and the representation-learning approach for treatment effects that the multi-task representation network is based on.","marker":"Shi et al. [2019]"},{"why":"Provides VCNet, the continuous-treatment representation learning method that informs the multi-treatment revenue head design.","marker":"Nie et al. [2021]"},{"why":"Defines decision-focused learning, the end-to-end paradigm that HRC compares against and that motivates the decision-loss gradient estimation challenges discussed in the paper.","marker":"Wilder et al. [2019]"},{"why":"The decision-focused causal learning with policy learning method that serves as the DFL-PL baseline in both offline and online comparisons.","marker":"Zhou et al. [2024]"},{"why":"Provides the Lagrangian duality and heuristic search two-stage baselines, and the observation that individual-level predict-then-optimize can underperform heuristics.","marker":"Zhou et al. [2023]"},{"why":"Gives the low-sensitivity Jacobian condition used in the appendix proof that hidden representations are more robust to input noise than outputs.","marker":"Hoffman et al. [2019]"},{"why":"Supplies the causal forest baseline and the expected outcome metric (EOM) used for offline evaluation of marketing optimization algorithms.","marker":"Ai et al. [2022]"}],"fun_headline_variants":["Clusters beat individuals for noisy budget allocation","Cluster-level optimization beats individual targeting","Hidden clusters improve noisy budget allocation","Group users by clusters for robust ad budgets","Multi-task clustering wins in noisy allocation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The cluster-level revenue and cost statistics, computed from historical training data, must remain representative of future traffic, and the paper does not verify that users within a cluster respond homogeneously to treatments, so if a cluster mixes users with opposite treatment responses the cluster-level optimal strategy can be systematically wrong.","fun_headline_variants_meta":{"raw":{"variants":["Clusters beat individuals for noisy budget allocation","Cluster-level optimization beats individual targeting","Hidden clusters improve noisy budget allocation","Group users by clusters for robust ad budgets","Multi-task clustering wins in noisy allocation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000618,"raw_usage":{"total_tokens":2865,"prompt_tokens":941,"completion_tokens":1924,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":1863}},"tokens_in":557,"tokens_out":1924,"duration_ms":12974,"temperature":1.0,"reasoning_tokens":1863,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:53:47.967411+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a held-out randomized dataset, compute the cluster-level optimal treatment assignment from HRC's clusters and compare its achieved revenue with the individual-level optimal assignment under the same budget; if within-cluster response variance is high, for example each cluster contains both users who do best under the smallest discount and users who do best under the largest discount, the cluster-level assignment will perform measurably worse, contradicting the claim that cluster statistics preserve near-optimality.","supporting_citations":[{"cited_title":"Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization","cited_arxiv_id":null,"evidence_quote":"Defines decision-focused learning, the end-to-end paradigm that HRC compares against and that motivates the decision-loss gradient estimation challenges discussed in the paper."},{"cited_title":"Direct heterogeneous causal learning for resource allocation problems in marketing","cited_arxiv_id":null,"evidence_quote":"Provides the Lagrangian duality and heuristic search two-stage baselines, and the observation that individual-level predict-then-optimize can underperform heuristics."}],"review_version":1}