REVIEW 4 major objections 6 minor 21 references
Personalized Tree-Based Progressive Regression Model for Watch-Time Prediction in Short Video Recommendation
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Pruning a global decision tree per user-item pair yields better watch-time prediction and a 0.428% online watch-time gain.
desk verdict Genuine extension of TPM with strong empirical results, but the UCL unbiasedness derivation has a real algebraic error that needs to be addressed before the debiasing claim can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a global full binary tree $T_g$ over the normalized watch-time interval $[0,1]$, together with a pruning model that maps user-item features to Bernoulli probabilities over non-leaf nodes. Sampling a pruning vector $a_T$ produces a sample-specific subtree $T = T_g \circledast a_T$, and the inherited classifier parameters $\theta^*(T)=\theta^*(T_g)\odot(1-a_T)$ are reused without fine-tuning because every subtree shares the same path semantics. The pruning policy is trained with a self-critical REINFORCE objective contrasting the pruned tree's reward $R_T-R_{T_g}$, where $R=\mathrm{XAUC}-\mathrm{MSE}$. For unbiased learning, the paper derives an inverse-propensity-scoring estimator: each conditional log-likelihood term along the path is weighted by the inverse propensity $1/p(\hat{y}_i \in n_{\phi_{l_k}}(j-1)|x_i)$, with the propensity computed as the product of ancestor classifier outputs. These mechanisms make the tree structure personalized and end-to-end trainable at latency comparable to TPM.
What would settle it
Train two versions of PTPM on the same data: the paper's inherited-parameter pruning versus re-training the classifiers of every sampled pruned subtree from scratch for the same number of steps. If the retrained subtree beats the inherited-parameter version by a margin comparable to PTPM's stated gains over TPM, then the no-fine-tuning assumption is the true source of the improvement. A second, cheaper check is to measure per-leaf calibration on out-of-parent-interval samples after pruning, comparing inherited versus retrained classifiers.
Extended reading notes
Core claim
On its own terms, the paper claims that the tree structure in ordinal watch-time regression is a first-class learnable object, not a fixed hyperparameter. PTPM starts from a globally trained full binary tree and, for each user-item pair, samples a pruning mask with a policy network trained by REINFORCE to maximize the reward of the pruned subtree relative to the global tree, where the reward is XAUC minus MSE. The paper also claims TPM's conditional modeling suffers from selection bias, since each internal classifier is trained only on samples whose watch time falls into its parent interval, and it derives an inverse-propensity-scoring correction whose propensity is obtained from the same path probabilities. The combination yields consistent offline gains on KuaiRec, CIKM16, and WeChat21, and a +0.428% watch-time lift in an online A/B test with essentially unchanged latency, on a production service reported to serve over 400 million users daily.
Load-bearing premise
The load-bearing premise is that a classifier trained on the global tree remains correct on a pruned subtree of it, so the inherited parameters need no fine-tuning; Section 4.1 asserts this but gives no experiment that isolates it, and if it fails, the sampled pruned trees would misestimate leaf probabilities and both the efficiency and accuracy gains would collapse.
Editorial extensions
If this is right
- If PTPM's claims hold, watch-time models can switch from fixed discretization to sample-adaptive discretization without extra inference latency, since only classifier selection changes.
- The UCL correction implies that TPM's conditional classifiers are systematically miscalibrated in tail intervals; debiasing them improves both distributional calibration and classifier AUC.
- The pruning formulation reduces bilevel tree-structure search to a one-pass policy-gradient problem, so deeper global trees provide a larger search space and better learned trees, up to diminishing returns.
- The online +0.428% watch-time lift suggests that ranking quality improves when prediction granularity adapts to context, and that this gain carries over to the deployed serving system.
Reading between the lines
- Beyond the paper: the same pruning-plus-inherited-parameters trick could apply to any hierarchical ordinal decomposition, such as CREAD-style threshold classifiers, turning static binning into per-request binning at near-zero added cost.
- Beyond the paper: because the IPS propensity is built from the tree's own path probabilities, any conditional classifier tree can in principle be debiased by reweighting with ancestor probabilities, not just watch-time trees.
- Beyond the paper: the average learned tree depth converges to about 4.6 for a global depth of 6, which hints that the benefit comes mainly from reshaping rather than deepening; a testable extension is learning depth per cohort.
- Beyond the paper: the reward $R_T = \mathrm{XAUC} - \mathrm{MSE}$ is one scalar choice; replacing it with ranking-aware or business-aware rewards could yield trees tuned for engagement trade-offs, which the paper does not test.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PTPM, an extension of the tree-based progressive regression model (TPM) for watch-time prediction in short-video recommendation. PTPM has two main additions: a Tree Structure Learning (TSL) module that learns a personalized, sample-dependent tree by pruning a global full binary tree via a REINFORCE-style self-critical objective, and an Unbiased Conditional Learning (UCL) module that claims to remove the sample-selection bias induced by training each node classifier only on samples that pass through its parent. Offline experiments on KuaiRec, CIKM16, and WeChat21 report improved MAE and XAUC over TPM and other baselines, ablations attribute gains to both modules, and a live A/B test on Kuaishou reports a +0.428% watch-time lift with negligible latency overhead. The paper concludes that PTPM has been fully deployed and serves over 400 million daily users.
Significance. If the technical claims hold, PTPM is a practically relevant contribution: it is, to my knowledge, the first end-to-end learned discretization for watch-time prediction, and the pruning-based joint training of tree structure and classifiers is an elegant way to avoid bilevel optimization. The paper also provides useful evidence beyond offline benchmarks: ablations, a live A/B test, and deployment-scale performance numbers. The strength of the empirical demonstration, however, depends critically on the correctness of the UCL derivation and on the inherited-parameter assumption in TSL. The current manuscript has a load-bearing algebraic inconsistency in Eq. (17) and an unverified assumption in Eq. (6); until these are resolved, the central claims of unbiased conditional learning and of parameter-free structure inheritance are not established.
major comments (4)
- [Sec. 4.2, Eq. (17)] The definition of L_ce-UCL is algebraically inconsistent. The first expression is -log p(y_i in l_k | x_i, T), which equals -sum_{j=1..d} log p_j, where p_j = p(y_i in n_{phi(j)} | y_i in n_{phi(j-1)}, x_i, T). The second expression is -sum_j log[ p_j / p(y_i in n_{phi(j-1)} | x_i) ]. Using the paper's own Eq. (14), p(y_i in n_{phi(j-1)} | x_i) = product_{l<=j-2} p_l, so the sum telescopes to -log p_d + log p_1 (for d=3 it is -log p_3 + log p_1), not -log(p_1 p_2 p_3). Thus Eq. (17) does not define the negative log-likelihood of the leaf, and the unbiasedness proof in Eq. (16) applies to a per-node IPS loss while Eq. (17) is a different quantity. This is load-bearing because UCL is one of the two core modules; the UCL ablation gains in Table 3 may be due to an unstated reweighting rather than to debiasing. The authors need to provide a correct derivation of the loss used in Algorithm 1, or remove the unbiasedness claim and reframe the module as a heuristic reweighting.
- [Sec. 4.1, Eq. (6) and following paragraph] The key assumption that inherited parameters theta*(T) = theta*(T_g) * (1 - a_T) 'remain well-adapted, and no additional fine-tuning is required' is asserted without direct evidence. This is load-bearing for the TSL module: if pruned-subtree classifiers are miscalibrated or poorly adapted, the personalized trees would produce degraded predictions, and the efficiency claim (no inner optimization, fast convergence in Fig. 6) would be weakened. The paper should isolate this assumption, e.g., by comparing (i) pruning with inherited parameters, (ii) pruning followed by fine-tuning on the pruned subtree, and (iii) training the pruned structure from scratch, on at least one offline dataset.
- [Sec. 4.2, Eq. (14)-(16)] The unbiasedness proof in Eq. (16) assumes that the propensity p(y_i in n_{phi(j-1)} | x_i) is the true conditional distribution, but the implementation uses the model's own predicted probabilities from Eq. (14), which are products of learned classifier outputs. IPS is unbiased only when propensities are known or consistently estimated under suitable overlap; using self-estimated propensities as if they were fixed oracle values is not justified, and the fact that these propensities are functions of the same parameters theta being optimized makes the 'unbiased' claim circular. The authors should state explicitly what conditions are needed for Eq. (16) and provide either a corrected analysis or empirical validation (e.g., with known synthetic propensities) that the claimed bias reduction actually occurs.
- [Sec. 5.2.2, Table 3 and Fig. 7] The paper reports that UCL improves AUC and prior-to-posterior calibration, but given the Eq. (17) issue, these results are not sufficient to establish that the improvement comes from unbiased conditional learning. The experiments should be repeated with the corrected loss (once derived) and, ideally, with a baseline that uses the same per-node weights but without the log-probability structure, to separate the effect of the reweighting from the effect of the claimed unbiased estimator.
minor comments (6)
- [Sec. 4.1.2] There is a typo in 'REINFORECE'; it should be 'REINFORCE'.
- [Sec. 4.3, Algorithm 1] Line 5 says 'Compute L_ce-UCL, the log-likelihood of (x_i,y_i) belonging to path phi, and E(y_i|x_i,T_g) and Var(y_i|x_i,T_g), as Eq. 17, 4.' This is unclear: Eq. 4 defines the TPM objective, not the UCL loss; please separate the references to Eq. 17 and the components of Eq. 4.
- [Sec. 4.2, Eq. (13)] The notation 'I[y_i in n_{phi(j)}]' is used both as a conditioning event and as an indicator in Eq. (16); please define the subscript on the expectation (over what distribution) and clarify that the propensity is evaluated at x_i.
- [Sec. 5.3, Table 4] The table caption states 'Statistically significant results (p-value < 0.05) are highlighted in bold', but the body of the table does not show confidence intervals or p-values; please report the actual p-values or confidence intervals for the watch-time metric and the constraints.
- [Sec. 5.2.2] The abbreviation 'SSB' is used in the sentence 'the debiasing of SSB improves the generalization of classifiers' without being defined; it should be spelled out (likely 'sample selection bias').
- [Fig. 6] The left panel labels both bars as 'TPM' which appears to be a typo; one bar should presumably be labeled 'PTPM' to match the text about parameter count and flops.
Circularity Check
UCL's 'unbiased conditional learning' uses the model's own predicted conditional probabilities as the propensity, so Eq. 16's unbiasedness proof assumes the quantity the correction is supposed to fix.
-
self definitional
[Section 4.2, Eqs. 14-16]
"Fortunately, we can directly derive this generative process from Eq. 1: p(ŷ_i∈n_{φ_{l_k}}(j−1)|x_i)=∏_{1≤l≤j−2}p(ŷ_i∈n_{φ_{l_k}}(l)|ŷ_i∈n_{φ_{l_k}}(l−1),x_i) (14) where p(...) is referred to as the propensity, modeling the probability that a sample belongs to a parent node."
The propensity in Eq. 14 is not an independent or true selection probability; it is the product of the model's own predicted conditional probabilities from Eq. 1, i.e., the very classifiers whose bias UCL is meant to correct. Eq. 16's proof cancels E_I[I] = p(y_i∈parent|x_i) against the denominator, but the denominator is p(ŷ_i∈parent|x_i). Cancellation is valid only if the predicted propensity equals the true conditional probability, which is exactly what training has not established and is the object being learned. The debiasing term is therefore defined in terms of the biased quantity it corrects, making the 'unbiased conditional learning' claim self-referential rather than derived from an external propensity model.
full rationale
PTPM's core TSL result is validated by external offline baselines and an online A/B test, and the TPM baseline is reproduced rather than merely cited, so the main tree-learning claim is not circular. The self-citation of TPM [10] is not load-bearing. The significant circularity is confined to Section 4.2: the propensity used for IPS is constructed from the model's own predicted edge probabilities (Eq. 14 derived from Eq. 1), while the unbiasedness proof (Eq. 16) requires the true conditional probability in the denominator; absent an independent propensity model, the proof assumes its conclusion. This is a self-definitional reduction of the debiasing claim. Separately (a correctness issue, not counted as circularity): Eq. 17's second expression is not algebraically equal to −log p(leaf|x,T); with p_j the edge probability, the RHS for depth 3 is log p_3 − log p_1, not log(p_1 p_2 p_3), so the UCL loss is not the negative log-likelihood claimed. This reinforces that the UCL derivation does not support the reported ablation gains, but the empirical offline and online comparisons still give the overall method independent content. Overall partial circularity: score 5.
Assumptions & free parameters
free parameters (3)
- Global tree depth d(T_g) =
6
- Reward weights in R_T =
1 (XAUC) and 1 (MSE)
- Inference pruning threshold =
not specified
assumptions (3)
- domain assumption Pruning a globally trained tree does not require retraining classifiers: θ*(T) = θ*(T_g)⊙(1-a_T) remain well-adapted.
- domain assumption The propensity p(y_i in n_{j-1}|x_i) in Eq 14 is the true conditional probability, so the IPS estimator in Eq 16 is unbiased.
- standard math The expected watch time for a leaf interval is approximated by the midpoint of that interval.
Cite this review
Pith. "Pith review of Personalized Tree-Based Progressive Regression Model for Watch-Time Prediction in Short Video Recommendation." pith.science (2026). https://pith.science/paper/ON7YBEZ7
@misc{pith2026250522153,
author = {Pith},
title = {Pith review of: Personalized Tree-Based Progressive Regression Model for Watch-Time Prediction in Short Video Recommendation},
year = {2026},
howpublished = {\url{https://pith.science/paper/ON7YBEZ7}},
note = {Machine review of arXiv:2505.22153}
}
read the original abstract
In online video platforms, accurate watch time prediction has become a fundamental and challenging problem in video recommendation. Previous research has revealed that the accuracy of watch time prediction highly depends on both the transformation of watch-time labels and the decomposition of the estimation process. TPM (Tree based Progressive Regression Model) achieves State-of-the-Art performance with a carefully designed and effective decomposition paradigm. TPM discretizes the watch time into several ordinal intervals and organizes them into a binary decision tree, where each node corresponds to a specific interval. At each non-leaf node, a binary classifier is used to determine the specific interval in which the watch time variable most likely falls, based on the prediction outcome at its parent node. The tree structure is central to TPM, as it defines the decomposition of watch time estimation and how ordinal intervals are discretized. However, TPM uses a predefined full binary tree, which may be sub-optimal for two reasons. First, full binary trees imply equal partitioning of the watch time space, which may fail to capture the complexity of real-world distributions. Second, rather than relying on a fixed global structure, we advocate for a personalized, data-driven tree that can be learned end-to-end. Thus, we propose PTPM to enable highly personalized decomposition of watch estimation with better efficacy and efficiency. Moreover, we show that TPM suffers from selection bias due to conditional modeling and propose a simple solution. We conduct extensive experiments on offline datasets and online environments. Offline results show improved watch time accuracy, and online A/B tests further validate the effectiveness of our framework. PTPM has been fully deployed in core traffic scenarios and now serves over 400 million users daily.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Gnana Anandalingam and Terry L Friesz. 1992. Hierarchical optimization: An introduction. Annals of Operations Research 34 (1992), 1–11
work page 1992
-
[2]
Kuang-Yu Chang, Chu-Song Chen, and Yi-Ping Hung. 2011. Ordinal hyperplanes ranker with cost sensitivities for age estimation. In CVPR 2011. IEEE, 585–592
work page 2011
-
[3]
Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . 785–794
2016
-
[4]
Benoît Colson, Patrice Marcotte, and Gilles Savard. 2007. An overview of bilevel optimization. Annals of operations research 153 (2007), 235–256
work page 2007
-
[5]
Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191–198
2016
-
[6]
Koby Crammer and Yoram Singer. 2001. Pranking with ranking. Advances in neural information processing systems 14 (2001)
work page 2001
-
[7]
Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. 2022. KuaiRec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management . 540–550
2022
-
[8]
Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social, and biomedical sciences. Cambridge university press
work page 2015
Show all 21 references
-
[9]
Justin M Johnson and Taghi M Khoshgoftaar. 2019. Survey on deep learning with class imbalance. Journal of big data 6, 1 (2019), 1–54
2019
-
[10]
Xiao Lin, Xiaokai Chen, Linfeng Song, Jingwei Liu, Biao Li, and Peng Jiang
-
[11]
Roderick JA Little and Donald B Rubin. 2019. Statistical analysis with missing data. Vol. 793. John Wiley & Sons
2019
-
[12]
Jie Sun, Zhaoying Ding, Xiaoshuang Chen, Qi Chen, Yincheng Wang, Kaiqiao Zhan, and Ben Wang. 2024. CREAD: A Classification-Restoration Framework with Error Adaptive Discretization for Watch Time Prediction in Video Recom- mender Systems. In Proceedings of the AAAI Conference o...
2024
-
[13]
Steven K Thompson. 2012. Sampling. Vol. 755. John Wiley & Sons
2012
-
[14]
Williams
Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 3-4 (1992), 229–256
1992
-
[15]
Hong Wu, Hanqing Lu, and Songde Ma. 2003. A practical SVM-based algorithm for ordinal regression in image retrieval. In Proceedings of the eleventh ACM international conference on Multimedia . 612–621
2003
-
[16]
Ruohan Zhan, Changhua Pei, Qiang Su, Jianfeng Wen, Xueliang Wang, Guanyu Mu, Dong Zheng, Peng Jiang, and Kun Gai. 2022. Deconfounding duration bias in watch-time prediction for video recommendation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Da...
2022
-
[17]
Yang Zhang, Yimeng Bai, Jianxin Chang, Xiaoxue Zang, Song Lu, Jing Lu, Fuli Feng, Yanan Niu, and Yang Song. 2023. Leveraging watch-time feedback for short-video recommendations: A causal labeling framework. InProceedings of the 32nd ACM International Conference on Information ...
2023
-
[18]
Han Zhu, Daqing Chang, Ziru Xu, Pengye Zhang, Xiang Li, Jie He, Han Li, Jian Xu, and Kun Gai. 2019. Joint optimization of tree-based index and deep model for recommender systems. Advances in Neural Information Processing Systems 32 (2019)
2019
-
[19]
Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie He, Han Li, and Kun Gai
-
[2018]
In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining
Learning tree-based deep model for recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1079–1088
-
[2023]
In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Tree based progressive regression model for watch-time prediction in short-video recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 4497–4506
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.