Pith. sign in

REVIEW 4 major objections 6 minor 70 references

This paper claims that recommender systems can be optimized for long-term retention by training on dense, early-observed downstream rewards—deeper session engagement, shallow-closeup penalties, and use-case adoption—rather than on sparse, d

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A unified downstream-reward framework, derived from observable session-level behaviors and validated in online A/B tests across four Pinterest surfaces, improves long-term engagement and retention metrics.

T0 review reviewed 2026-08-02 challenge →

load-bearing objection A serious, well-engineered industrial paper with real deployment evidence, but the retention claim is weaker than the engagement claim because the main metrics overlap with the optimized rewards and DAU/WAU are under-reported. the 4 major comments →

arxiv 2607.14192 v2 pith:LW73DEEK submitted 2026-07-15 cs.LG cs.IR

Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning

Dingsu Wang , Filip Ryzner , Kelly He , Armando Ordorica , David Woo , Aditya Mantha , Liyao Lu , Usha Amrutha Nookala
show 7 more authors
Haoran Guo Jiacong He Olafur Gudmundsson Matt Chun Krystal Benitez Dhruvil Deven Badani Yijie Dylan Wang
This is my paper
classification cs.LG cs.IR
keywords downstream rewardsuser retentionlong-term engagementrecommender systemssession-level behaviorP2P explorationnegative rewardsuse case adoption
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that long-term user retention in large-scale recommendation systems can be improved without task-specific reinforcement learning or hand-engineered reward functions. Its approach defines proxy downstream rewards from near-term session behaviors—deep P2P exploration, saves, avoided shallow closeups, and engagement with new use cases—and shows offline that these behaviors predict later revisitation better than raw activity volume. The paper then adds these rewards as auxiliary heads to production ranking models and reports statistically significant online gains in successful sessions, time spent, and DAU/WAU across Pinterest surfaces. If correct, the framework offers a practical, transferable way to align ranking with long-term user value using signals that are denser and less delayed than retention labels.

Core claim

On its own terms, the paper's central discovery is that 'how' a user engages—depth of P2P rabbit-hole exploration, saves, and deep engagement across diverse content—is more predictive of future retention than 'how much' content they consume, and that these session-level signals can serve as surrogate rewards for long-term value. The paper formalizes a proxy objective in which each recommendation is scored by the expected discounted downstream reward over the trajectory it triggers, then implements it with three reward families: deeper session engagement, negative rewards for shallow closeups with user-state-specific dwell-time thresholds, and use-case adoption rewards for engaging with conte

What carries the argument

The central object is the downstream reward R̂u,t = E_{p_enter}[ E_{p_φ}[ R(S) ] ], a proxy for the unobservable retention reward: it predicts whether a candidate item will draw the user into a downstream action trajectory and aggregates the actions in that trajectory into a discounted score. Work it does: it converts sparse, delayed retention labels into dense, early-observed supervised signals; the offline screening framework (pivot-day Random Forest and session-level Markov analysis) selects which behaviors qualify, and the reward definitions are model-agnostic auxiliary heads that can be added to any ranking model.

Load-bearing premise

The load-bearing premise is that deeper downstream engagement (P2P exploration, saves, deep diversity) actually causes users to return more often, and that the observed correlations with future revisitation reflect that causal link rather than user motivation or content quality.

What would settle it

A randomized controlled experiment that matches users on short-term engagement but serves items scoring high vs low on the downstream-reward proxy, then measures 30-day revisit rates: if revisit gains shrink to zero or reverse once short-term engagement is controlled, the proxy is not driving retention. Alternatively, a re-analysis of the pivot-day data using instrumental variables for deep engagement would test whether the association with future revisitation survives confounding control.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the framework is correct, recommendation systems can improve retention metrics by adding these three reward heads without re-engineering a reinforcement learning pipeline per surface.
  • The reported ~10× reduction in idea-to-experiment time from the infrastructure makes iterating on reward definitions practical at industrial scale.
  • The offline screening criteria (correlates with revisitation, observable at session level, incrementally predictive) provide a reusable recipe for discovering new downstream rewards on other platforms.
  • Cross-surface deployment results imply that optimizing each ranker for downstream value rather than the next click yields complementary gains across a non-linear user journey.
  • State-specific dwell-time thresholds for the negative reward suggest that personalized reward definitions outperform uniform ones, a principle that likely extends to other reward families.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's causal chain 'deeper actions → longer sessions → faster revisit → retention' rests on observational correlations; if unmeasured user motivation or content quality confounds that link, optimizing these proxies may inflate engagement metrics without increasing true retention.
  • A testable extension would be to run a randomized experiment that varies only the downstream-reward weights while holding immediate-engagement weights fixed, and measure 30-day real retention: the causal claim predicts WAU/DAU gains proportional to the proxy lifts.
  • The framework's principle—dense early-observed behaviors as surrogates for sparse delayed goals—could transfer to other domains such as video platforms or e-commerce, but the specific action definitions are category-specific, so the screening step would need to be repeated.
  • If shallow-closeup penalties change user behavior over time, users may learn to dwell just past the threshold, so ongoing monitoring of closeup duration distributions is needed to prevent reward hacking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a model-agnostic downstream reward framework for optimizing long-term user engagement and retention in large-scale recommendation systems. It first formulates a proxy objective based on downstream rewards, then uses an offline screening framework (pivot-day random forest, session-level Markov analysis, duration-threshold tests) to identify session-level behaviors predictive of future retention. It defines three reward families: deeper-session engagement rewards (discounted saves/downloads/screenshots), negative rewards (shallow-closeup penalties with user-state-specific dwell-time thresholds), and use-case adoption rewards (engagement with content outside established interest clusters). It also describes an infrastructure evolution from precomputed DRv1 tables to DRv2 on-demand label computation inside a Ray-based dataloader. Online A/B experiments on Homefeed and other surfaces report relative lifts in Successful Sessions, Total Time Spent, and some DAU/WAU trends, with deployment claimed across Homefeed, Related Pins, Search, and Notifications.

Significance. If the retention claim is substantiated, this is a practically valuable contribution: it provides a unified, model-agnostic way to inject long-term value into ranking models without task-specific RL reward engineering, and the DRv2 infrastructure (Section 4.2) with its 10x reduction in idea-to-experiment time is a concrete engineering achievement. The offline screening framework (Section 3) and the state-specific threshold analysis (Section 5.2.2) are also useful methodological contributions. The paper is explicit about the observational nature of its offline chain and about the directional alignment of its online metrics (Section 5.1), which is commendable, but those admissions also mark the central weakness: randomized evidence for true retention gains is sparse and partly confounded by metric overlap with the optimized rewards.

major comments (4)
  1. [§5.1 and Table 3] The primary online metrics, Successful Sessions and Total Time Spent, are defined using the same action types that the downstream rewards explicitly optimize: SS includes save, closeup, download, screenshot, etc. (§5.1), and the deeper-session rewards in §4.1.1 are sums over saves, downloads, and screenshots. Lifts in these metrics can therefore be mechanical: increasing the targeted actions increases the metric without necessarily increasing true retention. The paper acknowledges this ('We view these metrics as outcomes that are directionally aligned') but does not provide the DAU/WAU/MAU numbers needed to break the circularity. Only a single WAU lift (+0.1%, §5.2.1) and a qualitative 'DAU/WAU trends remain positive' (§5.2.2) are reported, with no confidence intervals. This is the load-bearing evidence for the central retention claim and must be supplied.
  2. [§3] The empirical chain 'deeper actions ⇒ longer sessions ⇒ faster revisit ⇒ retention↑' is explicitly observational, as the text states. The pivot-day RF has AUC 0.65–0.70, which is modest, and the t-tests comparing revisit times have extremely small p-values on what must be enormous samples, so they do not establish practically meaningful effect sizes. More importantly, user motivation and content quality are natural confounders: users who are about to return may browse differently regardless of the recommendations. The paper does not attempt to control for this beyond user-prevalence normalization. This concern is not fatal by itself, but it means the offline analysis cannot validate the reward proxies; only randomized experiments on retention-native metrics can, and those are under-reported.
  3. [§5.2.2] The negative-reward story involves a post-hoc specification search: a first uniform-threshold launch produced negative retention and engagement trends for non-core users and cross-surface cannibalization, after which the authors conducted offline analysis by user state and launched a second version with different thresholds (δ_core=1s, δ_non-core=0.5s). This is a legitimate engineering process, but as reported it raises multiple-comparison concerns, especially because the second launch is described as successful without reporting DAU/WAU numbers or any uncertainty. To support the claim that negative rewards 'improve both engagement and retention-related outcomes', the paper should report pre-registered or multiplicity-corrected analyses, or at minimum the DAU/WAU results for both launches.
  4. [§5.2, Table 3] The paper states 'All the metrics shown below are statistically significant in online experiments' but provides no p-values, confidence intervals, or standard errors. Given the tiny relative lifts (e.g., +0.10% SS, +0.10% Total Time Spent in Table 3), and the fact that some effects are in opposite directions for core vs. non-core users, statistical significance without effect-size uncertainty is not sufficient. The authors should report confidence intervals for at least the retention-native metrics and ideally for all reported lifts.
minor comments (6)
  1. [Definition 2.2] Typo: 'Here, 𝛾 and 𝜙 denote the learnable parameter sets of the entry model and the downstream trajectory model respectively' should refer to β and φ, since the entry model is denoted p_enter_β. Also, the discount factors in Definition 2.1 and Definition 2.3 use γ and β inconsistently; please unify notation.
  2. [§4.1.3] The threshold is denoted τ in §5.2.3 but η in the reward definition; please use one symbol consistently.
  3. [§4.1.1] 'Aeng = {save,downloads,screenshots}' should be 'A_eng = {save, download, screenshot}' for grammatical consistency with the definition of w_a.
  4. [§5.2.2] The sentence 'DAU/WAU trends remain positive for both core and non-core users' is too vague; please give actual percentages and, if possible, the time window.
  5. [Table 1] The table header mixes notation: 'Pctl. Norm.' and 'UP Norm.' are clear, but the meaning of 'Sign' and 'Str.' is explained only in the caption; consider adding a legend in the table itself. Also, 'RF Imp.' entries show '–' for some rows without explanation.
  6. [Abstract and §1] The phrase 'an proxy optimization signal' should be 'a proxy'. Also, 'Results from long-time online A/B testing experiment' is awkward; consider 'long-running online A/B experiments'.

Circularity Check

1 steps flagged

Primary online metrics overlap with the optimized reward actions; the only independent retention metrics are under-reported.

specific steps
  1. self definitional [Section 5.1 (metric definitions), Section 4.1.1 (R_eng), Section 5.2.1 (Homefeed results)]
    "Successful Sessions (SS): Number of user sessions on Pinterest with one or more of the following actions: save, search, closeup, create, download, click out etc. ... We view these metrics as outcomes that are directionally aligned with the downstream reward signals introduced in Section 2. ... Aeng = {save,downloads,screenshots}."

    SS is defined as sessions containing one or more of save/closeup/download/etc., and R_eng is the discounted sum of per-step rewards for exactly {save,downloads,screenshots}. Thus increasing the targeted actions (reported +0.7% downloads, +0.7% screenshots) directly adds sessions to the SS bucket by definition; the reported +0.36% SS lift is therefore partly a mechanical consequence of optimizing the reward, not an independent validation of long-term value. The paper's own wording—'directionally aligned with the downstream reward signals'—concedes this overlap. Total Time Spent is similarly aligned with the session-depth objective of R_eng. DAU/WAU are the only independent retention metrics, but they are reported only qualitatively or as a single +0.1% WAU, without effect sizes or confidenc

full rationale

The offline screening (Section 3) is not circular: it is a real predictive task (AUC 0.65–0.70 on held-out data) whose target is a transition from low engagement to sustained engagement, and the chosen rewards are not defined in terms of the success metrics. The paper also honestly labels the session-length chain 'observational rather than causal.' The circularity is in the online evaluation: the headline engagement metrics (Successful Sessions, Total Time Spent) are built from the same behaviors the deeper-session reward is designed to increase, so a portion of the measured lift is by construction. The only independent retention evidence (DAU/WAU/MAU) is seriously under-reported: no effect sizes or confidence intervals, one +0.1% WAU number, and a qualitative 'DAU/WAU trends remain positive.' There is no load-bearing self-citation chain; references to the authors' prior ranker work ([46]) and to related retention work ([24]) are contextual, not used to force the method. This is therefore partial circularity in the evaluation design rather than a derivation that equals its inputs; the framework's offline screening and engineering contributions stand independently, which is why the score is 6 rather than higher.

Axiom & Free-Parameter Ledger

8 free parameters · 5 axioms · 0 invented entities

The framework relies on a chain of empirically screened but correlational proxies (deep engagement, saves, shallow-closeup avoidance, new use case) rather than on a first-principles derivation. Many thresholds and weights are fitted to offline or online data; the evaluation metrics overlap with the optimized actions, introducing partial circularity. No new physical or conceptual entities are invented.

free parameters (8)
  • dwell-time threshold for shallow-closeup penalty (core users) = 1s
    Set from actionability curves in Fig 3b; offline screening to balance coverage/purity (Section 4.1.2).
  • dwell-time threshold for shallow-closeup penalty (non-core users) = 0.5s
    Set from actionability curves for non-core segment (Section 4.1.2).
  • dwell-time threshold for P2P negative reward = 2s
    Offline actionability analysis for P2P closeups; duration-only condition (Section 5.3).
  • cosine similarity threshold for new use case (tau) = 0.6
    Tuned over {0.5, 0.6, 0.7} to balance label volume and downstream performance (Section 5.2.3).
  • serving weights {w_i} for reward heads = not reported
    Tuned outside the ranker with HyperOPT; chosen to keep immediate engagement neutral (Section 4.3).
  • engagement action weights {w_a} (save, download, screenshot) = not reported
    Defined in Section 4.1.1; values not specified in the paper, presumably chosen by product value.
  • discount factor gamma for accumulated rewards = not reported
    Hyperparameter in Eq. (2) and (4); no value given.
  • negative reward strength lambda_SC = not reported
    Penalty strength in Eq. (6); implicit in serving weight tuning (Section 4.1.2).
axioms (5)
  • domain assumption Deep P2P exploration, saves, and deep diverse engagement are valid proxies for long-term retention
    Basis of the whole reward design; supported by correlational offline screening (Section 3, Table 1).
  • domain assumption The causal chain deeper actions → longer sessions → faster revisit → retention holds in deployment
    Established observationally only (Section 3, Fig 2); the online experiments test the downstream effect, but the proxy-causality link is assumed.
  • domain assumption User-state-specific thresholds generalize from offline analysis to online behavior
    After initial online failure for non-core users, thresholds were segmented based on offline actionability curves and assumed to transfer (Section 5.2.2).
  • domain assumption Random Forest screening model (AUC 0.65-0.70) captures predictive signal sufficient for ranking rewards
    Used to rank candidates; the modest AUC is assumed adequate (Section 3).
  • standard math Statistical tests (t-test, Pearson) are valid on the A/B and observational data
    Standard assumption underlying reported p-values (Sections 3, 5).

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning." pith.science (2026). https://pith.science/paper/LW73DEEK

@misc{pith2026260714192,
  author       = {Pith},
  title        = {Pith review of: Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LW73DEEK}},
  note         = {Machine review of arXiv:2607.14192}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

As recommender systems mature in the past few years, their optimization objectives have evolved from a primary focusing on short-term behavioral signals to a broader emphasis on long-term user engagement and retention. However, directly optimizing retention is difficult because return signals are sparse, delayed, and only partially attributable to earlier recommendations. Prior work has addressed this challenge with sequential modeling and reinforcement learning, but these approaches typically require task specific reward engineering, substantial computational overhead, and surface specific implementations that are difficult to generalize. In this paper, we present a unified, model-agnostic downstream reward framework for optimizing long-term user value in large-scale recommendation systems. First, we formulate the downstream reward learning problem and develop an offline screening framework to identify session level behaviors that are both observable early and predictive of future retention. We then propose several model-agnostic downstream rewards signals derived from observed user action patterns across multiple sources. We further discuss the engineering effort to productionize the proposed rewards derivations and challenges we faced when adding them to our ranking models. Online A/B experiments demonstrate consistent improvements in engagement and retention-related metrics, and the framework has been deployed across multiple Pinterest surfaces, including Homefeed, Related Pins, Search, and Notifications.

Figures

Figures reproduced from arXiv: 2607.14192 by Aditya Mantha, Armando Ordorica, David Woo, Dhruvil Deven Badani, Dingsu Wang, Filip Ryzner, Haoran Guo, Jiacong He, Kelly He, Krystal Benitez, Liyao Lu, Matt Chun, Olafur Gudmundsson, Usha Amrutha Nookala, Yijie Dylan Wang.

Figure 1
Figure 1. Figure 1: An illustration of user action patterns in P2P rabbit [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Median hours to next revisit for sessions above [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Shallow closeups data analysis. within the same trajectory. We define the shallow-closeup indicator I SC 𝑢,𝑡+𝑖 = I [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: DRv2 table structure illustration: per-user daily [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

70 extracted references · 1 linked inside Pith

  1. [1]

    Parag Agrawal. 2024. Building a Large-Scale Recommendation System: People You May Know. https://www.linkedin.com/blog/engineering/recommendations/ building-a-large-scale-recommendation-system-people-you-may-know

    2024

  2. [2]

    Ting Bai, Yudong Xiao, Bin Wu, Guojun Yang, Hongyong Yu, and Jian-Yun Nie

  3. [3]

    James Bergstra, Brent Komer, Chris Eliasmith, Dan Yamins, and David D Cox

  4. [4]

    James Bergstra, Daniel Yamins, and David Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. InInternational conference on machine learning. PMLR, 115–123

    2013

  5. [5]

    Qingpeng Cai, Shuchang Liu, Xueliang Wang, Tianyou Zuo, Wentao Xie, Bin Yang, Dong Zheng, Peng Jiang, and Kun Gai. 2023. Reinforcing user retention in a billion scale short video recommender system. InCompanion Proceedings of the ACM Web Conference 2023. 421–426

    2023

  6. [6]

    Yukuo Cen, Jianwei Zhang, Xu Zou, Chang Zhou, Hongxia Yang, and Jie Tang

  7. [7]

    Allison JB Chaney, Brandon M Stewart, and Barbara E Engelhardt. 2018. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. InProceedings of the 12th ACM conference on recommender systems. 224–232

    2018

  8. [8]

    Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, et al. 2023. TWIN: TWo-stage interest network for lifelong user behavior modeling in CTR prediction at kuaishou. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3785–3794

    2023

  9. [9]

    Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. InProceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785–794

    2016

  10. [10]

    Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach, and Lina Yao

  11. [11]

    Chen Cheng, Fen Xia, Tong Zhang, Irwin King, and Michael R Lyu. 2014. Gradient boosting factorization machines. InProceedings of the 8th ACM Conference on Recommender systems. 265–272

    2014

  12. [12]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al

  13. [13]

    Konstantina Christakopoulou, Can Xu, Sai Zhang, Sriraj Badam, Trevor Potter, Daniel Li, Hao Wan, Xinyang Yi, Ya Le, Chris Berg, et al. 2022. Reward shaping for user satisfaction in a REINFORCE recommender.arXiv preprint arXiv:2209.15166 (2022)

    Pith/arXiv arXiv 2022

  14. [14]

    Xiao Chunjing, Ranhao Guo, Zhang Yongwang, and Xiaoming Wu. 2025. Learn- ing Multi-interest Embedding with Dynamic Graph Cluster for Sequention Rec- ommendation. InConference on Uncertainty in Artificial Intelligence. PMLR, 4652– 4662

    2025

  15. [15]

    Robin Devooght and Hugues Bersini. 2017. Long and short-term recommenda- tions with recurrent neural networks. InProceedings of the 25th conference on user modeling, adaptation and personalization. 13–21

    2017

  16. [16]

    Shuyang Du, James Lee, and Farzin Ghaffarizadeh. 2019. Improve user retention with causal learning. InThe 2019 ACM SIGKDD Workshop on Causal Discovery. PMLR, 34–49

    2019

  17. [17]

    Yingpeng Du, Ziyan Wang, Zhu Sun, Yining Ma, Hongzhi Liu, and Jie Zhang

  18. [18]

    Huizhong Duan, Emre Kiciman, and ChengXiang Zhai. 2012. Click patterns: An empirical representation of complex query intents. InProceedings of the 21st ACM international conference on Information and knowledge management. 1035–1044

    2012

  19. [19]

    Ariel Evnine, Stratis Ioannidis, Dimitris Kalimeris, Shankar Kalyanaraman, Wei- wei Li, Israel Nir, Wei Sun, and Udi Weinsberg. 2024. Achieving a better tradeoff in multi-stage recommender systems through personalization. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4939–4950

    2024

  20. [20]

    Jerome H Friedman. 2001. Greedy function approximation: a gradient boosting machine.Annals of Statistics29, 5 (2001), 1189–1232

    2001

  21. [21]

    Xudong Gong, Qinlin Feng, Yuan Zhang, Jiangling Qin, Weijie Ding, Biao Li, Peng Jiang, and Kun Gai. 2022. Real-time short video recommendation on mobile devices. InProceedings of the 31st ACM international conference on information & knowledge management. 3103–3112

    2022

  22. [22]

    Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. 1725–1731

    2017

  23. [23]

    InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining

    Disentangled multi-interest representation learning for sequential rec- ommendation. InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining. 677–688

  24. [24]

    Weijie Jiang, Armando Ordorica, Jaewon Yang, Olafur Gudmundsson, Yucheng Tu, and Huizhong Duan. 2026. Save, Revisit, Retain: A Scalable Framework for Enhancing User Retention in Large-Scale Recommender Systems. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 39996–40005

    2026

  25. [25]

    Yangqin Jiang, Xubin Ren, Lianghao Xia, Da Luo, Kangyi Lin, and Chao Huang

  26. [26]

    Clark Mingxuan Ju, Liam Collins, Leonardo Neves, Bhuvesh Kumar, Louis Yufeng Wang, Tong Zhao, and Neil Shah. 2025. Generative Recommendation with Seman- tic IDs: A Practitioner’s Handbook. InProceedings of the 34th ACM International Conference on Information and Knowledge Management. 6420–6425

    2025

  27. [27]

    Komal Kapoor, Mingxuan Sun, Jaideep Srivastava, and Tao Ye. 2014. A hazard based approach to user return time prediction. InProceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 1719– 1728

    2014

  28. [28]

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems30 (2017)

    2017

  29. [29]

    Jianhua Han, Yong Yu, Feng Liu, Ruiming Tang, and Yuzhou Zhang. 2019. Op- timizing ranking algorithm in recommender system via deep reinforcement learning. In2019 international conference on artificial intelligence and advanced manufacturing (AIAM). IEEE, 22–26

    2019

  30. [30]

    Ziru Liu, Shuchang Liu, Zijian Zhang, Qingpeng Cai, Xiangyu Zhao, Kesen Zhao, Lantao Hu, Peng Jiang, and Kun Gai. 2024. Sequential recommendation for optimizing both immediate feedback and long-term retention. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1872–1882

    2024

  31. [31]

    Ziru Liu, Jiejie Tian, Qingpeng Cai, Xiangyu Zhao, Jingtong Gao, Shuchang Liu, Dayou Chen, Tonghao He, Dong Zheng, Peng Jiang, et al. 2023. Multi-task recommendations with reinforcement learning. InProceedings of the ACM web conference 2023. 1273–1282

    2023

  32. [32]

    Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, and Yongfeng Zhang. 2019. Value-aware recommendation based on reinforce- ment profit maximization. InThe World Wide Web Conference. 3123–3129

    2019

  33. [33]

    Paul Resnick and Hal R Varian. 1997. Recommender systems.Commun. ACM40, 3 (1997), 56–58

    1997

  34. [34]

    John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Jacob Menick, Jiayi Weng, Juan Felipe Ceron Uribe, Liam Fedus, Luke Metz, Michael Pokorny, et al

  35. [35]

    David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Diet- mar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems.Advances in neural information processing systems28 (2015)

    2015

  36. [36]

    Qi Liu, Zhilong Zhou, Gangwei Jiang, Tiezheng Ge, and Defu Lian. 2023. Deep task-specific bottom representation network for multi-task recommendation. InProceedings of the 32nd ACM International Conference on Information and Knowledge Management. 1637–1646

    2023

  37. [37]

    Gary Tang, Jiangwei Pan, Henry Wang, and Justin Basilico. 2023. Reward in- novation for long-term member satisfaction. InProceedings of the 17th ACM Conference on Recommender Systems. 396–399

    2023

  38. [38]

    Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. InProceedings of the 14th ACM conference on recommender systems. 269–278

    2020

  39. [39]

    Yu Tian, Jianxin Chang, Yanan Niu, Yang Song, and Chenliang Li. 2022. When multi-level meets multi-interest: A multi-grained neural model for sequential recommendation. InProceedings of the 45th international ACM SIGIR conference on research and development in information retrieval. 1632–1641

    2022

  40. [40]

    Chunqi Wang, Bingchao Wu, Zheng Chen, Lei Shen, Bing Wang, and Xiaoyi Zeng

  41. [41]

    Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. InProceedings of the ADKDD’17. 1–7

    2017

  42. [42]

    Introducing chatgpt.OpenAI blog(2022)

    2022

  43. [43]

    Yuyan Wang, Mohit Sharma, Can Xu, Sriraj Badam, Qian Sun, Lee Richardson, Lisa Chung, Ed H Chi, and Minmin Chen. 2022. Surrogate for long-term user experience in recommender systems. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 4100–4109

    2022

  44. [44]

    Zihua Si, Lin Guan, ZhongXiang Sun, Xiaoxue Zang, Jing Lu, Yiqun Hui, Xingchao Cao, Zeyu Yang, Yichen Zheng, Dewei Leng, et al . 2024. Twin v2: Scaling ultra-long user behavior sequence modeling for enhanced ctr predic- tion at kuaishou. InProceedings of the 33rd ACM International Conference on Information and Knowledge Management. 4890–4897

    2024

  45. [45]

    Qingyun Wu, Hongning Wang, Liangjie Hong, and Yue Shi. 2017. Returning is believing: Optimizing long-term user engagement in recommender systems. InProceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1927–1936

    2017

  46. [46]

    Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and Andrew Zhai. 2023. Transact: Transformer-based realtime user action model for recommendation at pinterest. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5249–5259

    2023

  47. [47]

    Ruiyang Xu, Jalaj Bhandari, Dmytro Korenkevych, Fan Liu, Yuchen He, Alex Nikulkov, and Zheqing Zhu. 2023. Optimizing long-term value for auction-based recommender systems via on-policy reinforcement learning. InProceedings of the 17th ACM Conference on Recommender Systems. 955–962

    2023

  48. [48]

    Wanqi Xue, Qingpeng Cai, Zhenghai Xue, Shuo Sun, Shuchang Liu, Dong Zheng, Peng Jiang, Kun Gai, and Bo An. 2023. Prefrec: Recommender systems with human preferences for reinforcing long-term user engagement. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2874–2884

    2023

  49. [49]

    InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    Scaling transformers for discriminative recommendation via generative pretraining. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 2893–2903

  50. [50]

    Hyunsik Yoo, Ruizhong Qiu, Charlie Xu, Fei Wang, and Hanghang Tong. 2025. Generalizable recommender system during temporal popularity distribution shifts. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 1833–1843

    2025

  51. [51]

    Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. 2021. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. InProceedings of the web conference 2021. 1785–1797. 9 Dingsu Wang et al

    2021

  52. [52]

    Qihua Zhang, Junning Liu, Yuzhuo Dai, Yiyan Qi, Yifan Yuan, Kunlun Zheng, Fan Huang, and Xianfeng Tan. 2022. Multi-task fusion via reinforcement learning for long-term user satisfaction in recommender systems. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 4510–4520

    2022

  53. [53]

    Yuyan Wang, Jing Zhong, Yuxin Cui, Zhaohui Guo, Chuanqi Wei, Yanchen Wang, and Zellux Wang. 2025. Not All Impressions Are Created Equal: Psychology- Informed Retention Optimization for Short-Form Video Recommendation. In Proceedings of the Nineteenth ACM Conference on Recommender Systems. 1022– 1025

    2025

  54. [54]

    Xiao Zhang, Haonan Jia, Hanjing Su, Wenhan Wang, Jun Xu, and Ji-Rong Wen

  55. [55]

    Yang Zhang, Dong Wang, Qiang Li, Yue Shen, Ziqi Liu, Xiaodong Zeng, Zhiqiang Zhang, Jinjie Gu, and Derek F Wong. 2021. User Retention: A Causal Approach with Triple Task Modeling.. InIJCAI. 3399–3405

    2021

  56. [56]

    Zijian Zhang, Shuchang Liu, Jiaao Yu, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Ziru Liu, Qidong Liu, Hongwei Zhao, Lantao Hu, et al . 2024. M3oe: Multi-domain multi-task mixture-of experts recommendation framework. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 893–902

    2024

  57. [57]

    Kesen Zhao, Lixin Zou, Xiangyu Zhao, Maolin Wang, and Dawei Yin. 2023. User retention-oriented recommendation with decision transformer. InProceedings of the ACM Web Conference 2023. 1141–1149

    2023

  58. [58]

    Zhenghai Xue, Qingpeng Cai, Bin Yang, Lantao Hu, Peng Jiang, Kun Gai, and Bo An. 2025. AURO: Reinforcement learning for adaptive user retention optimization in recommender systems. InProceedings of the ACM on Web Conference 2025. 391–401

    2025

  59. [59]

    Jie Zhu, Zhifang Fan, Xiaoxie Zhu, Yuchen Jiang, Hangyu Wang, Xintian Han, Haoran Ding, Xinmin Wang, Wenlin Zhao, Zhen Gong, et al. 2025. Rankmixer: Scaling up ranking models in industrial recommenders. InProceedings of the 34th ACM International Conference on Information and Knowledge Management. 6309–6316

    2025

  60. [60]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, et al. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommenda- tions. InInternational Conference on Machine Learning. PMLR, 58484–58509

    2024

  61. [62]

    Shengyu Zhang, Lingxiao Yang, Dong Yao, Yujie Lu, Fuli Feng, Zhou Zhao, Tat- Seng Chua, and Fei Wu. 2022. Re4: Learning to re-contrast, re-attend, re-construct for multi-interest recommendation. InProceedings of the ACM web conference

    2022

  62. [68]

    Yifei Zhao, Yu-Hang Zhou, Mingdong Ou, Huan Xu, and Nan Li. 2020. Maxi- mizing cumulative user engagement in sequential recommendation: An online optimization perspective. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2784–2792

    2020

  63. [70]

    Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, and Dawei Yin. 2019. Reinforcement learning to optimize long-term user engagement in recommender systems. InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2810–2818. 10

    2019

  64. [2015]

    Hyperopt: a python library for model selection and hyperparameter optimization.Computational Science & Discovery8, 1 (2015), 014008

    2015

  65. [2016]

    InProceedings of the 1st workshop on deep learning for recommender systems

    Wide & deep learning for recommender systems. InProceedings of the 1st workshop on deep learning for recommender systems. 7–10

  66. [2020]

    InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining

    Controllable multi-interest framework for recommendation. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2942–2951

  67. [2021]

    InProceedings of the 44th international ACM SIGIR conference on research and development in information retrieval

    Counterfactual reward modification for streaming recommendation with delayed feedback. InProceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 41–50

  68. [2022]

    InProceedings of the ACM web conference 2022

    A contrastive sharing model for multi-task recommendation. InProceedings of the ACM web conference 2022. 3239–3247

    2022

  69. [2024]

    On the opportunities and challenges of offline reinforcement learning for recommender systems.ACM Transactions on Information Systems42, 6 (2024), 1–26

    2024

  70. [2025]

    InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

    Recgpt: A foundation model for sequential recommendation. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 10140–10154

    2025

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.