Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a policy-gradient RL agent can replace manual tuning of ad-ranking utility weights, raising click-through rate by 9.7% and long click-through rate by 7.7% on the treated segment in a production A/B experiment.

desk verdict A deployed RL-for-utility-tuning paper with real online gains, but the training update is a heuristic surrogate without importance correction; still worth a serious referee. read the letter →

arxiv 2509.05292 v1 pith:H6X5JD7H submitted 2025-09-05 cs.LG

Xiao Yang , Mehdi Ben Ayed , Longyu Zhao , Fan Zhou , Yuchen Shen , Abe Engle , Jinfeng Zhuang , Ling Leng
show 3 more authors
Jiajing Xu Charles Rosenberg Prathibha Deshikachar
This is my paper
classification cs.LG
keywords RankingUtilityTuningAdRecommenderSystemDeepReinforcementLearningPolicyGradientREINFORCERewardFunctionPersonalizationOnlineA/BTesting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Manual tuning of an ad-ranking utility—a weighted sum of predicted engagement probabilities gated by a reserve price—is interpretable but struggles with combinatorial parameter counts, seasonal shifts, and user heterogeneity. DRL-PUT reframes this tuning as a one-step reinforcement learning problem: given the user and request features, an MLP policy outputs a probability distribution over a discretized, grouped set of hyperparameters, and is trained with a REINFORCE-style update on logged traffic. The paper's main evidence is a production A/B test in which the treated segment saw click-through rate rise 9.7% and long click-through rate rise 7.7% relative to the manually tuned baseline. A reward-function ablation shows that the composition of the reward—revenue versus user-engagement terms—is what steers the trade-off between business metrics, and the predicted weights vary sensibly with users' historical CTR and CVR, indicating personalization.

What carries the argument

The central object is the ranking utility function U = 1{Estimated_Revenue ≥ b}·(Estimated_Revenue + Σ p(engagement_i)·w_i), together with the policy π(a|s, θ) that selects the hyperparameter vector A = (b, {w_i}) from a discretized, grouped action space. The REINFORCE-style update θ ← θ + η·(1/B)Σ_i r_i ∇_θ log π(a_i|s_i, θ) carries the learning: it raises the log-likelihood of logged actions in proportion to their observed reward, with no value function and no discounting (γ=0). Action-space discretization—ten levels per weight, with semantically related engagement weights forced to share values—reduces the search space to 10^3 combinations and is what makes direct policy learning feasible

What would settle it

Take the logged state-action-reward data used for training and estimate the policy's value with an inverse-propensity scorer that accounts for the uniform behavior policy. If the learned policy's corrected expected reward is not higher than the uniform policy's, the reported online CTR gains cannot be attributed to the RL objective. Equivalently, in a live test, scramble the reward coefficients (α, β, γ) and observe whether served weights track the scramble; if they do not, the policy is optimizing something other than the stated reward.

Watch

Extended reading notes

Core claim

The paper claims that the hyperparameters in the ranking utility U = 1{Estimated_Revenue ≥ b}·(Estimated_Revenue + Σ_i p(engagement_i)·w_i) can be learned rather than hand-set. It models an incoming ad request as a state, the vector A = (b, {w_i}) as an action from a discretized and semantically grouped space of 10^3 candidates, and the immediate reward as a campaign-type-dependent weighted sum of expected revenue and user-engagement probabilities. The agent is a multilayer perceptron trained by the update θ ← θ + η·(1/B)Σ_i r_i ∇_θ log π(a_i|s_i, θ) on batches of about 0.5% of production traffic logged under a uniform behavioral policy. In the online A/B experiment, this approach beat the m

Load-bearing premise

The load-bearing premise is that the reward-weighted log-likelihood update on actions logged by a uniform random policy, with no importance weights and no baseline, actually maximizes the intended reward for the deployed policy.

Editorial extensions

If this is right

  • Utility weights no longer need to be static: the same framework can re-learn them continuously as user behavior and advertiser demand shift.
  • An RL agent can be layered on top of existing prediction models without retraining them, since it only re-weights their outputs.
  • Reward design, not the RL algorithm, is the main lever controlling the revenue-versus-engagement trade-off: dropping the user-value term flips CTR from +9.71% to −0.74%.
  • Grouping semantically related engagement weights into shared parameters reduces the action space enough (to 10^3) for stable convergence on modest logged data.
  • Personalization emerges from state features: high-CTR users get higher click weights and lower reserve prices, while high-CVR users get higher reserve prices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because Eq (4.1) uses reward-weighted log-likelihood without importance sampling or a baseline, the update is not the true policy gradient for the deployed policy; the same objective could be fit by straightforward reward-weighted supervised learning, so the online gains may reflect the reward signal more than RL-specific exploration.
  • A cheap falsification outside the paper: on the logged uniform data, compare the learned policy's expected reward against an inverse-propensity-weighted estimate; if the corrected estimate shows no gain over uniform sampling, the A/B lift cannot be credited to optimizing the stated reward.
  • The one-step formulation suggests a natural extension to multi-step or session-level tuning, which the paper lists as future work; reward design would then need to include delayed effects.
  • The discretization trick (m=10 levels, g=3 groups) is likely transferable to other ranking systems with linear utilities, but the choice of groups and ranges is currently manual and could be a source of bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DRL-PUT, a policy-gradient RL framework that selects discretized weights and a reserve price for a linear ad ranking utility. The policy is an MLP with softmax output; training uses a REINFORCE-style update on data logged from a uniform behavior policy. The method is evaluated in a production A/B test at Pinterest, reporting a +9.7% CTR increase and +7.7% CTR30 increase on the treated segment, with reward-function ablations and a visualization of personalized weight predictions.

Significance. If the empirical results hold, the paper demonstrates a practical framework for automated, personalized utility tuning in industrial ad recommender systems, where static manual weights are still common. The production-scale A/B test, reward ablations, and feature-based personalization analysis are valuable contributions. However, the theoretical grounding of the training objective is incomplete, and the online evidence lacks essential statistical detail; the strength of the central claim is therefore not yet commensurate with the evidence.

major comments (4)
  1. [Section 4.3, Eq. (4.1); Section 5] The update in Eq. (4.1) is a reward-weighted log-likelihood over actions logged from the uniform behavior policy π_B. This is not the policy gradient for J(θ)=E_s Σ_a π(a|s) r(s,a). The correct off-policy gradient requires the importance ratio π(a|s)/π_B(a|s); with π_B uniform this ratio is proportional to π(a|s). Omitting it means the update maximizes E_{s,a∼π_B}[r log π(a|s)], whose per-state solution is π(a|s) ∝ r(s,a), not a policy that selects the highest-reward action. Section 7 acknowledges the off-policy setting but does not correct the estimator. The central claim that DRL-PUT 'directly learn[s] an optimal policy' is therefore unsupported. Please add an importance-weighted correction, provide a simulation or off-policy evaluation showing the surrogate recovers the optimal action, or substantially soften the claim to a heuristic reward-weighted update.
  2. [Section 6.2.2, Table 2] The primary evidence for the paper's contribution is the online A/B experiment. However, no confidence intervals, p-values, or details of the test (duration, sample size, randomization unit, multiple-comparison handling) are reported. Only 'statistically insignificant' is marked for some metrics, and the headline CTR/CTR30 gains are unmarked. Given the large number of metrics in Tables 2–5, these numbers could be inflated by multiple comparisons. Please report uncertainty for the headline metrics and for each ablation.
  3. [Section 3.5, Eq. (3.4); Section 6.2.3] The reward function r in Eqs. (3.2)–(3.4) is a weighted sum of the same estimated engagement probabilities that the action weights w_i multiply in the ranking utility (3.1), with hand-tuned coefficients α, β, γ. Consequently, the reported CTR/CTR30 improvements are direct consequences of the chosen reward, not an independent demonstration that the RL agent discovered a good trade-off. The ablation in Tables 3–5 shows how sensitive the outcomes are to these coefficients. The paper should state explicitly that the agent optimizes an estimated proxy, and discuss the potential for reward hacking (e.g., exploiting correlations among estimated probabilities rather than improving actual user engagement).
  4. [Section 6.1, Eqs. (6.1)–(6.2)] The offline metrics are not adequate to validate the learned policy. Relative_Gain (6.2) is computed with the same reward r used in training and compares action probabilities to the uniform behavior policy; it therefore measures fit to the surrogate objective, not decision quality. Diversity (6.1) only checks that the policy is not degenerate. No held-out evaluation or counterfactual estimate is provided. Please include an off-policy evaluation (e.g., IPS on a logged validation set) or a small simulation to establish that the learned policy improves on the behavior policy.
minor comments (6)
  1. [Section 6.1] 'Reletive_Gain' should be 'Relative_Gain'.
  2. [Section 3.3] The phrase 'their values shall always be the same𝑖𝑡ℎ' is garbled; please rephrase.
  3. [Algorithm 1] Algorithm 1 uses α for the step size while Eq. (4.1) uses η; make the notation consistent. Also, the 'policy gradient w.r.t. θ: ∇θ L(θ)' in Algorithm 1 is the negative of the gradient used in Eq. (4.1); please align the signs.
  4. [Table 1] The success/failure indicators (✓/✗) in Table 1 are not legible; consider using text or a clearer table format.
  5. [Section 6.3, Figure 3] Hiding absolute values in Figure 3 makes it impossible to assess effect sizes; include scale labels or a table with the numerical values.
  6. [Section 1] The claim 'first published work in this area' is hard to verify; please soften it unless a thorough literature search is documented.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported CTR/CTR30 gains come from an online A/B experiment against a manual baseline, not from the reward function itself; the reward is a training objective, and self-citations are non-load-bearing.

full rationale

Step-by-step, the paper defines the ranking utility (3.1) with action weights w_i, defines a reward (3.2)-(3.4) as a linear combination of estimated engagement probabilities, trains a policy with Eq (4.1) to maximize that reward, and then measures actual business metrics in an online A/B experiment (Table 2). No step equates a 'prediction' with an input. The reward is not the predicted outcome: it uses model-estimated p(click), p(click30), p(conversion), while the reported CTR/CTR30 are measured from real user behavior in production. The claim 'DRL-PUT improved the click-through rate by 9.7%' is an external, online evaluation result, not a derived quantity from the reward definition. The hand-tuned coefficients (alpha, beta, gamma) are design choices in the reward, not parameters fitted to the reported metric deltas; the ablation study (R0-R4) explicitly explores trade-offs, so the reward's relation to the metrics is an empirical finding rather than a tautology. The off-policy concern (uniform behavior logging without importance sampling) is a potential bias in the training update, but that is a statistical correctness issue, not circularity of definitions. Self-citations (e.g., RecoMind, Rethinking Personalized Ranking) appear only in related work and are not load-bearing for the central result. Thus no circular step is exhibited.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several hand-tuned parameters (reward coefficients, action discretization, ranges, batch, learning rate) and on the assumptions that the linear utility form is correct, the reward proxy is aligned with business goals, and the off-policy training signal is valid. No new physical or formal entities are introduced.

free parameters (7)
  • reward coefficients (alpha, beta, gamma) = click-through: (1.0, 0.5, 0.0); conversion: (0.1, 0.4, 0.5); impression: (0.0, 0.0, 0.0)
    Hand-set and adjusted through online ablations R0-R4 in Section 6.2.3; these coefficients define the reward and therefore directly shape the learned policy and the reported metric changes.
  • discretization granularity m = 10
    Tuned hyperparameter in Section 3.3; controls the action space size and was chosen after experiments showed non-convergence with continuous actions.
  • number of weight groups g = 3
    Semantic grouping of engagement weights in Section 3.3; chosen by the authors to reduce the action space from m^n to g^n.
  • weight and reserve price ranges = not specified
    Predefined ranges [w_i,min, w_i,max] and reserve price bounds are based on 'prior information' (Section 3.3); these bound the action space and are not derived.
  • batch size B = not specified
    Mini-batch averaging in Eq (4.1) affects gradient variance and convergence; value not reported.
  • step size eta = not specified
    Learning rate in Eq (4.1); value not reported.
  • logging traffic fraction x% = no more than 0.5%
    Chosen to minimize production risk (Section 5); directly affects the amount of training data and exploration.
assumptions (5)
  • domain assumption The ranking utility is a linear weighted sum of estimated probabilities (Eq 3.1).
    The entire action space and policy are defined by this functional form; the RL agent only tunes weights, so the approach inherits the linearity assumption.
  • ad hoc to paper The reward, defined as estimated revenue plus a weighted sum of the same engagement probabilities (Eqs 3.2-3.4), is an adequate proxy for business success.
    The reward is a hand-constructed linear combination of model predictions with manually set coefficients; external validity depends on the alignment between this proxy and actual business goals.
  • domain assumption Increasing a weight w_i in the utility increases the overall frequency of the corresponding engagement_action_i.
    Stated in Section 3.2 as 'intuitively and empirically'; this monotonicity is assumed but not proven, and could fail under auction dynamics or user interference.
  • domain assumption A one-step myopic MDP with gamma = 0 is sufficient for utility tuning.
    The paper sets the discount factor to 0 in Section 4.3 and acknowledges in Section 7 that long-term rewards are a limitation.
  • ad hoc to paper The off-policy gradient update in Eq (4.1) using uniform-logged data without importance weighting learns the optimal policy.
    This is an unstated assumption; the update is reward-weighted log-likelihood on off-policy data, not the true policy gradient for the deployed policy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest." pith.science (2026). https://pith.science/paper/H6X5JD7H

@misc{pith2026250905292,
  author       = {Pith},
  title        = {Pith review of: Deep Reinforcement Learning for Ranking Utility Tuning in the Ad Recommender System at Pinterest},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H6X5JD7H}},
  note         = {Machine review of arXiv:2509.05292}
}
read the original abstract

The ranking utility function in an ad recommender system, which linearly combines predictions of various business goals, plays a central role in balancing values across the platform, advertisers, and users. Traditional manual tuning, while offering simplicity and interpretability, often yields suboptimal results due to its unprincipled tuning objectives, the vast amount of parameter combinations, and its lack of personalization and adaptability to seasonality. In this work, we propose a general Deep Reinforcement Learning framework for Personalized Utility Tuning (DRL-PUT) to address the challenges of multi-objective optimization within ad recommender systems. Our key contributions include: 1) Formulating the problem as a reinforcement learning task: given the state of an ad request, we predict the optimal hyperparameters to maximize a pre-defined reward. 2) Developing an approach to directly learn an optimal policy model using online serving logs, avoiding the need to estimate a value function, which is inherently challenging due to the high variance and unbalanced distribution of immediate rewards. We evaluated DRL-PUT through an online A/B experiment in Pinterest's ad recommender system. Compared to the baseline manual utility tuning approach, DRL-PUT improved the click-through rate by 9.7% and the long click-through rate by 7.7% on the treated segment. We conducted a detailed ablation study on the impact of different reward definitions and analyzed the personalization aspect of the learned policy model.

Figures

Figures reproduced from arXiv: 2509.05292 by the authors.

Figure 2
Figure 2. Workflow of data collection and model serving. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average of the predicted hyper parameters within each bucket. Top: learned weights by CTR bucket. Middle: learned [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 23 canonical work pages

  1. [1]

    Mehdi Ben Ayed, Fei Feng, Jay Adams, Vishwakarma Singh, Kritarth Anand, and Jiajing Xu. 2025. RecoMind: A Reinforcement Learning Framework for Optimizing In-Session User Satisfaction in Recommendation Systems. arXiv:2508.00201 [cs.LG] https://arxiv.org/abs/2508.00201

    arXiv 2025

  2. [2]

    Andrew G Barto, Richard S Sutton, and Charles W Anderson. 1983. Neuronlike adaptive elements that can solve difficult learning control problems.IEEE transactions on systems, man, and cybernetics5 (1983), 834–846

    work page 1983

  3. [3]

    Charles, D

    Léon Bottou, Jonas Peters, Joaquin Quiñonero-Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. 2013. Counterfactual Reasoning and Learning Systems: The Example of Computational Advertising.Journal of Machine Learning Research14, 101, 3207–3260

    2013

  4. [4]

    Han Cai, Kan Ren, Weinan Zhang, Kleanthis Malialis, Jun Wang, Yong Yu, and Defeng Guo. 2017. Real-time bidding by reinforcement learning in display advertising. InProceedings of the tenth ACM international conference on web search and data mining. 661–670

    2017

  5. [5]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al

  6. [6]

    Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep Neural Networks for YouTube Recommendations(RecSys ’16). Association for Computing Machinery, New York, NY, USA, 191–198

    work page 2016

  7. [7]

    Thomas Degris, Martha White, and Richard S. Sutton. 2012. Off-policy actor-critic. InProceedings of the 29th International Coference on International Conference on Machine Learning(Edinburgh, Scotland)(ICML’12). Madison, WI, USA, 179–186

    2012

  8. [8]

    Jun Feng, Heng Li, Minlie Huang, Shichen Liu, Wenwu Ou, Zhirong Wang, and Xiaoyan Zhu. 2018. Learning to collaborate: Multi-scenario ranking via multi-agent reinforcement learning. InProceedings of the 2018 World Wide Web Conference. 1939–1948

    work page 2018

Show all 43 references
  1. [9]

    Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (Melbourne, Australia)(IJCAI’17). AAAI Press,...

    2017

  2. [10]

    Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, and Joaquin Quiñonero Candela. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. InProceedings of the Eighth International Workshop on Data ...

    2014

  3. [11]

    Yujing Hu, Qing Da, Anxiang Zeng, Yang Yu, and Yinghui Xu. 2018. Reinforcement learning to rank in e-commerce search engine: Formalization, analysis, and application. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 368–377

    2018

  4. [12]

    Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. InProceedings of the 32nd International Conference on Machine Learning (ICML). PMLR, 448–456

    2015

  5. [13]

    Junqi Jin, Chengru Song, Han Li, Kun Gai, Jun Wang, and Weinan Zhang. 2018. Real-time bidding with multi-agent reinforcement learning in display advertising. InProceedings of the 27th ACM international conference on information and knowledge management. 2193–2201

    2018

  6. [14]

    Yushi Jing, David Liu, Dmitry Kislyuk, Andrew Zhai, Jiajing Xu, Jeff Donahue, and Sarah Tavel. 2015. Visual Search at Pinterest. InProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Sydney, NSW, Australia)(KDD ’15). Association ...

    2015

  7. [15]

    Guogang Liao, Xiaowen Shi, Ze Wang, Xiaoxu Wu, Chuheng Zhang, Yongkang Wang, Xingxing Wang, and Dong Wang. 2022. Deep page-level interest network in reinforcement learning for ads allocation. InProceedings of the 45th International ACM SIGIR Conference on Research and Developm...

    2022

  8. [16]

    Zhuoran Liu, Leqi Zou, Xuan Zou, Caihua Wang, Biao Zhang, Da Tang, Bolin Zhu, Yijie Zhu, Peng Wu, Ke Wang, and Youlong Cheng. 2022. Monolith: Real Time Recommendation System With Collisionless Embedding Table. arXiv:2209.07663 [cs.IR] https://arxiv.org/abs/2209.07663

    2022 arXiv

  9. [17]

    Quan Lu, Shengjun Pan, Liang Wang, Junwei Pan, Fengdan Wan, and Hongxia Yang. 2017. A Practical Framework of Conversion Rate Prediction for Online Display Advertising. InProceedings of the ADKDD’17(Halifax, NS, Canada) (ADKDD’17). Association for Computing Machinery, New York,...

    2017

  10. [18]

    Vinod Nair and Geoffrey E Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. InProceedings of the 27th International Conference on Machine Learning (ICML). Omnipress, 807–814

    2010

  11. [19]

    David Rohde, Stephen Bonner, Travis Dunlop, Flavian Vasile, and Alexandros Karatzoglou. 2018. Recogym: A reinforcement learning environment for the problem of product recommendation in online advertising.arXiv preprint arXiv:1808.00720(2018)

    2018 arXiv

  12. [20]

    Rosenblatt

    F. Rosenblatt. 1958. The perceptron: A probabilistic model for information storage and organization in the brain.Psychological Review65, 6 (1958), 386–408. https://doi.org/10.1037/h0042519

    1958 doi

  13. [21]

    Lili Shan, Lei Lin, and Chengjie Sun. 2018. Combined Regression and Tripletwise Learning for Conversion Rate Prediction in Real-Time Bidding Advertising. InThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval(Ann Arbor, MI, USA)(SIGIR ...

    2018

  14. [22]

    1984.Temporal credit assignment in reinforcement learning

    Richard Stuart Sutton. 1984.Temporal credit assignment in reinforcement learning. University of Massachusetts Amherst

    1984

  15. [23]

    Richard S Sutton. 2018. Reinforcement learning: An introduction.A Bradford Book(2018)

    2018

  16. [24]

    Aditya Srinivas Timmaraju, Mehdi Mashayekhi, Mingliang Chen, Qi Zeng, Quintin Fettes, Wesley Cheung, Yihan Xiao, Manojkumar Rangasamy Kannadasan, Pushkar Tripathi, Sean Gahagan, et al . 2023. Towards fairness in personalized ads using impression variance aware reinforcement le...

    2023

  17. [25]

    Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. InProceedings of the ADKDD’17. 1–7

    2017

  18. [26]

    Weixun Wang, Junqi Jin, Jianye Hao, Chunjie Chen, Chuan Yu, Weinan Zhang, Jun Wang, Xiaotian Hao, Yixi Wang, Han Li, Jian Xu, and Kun Gai. 2019. Learning Adaptive Display Exposure for Real-Time Advertising. InProceedings of the 28th ACM International Conference on Information ...

    2019

  19. [27]

    Penghui Wei, Yongqiang Chen, ShaoGuo Liu, Liang Wang, and Bo Zheng. 2023. RLTP: Reinforcement Learning to Pace for Delayed Impression Modeling in Preloaded Ads. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining(Long Beach, CA, USA). Associa...

    2023

  20. [28]

    1987.Reinforcement-learning connectionist systems

    Ronald J Williams. 1987.Reinforcement-learning connectionist systems. College of Computer Science, Northeastern University

    1987

  21. [29]

    Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Machine learning8 (1992), 229–256

    1992

  22. [30]

    Ian H Witten. 1977. An adaptive optimal controller for discrete-time Markov environments.Information and control34, 4 (1977), 286–295

    1977

  23. [31]

    Di Wu, Xiujun Chen, Xun Yang, Hao Wang, Qing Tan, Xiaoxun Zhang, Jian Xu, and Kun Gai. 2018. Budget constrained bidding by model-free reinforcement learning in display advertising. InProceedings of the 27th ACM International Conference on Information and Knowledge Management. ...

    2018

  24. [32]

    Ruobing Xie, Shaoliang Zhang, Rui Wang, Feng Xia, and Leyu Lin. 2021. Hierarchical reinforcement learning for integrated recommendation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 4521–4528

    2021

  25. [33]

    Jiajing Xu, Andrew Zhai, and Charles Rosenberg. 2022. Rethinking Personalized Ranking at Pinterest: An End-to-End Approach. InProceedings of the 16th ACM Conference on Recommender Systems(Seattle, WA, USA)(RecSys ’22). Association for Computing Machinery, New York, NY, USA, 50...

    2022

  26. [34]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, et al . 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations.arXiv preprint arXiv:2402.17152(2024)

    2024 arXiv

  27. [35]

    Jun Zhao, Guang Qiu, Ziyu Guan, Wei Zhao, and Xiaofei He. 2018. Deep reinforcement learning for sponsored search real-time bidding. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1021–1030

    2018

  28. [36]

    Xiangyu Zhao, Changsheng Gu, Haoshenglun Zhang, Xiwang Yang, Xiaobing Liu, Jiliang Tang, and Hui Liu. 2021. Dear: Deep reinforcement learning for online advertising impression in recommender systems. InProceedings of the AAAI conference on artificial intelligence, Vol. 35. 750–758

    2021

  29. [37]

    Deep reinforcement learning for search, recommendation, and online advertising: a survey

    Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. 2019. " Deep reinforcement learning for search, recommendation, and online advertising: a survey" by Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin with Martin Vesely as coordinator.ACM sigweb newsletter2019, Spring (20...

    2019

  30. [38]

    Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018. Deep reinforcement learning for page-wise recommendations. In Proceedings of the 12th ACM conference on recommender systems. 95–103

    2018

  31. [39]

    Xiangyu Zhao, Long Xia, Lixin Zou, Hui Liu, Dawei Yin, and Jiliang Tang. 2020. Whole-Chain Recommendations. InProceedings of the 29th ACM International Conference on Information & Knowledge Management. 1883–1891

    2020

  32. [40]

    Xiangyu Zhao, Xudong Zheng, Xiwang Yang, Xiaobing Liu, and Jiliang Tang

  33. [41]

    Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. InProceedings of the 24th ACM SIGKDD international conference Yang et al. on knowledge discovery & d...

    2018

  34. [2016]

    InProceedings of the 1st workshop on deep learning for recommender systems

    Wide & deep learning for recommender systems. InProceedings of the 1st workshop on deep learning for recommender systems. 7–10

  35. [2020]

    InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

    Jointly learning to recommend and advertise. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3319–3327

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.