Pith. sign in

REVIEW 4 major objections 5 minor 36 references

Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LAAC claims an untuned LLM, used as a reference policy in an adversarial actor-critic loop, improves recommender accuracy, diversity, and novelty on MovieLens-1M.

desk verdict LAAC is a plausible incremental extension of ATAC, but the empirical case is underpowered; the entropy/accuracy tension flagged in review is not the fatal flaw it first looks like. read the letter →

arxiv 2507.21274 v1 pith:HOWEAL6G submitted 2025-07-28 cs.LG

Jiin Woo , Alireza Bagheri Garakani , Tianchen Zhou , Zhishen Huang , Yan Gao This is my paper
classification cs.LG
keywords reinforcementlearningrecommendationsystemsdiversitynoveltylargelanguagemodelsadversarialactor-criticofflineMovieLens
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LAAC is a reinforcement-learning recipe for recommenders that tries to combine the relevance learned from logged ratings with the exploratory reach of a large language model. Instead of fine-tuning the LLM or using it as the recommender, the paper treats the LLM as a reference policy that proposes candidate items, and trains a small actor and critic against each other so the critic becomes selectively optimistic about LLM suggestions while staying anchored to well-measured dataset actions. The central claim is that on MovieLens-1M this yields a policy that beats GRU4Rec and SMORL on accuracy, reward, diversity, and novelty at the same time, and that the gains survive when training on a skewed male-user subset. The payoff, if the claim holds, is a cheap route to diverse recommendations: no LLM retraining, just guided policy training on existing logs.

What carries the argument

The load-bearing object is the minimax objective of Eq. (1): $\max_\pi \mathbb{E}[f(s,\pi)-f(s,\pi_{\mathrm{LLM}})]$, with the critic trained to minimize the same difference plus $\alpha E_g$ and $\beta E_{td}$. The LLM reference $\pi_{\mathrm{LLM}}$ is approximated operationally by prompting an LLM with 100 randomly sampled candidate items and treating its 10 returned suggestions as a uniform distribution. The grounding loss $E_g=\mathbb{E}[(f(s,a)-f(s,\pi_{\mathrm{LLM}}))^2]$ anchors critic values of LLM-suggested items to the values of observed actions, while the TD loss $E_{td}=\mathbb{E}[(f(s,a)-r-\gamma f(s',\pi))^2]$ keeps in-sample values realistic; together they are designed to keep the critic optimistic for novel items without overestimating them. The actor is then updated to beat the LLM policy according to the critic, using two critic networks as a double-Q heuristic.

What would settle it

Compare the trained critic's values for LLM-suggested items with their actual held-out ratings on MovieLens-1M; if the rank correlation is near zero or negative, the critic is not optimistic-but-grounded, and LAAC's improvement over baselines must come from something other than the mechanism the paper describes.

Watch

Extended reading notes

Core claim

The paper's central discovery is that an adversarial actor-critic loop, seeded by an untuned LLM reference policy, can replace the random exploration used by prior RL recommenders with targeted exploration of items the LLM deems plausible. In the training objective, the critic minimizes $f(s,\pi)-f(s,\pi_{\mathrm{LLM}})$ plus a grounding loss that pulls the critic values of LLM suggestions toward values of observed actions and a temporal-difference loss that keeps dataset actions Bellman-consistent, while the actor maximizes the same difference. LAAC reports that the resulting policy improves hit rate, NDCG, cumulative reward, coverage, and novel-item coverage over GRU4Rec and SMORL on MovieLens-1M, and that the gains hold when training on a male-only skewed subset and evaluating on the original distribution. The experiments also show a tunable trade-off: raising the grounding weight $\alpha$ improves accuracy while lowering novelty, and setting the TD weight $\beta$ too low hurts reward when the dataset contains mostly poor ratings.

Load-bearing premise

The load-bearing premise is that the trained critic values LLM-suggested items optimistically but not overestimated, so that steering the policy toward whatever the critic favors surfaces good new items rather than bad ones.

Editorial extensions

If this is right

  • If LAAC is right, recommender systems can add diversity and novelty without training or fine-tuning an LLM, which removes the main compute barrier to LLM-guided exploration.
  • The adversarial reference-policy recipe should transfer to other offline RL settings where a frozen policy proposes actions beyond the logged data and a critic decides which proposals to trust.
  • The coefficients $\alpha$ and $\beta$ give operators a direct accuracy-versus-novelty dial instead of requiring a new reward function or model for each objective.
  • Training on skewed user data still produced competitive accuracy and diversity when evaluated on the original distribution, indicating the LLM reference can supply coverage that majority-only training data lacks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cheap test of the mechanism: replace the LLM reference with a uniform-random policy over the same 100-candidate samples. If most of the diversity gain survives, the contribution is the adversarial grounding machinery rather than the LLM's semantic knowledge.
  • The policy entropy reported for LAAC is much higher than for GRU4Rec, so a deployed system would need to check ranking calibration and business metrics beyond rating reward before relying on this diversity.
  • At the scale tested (160 users, about 26,511 transitions), the 100-candidate sampling approximation of $\pi_{\mathrm{LLM}}$ may be a bottleneck; larger catalogs would require re-estimating how much of the LLM's knowledge actually enters training.
  • The same scheme is a natural fit for exploratory recommendation in non-entertainment domains, such as jobs, papers, or products, whenever an LLM can propose candidates and logged reward data exists.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LAAC, an offline RL method for sequential recommendation that uses an LLM as a reference policy within an adversarial actor-critic framework. The actor is trained to outperform the LLM's suggestions while the critic is regularized by a grounding loss and a TD loss, aiming to keep critic values for LLM-suggested items optimistic yet anchored to well-estimated dataset actions. The method is evaluated on a filtered MovieLens-1M dataset, comparing accuracy (HR@k, NDCG@k), reward (R@k), diversity (CV@k, entropy), and novelty (NCV@k, NC@k) against GRU4Rec and SMORL, as well as against raw LLM policies. The authors report that LAAC achieves simultaneous improvements in relevance, diversity, and novelty, and shows robustness when training data are skewed by gender.

Significance. If the reported results are correct, LAAC would be a practical and computationally cheap way to inject LLM knowledge into sequential recommendation without fine-tuning the LLM, addressing a real deployment bottleneck. The paper combines a principled adversarial offline-RL formulation with LLM-generated candidate actions, and it includes ablations of the two regularization coefficients, which is a useful design analysis. However, the central empirical claim rests on a single small dataset and, as detailed below, the headline numbers in Table 1 are mutually inconsistent under the stated metric definitions. The evaluation also lacks confidence intervals, significance tests, and LLM-based baselines, so the current evidence does not yet support the abstract's broad claims.

major comments (4)
  1. The reported numbers in Table 1 are internally inconsistent under the stated metric definitions. The LLM policy π_Llama3 has entropy 2.2747, which matches ln(10) and confirms that entropy is measured in natural units over the 10-item recommendation support. LAAC(Llama3) is reported with entropy 8.0594, which corresponds to an effective support of e^8.0594 ≈ 3,161 items, i.e., essentially uniform over the 3,503-item catalog. A near-uniform policy over the full catalog would have expected HR@5 ≈ 5/3,503 ≈ 0.0014, but the table reports HR@5 = 0.0458, about 33 times larger. The same policy would place a novel (bottom-50% popularity) item at rank 1 in roughly half of the 5,090 test sequences, i.e., NC@1 ≈ 2,545, but the table reports NC@1 = 268. Its CV@10 would be near 1 because 5,090 sequences of 10 near-random recommendations would cover essentially all items, but the table reports CV@10 = 0.69. No evaluation protocol described in Section 3.1.4 reconciles these discrepancies. Since the central claim of "diversity with aligned relevance" (Section 3.2.2) depends on exactly these numbers, the empirical evidence for the paper's main contribution is not currently supported.
  2. The double-Q implementation described in Section 2.3.2 is not a double-Q method. In Algorithm 1, both critics f1 and f2 are updated with the identical loss L(f,π) + α E_g(f,π_LLM) + β E_td(f,π), and the actor is updated using only f1. There is no minimum-of-two-critics target, no alternating use of the critics, and no target network. Standard double-Q heuristics (e.g., TD3 or SAC) reduce overestimation by using the minimum of two independently maintained critics in the target computation. Training two copies with the same objective and no interaction does not provide that correction. The paper identifies overestimation of LLM-suggested items' values as a core risk, so this implementation choice should be either fixed to a genuine double-Q formulation or explicitly framed as a heuristic that relies on regularization alone.
  3. The evaluation is too limited to support the abstract's claim that LAAC "outperforms existing baselines in diversity, novelty, and accuracy." The experiments use a single dataset (MovieLens-1M) filtered to 160 users and 26,511 samples, and only two non-LLM baselines (GRU4Rec and SMORL) are compared. There are no comparisons to LLM-based recommenders, such as zero-shot LLM prompting with a proper ranker, a LoRA-finetuned LLM, or the LLM-as-environment RL method of [32]. The paper also states that all models were trained 50 times with different seeds, but no error bars, standard deviations, or significance tests are reported anywhere. The robustness claim in Section 3.2.3 is based on a single artificial skew (male-only training data). I would ask the authors to report confidence intervals and significance tests, add at least one LLM-based baseline, and ideally validate on a second dataset or domain.
  4. The adversarial objective in Eq. (1) is stated without any theoretical analysis. The paper invokes the guarantees of ATAC [7] and its model-based extension [4] to justify the framework, but the objective here is modified by the grounding loss E_g and the TD regularization E_td. These terms change the fixed point of the game, so the cited guarantee that the learned policy outperforms the behavior/reference policy does not automatically transfer. The claim in Section 2.1.3 that "this adversarial training setup guarantees that the resulting policy provably outperforms the behavior policy" is therefore misleading when applied to the LAAC objective. Either the authors should provide a formal statement (with assumptions) of what the modified game optimizes, or they should clearly describe LAAC as a heuristic and remove the implication of a proof.
minor comments (5)
  1. Typo: "minimiax" should be "minimax".
  2. Formatting issue: "1,1091,687" should be "1,109 1,687".
  3. The TD loss E_td is defined in Eq. (1) with f(s',π), the expected value under the policy, but Algorithm 1 line 7 accumulates the squared error using f(s',a') for a single sample a' ~ π(s'). These are different targets. The authors should clarify which objective is actually optimized in the experiments and align the text with the code.
  4. The definition of R@k as "the sum of user ratings for the top-k recommendations" is ambiguous: it is not clear whether this is a per-user sum, a per-sequence sum, or a total over all test sequences. The reported magnitudes in Tables 1 and 2 should be reconciled with the rating scale [1,5].
  5. The filtering step leaves only 160 users and 26,511 samples from MovieLens-1M, which is a very small subset. The paper should state this more prominently and justify the random subsampling of ratings, as it may affect the diversity and novelty metrics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LAAC is an empirical method evaluated on held-out data; its LLM-guided adversarial objective is a design choice, not a tautology.

full rationale

The paper's core claim is empirical: LAAC improves diversity, novelty, and accuracy on MovieLens. The training objective in Eq. (1) is a design choice derived from the adversarial offline RL framework of [4,7]; it does not presuppose the reported metric values. The LLM reference policy is explicitly constructed as uniform over 10 items sampled from 100 candidates (Sec. 2.3.1), and its reported entropy is simply log 10 by construction; this is a definitional property of the baseline, not a prediction of LAAC. The actor and critic are trained with the stated losses and evaluated on a held-out split (Secs. 3.1.1, 3.1.4), so the reported gains are not fitted to the test set. The regularization terms E_g and E_td are hyperparameters (alpha, beta) that are varied in Sec. 3.2 and shown to trade off reward and novelty; this is an empirical analysis, not a tautology. There is no load-bearing self-citation: refs [4,7] are external and provide the adversarial formulation; the authors do not import a uniqueness theorem from their own prior work. The skeptic's observation that LAAC's reported entropy (~8.06 nats) is inconsistent with the reported HR@5 and NC@1 is a serious internal-consistency/correctness concern about the empirical tables, but it is not a case of a derivation reducing to its inputs; it does not make the method circular. Overall, no circular step is identifiable from the paper's equations or citations.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two tuned coefficients (alpha, beta), several chosen hyperparameters (gamma, n_c, n_r, learning rates), and the unproven premise that the regularized adversarial critic provides reliable values for LLM-suggested items.

free parameters (6)
  • alpha (grounding regularizer weight) = 1.0
    Tuned in Section 3.2.4; increasing alpha improves reward but reduces novelty.
  • beta (TD regularizer weight) = 1.0
    Tuned in Section 3.2.5; too low beta hurts reward on poor-quality datasets.
  • gamma (discount factor) = 0.99
    Set in Section 3.1.3 for LAAC, versus 0.5 for SMORL.
  • n_c (candidate set size) = 100
    Randomly sampled candidates provided to LLM prompt (Section 2.3.1).
  • n_r (response size) = 10
    Number of LLM recommendations used to form uniform pi_LLM (Section 2.3.1).
  • learning rates = eta_critic=0.01, eta_actor=0.001
    Chosen in Section 3.1.3; actor learns slower than critic.
assumptions (4)
  • domain assumption Sequential recommendation can be modeled as an MDP with state as the last five watched items and reward as the user rating.
    Section 2.1.1 and Section 3.1.1; this is the standard but strong assumption that ratings reflect immediate reward.
  • domain assumption The offline dataset D consists of valid (s,a,r,s') transitions and is representative enough to train a policy, despite coming from only 160 users.
    Section 3.1.1; the small user count makes representativeness questionable.
  • ad hoc to paper The adversarial game-theoretic framework of ATAC [7,4] with a reference policy remains valid when the reference is an LLM policy and the critic loss includes grounding and TD regularizers.
    Section 2.2; no proof is given that LAAC inherits ATAC's improvement guarantees.
  • domain assumption The LLM, when prompted with random candidate sets, provides a useful distribution over novel items that a small policy can profitably refine.
    Section 2.3.1; the experiments show the raw LLM policy has low accuracy, so this assumption is nontrivial.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations." pith.science (2026). https://pith.science/paper/HOWEAL6G

@misc{pith2026250721274,
  author       = {Pith},
  title        = {Pith review of: Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HOWEAL6G}},
  note         = {Machine review of arXiv:2507.21274}
}
read the original abstract

In recommendation systems, diversity and novelty are essential for capturing varied user preferences and encouraging exploration, yet many systems prioritize click relevance. While reinforcement learning (RL) has been explored to improve diversity, it often depends on random exploration that may not align with user interests. We propose LAAC (LLM-guided Adversarial Actor Critic), a novel method that leverages large language models (LLMs) as reference policies to suggest novel items, while training a lightweight policy to refine these suggestions using system-specific data. The method formulates training as a bilevel optimization between actor and critic networks, enabling the critic to selectively favor promising novel actions and the actor to improve its policy beyond LLM recommendations. To mitigate overestimation of unreliable LLM suggestions, we apply regularization that anchors critic values for unexplored items close to well-estimated dataset actions. Experiments on real-world datasets show that LAAC outperforms existing baselines in diversity, novelty, and accuracy, while remaining robust on imbalanced data, effectively integrating LLM knowledge without expensive fine-tuning.

Figures

Figures reproduced from arXiv: 2507.21274 by the authors.

Figure 1
Figure 1. Illustration of the critic function 𝑓 in movie recom￾mendation scenarios. The critic estimates realistic values for movies observed in dataset D based on actual rewards, while maintaining optimistic estimates for novel movies recom￾mended by 𝜋LLM but absent from the dataset. choices, though it restricts the policy to only recommend actions that were observed in the training dataset. Recently, [4] extended this adver… view at source ↗
Figure 2
Figure 2. Performance analysis of LAAC (Llama3) on MovieLens dataset. Left: Evaluated rewards (R@10) and novelty (NCV@10) for varying 𝛼 = 0, 1, 3, 5, 10. Higher 𝛼 improves reward but reduces novelty. Right: Evaluated rewards (R@10) for varying 𝛽 = 0, 1, 3, 5, 10 on the full dataset (𝑟 ∈ [1, 5]) and the filtered dataset only consisting of samples with poor ratings (𝑟 ∈ [1, 3)). Too low 𝛽 dcreases cumulative reward (R) when dat… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 23 canonical work pages

  1. [32]

    Jie Wang, Alexandros Karatzoglou, Ioannis Arapakis, and Joemon M. Jose. Rein- forcement learning-based recommender systems with large language models for state reward and action modeling. SIGIR ’24, page 375–385, 2024

    work page 2024

  2. [7]

    Adversarially trained actor critic for offline reinforcement learning

    Ching-An Cheng, Tengyang Xie, Nan Jiang, and Alekh Agarwal. Adversarially trained actor critic for offline reinforcement learning. In Proceedings of the 39th International Conference on Machine Learning , 2022

    work page 2022

  3. [4]

    Adversarial model for offline reinforcement learning

    Mohak Bhardwaj, Tengyang Xie, Byron Boots, Nan Jiang, and Ching-An Cheng. Adversarial model for offline reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, pages 1245–1269, 2023

    work page 2023

  4. [1]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024

    2024

  5. [2]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. 2024

    2024

  6. [3]

    Tallrec: An effective and efficient tuning framework to align large language model with recommendation

    Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. RecSys ’23, page 1007–1014, 2023

    work page 2023

  7. [5]

    Stochastic approximation with two time scales

    Vivek S Borkar. Stochastic approximation with two time scales. Systems & Control Letters, 29(5):291–294, 1997

    work page 1997

  8. [6]

    When large language models meet personalization: Perspectives of challenges and opportunities.arXiv preprint arXiv:2307.16376, 2023

    Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, et al. When large language models meet personalization: Perspectives of challenges and opportunities.arXiv preprint arXiv:2307.16376, 2023

    arXiv 2023

Show all 36 references
  1. [8]

    On the properties of neural machine translation: Encoder-decoder approaches

    Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259, 2014

    2014 arXiv

  2. [9]

    A review of modern recommender systems using generative models (gen-recsys)

    Yashar Deldjoo, Zhankui He, Julian McAuley, Anton Korikov, Scott Sanner, Arnau Ramisa, René Vidal, Maheswaran Sathiamoorthy, Atoosa Kasirzadeh, and Silvia LLM-Enhanced RL for Diverse and Novel Recommendations Milano. A review of modern recommender systems using generative mode...

    2024

  3. [10]

    Bert: Pre- training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  4. [11]

    Recommender systems in the era of large language models (llms)

    Wenqi Fan, Zihuai Zhao, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Jiliang Tang, and Qing Li. Recommender systems in the era of large language models (llms). arXiv preprint arXiv:2307.02046, 2023

    2023 arXiv

  5. [12]

    Addressing function approxi- mation error in actor-critic methods

    Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approxi- mation error in actor-critic methods. In International Conference on Machine Learning, pages 1587–1596. PMLR, 2018

    2018

  6. [13]

    Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochas- tic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochas- tic actor. In International conference on machine learning, pages 1861–1870. PMLR, 2018

    2018

  7. [14]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. , 5(4), 2015

    2015

  8. [15]

    Large lan- guage models as zero-shot conversational recommenders

    Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian Mcauley. Large lan- guage models as zero-shot conversational recommenders. CIKM ’23, 2023

    2023

  9. [16]

    Session-based recommendations with recurrent neural networks

    Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939, 2015

    2015 arXiv

  10. [17]

    Towards universal sequence representation learning for recommender systems

    Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 585–593, 2022

    2022

  11. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  12. [19]

    Human-centric dialog training via offline reinforcement learning

    Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, and Rosalind Picard. Human-centric dialog training via offline reinforcement learning. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of...

    2020

  13. [20]

    Cumulated gain-based evaluation of ir techniques

    Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS) , 2002

    2002

  14. [21]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations , 2015

    2015

  15. [22]

    How can recommender systems benefit from large language models: A survey

    Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Xiangyang Li, Chenxu Zhu, Huifeng Guo, Yong Yu, Ruiming Tang, et al. How can recommender systems benefit from large language models: A survey. arXiv preprint arXiv:2306.05817, 2023

    2023 arXiv

  16. [23]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Gra- ham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35, 2023

    2023

  17. [24]

    Diversity-promoting deep reinforcement learning for interactive recommendation

    Yong Liu, Zhiqi Shen, Yinan Zhang, and Lizhen Cui. Diversity-promoting deep reinforcement learning for interactive recommendation. ICCSE ’21, page 132–139, 2022

    2022

  18. [25]

    Convergent temporal-difference learning with arbitrary smooth function approximation

    Hamid Reza Maei, Csaba Szepesvari, Shalabh Bhatnagar, Doina Precup, David Silver, and Richard S Sutton. Convergent temporal-difference learning with arbitrary smooth function approximation. In NIPS, pages 1204–1212, 2009

    2009

  19. [26]

    Recent advances in natural language processing via large pre-trained language models: A survey

    Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2):1–40, 2023

    2023

  20. [27]

    Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems , 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems , 35:2773...

    2022

  21. [28]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint ArXiv:1707.06347, 2017

    2017 arXiv

  22. [29]

    Choosing the best of both worlds: Diverse and novel recom- mendations through multi-objective reinforcement learning

    Dusan Stamenkovic, Alexandros Karatzoglou, Ioannis Arapakis, Xin Xin, and Kleomenis Katevas. Choosing the best of both worlds: Diverse and novel recom- mendations through multi-objective reinforcement learning. WSDM, 2022

    2022

  23. [30]

    Learning to summarize with human feedback

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Inform...

    2020

  24. [31]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  25. [33]

    Transrec: Learning transferable recommendation from mixture-of-modality feedback

    Jie Wang, Fajie Yuan, Mingyue Cheng, Joemon M Jose, Chenyun Yu, Beibei Kong, Xiangnan He, Zhijin Wang, Bo Hu, and Zang Li. Transrec: Learning transferable recommendation from mixture-of-modality feedback. arXiv preprint arXiv:2206.06190, 2022

    2022

  26. [34]

    Behavior regularized offline rein- forcement learning

    Yifan Wu, George Tucker, and Ofir Nachum. Behavior regularized offline rein- forcement learning. arXiv preprint ArXiv:1911.11361, 2019

    1911 arXiv

  27. [35]

    A survey of large language models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023

    2023 arXiv

  28. [36]

    Drn: A deep reinforcement learning framework for news recommendation

    Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. Drn: A deep reinforcement learning framework for news recommendation. In Proceedings of the World Wide Web Conference , 2018

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.