Don't Let Bandit Feedback Pull Continual LLM-Recommender Updates Off Target

Chung Park; Hyeongjun Yun; Jaegul Choo; Taesan Kim

arxiv: 2605.18899 · v1 · pith:DX3BNS5Inew · submitted 2026-05-17 · 💻 cs.LG · cs.AI

Don't Let Bandit Feedback Pull Continual LLM-Recommender Updates Off Target

Taesan Kim , Hyeongjun Yun , Jaegul Choo , Chung Park This is my paper

Pith reviewed 2026-05-20 14:09 UTC · model grok-4.3

classification 💻 cs.LG cs.AI

keywords LLM recommenderscontinual learningbandit feedbackexposure biaspolicy optimizationGRPOinverse propensity scoringself-certainty

0 comments

The pith

Inserting prior policy exposures as anchors into group-relative optimization corrects exposure bias during continual LLM recommender updates.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LLM-based recommenders require continual updates from deployment logs, but these logs supply only partial bandit feedback shaped by the prior policy, which induces exposure bias and leaves no-responses ambiguous. The paper proposes Anchored Bandit Policy Optimization that inserts the actually exposed recommendation as a fixed anchor inside each GRPO rollout group so normalization stays calibrated to what the old policy did. Self-normalized inverse propensity scoring is applied to the anchor for both feedback types, while no-response penalties are softened by the model's own token-level confidence to avoid overreacting to uncertain signals. Experiments across five Amazon Reviews and MovieLens domains show consistent accuracy gains and better bias mitigation than earlier approaches.

Core claim

The central claim is that ABPO, by inserting the exposed recommendation as a logged anchor into each GRPO rollout group to calibrate group-relative normalization against the prior policy action, applying self-normalized inverse propensity scoring to the fixed anchor for both feedback types, and tempering no-response penalties with self-certainty from output-token confidence, produces consistent post-update gains in recommendation accuracy while mitigating prior-policy-induced exposure bias more effectively than prior baselines across five domains from Amazon Reviews and MovieLens.

What carries the argument

Insertion of the exposed recommendation as a logged anchor into each GRPO rollout group, which calibrates group-relative normalization against the prior policy's actual action rather than newly sampled rollouts alone.

If this is right

Recommendation accuracy improves consistently after updates across multiple domains.
Exposure bias induced by the prior policy is reduced more effectively than with previous baselines.
Asymmetric handling of positive and no-response signals prevents unstable updates from ambiguous feedback.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The anchoring technique may generalize to other continual learning systems that receive only policy-shaped partial feedback.
Token confidence could act as a lightweight, verifier-free reliability signal in additional online policy update settings.
Similar calibration steps might stabilize continual adaptation in sequential decision tasks beyond recommendation.

Load-bearing premise

Treating no-responses asymmetrically with self-certainty from token confidence avoids overly aggressive updates without introducing new biases, and anchor insertion correctly calibrates normalization for policy mismatch.

What would settle it

An ablation on the same Amazon and MovieLens datasets that removes either the anchor insertion or the self-certainty tempering and checks whether accuracy gains vanish or exposure bias increases.

Figures

Figures reproduced from arXiv: 2605.18899 by Chung Park, Hyeongjun Yun, Jaegul Choo, Taesan Kim.

**Figure 2.** Figure 2: Item-matching reward values for exposed items during GRPO update training [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Figure 3: Overview of Anchored Contextual Bandit Reinforcement Learning [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Comparison of in-domain recommendation per [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

read the original abstract

Generative LLM-based recommenders (LLM-Rec) require continual post-deployment updates, yet deployment logs provide only policy-shaped contextual bandit feedback: outcomes are observed solely for items exposed by a prior serving policy, inducing exposure bias and yielding partial, asymmetric signals consisting of relatively reliable positive responses and ambiguous no-responses. We propose an Anchored Bandit Policy Optimization (ABPO) framework for continual LLM-Rec updates that combines group-relative policy optimization (GRPO) with explicit treatment of exposure bias and feedback ambiguity. Specifically, we insert the exposed recommendation as a logged anchor into each GRPO rollout group, so that group-relative normalization is calibrated against the action actually exposed by the prior policy rather than against newly sampled rollouts alone. Because both positive- and no-responses are observed only through prior-policy exposure, we apply self-normalized inverse propensity scoring to the fixed anchor for both feedback types to correct for policy mismatch. At the same time, we treat the two feedback types asymmetrically in reliability: positive responses provide relatively direct endorsement signals, whereas no-responses remain ambiguous because they may reflect either true disinterest or unobserved external factors. To avoid overly aggressive updates from ambiguous no-responses, we temper their penalties with self-certainty, using the model's output-token confidence as a verifier-free reliability signal. Across five domains from Amazon Reviews and MovieLens, our method yields consistent post-update gains in recommendation accuracy while mitigating prior-policy-induced exposure bias more effectively than prior baselines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

ABPO anchors GRPO updates and tempers no-responses with token confidence to reduce exposure bias in continual LLM recommender training, but the confidence proxy is the part that needs checking.

read the letter

The paper's main move is to adapt group-relative policy optimization for the bandit logs that come from deployed LLM recommenders. They insert the actually exposed item as a fixed anchor in every rollout group so the normalization stays tied to what the prior policy served, then apply self-normalized IPS to that anchor for both positive and no-response cases. At the same time they scale down the penalty on no-responses using the model's own output-token confidence as a quick reliability weight. This produces measurable accuracy lifts on five Amazon and MovieLens domains while cutting the exposure bias that standard updates pick up. The anchoring step is a straightforward fix for policy mismatch and the asymmetric handling matches the real asymmetry in the data, where clicks are clearer signals than misses. The experiments report consistent post-update gains over baselines, which is the kind of result that matters for people running these systems in production. The soft spot is the self-certainty piece. Token confidence is easy to compute, but it can reflect popularity, prior exposure, or model overconfidence rather than actual user disinterest. If that correlation is weak, the tempering just trades one source of bias for another. The abstract and method do not show a direct check against ground-truth interest signals or an ablation that turns the tempering off, so it is hard to tell how much of the reported gain comes from the anchor versus the confidence scaling. This is aimed at groups that already run generative recommenders and need to keep updating them from live logs without full counterfactual data. The approach is incremental rather than revolutionary, but it is scoped tightly to the bandit setting and uses reproducible techniques. The math is clear and the datasets are standard, so the work is solid enough to go to referees who can ask for the missing validation on the confidence signal and any extra controls.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes the Anchored Bandit Policy Optimization (ABPO) framework for continual post-deployment updates of generative LLM-based recommenders. It augments group-relative policy optimization (GRPO) by inserting the prior-policy exposed item as a fixed anchor in each rollout group, applies self-normalized inverse propensity scoring (IPS) to the anchor to correct exposure bias, and treats feedback asymmetrically: positive responses are used as direct signals while no-responses are tempered by self-certainty computed from the model's output-token confidence to avoid overly aggressive updates from ambiguous signals. Experiments across five domains from Amazon Reviews and MovieLens report consistent post-update gains in recommendation accuracy and improved mitigation of prior-policy-induced exposure bias relative to baselines.

Significance. If the central empirical claims hold, the work addresses a practically important problem in maintaining LLM recommenders under realistic deployment constraints where only policy-shaped bandit feedback is available. The combination of anchor-based normalization and self-certainty tempering offers a concrete mechanism for handling exposure bias and feedback asymmetry without requiring additional supervision or verifiers. The multi-domain evaluation and explicit treatment of policy mismatch are strengths that could influence future continual-learning pipelines for generative recommenders.

major comments (2)

[§3.2] §3.2 (Self-certainty tempering): The central claim that output-token confidence serves as a reliable, verifier-free proxy for tempering no-response penalties rests on the untested assumption that confidence tracks actual user interest rather than training artifacts, item popularity, or prior-policy exposure. The manuscript provides no correlation analysis or ablation against ground-truth interest signals (e.g., explicit ratings or post-update click-through rates under policy shift), which is load-bearing for the asymmetric-handling argument.
[§4.2] §4.2 and Table 3 (Anchor + self-normalized IPS): The claim that inserting the logged anchor and restricting self-normalized IPS to it fully calibrates group-relative normalization for policy mismatch is not isolated in the ablations. The reported accuracy gains could be driven by other factors (e.g., GRPO hyperparameters or data filtering); a controlled removal of the anchor while keeping self-certainty would be required to substantiate the bias-mitigation contribution.

minor comments (2)

[§3.1] The notation for self-normalized IPS applied only to the anchor should be written explicitly as an equation rather than described in prose to avoid ambiguity when readers re-implement the method.
[Figure 2] Figure 2 (policy-shift curves) would benefit from error bars or shaded confidence intervals to make the consistency of gains across domains visually clearer.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The comments highlight important aspects of empirical validation for our proposed components. We address each major comment below and outline the revisions we will make to strengthen the manuscript.

read point-by-point responses

Referee: [§3.2] §3.2 (Self-certainty tempering): The central claim that output-token confidence serves as a reliable, verifier-free proxy for tempering no-response penalties rests on the untested assumption that confidence tracks actual user interest rather than training artifacts, item popularity, or prior-policy exposure. The manuscript provides no correlation analysis or ablation against ground-truth interest signals (e.g., explicit ratings or post-update click-through rates under policy shift), which is load-bearing for the asymmetric-handling argument.

Authors: We agree that an explicit correlation analysis would provide stronger support for using output-token confidence as a proxy. In the pure bandit-feedback setting targeted by the work, direct ground-truth interest labels are unavailable by design. Our existing ablations already compare performance with and without self-certainty tempering and show consistent gains in accuracy and bias mitigation. To address the referee's concern, we will add a new analysis in the revised §4 that correlates self-certainty scores with post-update metrics on the MovieLens domain (which contains explicit ratings usable as a proxy for interest) and report the resulting Spearman correlations. revision: yes
Referee: [§4.2] §4.2 and Table 3 (Anchor + self-normalized IPS): The claim that inserting the logged anchor and restricting self-normalized IPS to it fully calibrates group-relative normalization for policy mismatch is not isolated in the ablations. The reported accuracy gains could be driven by other factors (e.g., GRPO hyperparameters or data filtering); a controlled removal of the anchor while keeping self-certainty would be required to substantiate the bias-mitigation contribution.

Authors: The referee is correct that the current ablation suite does not fully isolate the anchored IPS component while holding self-certainty fixed. We will revise the experimental section to include a controlled ablation that removes the anchor (reverting group normalization to standard GRPO) while retaining self-certainty tempering and self-normalized IPS on the remaining rollouts. The updated Table 3 and accompanying text will report the incremental contribution of the anchor to bias mitigation and accuracy. revision: yes

Circularity Check

0 steps flagged

ABPO framework derivation remains self-contained without reduction to inputs

full rationale

The paper introduces ABPO by combining GRPO with explicit anchor insertion for calibration against prior-policy exposure and asymmetric feedback handling via self-certainty from token confidence. These components are presented as novel mechanisms to address exposure bias and ambiguity in bandit feedback. No equations or steps in the abstract reduce a claimed prediction or result to a fitted parameter or prior self-citation by construction. The central claims of post-update accuracy gains and bias mitigation rest on the proposed asymmetric treatment and normalization rather than re-deriving inputs. This qualifies as an honest non-finding per guidelines, as the derivation does not exhibit self-definitional, fitted-input, or load-bearing self-citation patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based on abstract description; full paper may detail additional parameters or assumptions.

axioms (1)

domain assumption Group-relative policy optimization provides effective normalization for policy updates in bandit settings.
The framework builds directly on GRPO.

pith-pipeline@v0.9.0 · 5805 in / 1094 out tokens · 69621 ms · 2026-05-20T14:09:54.723170+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages · 5 internal anchors

[1]

On softmax direct preference optimization for recommendation.Advances in Neural Information Processing Systems, 37:27463–27489, 2024

Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On softmax direct preference optimization for recommendation.Advances in Neural Information Processing Systems, 37:27463–27489, 2024

work page 2024
[2]

More than what was chosen: Llm-based explainable recommendation beyond noisy user preferences.arXiv preprint arXiv, 2026

Chung Park, Hyeongjun Yun, Taesan Kim, Junui Hong, Dongjoon Hong, Mira Myong, Jihoon Oh, MinCheol Cho, Kijung Park, Min sung Choi, Jihwan Seok, and Jaegul Choo. More than what was chosen: Llm-based explainable recommendation beyond noisy user preferences.arXiv preprint arXiv, 2026

work page 2026
[3]

Towards trustworthy llm-based recommendation via rationale integration.arXiv preprint arXiv:2601.02364, 2025

Chung Park, Taesan Kim, Hyeongjun Yun, Dongjoon Hong, Junui Hong, Kijung Park, MinCheol Cho, Mira Myong, Jihoon Oh, et al. Towards trustworthy llm-based recommendation via rationale integration.arXiv preprint arXiv:2601.02364, 2025

work page arXiv 2025
[4]

From clicks to preference: A multi-stage alignment framework for generative query suggestion in conversational system

Junhao Yin, Haolin Wang, Peng Bao, Ju Xu, and Yongliang Wang. From clicks to preference: A multi-stage alignment framework for generative query suggestion in conversational system. arXiv preprint arXiv:2508.15811, 2025

work page arXiv 2025
[5]

Ctr-guided generative query suggestion in conversational search

Erxue Min, Hsiu-Yuan Huang, Xihong Yang, Min Yang, Xin Jia, Yunfang Wu, Hengyi Cai, Junfeng Wang, Shuaiqiang Wang, and Dawei Yin. Ctr-guided generative query suggestion in conversational search. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 2624–2634, 2025

work page 2025
[6]

Reinforcement learning from user feedback.arXiv preprint arXiv:2505.14946, 2025

Eric Han, Jun Chen, Karthik Abinav Sankararaman, Xiaoliang Peng, Tengyu Xu, Eryk He- lenowski, Kaiyan Peng, Mrinal Kumar, Sinong Wang, Han Fang, et al. Reinforcement learning from user feedback.arXiv preprint arXiv:2505.14946, 2025

work page arXiv 2025
[7]

Leveraging unpaired feedback for long-term llm-based recommendation tuning

Jizhi Zhang, Chongming Gao, Wentao Shi, Xin Chen, Jingang Wang, Xunliang Cai, and Fuli Feng. Leveraging unpaired feedback for long-term llm-based recommendation tuning. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 24507–24521, 2025

work page 2025
[8]

Rec-r1: Bridging generative large language mod- els and user-centric recommendation systems via reinforcement learning.arXiv preprint arXiv:2503.24289, 2025

Jiacheng Lin, Tian Wang, and Kun Qian. Rec-r1: Bridging generative large language mod- els and user-centric recommendation systems via reinforcement learning.arXiv preprint arXiv:2503.24289, 2025

work page arXiv 2025
[9]

Think wise, collaborate effectively: A rationale-aware llm-based recommender with reinforcement learning from collaborative signals

Chung Park, Taesan Kim, Hyeongjun Yun, Dongjoon Hong, Junui Hong, Kijung Park, MinCheol Cho, Min Sung Choi, Jihwan Seok, and Jaegul Choo. Think wise, collaborate effectively: A rationale-aware llm-based recommender with reinforcement learning from collaborative signals. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 1560...

work page 2026
[10]

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[11]

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[12]

Learning to Reason without External Rewards

Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards.arXiv preprint arXiv:2505.19590, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025
[13]

Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders

Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation.arXiv preprint arXiv:2403.03952, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024
[14]

The movielens datasets: History and context.Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015

F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context.Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015

work page 2015
[15]

Lora: Low-rank adaptation of large language models.ICLR, 1 (2):3, 2022

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1 (2):3, 2022. 10

work page 2022
[16]

Data-efficient fine-tuning for llm-based recommendation

Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat-Seng Chua. Data-efficient fine-tuning for llm-based recommendation. InProceedings of the 47th interna- tional ACM SIGIR conference on research and development in information retrieval, pages 365–374, 2024

work page 2024
[17]

arXiv preprint arXiv:2408.00802

Alicia Y Tsai, Adam Kraft, Long Jin, Chenwei Cai, Anahita Hosseini, Taibai Xu, Zemin Zhang, Lichan Hong, Ed H Chi, and Xinyang Yi. Leveraging llm reasoning enhances personalized recommender systems.arXiv preprint arXiv:2408.00802, 2024

work page arXiv 2024
[18]

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization

Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Peter Belcak, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, et al. Gdpo: Group reward- decoupled normalization policy optimization for multi-reward rl optimization.arXiv preprint arXiv:2601.05242, 2026. 11 A Dataset We conduct experiments on fiveAmazon Review 2023 2 [13] do...

work page internal anchor Pith review Pith/arXiv arXiv 2026

[1] [1]

On softmax direct preference optimization for recommendation.Advances in Neural Information Processing Systems, 37:27463–27489, 2024

Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On softmax direct preference optimization for recommendation.Advances in Neural Information Processing Systems, 37:27463–27489, 2024

work page 2024

[2] [2]

More than what was chosen: Llm-based explainable recommendation beyond noisy user preferences.arXiv preprint arXiv, 2026

Chung Park, Hyeongjun Yun, Taesan Kim, Junui Hong, Dongjoon Hong, Mira Myong, Jihoon Oh, MinCheol Cho, Kijung Park, Min sung Choi, Jihwan Seok, and Jaegul Choo. More than what was chosen: Llm-based explainable recommendation beyond noisy user preferences.arXiv preprint arXiv, 2026

work page 2026

[3] [3]

Towards trustworthy llm-based recommendation via rationale integration.arXiv preprint arXiv:2601.02364, 2025

Chung Park, Taesan Kim, Hyeongjun Yun, Dongjoon Hong, Junui Hong, Kijung Park, MinCheol Cho, Mira Myong, Jihoon Oh, et al. Towards trustworthy llm-based recommendation via rationale integration.arXiv preprint arXiv:2601.02364, 2025

work page arXiv 2025

[4] [4]

From clicks to preference: A multi-stage alignment framework for generative query suggestion in conversational system

Junhao Yin, Haolin Wang, Peng Bao, Ju Xu, and Yongliang Wang. From clicks to preference: A multi-stage alignment framework for generative query suggestion in conversational system. arXiv preprint arXiv:2508.15811, 2025

work page arXiv 2025

[5] [5]

Ctr-guided generative query suggestion in conversational search

Erxue Min, Hsiu-Yuan Huang, Xihong Yang, Min Yang, Xin Jia, Yunfang Wu, Hengyi Cai, Junfeng Wang, Shuaiqiang Wang, and Dawei Yin. Ctr-guided generative query suggestion in conversational search. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 2624–2634, 2025

work page 2025

[6] [6]

Reinforcement learning from user feedback.arXiv preprint arXiv:2505.14946, 2025

Eric Han, Jun Chen, Karthik Abinav Sankararaman, Xiaoliang Peng, Tengyu Xu, Eryk He- lenowski, Kaiyan Peng, Mrinal Kumar, Sinong Wang, Han Fang, et al. Reinforcement learning from user feedback.arXiv preprint arXiv:2505.14946, 2025

work page arXiv 2025

[7] [7]

Leveraging unpaired feedback for long-term llm-based recommendation tuning

Jizhi Zhang, Chongming Gao, Wentao Shi, Xin Chen, Jingang Wang, Xunliang Cai, and Fuli Feng. Leveraging unpaired feedback for long-term llm-based recommendation tuning. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 24507–24521, 2025

work page 2025

[8] [8]

Rec-r1: Bridging generative large language mod- els and user-centric recommendation systems via reinforcement learning.arXiv preprint arXiv:2503.24289, 2025

Jiacheng Lin, Tian Wang, and Kun Qian. Rec-r1: Bridging generative large language mod- els and user-centric recommendation systems via reinforcement learning.arXiv preprint arXiv:2503.24289, 2025

work page arXiv 2025

[9] [9]

Think wise, collaborate effectively: A rationale-aware llm-based recommender with reinforcement learning from collaborative signals

Chung Park, Taesan Kim, Hyeongjun Yun, Dongjoon Hong, Junui Hong, Kijung Park, MinCheol Cho, Min Sung Choi, Jihwan Seok, and Jaegul Choo. Think wise, collaborate effectively: A rationale-aware llm-based recommender with reinforcement learning from collaborative signals. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 1560...

work page 2026

[10] [10]

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[11] [11]

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[12] [12]

Learning to Reason without External Rewards

Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards.arXiv preprint arXiv:2505.19590, 2025

work page internal anchor Pith review Pith/arXiv arXiv 2025

[13] [13]

Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders

Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation.arXiv preprint arXiv:2403.03952, 2024

work page internal anchor Pith review Pith/arXiv arXiv 2024

[14] [14]

The movielens datasets: History and context.Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015

F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context.Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015

work page 2015

[15] [15]

Lora: Low-rank adaptation of large language models.ICLR, 1 (2):3, 2022

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1 (2):3, 2022. 10

work page 2022

[16] [16]

Data-efficient fine-tuning for llm-based recommendation

Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat-Seng Chua. Data-efficient fine-tuning for llm-based recommendation. InProceedings of the 47th interna- tional ACM SIGIR conference on research and development in information retrieval, pages 365–374, 2024

work page 2024

[17] [17]

arXiv preprint arXiv:2408.00802

Alicia Y Tsai, Adam Kraft, Long Jin, Chenwei Cai, Anahita Hosseini, Taibai Xu, Zemin Zhang, Lichan Hong, Ed H Chi, and Xinyang Yi. Leveraging llm reasoning enhances personalized recommender systems.arXiv preprint arXiv:2408.00802, 2024

work page arXiv 2024

[18] [18]

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization

Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Peter Belcak, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, et al. Gdpo: Group reward- decoupled normalization policy optimization for multi-reward rl optimization.arXiv preprint arXiv:2601.05242, 2026. 11 A Dataset We conduct experiments on fiveAmazon Review 2023 2 [13] do...

work page internal anchor Pith review Pith/arXiv arXiv 2026