REVIEW 7 cited by
On Softmax Direct Preference Optimization for Recommendation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recommender systems aim to predict personalized rankings based on user preference data. With the rise of Language Models (LMs), LM-based recommenders have been widely explored due to their extensive world knowledge and powerful reasoning abilities. Most of the LM-based recommenders convert historical interactions into language prompts, pairing with a positive item as the target response and fine-tuning LM with a language modeling loss. However, the current objective fails to fully leverage preference data and is not optimized for personalized ranking tasks, which hinders the performance of LM-based recommenders. Inspired by the current advancement of Direct Preference Optimization (DPO) in human preference alignment and the success of softmax loss in recommendations, we propose Softmax-DPO (S-DPO) to instill ranking information into the LM to help LM-based recommenders distinguish preferred items from negatives, rather than solely focusing on positives. Specifically, we incorporate multiple negatives in user preference data and devise an alternative version of DPO loss tailored for LM-based recommenders, which is extended from the traditional full-ranking Plackett-Luce (PL) model to partial rankings and connected to softmax sampling strategies. Theoretically, we bridge S-DPO with the softmax loss over negative sampling and find that it has an inherent benefit of mining hard negatives, which assures its exceptional capabilities in recommendation tasks. Empirically, extensive experiments conducted on three real-world datasets demonstrate the superiority of S-DPO to effectively model user preference and further boost recommendation performance while providing better rewards for preferred items. Our codes are available at https://github.com/chenyuxin1999/S-DPO.
Forward citations
Cited by 7 Pith papers
-
LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation
LBR removes length bias in LLM recommenders via length-aware attention offsets and Trie-branching information-length normalization, improving accuracy and fairness with negligible cost.
-
Hierarchical Residual Policy Optimization for Generative Recommendations
HRPO decomposes item-level rewards into token-level 'residual credits' along semantic identifier hierarchies and optimizes the generator with a PPO-style objective, improving session utility in KuaiSim and production ...
-
Architecture is All You Need: Improving LLM Recommenders by Dropping the Text
A 2.2M-parameter Llama-style model using only item IDs beats a 7B-parameter text-based LLM recommender and SASRec on MovieLens-10M.
-
CORONA: A Coarse-to-Fine Framework for Graph-based Recommendation with Large Language Models
CORONA uses LLM-generated preference and intent queries to prune the interaction graph in two stages, then applies a GNN to the remaining subgraph, achieving state-of-the-art recommendation accuracy.
-
OneSug: The Unified End-to-End Generative Framework for E-commerce Query Suggestion
A unified encoder-decoder model with prefix representation enhancement and reward-weighted DPO outperforms Kuaishou's online multi-stage query suggestion system in offline and live A/B evaluations.
-
Listwise Preference Alignment Optimization for Tail Item Recommendation
The paper applies a listwise softmax preference loss with head-item negative sampling and tail reweighting to improve long-tail item recommendation.
-
GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models
A survey of LLM-based generative recommendation systems, covering application settings, training pipelines, industrial deployment challenges, and future directions.
Discussion (0). Sign in to comment.