REVIEW 18 cited by
M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Industrial recommender systems have been growing increasingly complex, may involve \emph{diverse domains} such as e-commerce products and user-generated contents, and can comprise \emph{a myriad of tasks} such as retrieval, ranking, explanation generation, and even AI-assisted content production. The mainstream approach so far is to develop individual algorithms for each domain and each task. In this paper, we explore the possibility of developing a unified foundation model to support \emph{open-ended domains and tasks} in an industrial recommender system, which may reduce the demand on downstream settings' data and can minimize the carbon footprint by avoiding training a separate model from scratch for every task. Deriving a unified foundation is challenging due to (i) the potentially unlimited set of downstream domains and tasks, and (ii) the real-world systems' emphasis on computational efficiency. We thus build our foundation upon M6, an existing large-scale industrial pretrained language model similar to GPT-3 and T5, and leverage M6's pretrained ability for sample-efficient downstream adaptation, by representing user behavior data as plain texts and converting the tasks to either language understanding or generation. To deal with a tight hardware budget, we propose an improved version of prompt tuning that outperforms fine-tuning with negligible 1\% task-specific parameters, and employ techniques such as late interaction, early exiting, parameter sharing, and pruning to further reduce the inference time and the model size. We demonstrate the foundation model's versatility on a wide range of tasks such as retrieval, ranking, zero-shot recommendation, explanation generation, personalized content creation, and conversational recommendation, and manage to deploy it on both cloud servers and mobile devices.
Forward citations
Cited by 18 Pith papers
-
BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models
BEAR is a cheap token-level top-B regularizer for LLM-based recommendation, but its central claim that this condition is necessary for beam-search survival is incorrect.
-
Hierarchical Latent Reasoning for LLM-based Recommendation
HiLaR aligns LLM latent reasoning states with temporally quantized user preference hierarchies and optimizes them with layer-aware process rewards, improving recommendation accuracy on four Amazon datasets.
-
Improving Rare Medication Recommendation with Counterfactual Data Augmentation and Large Language Models
GenRxR improves rare-medication recommendation using LLM-generated counterfactual data, instruction tuning, and sequential medication generation.
-
Probabilistic Residual Learning for Online Recommendations
PRL adds a cluster-aware, causality-adjusted residual correction layer to any base recommender, improving cold-start cross-domain recommendation accuracy in experiments.
-
GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models
A LoRA-tuned LLM with trie-constrained decoding improves grocery category recommendation and yields a 7.5% cart-add lift in production.
-
RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation
A from-scratch model that tokenizes items into hierarchical codes and predicts next-item codes reaches higher average zero-shot AUC on 8 datasets than LLM recommenders up to 7B parameters.
-
Generative Multi-Target Cross-Domain Recommendation
GMC uses shared discrete semantic item IDs and a unified generative recommender with domain-specific LoRA to improve multi-target cross-domain recommendation.
-
RecRankerEval: A Flexible and Extensible Framework for Top-k LLM-based Recommendation
A reimplementation of RecRanker shows its pointwise variant's high top-k scores come from ground-truth data leakage in the prompts, and the new RecRankerEval framework finds listwise tuning, DBSCAN sampling, XSimGCL, ...
-
KERAG_R: Knowledge-Enhanced Retrieval-Augmented Generation for Recommendation
KERAG_R improves LLM-based top-k recommendation by using a GAT to select relevant KG triples and incorporating them into instruction-tuned prompts, reporting gains over ten baselines on three datasets.
-
PageLLM: A Multi-Grained Reward Framework for Whole-Page Optimization with Large Language Models
A multi-grained reward framework fine-tunes an LLM with PPO to generate whole-page recommendations, showing that page-level and item-level reward heads are complementary.
-
Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendation
R2Rec combines graph-sampled interaction chains, masked stepwise reasoning prompts, and SFT plus GRPO training to improve LLM recommendation hit ratios.
-
Revisiting Self-attention for Cross-domain Sequential Recommendation
AutoCDSR improves cross-domain sequential recommendation by adding a Pareto-optimized penalty on cross-domain attention scores to the standard recommendation loss.
-
STARec: An Efficient Agent Framework for Recommender Systems via Autonomous Deliberate Reasoning
STARec trains LLM user agents to first rank fast, then reflect on mismatches and rewrite the user profile, using teacher distillation plus GRPO; on MovieLens-1M and Amazon CDs it reportedly beats full-data baselines w...
-
Membership Inference Attacks on In-Context Examples in LLM-based Recommender Systems
Simply asking a large language model 'have you seen this user?' or comparing its recommendations after prompt poisoning can reveal whether a user's interactions are in the hidden prompt of an ICL-based recommender.
-
TrackRec: Iterative Alternating Feedback with Chain-of-Thought via Preference Alignment for Recommendation
TrackRec trains a small LLM to generate user-preference summaries and a validator to score them, alternating the training so each improves the other, and reports gains on public and industrial recommendation benchmarks.
-
PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform
A single pretrained model over user activity sequences improves save rates in Pinterest's Home Feed and Related Items ranking when fine-tuned per application, while deduplication and quantization keep serving costs neutral.
-
RecCoT: Enhancing Recommendation via Chain-of-Thought
A slow-large, fast-small framework that generates chain-of-thought explanations from reviews and caches the resulting semantic embeddings improves Amazon rating prediction over several baselines.
-
Sharpness-aware Model Merging with Salience Recovery for LLM-based Cross-Domain Sequential Recommendation
SharpRec combines sharpness-aware fine-tuning with a nonlinear parameter reshape to merge LoRA adapters for cross-domain recommendation, but the reshape's claimed heavy-tail effect is mathematically backward.
Discussion (0). Sign in to comment.