REVIEW 21 cited by
Is ChatGPT a Good Recommender? A Preliminary Study
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Is ChatGPT a Good Recommender? A Preliminary Study
read the original abstract
Recommendation systems have witnessed significant advancements and have been widely used over the past decades. However, most traditional recommendation methods are task-specific and therefore lack efficient generalization ability. Recently, the emergence of ChatGPT has significantly advanced NLP tasks by enhancing the capabilities of conversational models. Nonetheless, the application of ChatGPT in the recommendation domain has not been thoroughly investigated. In this paper, we employ ChatGPT as a general-purpose recommendation model to explore its potential for transferring extensive linguistic and world knowledge acquired from large-scale corpora to recommendation scenarios. Specifically, we design a set of prompts and evaluate ChatGPT's performance on five recommendation scenarios. Unlike traditional recommendation methods, we do not fine-tune ChatGPT during the entire evaluation process, relying only on the prompts themselves to convert recommendation tasks into natural language tasks. Further, we explore the use of few-shot prompting to inject interaction information that contains user potential interest to help ChatGPT better understand user needs and interests. Comprehensive experimental results on Amazon Beauty dataset show that ChatGPT has achieved promising results in certain tasks and is capable of reaching the baseline level in others. We conduct human evaluations on two explainability-oriented tasks to more accurately evaluate the quality of contents generated by different models. And the human evaluations show ChatGPT can truly understand the provided information and generate clearer and more reasonable results. We hope that our study can inspire researchers to further explore the potential of language models like ChatGPT to improve recommendation performance and contribute to the advancement of the recommendation systems field.
Forward citations
Cited by 21 Pith papers
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Recommender systems are moving from raw IDs to semantic IDs, and the authors argue the next stage is 'semantic planning'—predicting an exposure's goal before choosing the item.
-
Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation
Retrieval coverage limits LLM rerankers in cold-start recommendation; a learned hybrid fusion improves pool quality but LLM reranking often degrades end-to-end performance while simpler rankers exploit the pool.
-
One Pass, Any Order: Position-Invariant Listwise Reranking for LLM-Based Recommendation
InvariRank achieves permutation-invariant listwise reranking for LLM-based recommendations via a structured attention mask that blocks cross-candidate interactions and shared positional framing under RoPE, enabling st...
-
Objective Shaping with Hard Negatives: Windowed Partial AUC Optimization for RL-based LLM Recommenders
Beam-search negatives induce partial AUC optimization in GRPO for LLM recommenders; Windowed Partial AUC and TAWin improve Top-K alignment on four datasets.
-
SRBench: A Comprehensive Benchmark for Sequential Recommendation with Large Language Models
SRBench is a multi-dimensional benchmark for sequential recommendation that uses prompt engineering and a coupled extraction mechanism to support fair evaluation of both neural-network and LLM-based models across accu...
-
BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models
BEAR adds a beam-search-aware regularization to LLM fine-tuning for recommendations that forces positive-item tokens to rank in the top-B candidates at each decoding step to avoid premature pruning.
-
GARDRec: Decision-Level Graph Grounding for Large Language Model Recommendation
GARDRec improves LLM-based next-item ranking by grounding decisions in knowledge-graph embeddings, personalized graph contexts, and late-stage scoring rather than prompt text.
-
GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models
A LoRA-tuned LLM with trie-constrained decoding improves grocery category recommendation and yields a 7.5% cart-add lift in production.
-
EvoRAG: Making Knowledge Graph-based RAG Automatically Evolve through Feedback-driven Backpropagation
EvoRAG adds a feedback-driven backpropagation step that attributes response quality to individual knowledge-graph triplets and updates the graph to raise reasoning accuracy by 7.34 percent over prior KG-RAG methods.
-
LWGR: Lagrangian-Constrained Personalized World Knowledge for Generative Recommendation
LWGR applies personalized soft instructions for LLM knowledge extraction and Lagrangian primal-dual optimization to selectively fuse beneficial world knowledge into generative recommendation while bounding degradation.
-
TSUBASA: Improving Long-Horizon Personalization via Evolving Memory and Self-Learning with Context Distillation
TSUBASA improves long-horizon personalization in LLMs via dynamic memory evolution for writing and context-distillation self-learning for reading, outperforming Mem0 and Memory-R1 on Qwen-3 benchmarks while reducing t...
-
BEAR: Towards Beam-Search-Aware Optimization for Recommendation with Large Language Models
BEAR is a cheap token-level top-B regularizer for LLM-based recommendation, but its central claim that this condition is necessary for beam-search survival is incorrect.
-
A Survey on Generative Recommendation: Data, Model, and Tasks
This survey organizes generative recommendation into data, model, and task dimensions, identifying five advantages including world knowledge integration and creative generation while noting challenges in benchmarks an...
-
RouteRec: Strict Evaluation of Recommender-Agent Selection and Aggregation
On MovieLens-1M, request-level hard selection of recommender agents underperforms BM25, while item-level learned shortlist aggregation matches or beats it and improves further with gated LLM use.
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Recommender systems are moving from raw IDs to semantic IDs, and the next step should be semantic planning that first predicts an exposure's purpose before choosing or generating content.
-
LERA: LLM-Enhanced RAG for Ad Auction in Generative Chatbots
LERA is a retrieve-then-generate auction system that refines ad candidate ranking with LLM logits and applies a threshold-aware critical-value payment rule to maintain truthfulness in chatbot ad insertion.
-
Why Thinking Hurts: Diagnosing and Rectifying Linguistic Inertia in Large Language Models for Recommendation
Chain-of-thought reasoning degrades semantic-ID recommendation accuracy through 'linguistic inertia,' and a training-free compression-plus-contrastive decoding fix restores and often improves accuracy.
-
TRWH: A Text-Driven Random Walk Heterogeneous GNN for Semantic-Aware Sparse Recommendation
TRWH combines LLM profiles, a heterogeneous GNN, and one-hop random walk edges, but its headline performance claims are contradicted by its own results table.
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Industrial recommenders are evolving from raw IDs through semantic IDs toward semantic planning, where the system predicts a semantic next-exposure target before choosing or generating a concrete item.
-
A Reproducibility Analysis of PO4ISR: Diagnosing and Mitigating Semantic Drift in LLM-Based Session Recommendation
Reproducibility study diagnoses semantic drift in PO4ISR and introduces PO4ISR++ with reflexive prompting that restores performance with gains up to 54% on Games and 96% on Bundle.
-
T-POP: Test-Time Personalization with Online Preference Feedback
T-POP uses dueling-bandit token selection to learn a reward function online from pairwise user feedback, enabling test-time personalization of a frozen LLM without fine-tuning.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.