Pith. sign in

REVIEW 10 cited by

Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.06474 v1 pith:D3GMC3OV submitted 2023-05-10 cs.IR cs.LG

Wang-Cheng Kang , Jianmo Ni , Nikhil Mehta , Maheswaran Sathiamoorthy , Lichan Hong , Ed Chi , Derek Zhiyuan Cheng This is my paper
classification cs.IRcs.LG
keywords llmsuserdataratingzero-shotconductfew-shotfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) have demonstrated exceptional capabilities in generalizing to new tasks in a zero-shot or few-shot manner. However, the extent to which LLMs can comprehend user preferences based on their previous behavior remains an emerging and still unclear research question. Traditionally, Collaborative Filtering (CF) has been the most effective method for these tasks, predominantly relying on the extensive volume of rating data. In contrast, LLMs typically demand considerably less data while maintaining an exhaustive world knowledge about each item, such as movies or products. In this paper, we conduct a thorough examination of both CF and LLMs within the classic task of user rating prediction, which involves predicting a user's rating for a candidate item based on their past ratings. We investigate various LLMs in different sizes, ranging from 250M to 540B parameters and evaluate their performance in zero-shot, few-shot, and fine-tuning scenarios. We conduct comprehensive analysis to compare between LLMs and strong CF methods, and find that zero-shot LLMs lag behind traditional recommender models that have the access to user interaction data, indicating the importance of user interaction data. However, through fine-tuning, LLMs achieve comparable or even better performance with only a small fraction of the training data, demonstrating their potential through data efficiency.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 24 citations worldwide. Full citation record

  1. GrocLM: Grocery Category Recommendation in E-Commerce with Large Language Models

    cs.AI 2026-06 conditional novelty 6.0 of 10

    A LoRA-tuned LLM with trie-constrained decoding improves grocery category recommendation and yields a 7.5% cart-add lift in production.

  2. VENOMREC: Cross-Modal Interactive Poisoning for Targeted Promotion in Multimodal LLM Recommender Systems

    cs.CR 2026-02 conditional novelty 6.0 of 10

    Synchronized text+image poisoning steers multimodal LLM recommenders to promote target items, reaching 0.73 mean exposure@20.

  3. Beyond Offline A/B Testing: Context-Aware Agent Simulation for Recommender System Evaluation

    cs.IR 2026-01 unverdicted novelty 6.0 of 10

    ContextSim uses life-simulation and explicit thought-training to make LLM user proxies more realistic, and claims that recommender systems tuned with these proxies improve real-world engagement.

  4. KERAG_R: Knowledge-Enhanced Retrieval-Augmented Generation for Recommendation

    cs.IR 2025-07 conditional novelty 6.0 of 10

    KERAG_R improves LLM-based top-k recommendation by using a GAT to select relevant KG triples and incorporating them into instruction-tuned prompts, reporting gains over ten baselines on three datasets.

  5. PageLLM: A Multi-Grained Reward Framework for Whole-Page Optimization with Large Language Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A multi-grained reward framework fine-tunes an LLM with PPO to generate whole-page recommendations, showing that page-level and item-level reward heads are complementary.

  6. GORACS: Group-level Optimal Transport-guided Coreset Selection for LLM-based Recommender Systems

    cs.IR 2025-06 conditional novelty 6.0 of 10

    GORACS selects small groups of fine-tuning examples via an optimal-transport and gradient-norm proxy objective, outperforming prior coreset methods for LLM-based recommendation.

  7. Personas within Parameters: Fine-Tuning Small Language Models with Low-Rank Adapters to Mimic User Behaviors

    cs.IR 2025-08 conditional novelty 5.0 of 10

    Persona-level LoRA fine-tuning lets a 3.8B small language model simulate MovieLens users about as accurately as a much larger frozen LLM, at lower cost.

  8. CLAImate: AI-Enabled Climate Change Communication through Personalized and Localized Narrative Visualizations

    cs.HC 2025-07 conditional novelty 5.0 of 10

    A personalized, localized AI conversation system for climate communication shows modest factual accuracy and positive early feedback from 10 UK users.

  9. Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing

    cs.LG 2025-09 conditional novelty 4.0 of 10

    LLM tutoring modestly accelerates RL convergence on average, with advice reuse saving wall-clock time but reducing stability.

  10. Improving the Performance of Sequential Recommendation Systems with an Extended Large Language Model

    cs.IR 2025-07 conditional novelty 2.0 of 10

    Replacing Llama2 with Llama3.1 in the LlamaRec framework improved recommendation metrics by 8 to 39 percent on three public datasets.

Pith tools