RecRankerEval: A Flexible and Extensible Framework for Top-k LLM-based Recommendation

Iadh Ounis; Zeyuan Meng; Zixuan Yi

REVIEW 4 major objections 5 minor 51 references

RecRankerEval: A Flexible and Extensible Framework for Top-k LLM-based Recommendation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper reproduces RecRanker and finds that its pointwise variant's apparent superiority is an artifact of data leakage in prompts.

desk verdict A useful reproduction study that exposes a real leakage problem in RecRanker's pointwise prompts, but the central attribution is under-supported and the missing side-by-side numbers weaken the reproducibility claim. read the letter →

arxiv 2507.05880 v1 pith:S7C6DAYZ submitted 2025-07-08 cs.IR

Zeyuan Meng , Zixuan Yi , Iadh Ounis This is my paper

classification cs.IR

keywords RecRankerLLM-basedrecommendationtop-kdataleakageinstructiontuningreproducibilityusersamplingEval

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether RecRanker's reported top-k recommendation results survive reimplementation and which parts of the design actually matter. It finds that pairwise and listwise variants reproduce, but the pointwise variant's high scores come from data leakage: the prompts embed ground-truth ratings as hint scores. A corrected version with those scores removed drops from 0.0807 to 0.0488 in hit rate at rank 3 on ML-100K, aligning with the other variants. The paper contributes RecRankerEval, a modular evaluation framework spanning user sampling, initial recommender, LLM backbone, dataset, and tuning method, and uses it to show that RecRanker transfers to Amazon-Music but not BookCrossing, and that stronger initial recommenders, DBSCAN sampling, and Llama3 all improve results.

What carries the argument

The load-bearing mechanism is the pointwise prompt's hint field, which contains a score that is not a neutral recommendation signal but a value derived directly from the user's ground-truth rating. That turns what should be a learned ranking task into a lookup of the correct answer inside the input, and it is what inflates the pointwise numbers. The corrective machinery is RecRankerEval's modular substitution setup, which lets each component of user sampling, initial recommender, LLM backbone, dataset, and instruction tuning method be swapped independently so that the effect of removing the leaking hint, changing the sampler, or upgrading the base model can be measured in isolation.

What would settle it

Run RecRanker in the pointwise setting on ML-100K with the hint field preserved but its value changed from the true user rating to the initial recommender's predicted score; if hit rate at rank 3 stays near the leaked 0.0807 instead of falling to the corrected 0.0488, the leakage attribution is wrong, while a fall to about 0.0488 would confirm that the ground-truth content of the hint, not the hint structure, was carrying the performance.

Watch

Extended reading notes

Core claim

The paper's central claim is that RecRanker's pointwise instruction-tuning variant achieves its unusually high performance by data leakage: the pointwise prompts include explicit scores derived from user ratings, so interacted or ground-truth items receive high scores while non-interacted items receive low or zero scores, effectively handing the model the answers inside the prompt. After removing these hint scores, the corrected pointwise variant's hit rate at rank 3 on ML-100K falls from 0.0807 to 0.0488, and listwise and corrected hybrid variants become the strongest instruction-tuning strategies. The paper further proposes RecRankerEval, an extensible framework that modularizes five components of the RecRanker paradigm, and reports that RecRanker-style models generalize to a new domain (Amazon-Music) but fail on BookCrossing when timestamps must be simulated, while swapping in stronger initial recommenders and newer LLM backbones improves overall performance.

Load-bearing premise

The leakage conclusion assumes that removing the hint scores isolates the leaked ground-truth signal, but the paper has no control that keeps a hint whose content is not ground truth (such as the initial recommender's own predicted score), so part of the drop from 0.0807 to 0.0488 could come from losing any hint at all; a second load-bearing premise is that single-GPU LoRA fine-tuning faithfully represents RecRanker's original full fine-tuning on 16 A800 GPUs.

Editorial extensions

If this is right

RecRanker's originally reported pointwise results should not be treated as a fair baseline; the corrected pointwise run drops to 0.0488 hit rate at rank 3 on ML-100K.
After the correction, the listwise variant is the strongest instruction-tuning strategy in most configurations, suggesting that ranking-oriented prompts beat score-prediction prompts for LLM-based top-k recommendation.
RecRanker's design transfers to a new domain such as Amazon-Music with real timestamps, but fails on BookCrossing when timestamps must be fabricated, indicating that temporal data quality is essential for this pipeline.
RecRanker's performance improves when the initial recommender is upgraded from MF or LightGCN to XSimGCL, and when the backbone is upgraded from Llama2 to Llama3.
DBSCAN-based user sampling generally outperforms both K-means and random sampling across instruction-tuning variants, pointing to density-based denoised sampling as a better default.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct extension the paper does not run is to keep the pointwise hint structure but replace the ground-truth rating with the initial recommender's predicted score; this control would separate the leakage effect from the effect of losing any hint at all.
If the leakage finding generalizes, other LLM-based recommenders that encode ratings or predicted scores inside pointwise-style prompts may have similarly inflated results, and a simple prompt rewrite would serve as a cheap leakage audit.
The framework's modular swaps imply a testable recipe: compare two configurations that differ only in one component, and treat any large performance jump as a warning that the changed component is carrying information it should not.
Because the reproduction relies on single-GPU LoRA fine-tuning rather than the original full fine-tuning on 16 GPUs, a full-resource replication would be the natural next step to confirm that the corrected ranking of variants is not an artifact of the cheaper training regime.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Zeyuan, here's my read on 2507.05880.

The paper does two useful things. First, it identifies a likely data-leakage bug in RecRanker's pointwise prompt: the 'hint' score is literally the ground-truth rating for interacted items and zero for non-interacted ones. That is a real finding and, as far as I know, new. The corrected pointwise variant drops from 0.0807 to 0.0488 H@3 on ML-100K, and the corrected ranking puts listwise on top. Second, it ships RecRankerEval, a modular harness for five design dimensions, and contributes a clean Amazon-Music evaluation. The component ablations—DBSCAN over K-means, XSimGCL as initial recommender, Llama3 as backbone—are useful for anyone building on this paradigm.

The soft spots are real but not fatal. The leakage attribution is confounded: removing the hint removes both ground-truth content and any numeric hint. The paper doesn't run the control that would separate these, e.g., replacing the true rating with the initial recommender's predicted score. Given that the pointwise hint is literally the relevance label, I still think leakage is the most plausible explanation, but the causal claim as written is underdetermined. Also, for a reproducibility paper, Table 3 shows only the reimplementation—there's no side-by-side column of the original RecRanker numbers. The claim 'we reproduce the original results' is asserted, not demonstrated. The BookCrossing failure is blamed on simulated timestamps, but that's an assumption, not a verified cause. And the code isn't linked, which makes the framework hard to adopt right now.

On balance, I'd send this to peer review. The leakage finding is important enough to warrant a careful look even if the current evidence is short of conclusive. A good referee would ask for the predicted-score control, the side-by-side table, and the code release. The paper's own conclusions—especially that RecRanker's pointwise advantage was an artifact—are probably right, but they need to be argued more carefully.

I'd bring this to our reading group and likely cite the leakage result in my own work once it clears review.

Referee Report

4 major / 5 minor

Summary. The paper reimplements RecRanker, an LLM-based top-k recommender, and packages the reimplementation as RecRankerEval, a modular framework spanning user sampling, initial recommendation models, LLM backbones, datasets, and instruction tuning variants. The authors report that their reimplementation reproduces RecRanker's pairwise and listwise results, while the pointwise variant's high performance is due to data leakage from ground-truth scores in the prompts; removing those scores (pointwise_Fix) causes a sharp drop in H@3/N@3. They further compare alternative samplers, stronger initial recommenders, and different LLMs, concluding that DBSCAN sampling, listwise tuning, and stronger backbones generally improve performance, and that BookCrossing is not reproducible because the original paper lacks timestamp information.

Significance. The paper is a useful reproducibility and analysis contribution: it provides a modular framework with preprocessing and prompt-construction scripts, covers a broad configuration space (three samplers, three initial recommenders, three LLMs, four datasets), and identifies a plausible leakage risk in RecRanker's pointwise prompts that, if confirmed, would be an important caution for instruction-tuning evaluations in LLM-based recommendation. The comparative results across sampling strategies and initial recommenders are also a useful reference for practitioners. The main weakness is that the two central claims—the leakage explanation and the reproduction of the original numbers—are not yet backed by fully controlled comparisons: the leakage experiment removes the hint signal entirely, and no side-by-side table of reproduced versus original RecRanker numbers is provided.

major comments (4)

[Section 5.4, Table 4] The evidence for the leakage claim is the drop from RecRanker_pointwise (H@3 0.0807) to pointwise_Fix (H@3 0.0488) when hint scores are removed from the pointwise prompts. This manipulation removes both the ground-truth content and the numeric hint signal simultaneously. Because the listwise and pairwise prompts retain hints from the initial recommender and are not treated as leakage, the paper should include a control that replaces the ground-truth score with the initial recommender's predicted score, keeping a non-ground-truth hint in the prompt. Without that control, the performance drop cannot be attributed to ground-truth leakage rather than to the general usefulness of a numeric hint, so the central causal claim of Section 5.4 is underdetermined.
[Section 5.3, Table 3, RQ1] The reproduction claim is not backed by a direct numerical comparison: Table 3 reports only the authors' reimplemented numbers, and the original RecRanker values are never shown in a table or listed against the reproduced ones. The reimplementation also uses LoRA on a single NVIDIA RTX A6000, whereas the original RecRanker uses full fine-tuning on 16 A800 GPUs (Section 5.3). The paper should provide a side-by-side table of reproduced versus original H@k and N@k values and discuss how LoRA and the reduced GPU setup might affect the comparison; otherwise RQ1 cannot be assessed from the presented evidence.
[Section 5.1, Table 2] The BookCrossing preprocessing is claimed to follow the original paper, but the reproduced dataset differs materially in size: 1,777 vs 1,820 users, 22,288 vs 2,030 items, and 90,818 vs 41,456 interactions. This is not a small discrepancy, and it may explain why all RecRanker variants underperform the baseline on BookCrossing in Table 4. The paper should clarify whether these statistics result from a different filtering rule or whether Table 2 contains an error, and should not attribute the failure solely to timestamp simulation without addressing this inconsistency.
[Section 5.4, Tables 3 and 4] The paper reports that pointwise_Fix aligns with pairwise and listwise performance, but it does not provide a statistical test of the difference between RecRanker_pointwise and pointwise_Fix; the asterisks in Tables 3 and 4 are comparisons to the base model only. Since the leakage conclusion rests entirely on this drop, the authors should report a paired significance test (or confidence interval) for the pointwise versus pointwise_Fix comparison, and confirm that the drop is not driven by the changed prompt format alone.

minor comments (5)

[Section 5.6] There is a typo: 'best perfrmance' should be 'best performance'.
[Section 5.7] The sentence 'Furthermore, they the results show the importance...' is grammatically broken and should be rewritten.
[Table 4] The table is split into three subtables with repeated column headers, which makes cross-model comparison difficult; a unified table with the initial recommender as a row or column group would be clearer.
[Section 4.5] The text says ML-100K and ML-1M results are 'directly reproduced from the original RecRanker's supplied resources,' but Section 4.1 describes RecRankerEval as a reimplementation; the wording should clarify that these are reimplemented configurations, not copied outputs.
[Abstract and Section 5.1] The abstract says BookCrossing is not reproducible 'due to the lack of timestamp information,' while Section 5.1 says the authors simulate timestamps and Section 5.4 says the lack of real timestamps 'may introduce noise'; the phrasing should be consistent about whether the issue is absence of timestamps or noisy simulated timestamps.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical reproducibility study whose claims rest on measured comparisons, not on definitions or self-citations that reduce to inputs.

full rationale

The paper's conclusions are empirical: it reimplements RecRanker, measures H@k/N@k across prompt variants, and attributes the pointwise advantage to ground-truth leakage after removing hint scores. None of these steps defines a target quantity in terms of itself, fits a parameter and then calls a related quantity a prediction, or imports a load-bearing conclusion from the authors' prior work. The leakage attribution is underdetermined---removing the hint scores changes two variables at once (ground-truth content and the presence of any numeric hint), and a predicted-score control would be needed to isolate the leakage effect---but this is a threat to causal inference, not circularity. Self-citations in the related work are not load-bearing for any experimental claim. Since the reported results are measured against external baselines and datasets rather than derived from a fitted input, the circularity score is 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper's contributions are empirical; its central leakage finding rests on inspecting the original prompt design and on the pointwise_Fix comparison. Reproduction depends on many hyperparameters inherited from the original RecRanker code (utility constants C1-C3, hybrid weights alpha, K-means K, repetition penalty C, and the new framework's DBSCAN settings and LoRA configuration), none of which are reported. The evaluation assumes the clustering-to-prompt-to-LLM pipeline (user sampling, initial recommender scores, instruction-tuned Llama) is a valid vehicle for top-k ranking, and that simulated timestamps on BookCrossing permit a valid temporal split, an assumption the paper itself says harmed training. No new entities are introduced.

free parameters (6)

Utility constants C1, C2, C3 in hybrid scoring = Not reported (inherited from RecRanker)
Section 3.4 defines U_pointwise = P - m*C1, U_pairwise = C2, U_listwise = -m'*C3; all RecRanker variant scores depend on these, yet no values appear in the paper, so results cannot be regenerated independently.
Hybrid weights alpha_1, alpha_2, alpha_3 = Not reported (alpha_1+alpha_2+alpha_3=1)
Section 3.4: U_hybrid = alpha_1 U_pointwise + alpha_2 U_pairwise + alpha_3 U_listwise. The hybrid and hybrid_Fix results in Tables 3-4 depend on these weights.
Number of K-means clusters K (user sampling) = Not reported
Section 3.1 clustering-based sampling and Section 5.6 KMeans comparisons depend on K, which is never specified.
DBSCAN eps and min_samples = Not reported
Section 4.2 introduces DBSCAN sampling; the Section 5.6 conclusion that DBSCAN is most effective depends on hyperparameters that are not given.
Repetition penalty constant C in (0,1) = Not reported
Section 3.1 penalty weight psi_u = C^{M3(u)} governs the merged sampling multiset.
LoRA rank and alpha = Not reported
Section 5.3 applies LoRA for efficient fine-tuning; rank and alpha are not stated, and the reproduction relies on LoRA instead of the original full fine-tuning.

assumptions (5)

domain assumption Unobserved items are more likely to be preferred than items explicitly disliked, so negative samples should come from unobserved items.
Section 3.2 uses this to build training ranking lists R_train = I+ union I-.
domain assumption Clustering users by interaction embeddings produces intent groups that improve prompt construction.
Sections 3.1 and 4.2: K-means and DBSCAN sampling claims depend on embedding-space clusters being meaningful.
domain assumption Instruction-tuned LLMs with LoRA can improve top-k recommendation quality from prompt data.
The whole RecRankerEval evaluation (Sections 4.4, 5.3) presumes this; the reimplementation itself is the test subject, so the premise is not independently verified.
domain assumption Randomly simulated timestamps on BookCrossing support a valid temporal data split.
Sections 5.1 and 5.4: the paper acknowledges this 'may introduce noise and hinder training', and indeed BookCrossing results do not reproduce, which is consistent with this assumption being false or fragile.
standard math Paired t-test with Holm-Bonferroni correction is a valid significance test on the evaluation protocol.
Section 5.3 uses it for all asterisk annotations in Tables 3-5.

how reviews work

0 comments

Cite this review

Pith. "Pith review of RecRankerEval: A Flexible and Extensible Framework for Top-k LLM-based Recommendation." pith.science (2026). https://pith.science/paper/S7C6DAYZ

@misc{pith2026250705880,
  author       = {Pith},
  title        = {Pith review of: RecRankerEval: A Flexible and Extensible Framework for Top-k LLM-based Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S7C6DAYZ}},
  note         = {Machine review of arXiv:2507.05880}
}

read the original abstract

A recent Large language model (LLM)-based recommendation model, called RecRanker, has demonstrated a superior performance in the top-k recommendation task compared to other models. In particular, RecRanker samples users via clustering, generates an initial ranking list using an initial recommendation model, and fine-tunes an LLM through hybrid instruction tuning to infer user preferences. However, the contribution of each core component remains underexplored. In this work, we inspect the reproducibility of RecRanker, and study the impact and role of its various components. We begin by reproducing the RecRanker pipeline through the implementation of all its key components. Our reproduction shows that the pairwise and listwise methods achieve a performance comparable to that reported in the original paper. For the pointwise method, while we are also able to reproduce the original paper's results, further analysis shows that the performance is abnormally high due to data leakage from the inclusion of ground-truth information in the prompts. To enable a fair and comprehensive evaluation of LLM-based top-k recommendations, we propose RecRankerEval, an extensible framework that covers five key dimensions: user sampling strategy, initial recommendation model, LLM backbone, dataset selection, and instruction tuning method. Using the RecRankerEval framework, we show that the original results of RecRanker can be reproduced on the ML-100K and ML-1M datasets, as well as the additional Amazon-Music dataset, but not on BookCrossing due to the lack of timestamp information in the original RecRanker paper. Furthermore, we demonstrate that RecRanker's performance can be improved by employing alternative user sampling methods, stronger initial recommenders, and more capable LLMs.

Figures

Figures reproduced from arXiv: 2507.05880 by the authors.

**Figure 1.** Overview of the RecRankerEval Framework. top-𝑘 recommendation models. One key objective of RecRankerEval is to support modular experimentation and a comparative analysis of different alternative choices when deploying an LLM-based recommender such as RecRanker. Such choices pertain to five key components: user sampling strategy, initial recommendation model, LLM backbone, dataset selection, and instruction tuning m… view at source ↗

**Figure 2.** Zero-shot performance comparison of Llama2, GPT [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Instruction-tuned performance comparison of [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 35 canonical work pages

[1]

Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He
[2]

Zeyu Cui, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. M6-rec: Generative pretrained language models are open-ended recommender systems.arXiv preprint arXiv:2205.08084(2022)

arXiv 2022
[3]

Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongx- iang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. InProc. of RecSys

work page 2023
[4]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783(2024)

arXiv 2024
[5]

Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang
[6]

John A Hartigan and Manchek A Wong. 1979. Algorithm AS 136: A k-means clustering algorithm.Journal of the royal statistical society(1979)

work page 1979
[7]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. InProc. of SIGIR

work page 2020
[8]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. InProc. of ICLR

work page 2022

Show all 51 references

[9]

Jianchao Ji, Zelong Li, Shuyuan Xu, Wenyue Hua, Yingqiang Ge, Juntao Tan, and Yongfeng Zhang. 2024. Genrec: Large language model for generative recommen- dation. InProc. of ECIR

2024
[10]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization tech- niques for recommender systems.Computer(2009)

2009
[11]

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proc. of SOSP

2023
[12]

Ruyu Li, Wenhao Deng, Yu Cheng, Zheng Yuan, Jiaqi Zhang, and Fajie Yuan
[13]

Yaoyiran Li, Xiang Zhai, Moustafa Alzantot, Keyi Yu, Ivan Vulić, Anna Korhonen, and Mohamed Hammad. 2024. Calrec: Contrastive alignment of generative llms for sequential recommendation. InProc. of RecSys

2024
[14]

Xinyu Lin, Wenjie Wang, Yongqi Li, Shuo Yang, Fuli Feng, Yinwei Wei, and Tat- Seng Chua. 2024. Data-efficient Fine-tuning for LLM-based Recommendation. In Proc. of SIGIR

2024
[15]

Exploring the upper limits of text-based collaborative filtering using large language models: Discoveries and insights.arXiv preprint arXiv:2305.11700(2023)

2023
[16]

Zeyuan Meng, Iadh Ounis, Craig Macdonald, and Zixuan Yi. 2024. Knowledge Graph Cross-View Contrastive Learning for Recommendation. InEuropean Con- ference on Information Retrieval. Springer, 3–18

2024
[17]

Aleksandr V Petrov and Craig Macdonald. 2023. Generative sequential recom- mendation with gptrec.arXiv preprint arXiv:2306.11114(2023)

2023 arXiv
[18]

Sichun Luo, Bowei He, Haohan Zhao, Wei Shao, Yanlin Qi, Yinya Huang, Ao- jun Zhou, Yuxuan Yao, Zongpeng Li, Yuanzhang Xiao, et al. 2023. Recranker: Instruction tuning large language model as ranker for top-k recommendation. Transactions on Information Systems(2023)

2023
[19]

Kyuyong Shin, Hanock Kwak, Kyung-Min Kim, Minkyu Kim, Young-Jin Park, Jisu Jeong, and Seungjae Jung. 2021. One4all user representation for recommender systems in e-commerce.arXiv preprint arXiv:2106.00573(2021)

2021 arXiv
[20]

Zhu Sun, Hongyang Liu, Xinghua Qu, Kaidong Feng, Yan Wang, and Yew Soon Ong. 2024. Large language models for intent-driven session recommendations. InProc. of SIGIR

2024
[21]

Wentao Shi, Xiangnan He, Yang Zhang, Chongming Gao, Xinyue Li, Jizhi Zhang, Qifan Wang, and Fuli Feng. 2024. Large language models are learnable planners for long-term recommendation. InProc. of SIGIR

2024
[22]

Lei Wang and Ee-Peng Lim. 2023. Zero-shot next-item recommendation using large pretrained language models.arXiv preprint arXiv:2304.03153(2023)

2023 arXiv
[23]

Wei-Tung Wang, Yi-Leh Wu, Cheng-Yuan Tang, and Maw-Kae Hor. 2015. Adap- tive density-based spatial clustering of applications with noise (DBSCAN) ac- cording to data. InProc. of ICMLC

2015
[24]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

2023 arXiv
[25]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. InProc. of SIGIR

2019
[26]

Zhoumeng Wang. 2024. Empowering few-shot recommender systems with large language models-enhanced representations.Access(2024)

2024
[27]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. Kgat: Knowledge graph attention network for recommendation. InProc. of WSDM

2019
[28]

Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al . 2024. A survey on large language models for recommendation. InProc. of WWW

2024
[29]

Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, et al. 2023. A comprehensive capability analysis of gpt-3 and gpt-3.5 series models.arXiv preprint arXiv:2303.10420(2023)

2023 arXiv
[30]

Wei Wei, Xubin Ren, Jiabin Tang, Qinyong Wang, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, and Chao Huang. 2024. Llmrec: Large language models with graph augmentation for recommendation. InProc. of WSDM

2024
[31]

Zixuan Yi, Zijun Long, Iadh Ounis, Craig Macdonald, and Richard Mccreadie
[32]

Zixuan Yi and Iadh Ounis. 2024. A unified graph transformer for overcom- ing isolations in multi-modal recommendation. InProceedings of the 18th ACM Conference on Recommender Systems. 518–527

2024
[33]

Zixuan Yi, Zijun Long, Iadh Ounis, Craig Macdonald, and Richard Mccreadie. 2023. Large multi-modal encoders for recommendation.arXiv preprint arXiv:2310.20343 (2023)

2023 arXiv
[34]

Zixuan Yi, Iadh Ounis, and Craig Macdonald. 2023. Contrastive graph prompt- tuning for cross-domain recommendation.ACM Transactions on Information Systems42, 2 (2023), 1–28

2023
[35]

Zixuan Yi, Iadh Ounis, and Craig Macdonald. 2023. Graph contrastive learning with positional representation for recommendation. InEuropean Conference on Information Retrieval. Springer, 288–303

2023
[36]

Zixuan Yi, Xi Wang, and Iadh Ounis. 2024. A directional diffusion graph trans- former for recommendation.arXiv preprint arXiv:2404.03326(2024)

2024 arXiv
[37]

Zixuan Yi and Iadh Ounis. 2025. A multi-modal recipe for improved multi-domain recommendation. InEuropean Conference on Information Retrieval. Springer, 421– 436

2025
[38]

Junliang Yu, Xin Xia, Tong Chen, Lizhen Cui, Nguyen Quoc Viet Hung, and Hongzhi Yin. 2023. XSimGCL: Towards extremely simple graph contrastive learning for recommendation.Transactions on Knowledge and Data Engineering (2023)

2023
[39]

Junjie Zhang, Ruobing Xie, Yupeng Hou, Xin Zhao, Leyu Lin, and Ji-Rong Wen
[40]

Kaike Zhang, Qi Cao, Yunfan Wu, Fei Sun, Huawei Shen, and Xueqi Cheng
[41]

Zixuan Yi, Xi Wang, Iadh Ounis, and Craig Macdonald. 2022. Multi-modal graph contrastive learning for micro-video recommendation. InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1807–1811

2022
[42]

Yuhui Zhang, Hao Ding, Zeren Shui, Yifei Ma, James Zou, Anoop Deoras, and Hao Wang. 2021. Language models as recommender systems: Evaluations and limitations. (2021)

2021
[43]

Zhilu Zhang and Mert Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels.Advances in neural information processing systems(2018)

2018
[44]

Recommendation as instruction following: A large language model em- powered recommendation approach.Transactions on Information Systems(2023)

2023
[45]

Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. InProc. of WWW

2005
[47]

Xiaotong Zhang, Xinyi Li, Han Liu, Xinyue Liu, and Xianchao Zhang. 2024. Label Hierarchical Structure-Aware Multi-Label Few-Shot Intent Detection via Prompt Tuning. InProc. of SIGIR

2024
[50]

Zihuai Zhao, Wenqi Fan, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Zhen Wen, Fei Wang, Xiangyu Zhao, Jiliang Tang, et al. 2024. Recommender systems in the era of large language models (llms).Transactions on Knowledge and Data Engineering(2024)

2024
[2022]

Recommendation as language processing: A unified pretrain, personalized prompt & predict paradigm (p5). InProc. of RecSys
[2023]

Tallrec: An effective and efficient tuning framework to align large language model with recommendation. InProc. of RecSys
[2024]

Lorec: Combating poisons with large language model for robust sequential recommendation. InProc. of SIGIR
[2025]

Enhancing recommender systems: Deep modality alignment with large multi-modal encoders.ACM Transactions on Recommender Systems3, 4 (2025), 1–25

2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He

[2] [2]

Zeyu Cui, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. M6-rec: Generative pretrained language models are open-ended recommender systems.arXiv preprint arXiv:2205.08084(2022)

arXiv 2022

[3] [3]

Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongx- iang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. InProc. of RecSys

work page 2023

[4] [4]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783(2024)

arXiv 2024

[5] [5]

Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang

[6] [6]

John A Hartigan and Manchek A Wong. 1979. Algorithm AS 136: A k-means clustering algorithm.Journal of the royal statistical society(1979)

work page 1979

[7] [7]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. InProc. of SIGIR

work page 2020

[8] [8]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. InProc. of ICLR

work page 2022