Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?

T0 review · 4 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In-domain benchmark data leakage causes large but spurious gains in LLM-based recommender evaluation, while out-of-domain leakage degrades accuracy, and the effect varies with architecture and contamination mix.

desk verdict A plausible but unverified first probe of benchmark leakage in LLM-based recommenders: the in-domain leakage sample may include test interactions, making the headline gains possibly answer-key memorization. read the letter →

arxiv 2602.13626 v3 pith:BM4LPQ5W submitted 2026-02-14 cs.LG

Mingqiao Zhang , Qiyao Peng , Yinghui Wang , Hongtao Liu , Yumeng Wang This is my paper
classification cs.LG
keywords LLM-basedrecommendationbenchmarkdataleakageperformanceinflationevaluationreliabilityin-domainout-of-domaincontaminationLoRAfine-tuningrecommendersystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that LLM-based recommender systems can be secretly contaminated by benchmark data that leaked into the model's training, and that this contamination makes standard evaluation metrics unreliable. By fine-tuning a frozen open 7-billion-parameter language model with low-rank adapters on a mixture of in-domain (from the target benchmark) and out-of-domain (from six unrelated datasets) user–item interactions, the authors create 'dirty' versions of the model and compare them against the 'clean' original across seven recommender architectures. They find that in-domain leakage produces large, spurious performance gains (up to 25% relative AUC improvement), while out-of-domain leakage typically degrades accuracy. The authors conclude that in-domain leakage inflates metrics, misleadingly exaggerating model capability, and that the effect depends on both the leakage composition and the recommender architecture.

What carries the argument

The central mechanism is a controlled contamination pipeline: a frozen base LLM (θ₀) serves as the 'clean' backbone, and a 'dirty' version is created by training low-rank adaptation (LoRA) adapters—small additive matrices trained while base weights stay frozen—on a mixed leakage corpus D_leak = D_ID ∪ D_OOD, where D_ID is a 10% uniform sample of the target dataset and D_OOD is an equal-sized union from six external datasets (60% total). Because only the adapter parameters change, any downstream performance shift can be attributed to the injected memorized interactions. The paper organizes results with a 'triple effect' taxonomy—spurious gains (in-domain), stability, and degradation (out-of-d

What would settle it

Rerun the leakage simulation after removing from the in-domain leak sample any interactions that appear in the validation or test splits; if the in-domain gains vanish or shrink substantially, the central 'spurious gains' conclusion is an artifact of answer-key contamination, not a general leakage phenomenon.

Watch

Extended reading notes

Core claim

The paper argues that benchmark data leakage—user–item interactions from a target evaluation dataset appearing in the LLM's training or fine-tuning data—causes substantial but spurious performance inflation in downstream recommender systems, whereas leakage of semantically or structurally unrelated data generally degrades performance. To demonstrate this, the authors fine-tune a frozen open 7-billion-parameter language model with low-rank adapters on a mixed corpus of 10% in-domain interactions sampled from the target benchmark plus 60% out-of-domain interactions drawn equally from six external datasets, producing 'dirty' backbones that are compared against the 'clean' original across seven

Load-bearing premise

The paper never states whether the 10% in-domain leakage sample is disjoint from the validation and test splits used to compute AUC/UAUC; if it overlaps the test set, the reported spurious gains are direct memorization of the answer key rather than a subtle contamination effect.

Editorial extensions

If this is right

  • If an LLM backbone has been fine-tuned on any in-domain benchmark interactions, its reported AUC/UAUC gains can be inflated by memorization rather than genuine recommendation ability.
  • If out-of-domain contamination degrades performance, adding extra fine-tuning data without checking semantic overlap can silently hurt real-world accuracy.
  • If collaborative-augmented architectures are more resilient, future recommender designs should incorporate collaborative signals to buffer against backbone leakage.
  • Since even small in-domain proportions (5%) show measurable effects, evaluation reports should disclose the backbone's training provenance to allow leakage audits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Because LoRA touches only a small fraction of the model's weights, the large in-domain gains likely represent a lower bound; full-scale pre-training contamination could produce even more severe inflation.
  • Editorial inference: A concrete design rule follows from the resilience result: fuse external collaborative embeddings into the recommender to reduce vulnerability to backbone memorization.
  • Editorial inference: The paper's findings suggest a practical test: build a benchmark with deliberately leaked interactions and verify whether a recommender's performance improves on held-out users who share no items with the leaked set; that would separate memorization from adaptation.
  • Editorial inference: If the 10% in-domain sample is later confirmed to be disjoint from validation/test, the reported gains quantify true contamination risk; if not, the inflation measure would need to be recalibrated against answer-key memorization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper investigates benchmark data leakage in LLM-based recommender systems by simulating contamination of Vicuna-7B through LoRA fine-tuning on blended in-domain and out-of-domain interaction data. The authors construct a 'dirty LLM' trained on a leakage corpus (Eq. 4), use it as the backbone for seven recommender models spanning two architectural families (LLMRec and LLMRec+Collab.), and compare AUC/UAUC against a 'clean LLM' baseline on ML-1M and Amazon-Book. The central claim is a dual/triple effect of leakage: in-domain leakage produces large but spurious performance gains, out-of-domain leakage typically degrades performance, and some settings remain relatively stable. The paper concludes that LLM-based recommendation evaluations are unreliable unless the backbone LLM is audited for prior benchmark exposure.

Significance. If the central claim is established, the paper addresses a timely and important problem for the LLM-based recommendation community: evaluation results may be inflated by benchmark contamination. The experimental design is a controlled intervention, comparing clean vs. dirty backbones while freezing the base weights, and the coverage of several model architectures and six OOD sources is a strength. The release of code is also a positive step. However, the main empirical conclusion currently rests on an unverified dataset-split assumption: if the in-domain leakage sample overlaps with the test split, the reported gains are direct answer-key memorization, not a newly discovered 'spurious gains' phenomenon. The paper also lacks error bars, significance tests, and memorization checks. The claim is important but not yet sufficiently supported; with a straightforward split-restriction fix and appropriate re-running, the contribution could be valuable.

major comments (4)
  1. [§3.2, Eq. (1)] The manuscript states that each dataset is split 7:2:1 into training/validation/test, but then defines DID as Sample(Dtarget, p=0.1) with no statement that DID is drawn only from the training split or is otherwise disjoint from validation/test. Under the literal reading, DID is sampled from the full target dataset, so with p=0.1 and a 10% test split the expected overlap is about 1% of the full dataset (≈10k interactions on ML-1M). If those exact user–item pairs appear in both the LoRA leakage corpus and the evaluation set, the large positive ΔAUC/ΔUAUC in Tables 3 and 6 (e.g., TALLRec +25.0%) are direct memorization of the answer key, and the 'spurious gains' conclusion becomes a restatement of the setup rather than an empirical discovery. The authors must restrict DID to the training split (or prove disjointness) and re-run Tables 2, 3, 5, and 6. This is the load-bearing point for the p
  2. [§3.3 and §6] The abstract and conclusion generalize to benchmark leakage during 'pre-training or fine-tuning,' and the abstract describes the method as 'continued pre-training.' The experiments, however, only train LoRA adapters on the leakage corpus (Eq. 6); they do not perform continued pretraining of the base model. LoRA fine-tuning on user–item interaction text is a plausible controlled proxy for fine-tuning contamination, but it is not equivalent to pretraining contamination, and no memorization checks (e.g., extraction or membership tests) are provided to show that the dirty LLM actually memorized the leaked interactions. The authors should either narrow all claims to fine-tuning contamination or supply evidence that the LoRA intervention faithfully reproduces pretraining leakage.
  3. [§4.2, Tables 2–6] No error bars, confidence intervals, or significance tests are reported; every table appears to contain a single run. This is particularly problematic for the 'remains stable' condition in Table 5, where the largest observed change is 5.9% and differences of this magnitude may be within run-to-run noise. The claims of 'significant performance variations' based on a |Δ|≥8% threshold (Table 2 caption) are not statistical significance. At minimum, the authors should report multiple seeds with means and standard deviations for the main leakage conditions, especially for the mixed 5%+30% condition used to support the 'smaller impact' conclusion.
  4. [§4.2.1, architecture-resilience claim] The paper concludes that LLMRec+Collab. models are 'more resilient' than pure LLMRec models, but Table 2 does not uniformly support this: CoLLM-DIN drops 4.4% on Amazon-Book, PersonPrompt drops 5.1% on ML-1M, and BinLLM drops 9.1% on Amazon-Book, while TALLRec improves 7.1% on Amazon-Book. The aggregate tendency may hold, but the text asserts it as a general finding. Please qualify the claim and, ideally, provide a per-method/per-dataset breakdown with an appropriate significance or effect-size analysis.
minor comments (4)
  1. [§4.2, text] There are several typos and unclear abbreviations: 'LM-1M' should be 'ML-1M'; 'ODD data' should be 'OOD data'; 'AT dataset' and 'B' in the second bullet of §4.2.1 are undefined; 'the mode' should be 'the model.' Also, the note in §4.2.2 that the results 'may not logically correspond exactly with the data in Table 2' undermines reproducibility — clarify whether the same data splits and seeds are used across tables.
  2. [Figures 2 and 3] Both figure captions and the plots contain rendering artifacts such as '/uni00000037/uni00000024/...' which make the figures partially unreadable. The figures should be regenerated and the source data for the curves should be described in the caption or text.
  3. [Abstract vs. §3.1] The abstract refers to a 'dual-effect' while §3.1 and Figure 1 introduce a 'Triple Effect of Leakage' (spurious gains, remaining stable, degradation). Please align the terminology throughout. Also, Figure 1 contains the typo 'Spurious Grains' for 'Spurious Gains.'
  4. [References] Several references use 'and 1 others' (e.g., Achiam et al., Bommasani et al., Hu et al.), which is unconventional and should be formatted consistently as 'et al.'

Circularity Check

1 steps flagged · score 6.0 of 10

In-domain leakage is sampled from the full target dataset (including the test split), so the reported 'spurious gains' are answer-key memorization by construction.

  1. self definitional [Section 3.2, Eq. (1); Section 4.2.2; Tables 3 and 6; Conclusion Section 6]
    "Each dataset is split into training, validation, and test sets following a 7:2:1 ratio. First, we randomly sample 10% from the target dataset as in-domain leakage: DID ∼Sample(Dtarget, p=0.1). (1)"

    The paper never restricts DID to the training split; the literal equation samples from the full Dtarget, which contains the test split later used for AUC/UAUC evaluation. The paper's own analysis admits the consequence: 'Although LLMs remember them, they don’t know the answers like they do with test data.' LoRA next-token training on DID therefore places the target benchmark's test answers into the dirty LLM. The large positive deltas in Tables 3 and 6 (e.g., TALLRec +25.0% AUC under pure ID leakage) are then a direct memorization effect, not evidence that domain-relevant exposure per se 'misrepresents true model capability.' The central conclusion is a restatement of the experimental setup: the in-domain leakage set is defined to include evaluation data, so inflated test performance is fo

full rationale

The paper's central claim (ID leakage inflates performance; OOD degrades) rests on Eq. (1), which samples DID from Dtarget without excluding the validation/test splits. Section 4.2.2's remark that OOD data 'don't know the answers like they do with test data' confirms the authors treat ID leakage as test-data exposure. Consequently, the headline finding is at least partially circular: training the LoRA adapter on test interactions and then measuring test AUC guarantees memorization-based gains; the ID-vs-OOD contrast is partly a contrast between answer-key leakage and irrelevant data. The non-ID parts of the study (OOD degradation, architecture sensitivity, mixed proportions) are independent interventions and not circular. No load-bearing self-citations or imported uniqueness theorems appear. Because the central empirical generalization reduces, in its pure-ID form, to the construction of the leakage set, the score is 6 rather than 0-2. If the authors can show DID was drawn only from the training split and disjoint from validation/test, the circularity would be resolved; the current text does not do so.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central result does not rest on fitted constants; the hand-chosen leakage ratios and the unvalidated clean-proxy assumption carry the analytical weight. No new theoretical entities are introduced.

free parameters (5)
  • in-domain leakage ratio p = 0.1 (10%)
    Chosen by hand in Eq. (1); sets the scale of the leaked in-domain corpus and all OOD sampling.
  • OOD multiplier = 6 × |D_ID|
    Eqs. (2)-(3); the out-of-domain corpus is fixed to six times the in-domain size, chosen to balance domains.
  • ablation leakage ratios = 5%+30%, 25%+45%, 10%+60%, pure 10%, pure 60%
    Tables 2-6; manual control levels, and the observed effects are sensitive to these ratios.
  • LoRA rank and scale = r=8, alpha=16
    Section 4.1.2; these hyperparameters determine how much contamination is encoded into the low-rank adapters.
  • significance threshold for bold values = |Δ| ≥ 8%
    Table 2 caption; arbitrary cutoff used to label 'significant performance variations' without statistical testing.
assumptions (3)
  • domain assumption Vicuna-7B is a clean LLM with no prior leakage of ML-1M or Amazon-Book
    Section 3.1 treats the base model as leakage-free, but no audit or membership inference is provided; if Vicuna already memorized these benchmarks, the clean/dirty contrast is blurred.
  • ad hoc to paper LoRA fine-tuning on interaction text faithfully simulates benchmark leakage from pre-training
    Section 3.3 justifies LoRA as a 'controlled proxy' for real leakage; this is an unvalidated modeling assumption that the downstream effect of full pre-training contamination is equivalent in sign and magnitude.
  • domain assumption The in-domain leakage sample is disjoint from the evaluation test split
    Eq. (1) samples 10% from D_target without stating whether validation/test pairs are excluded; the interpretation of 'spurious gains' depends on this unstated assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?." pith.science (2026). https://pith.science/paper/BM4LPQ5W

@misc{pith2026260213626,
  author       = {Pith},
  title        = {Pith review of: Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BM4LPQ5W}},
  note         = {Machine review of arXiv:2602.13626}
}
read the original abstract

The expanding integration of Large Language Models (LLMs) into recommender systems poses critical challenges to evaluation reliability. This paper identifies and investigates a previously overlooked issue: benchmark data leakage in LLM-based recommendation. This phenomenon occurs when LLMs are exposed to and potentially memorize benchmark datasets during pre-training or fine-tuning, leading to artificially inflated performance metrics that fail to reflect true model performance. To validate this phenomenon, we simulate diverse data leakage scenarios by conducting continued pre-training of foundation models on strategically blended corpora, which include user-item interactions from both in-domain and out-of-domain sources. Our experiments reveal a dual-effect of data leakage: when the leaked data is domain-relevant, it induces substantial but spurious performance gains, misleadingly exaggerating the model's capability. In contrast, domain-irrelevant leakage typically degrades recommendation accuracy, highlighting the complex and contingent nature of this contamination. Our findings reveal that data leakage acts as a critical, previously unaccounted-for factor in LLM-based recommendation, which could impact the true model performance. We release our code at https://github.com/yusba1/LLMRec-Data-Leakage.

Figures

Figures reproduced from arXiv: 2602.13626 by the authors.

Figure 1
Figure 1. Overview of the Experimental Framework. The workflow illustrates the construction of a Mixed Leakage [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. AUC under different data injection methods [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. $\tau$-Rec: A Verifiable Benchmark for Agentic Recommender Systems

    cs.IR 2026-06 unverdicted novelty 7.0 of 10

    τ-Rec is a verifiable benchmark for agentic recommender systems using RTE and pass^k metrics that exposes low consistent-reasoning reliability in current LLMs.

Reference graph

Works this paper leans on

38 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Bao and 1 others

    H. Bao and 1 others. 2023. Tallrec: Aligning large language models with recommendation through task-specific instruction tuning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2023

  2. [2]

    Hudson, Ehsan Adeli, and 1 others

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, and 1 others. 2021. https://arxiv.org/abs/2108.07258 On the opportunities and risks of foundation models . arXiv preprint arXiv:2108.07258

    arXiv 2021

  3. [3]

    Brown, Benjamin Mann, Nick Ryder, and 1 others

    Tom B. Brown, Benjamin Mann, Nick Ryder, and 1 others. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 1877--1901

    2020

  4. [4]

    Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Chen, Matthew Jagielski, Vikash Sehwag, Florian Tramer, Borja Balle, Daphne Ippolito, and Eric Wallace. 2023. Extracting training data from diffusion models. arXiv preprint arXiv:2301.13188

    arXiv 2023

  5. [5]

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tram \`e r, Chiyuan Zhang, Eric Wallace, Nicolas Papernot, \'U lfar Erlingsson, Klas Leino, and Jacob Steinhardt. 2022. https://arxiv.org/abs/2202.07646 Quantifying memorization across neural language models . In Advances in Neural Information Processing Systems (NeurIPS)

    arXiv 2022

  6. [6]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, and 1 others. 2021. Extracting training data from large language models. 30th USENIX Security Symposium (USENIX Security 21), pages 2633--2650

    2021

  7. [7]

    \`O scar Celma. 2010. Last.fm dataset --- 360k users. https://ocelma.net/MusicRecommendationDataset/lastfm-360K.html. Accessed: 2025-10-25

    2010

  8. [8]

    Dai and 1 others

    X. Dai and 1 others. 2023. Why can GPT do recommendation without training? in-context learning for recommendation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2023

Show all 38 references
  1. [9]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022 a . https://arxiv.org/abs/2203.13366 P5: Pretrain, personalized, prompt: A unified framework for recommendation . arXiv preprint arXiv:2203.13366

    2022 arXiv

  2. [10]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022 b . Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). Proceedings of the 16th ACM Conference on Recommender Systems, pages 299--315

    2022

  3. [11]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. 2015. https://doi.org/10.1145/2827872 The movielens datasets: History and context . ACM Transactions on Interactive Intelligent Systems (TiiS), 5(4)

    2015 doi

  4. [12]

    Shohei Hisamoto, Matt Post, and Kevin Duh. 2020. https://aclanthology.org/2020.acl-main.101 Membership inference attacks on sequence-to-sequence models: Is my data in your machine translation system? In Proceedings of the 58th Annual Meeting of the Association for Computationa...

    2020

  5. [13]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  6. [14]

    Brown, and 1 others

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, and 1 others. 2020. https://arxiv.org/abs/2001.08361 Scaling laws for neural language models . arXiv preprint arXiv:2001.08361

    2020 arXiv

  7. [15]

    Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Pieter Abbeel, and Florian Tramer. 2022. Deduplicating training data mitigates privacy risks in language models. International Conference on Machine Learning, pages 12997--13005

    2022

  8. [16]

    Lei Li, Yongfeng Zhang, Dugang Liu, and Li Chen. 2023 a . https://arxiv.org/abs/2309.01157 Large language models for generative recommendation: A survey and visionary discussions . arXiv preprint arXiv:2309.01157

    2023 arXiv

  9. [17]

    Li and 1 others

    Y. Li and 1 others. 2023 b . Personprompt: Personalized soft prompting for large language model recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys)

    2023

  10. [18]

    Junling Lin, Wayne Xin Zhao, and 1 others. 2023. https://arxiv.org/abs/2306.05817 How can recommender systems benefit from large language models: A survey . arXiv preprint arXiv:2306.05817

    2023 arXiv

  11. [19]

    Pengjie Liu and 1 others. 2023. https://doi.org/10.1162/tacl_a_00619 Pre-train, prompt, and recommendation: A systematic survey of PLM -based recommender systems . Transactions of the Association for Computational Linguistics, 11:864--889

    2023 doi

  12. [20]

    Yiyu Liu, Qian Liu, Yu Tian, Changping Wang, Yanan Niu, Yang Song, and Chenliang Li. 2021. https://doi.org/10.1145/3459637.3482417 Concept-aware denoising graph neural network for micro-video recommendation . In Proceedings of the 30th ACM International Conference on Informati...

    2021

  13. [21]

    Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019 a . Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In EMNLP

    2019

  14. [22]

    Jingjing Ni, Jiacheng Li, and Julian McAuley. 2019 b . Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2019

  15. [23]

    OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    2023 arXiv

  16. [24]

    Steffen Rendle. 2012. https://doi.org/10.1145/2168752.2168771 Factorization machines . ACM Transactions on Intelligent Systems and Technology, 3(3):57:1--57:22

    2012

  17. [25]

    Stanford Network Analysis Project (SNAP). 2010. Epinions social network dataset. https://snap.stanford.edu/data/soc-Epinions1.html

    2010

  18. [26]

    Stanford Network Analysis Project (SNAP). 2011. Gowalla social network and check-in dataset. https://snap.stanford.edu/data/loc-gowalla.html

    2011

  19. [27]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  20. [28]

    Qingyun Wang and 1 others. 2024. https://arxiv.org/abs/2410.19744 Towards next-generation llm-based recommender systems: A survey of advances and challenges . arXiv preprint arXiv:2410.19744

    2024 arXiv

  21. [29]

    Jason Wei, Yi Tay, Rishi Bommasani, and 1 others. 2022. https://arxiv.org/abs/2206.07682 Emergent abilities of large language models . arXiv preprint arXiv:2206.07682

    2022 arXiv

  22. [30]

    Fangzhao Wu, Ying Qiao, and 1 others. 2020. Mind: A large-scale dataset for news recommendation. In ACL

    2020

  23. [32]

    Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, and Enhong Chen. 2023 b . https://arxiv.org/abs/2305.19860 A survey on large language models for recommendation . arXiv preprint arXiv:2305.19860

    2023 arXiv

  24. [33]

    Zhang and X

    X. Zhang and X. Wang. 2023. Prompt4nr: Prompt learning for neural recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

    2023

  25. [34]

    Yang Zhang, Keqin Bao, Ming Yan, Wenjie Wang, Fuli Feng, and Xiangnan He. 2024. https://arxiv.org/abs/2406.03210 Text-like encoding of collaborative information in large language models for recommendation . Preprint, arXiv:2406.03210

    2024 arXiv

  26. [35]

    Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, and Xiangnan He. 2025. https://doi.org/10.1109/TKDE.2025.3540912 Collm: Integrating collaborative embeddings into large language models for recommendation . IEEE Transactions on Knowledge and Data Engineering, 37(5):2329--2340

    2025

  27. [36]

    Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining,...

    2018

  28. [37]

    Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. 2023. https://arxiv.org/abs/2311.01964 Don't make your llm an evaluation benchmark cheater . Preprint, arXiv:2311.01964

    2023 arXiv

  29. [38]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  30. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.