Pith. sign in

REVIEW 4 major objections 5 minor 54 references

LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Injecting personalized knowledge-graph paths selected by a learned user-preference module into LlamaRec's prompt improves ranking accuracy on MovieLens-100K and Amazon Beauty, with the largest gains on the denser movie dataset.

desk verdict A sensible incremental extension of LlamaRec with a relation-preference path scorer, but the evaluation is undermined by a likely label-leakage problem in the static KG and an internally inconsistent ablation table. read the letter →

arxiv 2506.07449 v1 pith:JZ7CRWQ6 submitted 2025-06-09 cs.IR cs.AIcs.CL

Vahid Azizi , Fatemeh Koochaki This is my paper
classification cs.IRcs.AIcs.CL
keywords knowledgegraphretrieval-augmentedgenerationLLM-basedrankingrecommendersystemsuserpreferencemodulesingle-passinferenceLlamaRecpersonalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that adding structured knowledge-graph context to an LLM-based recommender improves ranking beyond what flat, similarity-based retrieval achieves. It builds on LlamaRec, a two-stage pipeline where a lightweight retriever proposes candidates and a fine-tuned Llama-2 model ranks them by outputting a letter index. The new framework, LlamaRec-LKG-RAG, inserts a learned user-preference module that scores relation types in a heterogeneous knowledge graph and selects a small set of personalized paths between each history item and each candidate. Those paths are added to the prompt, and the whole system is trained end-to-end. On MovieLens-100K and Amazon Beauty, the authors report consistent gains over LlamaRec across MRR, NDCG, and Recall, with the largest gains on MovieLens.

What carries the argument

The load-bearing components are (1) a heterogeneous knowledge graph with users, items, and metadata entities (genres, years, directors, actors for movies; brands, categories, and co-purchase/co-view edges for beauty products); (2) a lightweight user preference module, a small network that maps a user embedding to a distribution over relation types; and (3) a TF-IDF-inspired path scoring rule that combines the learned relation scores with the informativeness of each relation in the current query context. The top-scored paths are added to the existing LlamaRec prompt template, and a verbalizer converts the LLM's output logits for candidate index letters into ranking scores, so inference remains a single pass.

What would settle it

Re-run the MovieLens and Beauty experiments with knowledge graphs restricted to information timestamped before each prediction time (for example, dropping directors, actors, release year, and co-purchase edges that post-date the target interaction); if the reported gains over LlamaRec shrink or disappear, the improvements are not due to personalized structured reasoning at inference time.

Watch

Extended reading notes

Core claim

The central discovery is that personalized, relation-scored knowledge-graph paths, selected by a lightweight neural network and weighted by a TF-IDF-style scheme, can be injected into the LlamaRec prompt in a single forward pass, and this improves ranking accuracy. The paper argues that what matters is not just the presence of KG context but its personalization: an ablation shows that including unfiltered shortest paths between history and candidate items (LlamaRec-KG-RAG) actually hurts performance relative to LlamaRec, while the preference-module-filtered version helps. The selected paths are reported to give the LLM semantically coherent signals, such as a release-year relation matching the user's inferred taste, that support its ranking decision.

Load-bearing premise

The knowledge graph is built from the full datasets, including metadata and co-purchase edges, and the authors assume that none of these paths leak information that would be unavailable at inference time; they acknowledge this in the Discussion but do not test a temporally filtered graph.

Editorial extensions

If this is right

  • LLM-based rankers can be augmented with structured knowledge in a single forward pass, keeping inference latency close to LlamaRec while adding relational context.
  • The user preference module is what makes KG context helpful; dumping unfiltered graph paths into the prompt can degrade ranking, so selective retrieval matters.
  • The approach inherits interpretability benefits: the paths included in the prompt can serve as explanations for a ranking decision.
  • Because the framework is trained end-to-end and uses lightweight modules, it scales to datasets with larger item catalogs than the two tested, provided candidate generation remains cheap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gains survive a temporally filtered graph, the same path-selection mechanism could be applied to other structured signals, such as social networks, item taxonomies, or review-derived relations.
  • A direct test the paper leaves implicit is whether the user preference module could be replaced by an in-prompt scoring instruction to the LLM itself, which would isolate whether the benefit comes from the learned filter or from the structured context per se.
  • The TF-IDF weighting suggests a broader design pattern: personalize which relations to attend to, not just which items, which may transfer to non-recommendation RAG tasks where users have stable preferences.
  • In the paper's exploratory finding that larger models do not need explicit filtering, the filter's role is partly a cost-control device; a measured trade-off between path count, model size, and ranking quality would be a concrete follow-up.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LlamaRec-LKG-RAG, a single-pass, end-to-end trainable framework that augments the LlamaRec two-stage LLM ranking pipeline with personalized knowledge-graph context. A user preference module scores relation types and selects top-K shortest paths between history items and candidate items; these paths are added to the prompt of a fine-tuned Llama-2-7b model. Experiments on ML-100K and Amazon Beauty report improved MRR, NDCG, and Recall over LlamaRec, and an ablation on ML-100K suggests that unfiltered KG paths hurt performance, motivating the preference module.

Significance. If the results are valid, the paper would demonstrate a practical way to inject structured KG evidence into LLM-based ranking in a single forward pass, with a lightweight personalization module and a code release. The core idea is plausible and the engineering contribution is useful: it combines candidate retrieval, relation-path scoring, and verbalizer-based ranking in one pipeline. However, the empirical evidence currently does not establish the central claim because the knowledge graph is built from the complete datasets without temporal filtering, which creates a direct leakage channel through user-item rating edges and future co-purchase/co-view edges. The ablation also contains internally inconsistent numbers. With correction of these evaluation issues, the approach could still be of interest, but as presented the main quantitative conclusions are not supported.

major comments (4)
  1. [§3.0.2, §3.0.3, §3.0.7] The knowledge graph is constructed once from the complete datasets and no temporal filtering is described. For ML-100K, the (User, RATED, Movie) edges include the held-out last interaction that is used as the test label; for Beauty, the item-item relations (ALSO_BOUGHT, ALSO_VIEWED, BOUGHT_TOGETHER) are derived from all user behavior, including behavior after the prediction point. Because shortest-path traversal in Neo4j is undirected and the paper does not state that target edges are removed or that path direction is restricted, a path from a history item to the correct candidate can pass through the user node via the RATED edge that encodes the target label, or through item-item edges that only exist in the future. The Discussion (Section 3.0.7) acknowledges that metadata could introduce future information, but it does not address this more direct label-edge leak. To support the central claim, the authors must rerun experiments with a temporally filtered KG (e.g., only edges timestamped before the prediction point) and with target rating and co-purchase edges removed, then compare results.
  2. [Table 3] The ablation table is internally inconsistent. For LlamaRec-KG-RAG, Table 3 reports Recall@10 = 0.9672 and MRR@10 = 0.0380. Under leave-one-out evaluation with candidates ranked within a top-10 set, every recalled item must be ranked no lower than 10, so MRR@10 must be at least Recall@10 / 10 = 0.0967. The reported MRR@10 of 0.0380 is below this lower bound, so the two numbers cannot both be correct. Additionally, a Recall@10 of 0.9672 would be an approximately 9x improvement over LlamaRec's 0.1065, which contradicts Table 2 where LlamaRec-LKG-RAG improves Recall@10 by only about 3%. This suggests a typo, an incorrect computation, or a different evaluation protocol, and it invalidates the conclusion that unfiltered KG context degrades performance.
  3. [§3.0.5, Tables 2 and 3] The main comparison lacks any measure of variance or statistical significance. Tables 2 and 3 report single runs without error bars, multiple seeds, or significance tests. On the Beauty dataset the absolute gains are very small (for example, MRR@10 0.0386 vs. 0.0380, NDCG@10 0.0498 vs. 0.0491, Recall@10 0.0868 vs. 0.0855), and the abstract's claim of "consistent and significant improvements" is not supported by any inferential statistic. Please report means and standard deviations over multiple random seeds and apply a paired significance test (e.g., paired bootstrap or Wilcoxon signed-rank test) to justify the word 'significant.'
  4. [§2.0.3, §3.0.4] The TF-IDF-inspired path weighting scheme is central to the method but is not defined precisely. The text states that learned relation scores are scaled by the TF-IDF score of each relation in the context of the current query, but no formula is given for the relation-level term frequency, document frequency, or normalization, nor is it specified how relations are tokenized or counted over paths. This makes the method non-reproducible as written. Please provide the exact scoring equation or pseudocode for the path selection procedure.
minor comments (5)
  1. [§1] The introduction contains a duplicated sentence: 'The rapid advancement of LLMs has positioned them as pivotal components in modern recommender systems.' appears twice in consecutive paragraphs.
  2. [Table 3] The table caption says 'Bold values indicate the lowest performance scores across models,' but in several rows the bold value is not the lowest (e.g., Recall@1 for LlamaRec-KG-RAG is 0.1080, which is higher than the other models' values). Please clarify the intended typeface or use a different marker for the statistically best/worst result.
  3. [§2.0.1, §3.0.3] The symbol K is overloaded: it denotes the candidate-set size (K=20 in Section 2.0.1) and the metric cutoff (K∈{1,5,10} in Section 3.0.3). Please use distinct notation, such as K_cand and K_metric, to avoid ambiguity.
  4. [§3.0.2] The paper should explicitly state whether shortest-path traversal is undirected or directionally restricted. Currently the description of relation types and the Neo4j implementation leaves this ambiguous, which matters for both causal validity and interpretability.
  5. [References] Several references are incomplete or contain placeholder IDs, such as Wang et al. (2024b) with 'https://dl.acm.org/doi/10.1145/XXXXXX' and Zhang et al. (2025) with a similarly incomplete DOI. These should be filled in or replaced with a stable citation.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation-level circularity: the ranking claim is empirical and self-contained against LlamaRec; the sole self-citation (Ai et al. 2018) is a passing reference, and the KG leakage concern is a correctness issue, not a circular reduction.

full rationale

This is an empirical systems paper, not a formal derivation, so the circularity patterns that apply to fitted-parameter derivations do not arise. The user preference module is trained on the benchmark data, but it only gates which KG paths are retrieved; the final ranking is produced by a separately fine-tuned Llama-2 model on the held-out last interaction, and the reported metrics are not the output of the preference module's own loss. The only self-citation is Ai et al. (2018), co-authored by Vahid Azizi, in the Discussion's paragraph on explainable recommendation; it is a passing pointer and does not carry the load of the MRR/NDCG/Recall claims. The Discussion (Section 3.0.7) itself flags that building the KG offline with metadata could inadvertently introduce future information and thereby violate causal constraints; this is a real evaluation-causality risk for the empirical comparison, but it is a correctness concern rather than a circular reduction of the method's derivation to its inputs. Accordingly, no specific circular step can be exhibited under the hard-evidence rule, and the score is 2 for the minor non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The contribution rests on a hand-designed KG schema, a hand-chosen path scoring scheme, and standard sequential recommendation assumptions. No new physical entities are introduced. The main unexamined assumption is temporal safety of the static KG.

free parameters (4)
  • Candidate set size K = 20
    Matches LlamaRec; controls context length and path count. Chosen by hand, not derived.
  • Preference module embedding dimension = 128 (ML-100K), 512 (Beauty)
    Set manually; impacts the capacity of the relation scoring module.
  • Relation output classes = 4 (ML-100K), 5 (Beauty)
    Determined by the manually designed KG relation types, not learned from data.
  • TF-IDF path weighting scheme = TF-IDF weighting (selected after trials)
    The Discussion states several heuristics were tested before settling on TF-IDF; this choice affects which paths enter the prompt.
assumptions (5)
  • domain assumption The preprocessing and evaluation protocol of Yue et al. (2023a) is valid for sequential recommendation.
    Used to filter data and split training, validation, and test sets in Section 3.0.1.
  • domain assumption Shortest paths between history items and candidate items summarize useful semantic context for ranking.
    Adopted in Section 2.0.3 to bound context size without evidence that shortest paths are the most informative.
  • ad hoc to paper A user's preference over relation types is well approximated by a feedforward softmax network over a user embedding.
    Core to the user preference module; no theoretical or empirical justification beyond the reported results.
  • ad hoc to paper TF-IDF scaling of relation scores improves path selection over naive summation.
    Reported as empirically chosen after testing alternatives; no derivation is provided.
  • domain assumption The statically built KG does not create temporal leakage that inflates results.
    The authors flag in the Discussion that future information may enter paths, but they do not test a temporally filtered graph.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking." pith.science (2026). https://pith.science/paper/JZ7CRWQ6

@misc{pith2026250607449,
  author       = {Pith},
  title        = {Pith review of: LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JZ7CRWQ6}},
  note         = {Machine review of arXiv:2506.07449}
}
read the original abstract

Recent advances in Large Language Models (LLMs) have driven their adoption in recommender systems through Retrieval-Augmented Generation (RAG) frameworks. However, existing RAG approaches predominantly rely on flat, similarity-based retrieval that fails to leverage the rich relational structure inherent in user-item interactions. We introduce LlamaRec-LKG-RAG, a novel single-pass, end-to-end trainable framework that integrates personalized knowledge graph context into LLM-based recommendation ranking. Our approach extends the LlamaRec architecture by incorporating a lightweight user preference module that dynamically identifies salient relation paths within a heterogeneous knowledge graph constructed from user behavior and item metadata. These personalized subgraphs are seamlessly integrated into prompts for a fine-tuned Llama-2 model, enabling efficient and interpretable recommendations through a unified inference step. Comprehensive experiments on ML-100K and Amazon Beauty datasets demonstrate consistent and significant improvements over LlamaRec across key ranking metrics (MRR, NDCG, Recall). LlamaRec-LKG-RAG demonstrates the critical value of structured reasoning in LLM-based recommendations and establishes a foundation for scalable, knowledge-aware personalization in next-generation recommender systems. Code is available at~\href{https://github.com/VahidAz/LlamaRec-LKG-RAG}{repository}.

Figures

Figures reproduced from arXiv: 2506.07449 by the authors.

Figure 1
Figure 1. Overview of the LlamaRec-LKG-RAG framework. A sequential recommendation module [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An illustrative subgraph of the knowledge graph constructed for the MovieLens dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. This is a representative subgraph of the knowledge graph constructed for the Beauty [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: An illustrative example of a model query enriched with KG context selected using the user [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: A prompt generated using KG context from [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 21 canonical work pages

  1. [1]

    Learning heterogeneous knowledge base embeddings for explainable recommendation

    Qingyao Ai, Vahid Azizi, Xu Chen, and Yongfeng Zhang. Learning heterogeneous knowledge base embeddings for explainable recommendation. Algorithms, 11 0 (9), 2018. ISSN 1999-4893. doi:10.3390/a11090137. URL https://www.mdpi.com/1999-4893/11/9/137

    work page doi:10.3390/a11090137 2018

  2. [2]

    Edgeformer: A graph-based framework for multi-hop question answering

    Akari Asai and Hannaneh Hajishirzi. Edgeformer: A graph-based framework for multi-hop question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 1414--1431, 2021

    work page 2021

  3. [3]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016. URL https://arxiv.org/abs/1607.06450

    arXiv 2016

  4. [4]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  5. [5]

    Typed Design Patterns for the Functional Era

    Yixin Cao, Xiangnan He, Yongfeng Zhang, et al. A survey of large language models for recommender systems. arXiv preprint arXiv:2307.07069, 2023

    work page Pith review arXiv 2023

  6. [6]

    A survey on llm-powered agents for recommender systems

    Hao Chen, Wei Xu, and Ming Zhang. A survey on llm-powered agents for recommender systems. arXiv preprint arXiv:2502.10050, 2025. URL https://arxiv.org/abs/2502.10050

    arXiv 2025

  7. [7]

    L. Chen, W. Huang, and S. Liu. Think-on-graph 2.0: Deep and faithful large language model reasoning with knowledge-guided retrieval augmented generation. arXiv preprint arXiv:2407.10805, 2024. URL https://arxiv.org/abs/2407.10805

    arXiv 2024

  8. [8]

    Llmrec: Personalized recommendation via prompting large language models

    Yasin Cinar, Pinar Karagoz, and Jundong Li. Llmrec: Personalized recommendation via prompting large language models. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.\ 3057--3061, 2023

    work page 2023

Show all 54 references
  1. [9]

    Qlora: efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: efficient finetuning of quantized llms. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc

    2023

  2. [10]

    Chatgpt is all you need for conversational recommendation: Towards personalized preference elicitation with large language models

    Qifan Gao, Jiaqi Tang, and Minlie Huang. Chatgpt is all you need for conversational recommendation: Towards personalized preference elicitation with large language models. arXiv preprint arXiv:2302.09104, 2023

    2023 arXiv

  3. [11]

    Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, and Jiliang Tang

    Haoyu Han, Yu Wang, Harry Shomer, Kai Guo, Jiayuan Ding, Yongjia Lei, Mahantesh Halappanavar, Ryan A. Rossi, Subhabrata Mukherjee, Xianfeng Tang, Qi He, Zhigang Hua, Bo Long, Tong Zhao, Neil Shah, Amin Javari, Yinglong Xia, and Jiliang Tang. Retrieval-augmented generation with...

    2025 arXiv

  4. [12]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst., 5 0 (4), December 2015. ISSN 2160-6455. doi:10.1145/2827872. URL https://doi.org/10.1145/2827872

    2015 doi

  5. [13]

    Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering

    Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, WWW '16, pp.\ 507–517, Republic and Canton of Geneva, CHE, 2016. Interna...

    2016

  6. [14]

    Towards reasoning-enhanced recommender systems: A survey and future directions

    Yifan Hou, Yajing Qi, Zhiwen Yu, Yanfang Song, Zhu Li, and Chengzhong Zhang. Towards reasoning-enhanced recommender systems: A survey and future directions. arXiv preprint arXiv:2305.15706, 2023

    2023 arXiv

  7. [15]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https://arxiv.org/abs/2106.09685

    2021 arXiv

  8. [16]

    Leveraging passage retrieval with generative models for open domain question answering

    Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282, 2021

    2007 arXiv

  9. [17]

    D. Lee, J. Park, and S. Kim. Gfm-rag: Graph foundation model for retrieval augmented generation. arXiv preprint arXiv:2502.01113, 2025. URL https://arxiv.org/abs/2502.01113

    2025

  10. [18]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kulkarni, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in neural information processing...

    2020

  11. [19]

    H. Li, Y. Chen, and M. Zhang. Grag: Graph retrieval-augmented generation for multi-hop reasoning. In Findings of NAACL, 2025. URL https://aclanthology.org/2025.findings-naacl.232

    2025

  12. [21]

    Llm-rec: Large language models for recommendation with interaction and reasoning

    Jing Liu, Wei Chen, and Mei Huang. Llm-rec: Large language models for recommendation with interaction and reasoning. arXiv preprint arXiv:2401.01234, 2024 a . URL https://arxiv.org/abs/2401.01234

    2024 arXiv

  13. [22]

    Chat-rec: Towards interactive and explainable llms-augmented recommendation

    Yihong Liu, Lei Zheng, Yong Ge, Qi Li, and Enhong Zhang. Chat-rec: Towards interactive and explainable llms-augmented recommendation. arXiv preprint arXiv:2305.14251, 2023

    2023 arXiv

  14. [23]

    Large language model enhanced recommender systems: Taxonomy, trend, application and future

    Zhen Liu, Lei Zhang, and Jing Gao. Large language model enhanced recommender systems: Taxonomy, trend, application and future. arXiv preprint arXiv:2412.13432, 2024 b . URL https://arxiv.org/abs/2412.13432

    2024 arXiv

  15. [24]

    Large language models are not stable recommender systems

    Tianhui Ma, Yuan Cheng, Hengshu Zhu, and Hui Xiong. Large language models are not stable recommender systems. arXiv preprint arXiv:2312.15746, 2023

    2023 arXiv

  16. [25]

    Image-based recommendations on styles and substitutes

    Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '15, pp.\ 43–52, New Yor...

    2015

  17. [26]

    Cypher Query Language - Introduction, 2024 a

    Neo4j . Cypher Query Language - Introduction, 2024 a . URL https://neo4j.com/docs/cypher-manual/current/introduction/. Accessed: 2025-05-12

    2024

  18. [27]

    Neo4j - the world's leading graph database, 2024 b

    Neo4j . Neo4j - the world's leading graph database, 2024 b . URL https://neo4j.com/. Accessed: 2025-05-12

    2024

  19. [28]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. Technical report, OpenAI, 2023. URL https://openai.com/research/gpt-4

    2023

  20. [29]

    Chatgpt (may 2025 version)

    OpenAI. Chatgpt (may 2025 version). https://chat.openai.com, 2025. Large language model developed by OpenAI

    2025

  21. [30]

    Graph retrieval-augmented generation: A survey

    Boci Peng et al. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921, 2024. URL https://arxiv.org/abs/2408.08921

    2024 arXiv

  22. [31]

    On explaining recommendations with large language models: A review

    Alan Said. On explaining recommendations with large language models: A review. arXiv preprint arXiv:2411.19576, 2024

    2024 arXiv

  23. [32]

    Smith, R

    J. Smith, R. Kumar, and L. Zhang. Rgl: A graph-centric, modular framework for efficient retrieval-augmented generation on graphs. arXiv preprint arXiv:2503.19314, 2025. URL https://arxiv.org/abs/2503.19314

    2025 arXiv

  24. [33]

    Yu, and Ji-Rong Wen

    Yujia Sun, Can Xu, Wayne Xin Zhao, Da Yin, Philip S. Yu, and Ji-Rong Wen. Think-on-graph: Structured reasoning with knowledge graphs for large language models. arXiv preprint arXiv:2312.02948, 2023 a

    2023 arXiv

  25. [34]

    Graphrag: Retrieval-augmented generation meets knowledge graph for recommendation

    Ziqian Sun, Xinyu Wang, Wenqiang Zhang, and Dawei Yin. Graphrag: Retrieval-augmented generation meets knowledge graph for recommendation. arXiv preprint arXiv:2310.06674, 2023 b

    2023 arXiv

  26. [35]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  27. [36]

    J. Wang, X. Li, and Y. Zhao. Gnn-rag: Graph neural retrieval for large language model reasoning. arXiv preprint arXiv:2405.20139, 2024 a . URL https://arxiv.org/abs/2405.20139

    2024 arXiv

  28. [37]

    A survey on large language models for recommendation

    Rui Wang, Fan Li, and Ming Zhao. A survey on large language models for recommendation. arXiv preprint arXiv:2305.19860, 2023. URL https://arxiv.org/abs/2305.19860

    2023 arXiv

  29. [38]

    Knowledge graph retrieval-augmented generation for llm-based recommendation

    Shijie Wang, Wenqi Fan, Yue Feng, Xinyu Ma, Shuaiqiang Wang, and Dawei Yin. Knowledge graph retrieval-augmented generation for llm-based recommendation. arXiv preprint arXiv:2501.02226, 2025. URL https://arxiv.org/abs/2501.02226

    2025 arXiv

  30. [40]

    Llmr: Large language model-augmented recommender system

    Canran Xu, Bowen Du, Qinming Zhan, et al. Llmr: Large language model-augmented recommender system. arXiv preprint arXiv:2306.05817, 2023

    2023 arXiv

  31. [41]

    Palr: Personalization aware llms for recommendation, 2023

    Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, and Yanbin Lu. Palr: Personalization aware llms for recommendation, 2023. URL https://arxiv.org/abs/2305.07622

    2023 arXiv

  32. [42]

    Retrieval-augmented generation for recommendation: A survey

    Hanxiong Yao, Chuhan Wu, and Yongfeng Zhang. Retrieval-augmented generation for recommendation: A survey. arXiv preprint arXiv:2305.01944, 2023

    2023 arXiv

  33. [43]

    Qa-gnn: Reasoning with language models and knowledge graphs for question answering

    Michihiro Yasunaga, Xiang Ren, Percy Liang, and Jure Leskovec. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua...

    2022

  34. [44]

    Defending substitution-based profile pollution attacks on sequential recommenders

    Zhenrui Yue, Huimin Zeng, Ziyi Kou, Lanyu Shang, and Dong Wang. Defending substitution-based profile pollution attacks on sequential recommenders. In Proceedings of the 16th ACM Conference on Recommender Systems, RecSys '22, pp.\ 59–70, New York, NY, USA, 2022. Association for...

    2022

  35. [45]

    Llamarec: Two-stage recommendation using large language models for ranking

    Zhenrui Yue, Sara Rabhi, Gabriel de Souza Pereira Moreira, Dong Wang, and Even Oldridge. Llamarec: Two-stage recommendation using large language models for ranking. arXiv preprint arXiv:2311.02089, 2023 a

    2023 arXiv

  36. [46]

    Linear recurrent units for sequential recommendation

    Zhenrui Yue, Yueqi Wang, Zhankui He, Huimin Zeng, Julian McAuley, and Dong Wang. Linear recurrent units for sequential recommendation. arXiv preprint arXiv:2310.02367, 2023 b

    2023 arXiv

  37. [47]

    Generative recommendation: A survey of large language models in recommender systems

    Lei Zhang, Rui Sun, and Hao Tan. Generative recommendation: A survey of large language models in recommender systems. ACM Computing Surveys, 58 0 (3): 0 1--35, 2025. doi:10.1145/XXXXXX

    2025 doi

  38. [48]

    Prompt-llmrec: Towards personalized recommendation via prompt tuning of large language models

    Rui Zhang et al. Prompt-llmrec: Towards personalized recommendation via prompt tuning of large language models. Proceedings of the Web Conference 2024, 2024

    2024

  39. [49]

    Opt: Open pre-trained transformer language models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Sid Dewan, Murtaza Genc, Sunita Goel, Christina Guta, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  40. [50]

    Llm agents for personalized recommendations: Reasoning, planning, and acting

    Yongfeng Zhang et al. Llm agents for personalized recommendations: Reasoning, planning, and acting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023

    2023

  41. [51]

    Llm4rec: Revolutionizing recommendation with large language models

    Wayne Xin Zhao, Yiheng Chang, Jianing Yang, Jun Wang, and Ji-Rong Wen. Llm4rec: Revolutionizing recommendation with large language models. arXiv preprint arXiv:2307.10649, 2023 a

    2023 arXiv

  42. [52]

    Recommender systems in the era of large language models (llms)

    Zihuai Zhao, Wenqi Fan, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Zhen Wen, Fei Wang, Xiangyu Zhao, Jiliang Tang, and Qing Li. Recommender systems in the era of large language models (llms). arXiv preprint arXiv:2307.02046, 2023 b

    2023 arXiv

  43. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  44. [54]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  45. [55]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  46. [56]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.