REVIEW 3 major objections 5 minor 44 references
SemSR: Semantics aware robust Session-based Recommendations
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper argues that fusing frozen LLM item embeddings with trainable interaction embeddings in session-based recommenders such as MSGAT and NISER raises Recall@K substantially, while a separate re-ranking step by vanilla NISER recovers…
desk verdict SemSR's fusion of frozen LLM embeddings with GNN session encoders gives real Recall gains, but the MRR claim only holds after an external NISER re-ranker, so the abstract overstates what the fused model itself does. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a fusion block placed between the item representation and the scorer. Each item gets a trainable interaction embedding $\mathbf{i}^m_j \in \mathbb{R}^{d_1}$ and a frozen LLM embedding $\mathbf{i}^l_j \in \mathbb{R}^{d_2}$; the two are concatenated and passed through a linear projection to form the semantic item embedding. The session is represented twice, once by the base SR model as $\mathbf{s}^m$ and once by a soft-attention aggregation of the LLM embeddings with the most recent item highlighted, giving $\mathbf{s}^l$; the two session vectors are concatenated and projected. Relevance is then the softmax of $\mathbf{i}_k^\top \mathbf{s}$. The same block is dropped into MSGAT and NISER, and a final re-ranking pass by vanilla NISER is what the paper credits for the MRR improvements.
What would settle it
Reproduce Table 2 on Amazon-M2(UK) without any re-ranking step. If SemNISER-F or SemMSGAT-F fails to beat its vanilla counterpart on MRR@20 while still improving Recall@20, then the recall claim stands but the claim that fusion improves fine-grained ranking is false and depends entirely on the external ranker.
Extended reading notes
Core claim
The paper's central claim is that the fusion variant, SemSR-F, dominates both data-driven baselines and LLM-only recommenders: on Amazon-M2(UK), Recall@20 rises from 38.86 for MSGAT to 52.66 for SemMSGAT-F with Llama-3 embeddings, and from 49.84 for NISER to 54.98 for SemNISER-F with BERT-based embeddings. The authors attribute the recall gain to semantic item representations capturing title, category, brand, and price signals that interaction-only embeddings miss. They also report that re-ranking the fused model's top-K list with an unmodified NISER model raises MRR for every variant, and they interpret that re-ranking step as part of the SemSR pipeline. The paper is explicit that LLM-derived knowledge helps at coarse retrieval while interaction-based signals carry fine-grained ranking, and that the two are complementary.
Load-bearing premise
The headline MRR improvement depends on counting an external vanilla NISER re-ranking step as part of SemSR; for the fused model alone, MRR@20 can fall below plain NISER (SemNISER-F on Beauty: 6.10 vs 6.95).
Editorial extensions
If this is right
- Any graph-attention or normalization-based SR model can receive semantic awareness by concatenating frozen LLM embeddings to its item representations, with no LLM retraining.
- A two-stage design, semantic retrieval for candidate generation and interaction-based re-ranking for final order, should beat either stage alone.
- LLM prompting alone is not competitive on fine-grained ranking, so systems relying on in-context LLMs for next-item ranking would lose MRR.
- Semantic initialization alone yields its largest recall advantage at high K (200 to 500), meaning it is suited to large candidate sets.
Reading between the lines
- A natural production architecture, which the paper does not develop, would run SemSR-F as candidate generator and a cheap interaction model as final ranker; the paper's own numbers suggest MRR would match or beat the fused model alone.
- The drop in MRR when semantic embeddings are used without re-ranking suggests the frozen text vectors distort the learned ranking geometry; a testable extension is to constrain the fusion projection so the interaction embedding lives in a subspace complementary to the semantic embedding, to see whether recall gains can be had without an external ranker.
- Because the paper reports only aggregate Recall and MRR, a testable extension is to split results by item frequency: if the recall gains concentrate on long-tail or cold items, the method is a cheap way to address cold-start, which the paper does not explicitly claim.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces SemSR, a family of methods that incorporate frozen LLM-generated item embeddings into two session-based recommender models, MSGAT and NISER. Variants include semantic initialization (SemSR-I), feature fusion (SemSR-F), and re-ranking of the top-K list by a vanilla NISER model (SemSR-I+ and SemSR-F+). Experiments on Amazon-M2 (UK) and Amazon-Beauty report Recall@K and MRR@K. The paper claims that LLM-based in-context methods are strong at coarse retrieval but weak at ranking, and that the integration of LLM embeddings with data-driven SR models significantly improves both Recall and MRR over all baselines.
Significance. If the central claim were fully supported, the contribution would be practically useful: it would give a model-agnostic, end-to-end recipe for injecting frozen semantic embeddings into established SR models, evaluated on public benchmarks with two LLM encoders. The recall gains in Table 2 are large and consistent, e.g., MSGAT-F on Amazon-M2 raises R@20 from 38.86 to 52.66, and the framework is clearly described in Section 3.3. However, the MRR improvements are not consistent for the fused models alone and depend on an external re-ranking step that is not part of the stated architecture; the paper also provides no uncertainty quantification, so the rhetorical claim of 'significantly outperforms' is currently unsupported.
major comments (3)
- [Abstract and RQ2; Table 2] The abstract claims that the integration of LLMs with data-driven SR models significantly outperforms baseline SR models in both Recall and MRR, but Table 2 shows that the fused models alone do not consistently improve MRR. For example, SemNISER-F with Angle embeddings on Beauty has MRR@20 6.10 versus NISER's 6.95, and SemNISER-F with Llama 3 has 5.42 versus 6.95; SemNISER-I on Amazon-M2 has MRR@20 14.25 versus NISER's 26.45. The consistent MRR improvements appear only in the '+' variants, which additionally use re-ranking. The claim must either be restricted to Recall for the fused models, or the re-ranking step must be explicitly treated as part of the method and evaluated symmetrically.
- [RQ3 and Section 3.3] The re-ranking procedure that produces SemSR-I+ and SemSR-F+ is not part of the architecture described in Section 3.3, and no algorithmic details are given for it: Section 4 only states that the lists 'obtained from SemSR-I, and SemSR-F' are re-ranked 'using NISER model.' This is load-bearing because re-ranking is the mechanism by which the paper obtains its claimed MRR improvements. Without a description of the candidate-list size, whether the NISER re-ranker is trained or frozen, and how re-ranking interacts with the fusion, the results cannot be reproduced. Since vanilla NISER is both the baseline and the re-ranker embedded in the pipeline, the comparison is also asymmetric.
- [Section 4.1; Table 3] No error bars, standard deviations, or significance tests are reported anywhere in Section 4, so the repeated use of 'significantly' in RQ2 and RQ3 is not substantiated. In addition, the LLM-as-RS comparison in Table 3 is performed on a 10,000-session subset of Amazon-M2, as described in Section 4, whereas Table 2 uses the full test set; the paper should state explicitly that all models in Table 3 are evaluated on this subset and should temper the RQ1 conclusions accordingly, because the comparison to the full-test-set numbers in Table 2 is not direct.
minor comments (5)
- [Section 2] There are typos: 'knowlege' should be 'knowledge', and 'utilizes a a two step strategy' contains a duplicated article.
- [Section 3.2] The footnote markers after 'Llama3-8B-Instruct' and 'Chromadb' render as asterisks that are easy to confuse with mathematical symbols; these should be replaced with standard numbered citations or footnotes.
- [Figure 1] The caption refers to Step-1 and Step-2, but the figure does not clearly delineate these steps; adding visual separators or labels would improve readability.
- [References] Reference [2] appears to be a mistaken or secondary citation for SRGNN; the original work is cited as [36], so [2] should be corrected or removed.
- [Hyperparameter Setup, Section 4] The statement 'adam optimizer with mini-batch size 100, momentum 0.9' is ambiguous for Adam; please report the exact optimizer settings such as beta1, beta2, and epsilon.
Circularity Check
No circularity: LLM embeddings are frozen pretrained text encoders and evaluation is on held-out sessions; the self-cited NISER baseline is a design choice, not the source of the semantic gain.
full rationale
No circular reasoning is present. The LLM-based item embeddings (Angle and LLM2Vec/Llama-3.1-8B) come from frozen, pretrained text encoders; they are not trained on the next-item labels and no target-derived quantity is used to build them. The semantic session embedding s_l is a learned attention-weighted aggregation of these frozen embeddings, and the fusion weights and trainable interaction embeddings are fit on training sessions using cross-entropy, which is standard supervised training rather than circularity: the reported Recall@20, Recall@100, MRR@20, and MRR@100 are computed on held-out test sessions. Hyperparameters are selected on validation data using Recall@100, not on the test set. The only self-citation is NISER [11] as a base architecture and as the re-ranker in the '+' variants; this is a design choice and does not reduce the claimed semantic improvement to that citation. The paper itself exposes a limitation in RQ3: the fused models alone do not consistently improve MRR over vanilla baselines (e.g., SemNISER-F has MRR@20 6.10 vs NISER 6.95 on Beauty), so the abstract's blanket 'both Recall and MRR' claim is empirically overstated for the fusion alone. That is an attribution and correctness issue, not circularity: the LLM embeddings are external, the held-out evaluation is independent of the fitted parameters, and no equation is defined in terms of the quantity it claims to predict.
Assumptions & free parameters
free parameters (4)
- Embedding dimensions (d1=100, d2=1024, d=100) =
d1=100, d2=1024, d=100
- NISER scaling factor =
16.0
- Learning rate for MSGAT and NISER =
0.001
- Mini-batch size and momentum =
batch=100, momentum=0.9
assumptions (4)
- domain assumption Frozen LLM embeddings computed from item metadata such as title, brand, price, color, and category preserve enough semantics to improve session modeling.
- domain assumption The datasets' item metadata is complete and consistent for all items in train and test sessions.
- domain assumption Cosine similarity in the ChromaDB vector store between the LLM output title and item embeddings is a valid retrieval mechanism for LLM-as-RS.
- domain assumption Single ground-truth next-item evaluation with Recall@K and MRR@K is the appropriate measure of recommendation quality.
Cite this review
Pith. "Pith review of SemSR: Semantics aware robust Session-based Recommendations." pith.science (2026). https://pith.science/paper/6HEDHZKY
@misc{pith2026250820587,
author = {Pith},
title = {Pith review of: SemSR: Semantics aware robust Session-based Recommendations},
year = {2026},
howpublished = {\url{https://pith.science/paper/6HEDHZKY}},
note = {Machine review of arXiv:2508.20587}
}
read the original abstract
Session-based recommendation (SR) models aim to recommend items to anonymous users based on their behavior during the current session. While various SR models in the literature utilize item sequences to predict the next item, they often fail to leverage semantic information from item titles or descriptions impeding session intent identification and interpretability. Recent research has explored Large Language Models (LLMs) as promising approaches to enhance session-based recommendations, with both prompt-based and fine-tuning based methods being widely investigated. However, prompt-based methods struggle to identify optimal prompts that elicit correct reasoning and lack task-specific feedback at test time, resulting in sub-optimal recommendations. Fine-tuning methods incorporate domain-specific knowledge but incur significant computational costs for implementation and maintenance. In this paper, we present multiple approaches to utilize LLMs for session-based recommendation: (i) in-context LLMs as recommendation agents, (ii) LLM-generated representations for semantic initialization of deep learning SR models, and (iii) integration of LLMs with data-driven SR models. Through comprehensive experiments on two real-world publicly available datasets, we demonstrate that LLM-based methods excel at coarse-level retrieval (high recall values), while traditional data-driven techniques perform well at fine-grained ranking (high Mean Reciprocal Rank values). Furthermore, the integration of LLMs with data-driven SR models significantly out performs both standalone LLM approaches and data-driven deep learning models, as well as baseline SR models, in terms of both Recall and MRR metrics.
Figures
Reference graph
Works this paper leans on
-
[1]
Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961 (2024)
arXiv 2024
-
[2]
Juan Belieni and Diego Mesquita. 2025. SRGNN: simple recurrent graph neural network. Proceeding Series of the Brazilian Society of Computational and Applied Mathematics 11, 1 (2025), 1–2
work page 2025
-
[3]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901
2020
-
[4]
Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongx- iang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. In Proceedings of the 17th ACM Conference on Recom- mender Systems. 1126–1132
2023
-
[5]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186
2019
-
[6]
Michael D Ekstrand, John T Riedl, Joseph A Konstan, et al. 2011. Collaborative filtering recommender systems. Foundations and Trends® in Human–Computer Interaction 4, 2 (2011), 81–173
2011
-
[7]
Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In Proceedings of the 16th ACM conference on recommender systems. 299–315
work page 2022
-
[8]
Naicheng Guo, Hongwei Cheng, Qianqiao Liang, Linxun Chen, and Bing Han
Show all 44 references
-
[9]
Muskan Gupta, Priyanka Gupta, Jyoti Narwariya, Lovekesh Vig, and Gautam Shroff. 2024. SCM4SR: Structural Causal Model-based Data Augmentation for Robust Session-based Recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in I...
2024
-
[10]
Muskan Gupta, Priyanka Gupta, and Lovekesh Vig. 2024. Guided Diffusion-based Counterfactual Augmentation for Robust Session-based Recommendation. arXiv preprint arXiv:2410.21892 (2024)
2024 arXiv
-
[11]
Priyanka Gupta, Diksha Garg, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff
-
[12]
Priyanka Gupta, Ankit Sharma, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. 2021. Causer: Causal session-based recommendations for handling popu- larity bias. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3048–3052
2021
-
[13]
Ruining He and Julian McAuley. 2016. Fusing similarity models with markov chains for sparse sequential recommendation. In 2016 IEEE 16th international conference on data mining (ICDM) . IEEE, 191–200
2016
-
[14]
Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian McAuley. 2023. Large language models as zero-shot conversational recommenders. InProceedings of the 32nd ACM international conference on informatio...
2023
-
[15]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk
-
[16]
Yupeng Hou, Binbin Hu, Zhiqiang Zhang, and Wayne Xin Zhao. 2022. Core: simple and effective session-based recommendation within consistent represen- tation space. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval...
2022
-
[17]
Guoqing Hu, An Zhang, Shuo Liu, Zhibo Cai, Xun Yang, and Xiang Wang. 2025. Alphafuse: Learn id embeddings for sequential recommendation in null space of language embeddings. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Informatio...
2025
-
[18]
Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. 2024. Enhancing sequential recommendation via llm-based semantic embedding learning. In Companion Proceedings of the ACM Web Conference 2024. 103–111
2024
-
[19]
Mohsen Jamali and Martin Ester. 2010. A matrix factorization technique with trust propagation for recommendation in social networks. In Proceedings of the fourth ACM conference on Recommender systems . 135–142
2010
-
[20]
Jianchao Ji, Zelong Li, Shuyuan Xu, Wenyue Hua, Yingqiang Ge, Juntao Tan, and Yongfeng Zhang. 2024. Genrec: Large language model for generative recommen- dation. In European Conference on Information Retrieval . Springer, 494–502
2024
-
[21]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In 2018 IEEE international conference on data mining (ICDM) . IEEE, 197–206
2018
-
[22]
Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management . 1419–1428
2017
-
[23]
Xianming Li and Jing Li. 2023. Angle-optimized text embeddings. arXiv preprint arXiv:2309.12871 (2023)
2023 arXiv
-
[24]
Qidong Liu, Xian Wu, Wanyu Wang, Yejing Wang, Yuanshao Zhu, Xiangyu Zhao, Feng Tian, and Yefeng Zheng. 2025. LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 39...
2025
-
[25]
Qidong Liu, Xian Wu, Yejing Wang, Zijian Zhang, Feng Tian, Yefeng Zheng, and Xiangyu Zhao. 2024. Llm-esr: Large language models enhancement for long- tailed sequential recommendation. Advances in Neural Information Processing Systems 37 (2024), 26701–26727
2024
-
[26]
Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 2018. STAMP: short- term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD international conference on knowledge dis- covery & data mining . 1831–1839
2018
-
[27]
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2023. Recent advances in natural language processing via large pre-trained language models: A survey. Comput. Surveys 56, 2 (2023), 1–40
2023
-
[28]
Shutong Qiao, Chen Gao, Junhao Wen, Wei Zhou, Qun Luo, Peixuan Chen, and Yong Li. 2024. LLM4SBR: A Lightweight and Effective Framework for Integrat- ing Large Language Models in Session-based Recommendation. arXiv preprint arXiv:2402.13840 (2024)
2024 arXiv
-
[29]
Shutong Qiao, Wei Zhou, Junhao Wen, Hongyu Zhang, and Min Gao. 2023. Bi-channel multiple sparse graph attention networks for session-based recom- mendation. In Proceedings of the 32nd ACM international conference on information and knowledge management. 2075–2084
2023
-
[30]
Scott Sanner, Krisztian Balog, Filip Radlinski, Ben Wedin, and Lucas Dixon
-
[31]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)
2023 arXiv
-
[32]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)
2017
-
[33]
Meirui Wang, Pengjie Ren, Lei Mei, Zhumin Chen, Jun Ma, and Maarten De Rijke
-
[34]
Shoujin Wang, Liang Hu, Yan Wang, Xiangnan He, Quan Z Sheng, Mehmet A Orgun, Longbing Cao, Francesco Ricci, and Philip S Yu. 2021. Graph learning based recommender systems: A review. arXiv preprint arXiv:2105.06339 (2021)
2021 arXiv
-
[35]
Ziyan Wang, Yingpeng Du, Zhu Sun, Haoyan Chua, Kaidong Feng, Wenya Wang, and Jie Zhang. 2024. Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation. arXiv preprint arXiv:2403.16427 (2024)
2024 arXiv
-
[36]
Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based Recommendation with Graph Neural Networks. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence
2019
-
[37]
Yunjia Xi, Weiwen Liu, Jianghao Lin, Xiaoling Cai, Hong Zhu, Jieming Zhu, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. 2024. Towards open-world recommendation with knowledge augmentation from large language models. In Proceedings of the 18th ACM Conference on Recommender ...
2024
-
[38]
In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval
A collaborative session-based recommendation approach with parallel memory modules. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval . 345–354
-
[39]
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223 1, 2 (2023)
2023 arXiv
-
[43]
Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive learning for sequential recommendation. In 2022 IEEE 38th international conference on data engineering (ICDE) . IEEE, 1259– 1273
2022
-
[2015]
arXiv preprint arXiv:1511.06939 (2015)
Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015)
2015 arXiv
-
[2019]
arXiv preprint arXiv:1909.04276 (2019)
NISER: Normalized Item and Session Representations with Graph Neural Networks. arXiv preprint arXiv:1909.04276 (2019)
2019 arXiv
-
[2023]
In Proceedings of the 17th ACM conference on recommender systems
Large language models are competitive near cold-start recommenders for language-and item-based preferences. In Proceedings of the 17th ACM conference on recommender systems. 890–896
-
[2024]
arXiv preprint arXiv:2402.16539 (2024)
Integrating Large Language Models with Graphical Session-Based Recom- mendation. arXiv preprint arXiv:2402.16539 (2024)
2024 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.