Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Exploring the Potential of LLMs for Serendipity Evaluation in Recommender Systems

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Plain prompted LLMs match or beat proxy metrics for judging recommendation serendipity.

desk verdict Useful systematic benchmark of LLM serendipity evaluators, but the headline 21.5% is test-set selection and the movie gold standard is an unexpectedness proxy. read the letter →

arxiv 2507.17290 v1 pith:PG73P64T submitted 2025-07-23 cs.IR

Li Kang , Yuhan Zhao , Li Chen This is my paper
classification cs.IR
keywords recommendersystemsserendipitylargelanguagemodelsLLMevaluationmeta-evaluationproxymetricsuserstudiesmulti-LLMensemble
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that large language models can evaluate recommendation serendipity—whether a suggested item is an unexpected but pleasant surprise—as well as or better than the formula-based proxy metrics now used in place of user studies. The authors compare LLM ratings, produced by prompting models with a user's behavior history, against ratings from two real user studies and against four conventional proxy metrics. They report that even zero-shot LLMs match or surpass the best proxy metrics, and that adding auxiliary data and averaging scores across several LLMs raises the correlation with user-study ratings to about 21.5%. If correct, this gives researchers a cheap, reproducible way to measure serendipity without running costly user studies.

What carries the argument

The central object is SerenEva, a meta-evaluation framework that measures how well any candidate evaluator aligns with user-study ground truth. SerenEva feeds a constrained prompt—user behavior history, item information, and a 5-point Likert question about serendipity—to an LLM, then compares the LLM's numeric ratings with real user ratings using Pearson correlation, MAE, and RMSE; proxy metrics are first rescaled to the same 5-point Likert format. The framework also varies the auxiliary data injected into the prompt (user demographics and psychology, item popularity and similarity, interaction length and type) and supports multi-LLM ensembles that average scores across models. The load-bearing comparisons are the four proxy metrics (SOG, PURS, DESR, SNPR) and the two user-study datasets treated as gold standard.

What would settle it

Run the same SerenEva evaluation on a dataset where users directly rate serendipity, or where the gold standard is independently validated by a second user study; if LLM correlations against that direct gold standard fall below the proxy metrics, or below the 21.5% reported here, the paper's central claim fails. A simpler check is to recompute the Serendipity-2018 gold standard from all eight original survey statements instead of the three unexpectedness variables and see whether the LLM-versus-proxy ranking survives.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that serendipity evaluation can be performed by LLMs with no task-specific training. In a meta-evaluation across an e-commerce dataset (Taobao) and a movie dataset (Serendipity-2018), the authors find that a zero-shot GPT-4 or Qwen2.5-72B prompt reaches Pearson correlations of roughly 10.5–10.9% against user-study ratings, about double the 4.6–5.7% of the strongest proxy metric (SOG). Few-shot prompting improves most models further, and combining auxiliary data with a score-averaging ensemble of Qwen2.5-14B, Qwen2.5-72B, and GPT-4 yields Pearson correlations of 20.23% (Taobao) and 21.51% (Serendipity-2018), with correspondingly lower MAE and RMSE. The paper interprets these results as evidence that LLMs can combine the accuracy of user studies with the efficiency of proxy metrics.

Load-bearing premise

The load-bearing premise is that the user-study ratings used as gold standard genuinely measure serendipity; in the Taobao dataset that means a single 'pleasant surprise' Likert response, and in Serendipity-2018 an average of three unexpectedness-related survey variables rather than a direct serendipity rating.

Editorial extensions

If this is right

  • Zero-shot or few-shot LLM prompts can serve as a drop-in replacement for proxy metrics when user studies are infeasible, giving similar or better alignment with human judgments.
  • Including a small amount of user-study data as few-shot examples lets smaller models like Qwen2.5-7B approach the accuracy of much larger models, lowering cost.
  • Auxiliary data helps only when matched to the domain: curiosity data matters for e-commerce, item popularity and long-term profiles matter for movies, and indiscriminate additions can hurt.
  • Multi-LLM ensembles with simple score averaging improve alignment with human ratings, so evaluation systems can trade API cost for accuracy by selecting diverse models.
  • LLM-based serendipity evaluation could make serendipity research more reproducible, since prompts and models are fixed and rerunnable, unlike one-off user studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported correlations are modest in absolute terms (around 0.2), so the practical claim is not that LLMs capture serendipity fully, but that they capture it as well as any existing cheap proxy; a reader should interpret 'match or surpass' relative to weak baselines.
  • Because the gold standard itself contains noise—the Taobao label is a single 'pleasant surprise' item and Serendipity-2018 averages three unexpectedness variables—the true human-alignment ceiling may be higher or lower than 21.5%; a direct serendipity-rating study would be needed to know.
  • The same SerenEva protocol could be applied to other subjective evaluation targets—novelty, diversity, trust, explanation quality—where proxy metrics are known to diverge from user perception.
  • The finding that multi-LLM averaging helps suggests that ensemble diversity, not raw model size, drives alignment; future work could test whether a small set of deliberately diverse open-weight models matches the GPT-4-inclusive ensemble.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper proposes SerenEva, a meta-evaluation framework for assessing whether large language models (LLMs) can serve as serendipity evaluators in recommender systems. The authors compare zero-shot and few-shot LLM ratings against two user-study-derived gold standards (Taobao e-commerce and MovieLens/Serendipity-2018), benchmark them against four conventional proxy metrics (SOG, SNPR, PURS, DESR), investigate the effect of auxiliary user, item, and interaction data, and evaluate multi-LLM ensembles with score averaging. The main reported findings are that even basic zero-shot LLMs match or surpass proxy metrics, that auxiliary data improve performance in a domain-dependent manner, and that the best multi-LLM configuration reaches Pearson correlations of 20.23% (Taobao) and 21.51% (Serendipity-2018) with the user-study ground truth. The paper provides its code and uses publicly available models.

Significance. The study addresses a timely and practical question: whether LLMs can be inexpensive, reproducible evaluators of a subjective quantity like serendipity. The strengths of the manuscript are its clear meta-evaluation protocol, the use of real user-study data, the inclusion of several LLM families and sizes, and the public release of code. The zero-shot parity claim is supported by the results in Table 2, and the analysis of auxiliary data is a useful mapping of when such data help. However, the headline 'optimal' result is based on test-set selection, and the movie-domain gold standard is an unexpectedness composite rather than direct serendipity; both issues temper the strength of the central claims as currently presented.

major comments (3)
  1. [Section 5, Table 5] The reported headline Pearson correlations (20.23% and 21.51%) are the maximum over a grid search of auxiliary data types and multi-LLM ensemble compositions evaluated on the same test set used to compute the correlations. No held-out validation or multiple-testing correction is described; this makes the optimal performance estimate optimistically biased. Please separate configuration selection from final evaluation (e.g., with a validation split) and temper the abstract and conclusions accordingly.
  2. [Section 2.2] The Serendipity-2018 gold standard is not a direct user serendipity rating; the paper states that the dataset 'lacks direct user ratings on serendipity' and that the score was computed by averaging three unexpectedness-related variables. Since the LLM prompt asks for 'pleasant surprise', the correlations on this dataset measure alignment with an unexpectedness composite, which is a construct mismatch. Please justify this operationalization, discuss its validity, or provide sensitivity analyses with alternative gold-standard definitions.
  3. [Section 2.5] The paper reports only results that are statistically significant at p < 0.05 in a two-sided t-test, without any correction for the many models, prompts, auxiliary-data conditions, and ensemble configurations tested. Selective reporting of significant results inflates effect sizes and false-positive rates. Please report the full result matrix or apply a multiple-comparison correction, and distinguish exploratory from confirmatory findings.
minor comments (5)
  1. [Figure 5 caption] The third panel caption contains a typo: 'Nummber' should be 'Number'.
  2. [Section 2.2] The sentence 'Because user studies provide real user feedback, making them the closest approximation to actual user perceptions. Therefore, we regard these results as the gold standard' is a fused sentence; consider splitting or rephrasing.
  3. [Section 2.5, Eq. (6)] The min-max normalization uses the distribution of outputs over the full evaluation set; this is acceptable for correlation but should be clarified for MAE/RMSE, since the normalized scores depend on the test set's min and max.
  4. [Section 4.1] The paper states that Big-Five traits, age, and gender 'did not help enhance performance and even reduced it'; providing the actual numbers (even in an appendix) would strengthen this negative result.
  5. [Section 5] The claim that 'multi-LLM techniques show performance improvements as the number of LLMs increases' is based on a limited set of ensembles; a direct comparison of ensemble sizes with error bars would clarify the trend.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: LLM ratings are genuine model outputs compared against external user-study gold standards; minor self-citation overlap and grid-search selection are caveats, not circularity.

full rationale

The claimed derivation chain is not circular. The LLM evaluator outputs are real model predictions obtained from prompts (Section 2.4); no model parameter or prompt is fitted to the gold-standard serendipity labels. The meta-evaluation in Eq. (5) and the Likert rescaling in Eq. (6) are standard comparison operations, not identities that define the LLM scores in terms of the user-study scores. The proxy metrics are independently defined in Section 2.3 and then transformed for comparison. The Taobao and Serendipity-2018 gold standards come from external user studies, and the paper transparently states that Serendipity-2018 lacks direct serendipity ratings and uses an unexpectedness composite 'as validated in [31]'. Because [31] and [5] share author Li Chen, there is a legitimate independence caveat, and the composite may measure unexpectedness rather than serendipity, but this is a construct-validity limitation of the input data, not a reduction of the paper's conclusions to its own assumptions. Similarly, the headline 21.5% Pearson value is the best result of a grid search over multi-LLM and auxiliary-data combinations (Section 5), so it is a selected maximum rather than a held-out prediction; this is post-selection optimism, not circularity. The central comparative finding—that zero-shot/few-shot LLMs match or beat conventional proxy metrics—is evaluated against the same external labels for both sides and therefore has independent content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper is an empirical meta-evaluation, not a derivation, so free parameters are experimental choices rather than fitted constants. The main threat is that the best-performing configuration is selected on the test set via grid search, making the headline 21.5% an optimistic maximum rather than an unbiased estimate. The axioms are domain assumptions about the validity of user-study labels and the comparability of transformed metrics.

free parameters (4)
  • Auxiliary data configuration = Taobao: curiosity; Serendipity-2018: popularity
    Selected by grid search over auxiliary data types using the user-study test labels (Section 4, Table 5). This selection inflates the reported performance because the same data are used to choose and to evaluate the configuration.
  • Multi-LLM ensemble composition = Qwen2.5-14B, Qwen2.5-72B, GPT-4
    Selected by grid search over all model combinations on the test set (Section 5, Table 5). The best result of 21.5% is the maximum over searched configurations, not a held-out estimate.
  • Few-shot example count = 5
    Default configuration in RQ2 and RQ3; the choice of five examples is not justified by a separate validation step (Section 4).
  • Interaction history length = 10, with 20 for the Taobao interaction-type experiment
    The default top-k recent interactions is used throughout, and the paper reports that shorter histories generally perform better, but the default itself is an arbitrary modeling choice (Section 4.3).
assumptions (5)
  • domain assumption User-study serendipity ratings in Taobao and Serendipity-2018 are valid ground truth for serendipity.
    Section 2.2 treats these ratings as the gold standard. If they are noisy or inconsistent, all correlation comparisons lose meaning.
  • domain assumption Pearson correlation, MAE, and RMSE between predicted and reported ratings are appropriate measures of evaluator quality.
    Section 2.5 defines SerenEva with these metrics. Pearson only captures linear agreement and can be low even when ranks are reasonable.
  • domain assumption Min-max normalization of proxy metric outputs to a 5-point scale (Eq. 6) yields a fair comparison with LLM ratings.
    Section 2.5 applies this transformation. It is monotonic so Pearson correlation is invariant, but it changes MAE and RMSE and assumes all metrics are comparable after rescaling.
  • domain assumption LLM ratings generated by constrained prompts are meaningful psychometric responses, not artifacts of prompt wording or output format.
    The paper relies on LLM outputs as user-simulation ratings without calibration or validation against human response distributions.
  • domain assumption The two datasets, e-commerce and movies, are representative enough to support general conclusions about LLM serendipity evaluation.
    Section 2.2 selects these domains for diversity, but the results are domain-dependent and only two domains are tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring the Potential of LLMs for Serendipity Evaluation in Recommender Systems." pith.science (2026). https://pith.science/paper/PG73P64T

@misc{pith2026250717290,
  author       = {Pith},
  title        = {Pith review of: Exploring the Potential of LLMs for Serendipity Evaluation in Recommender Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PG73P64T}},
  note         = {Machine review of arXiv:2507.17290}
}
read the original abstract

Serendipity plays a pivotal role in enhancing user satisfaction within recommender systems, yet its evaluation poses significant challenges due to its inherently subjective nature and conceptual ambiguity. Current algorithmic approaches predominantly rely on proxy metrics for indirect assessment, often failing to align with real user perceptions, thus creating a gap. With large language models (LLMs) increasingly revolutionizing evaluation methodologies across various human annotation tasks, we are inspired to explore a core research proposition: Can LLMs effectively simulate human users for serendipity evaluation? To address this question, we conduct a meta-evaluation on two datasets derived from real user studies in the e-commerce and movie domains, focusing on three key aspects: the accuracy of LLMs compared to conventional proxy metrics, the influence of auxiliary data on LLM comprehension, and the efficacy of recently popular multi-LLM techniques. Our findings indicate that even the simplest zero-shot LLMs achieve parity with, or surpass, the performance of conventional metrics. Furthermore, multi-LLM techniques and the incorporation of auxiliary data further enhance alignment with human perspectives. Based on our findings, the optimal evaluation by LLMs yields a Pearson correlation coefficient of 21.5\% when compared to the results of the user study. This research implies that LLMs may serve as potentially accurate and cost-effective evaluators, introducing a new paradigm for serendipity evaluation in recommender systems.

Figures

Figures reproduced from arXiv: 2507.17290 by the authors.

Figure 1
Figure 1. Comparison w.r.t. Pearson correlation coefficient [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Comparison w.r.t. Pearson correlation coefficient, MAE, and RMSE for different profile types (“S-prof.” stands for [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison w.r.t. Pearson correlation coefficient, MAE, and RMSE for different item data types (“Sim” stands for item [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison w.r.t. Pearson correlation coefficient, MAE, and RMSE with varying interaction history length across the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparison w.r.t. Pearson correlation coefficient, MAE, and RMSE regarding different ensemble sizes on the Taobao. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison w.r.t. Pearson correlation coefficient, MAE, and RMSE regarding different ensemble sizes on the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 36 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, et al. 2023. GPT-4 Technical Report. arXiv (2023)

    work page 2023

  2. [2]

    Zahra Ashktorab, Michael Desmond, Qian Pan, James M Johnson, Martin Santillan Cooper, Elizabeth M Daly, Rahul Nair, Tejaswini Pedapati, Swapnaja Achintalwar, and Werner Geyer. 2024. Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences. arXiv preprint arXiv:2410.00873 (2024)

    arXiv 2024

  3. [3]

    Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexan- der Koller, et al. 2024. LLMs Instead of Human Judges? A Large-Scale Empirical Study Across 20 NLP Evaluation Tasks. arXiv preprint arXiv:2406.18403 (2024)

    arXiv 2024

  4. [4]

    Brett Binst. 2024. How to Evaluate Serendipity in Recommender Systems: the Need for a Serendiptionnaire. In RecSys. 1335–1341

    work page 2024

  5. [5]

    Li Chen, Yonghua Yang, Ningxia Wang, Keping Yang, and Quan Yuan. 2019. How Serendipity Improves User Satisfaction with Recommendations? A Large-Scale User Evaluation. In WWW. 240–250

    work page 2019

  6. [6]

    Zhe Fu and Xi Niu. 2023. Modeling Users’ Curiosity in Recommender Systems. TKDD 18, 1 (2023), 1–23

    work page 2023

  7. [7]

    Zhe Fu and Xi Niu. 2024. The Art of Asking: Prompting Large Language Models for Serendipity Recommendations. In ICTIR. 157–166

    work page 2024

  8. [8]

    Zhe Fu, Xi Niu, and Mary Lou Maher. 2023. Deep Learning Models for Serendipity Recommendations: A Survey and New Perspectives. Comput. Surveys 56, 1 (2023), 1–26

    work page 2023

Show all 46 references
  1. [9]

    Zhe Fu, Xi Niu, Xiangcheng Wu, and Ruhani Rahman. 2025. A Deep Learning Model for Cross-Domain Serendipity Recommendations. TORS 3, 3 (2025), 1–21

    2025

  2. [10]

    Zhe Fu, Xi Niu, and Li Yu. 2023. Wisdom of Crowds and Fine-grained Learning for Serendipity Recommendations. In SIGIR. 739–748

    2023

  3. [11]

    Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. 2003. A Very Brief Measure of the Big-Five Personality Domains. Journal of Research in personality 37, 6 (2003), 504–528

    2003

  4. [12]

    Tonmoy Hasan and Razvan Bunescu. 2023. Topic-Level Bayesian Surprise and Serendipity for Recommender Systems. In RecSys. 933–939

    2023

  5. [13]

    Todd B Kashdan, Matthew W Gallagher, Paul J Silvia, Beate P Winterstein, William E Breen, Daniel Terhar, and Michael F Steger. 2009. The Curiosity And Exploration Inventory-II: Development, Factor Structure, And Psychometrics. Journal of research in personality 43, 6 (2009), 987–998

    2009

  6. [14]

    Denis Kotkov, Joseph A Konstan, Qian Zhao, and Jari Veijalainen. 2018. Investi- gating Serendipity in Recommender Systems Based on Real User Feedback. In SAC. 1341–1350

    2018

  7. [15]

    Denis Kotkov, Alan Medlar, and Dorota Glowacka. 2023. Rethinking Serendipity in Recommender Systems. In CHIIR. 383–387

    2023

  8. [16]

    Denis Kotkov, Alan Medlar, Triin Kask, and Dorota Glowacka. 2024. The dark matter of serendipity in recommender systems. In CHIIR. 108–118

    2024

  9. [17]

    Denis Kotkov, Jari Veijalainen, and Shuaiqiang Wang. 2020. How Does Serendip- ity Affect Diversity in Recommender Systems? A Serendipity-Oriented Greedy Algorithm. Computing 102 (2020), 393–411

    2020

  10. [18]

    Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. LLMs-as-Judges: A Comprehensive Survey on LLM-Based Evaluation Methods. arXiv preprint arXiv:2412.05579 (2024)

    2024 arXiv

  11. [19]

    Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2023. LooGLE: Can Long-Context Language Models Understand Long Contexts? arXiv preprint arXiv:2311.04939 (2023)

    2023 arXiv

  12. [20]

    Pan Li, Maofei Que, Zhichao Jiang, Yao Hu, and Alexander Tuzhilin. 2020. PURS: Personalized Unexpected Recommender System for Improving User Satisfaction. In RecSys. 279–288

    2020

  13. [21]

    Xueqi Li, Wenjun Jiang, Weiguang Chen, Jie Wu, Guojun Wang, and Kenli Li. 2020. Directional and Explainable Serendipity Recommendation. In WWW. 122–132

    2020

  14. [22]

    Xiang Liu, Peijie Dong, Xuming Hu, and Xiaowen Chu. 2024. Longgenbench: Long-context Generation Benchmark. arXiv preprint arXiv:2410.04199 (2024)

    2024 arXiv

  15. [23]

    Gaurav Pandey, Denis Kotkov, and Alexander Semenov. 2018. Recommending Serendipitous Items Using Transfer Learning. In CIKM. 1771–1774

    2018

  16. [24]

    Bhrij Patel, Souradip Chakraborty, Wesley A Suttle, Mengdi Wang, Amrit Singh Bedi, and Dinesh Manocha. 2024. AIME: AI System Optimization via Multiple LLM Evaluators. arXiv preprint arXiv:2410.03131 (2024)

    2024

  17. [25]

    Pearl Pu, Li Chen, and Rong Hu. 2011. A User-Centric Evaluation Framework for Recommender Systems. In Proceedings of the fifth ACM conference on Recom- mender systems. 157–164

    2011

  18. [26]

    Yu Tokutake and Kazushi Okamoto. 2024. Can Large Language Models Assess Serendipity in Recommender Systems? JACIII 28, 6 (2024), 1263–1272

    2024

  19. [27]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288 (2023)

    2023 arXiv

  20. [28]

    Yu-Min Tseng, Wei-Lin Chen, Chung-Chi Chen, and Hsin-Hsi Chen. 2024. Are Expert-Level Language Models Expert-Level Annotators? arXiv preprint arXiv:2410.03254 (2024)

    2024 arXiv

  21. [29]

    Jianling Wang, Haokai Lu, Yifan Liu, He Ma, Yueqi Wang, Yang Gu, Shuzhou Zhang, Ningren Han, Shuchao Bi, Lexi Baugher, et al. 2024. LLMs for User Interest Exploration in Large-Scale Recommendation Systems. In RecSys. 872–877

    2024

  22. [30]

    Lei Wang, Jingsen Zhang, Hao Yang, Zhi-Yuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Hao Sun, Ruihua Song, et al. 2025. User Behavior Simulation with Large Language Model-Based Agents. TOIS 43, 2 (2025), 1–37

    2025

  23. [31]

    Ningxia Wang and Li Chen. 2023. How Do Item Features and User Characteristics Affect Users’ Perceptions of Recommendation Serendipity? A Cross-Domain Analysis. UMUAI 33, 3 (2023), 727–765

    2023

  24. [32]

    Ningxia Wang, Li Chen, et al . 2020. The Impacts of Item Features and User Characteristics on Users’ Perceived Serendipity of Recommendations. In UMAP

    2020

  25. [33]

    Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do Llamas Work in English? On the Latent Language of Multilingual Transformers. In ACL. 15366–15394

    2024

  26. [34]

    Yunjia Xi, Muyan Weng, Wen Chen, Chao Yi, Dian Chen, Gaoyang Guo, Mao Zhang, Jian Wu, Yuning Jiang, Qingwen Liu, et al. 2025. Bursting Filter Bubble: Enhancing Serendipity Recommendations with Aligned Large Language Models. arXiv preprint arXiv:2502.13539 (2025)

    2025 arXiv

  27. [35]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2.5 Technical Report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  28. [36]

    Mingwei Zhang, Yang Yang, Rizwan Abbas, Ke Deng, Jianxin Li, and Bin Zhang

  29. [37]

    Xiaoyu Zhang, Yishan Li, Jiayin Wang, Bowen Sun, Weizhi Ma, Peijie Sun, and Min Zhang. 2024. Large Language Models as Evaluators for Recommendation Explanations. In RecSys. 33–42

    2024

  30. [38]

    Jun Zhao, Zhihao Zhang, Luhui Gao, Qi Zhang, Tao Gui, and Xuanjing Huang

  31. [39]

    Pengfei Zhao and Dik Lun Lee. 2016. How Much Novelty Is Relevant? It Depends on Your Curiosity. In SIGIR. 315–324

    2016

  32. [40]

    Yuhan Zhao, Rui Chen, Li Chen, Shuang Zhang, Qilong Han, and Hongtao Song

  33. [41]

    Yuhan Zhao, Rui Chen, Qilong Han, Hongtao Song, and Li Chen. 2024. Unlocking the Hidden Treasures: Enhancing Recommendations with Unlabeled Data. In RecSys. 247–256

    2024

  34. [42]

    Lixi Zhu, Xiaowen Huang, and Jitao Sang. 2024. How Reliable Is Your Simu- lator? Analysis on the Limitations of Current LLM-Based User Simulators for Conversational Recommendation. In WWW. 1726–1732

    2024

  35. [43]

    Reza Jafari Ziarani and Reza Ravanmehr. 2021. Serendipity in Recommender Systems: A Systematic Literature Review. Journal of Computer Science and Technology 36 (2021), 375–396

    2021

  36. [2021]

    SNPR: A Serendipity-Oriented Next POI Recommendation Model. In CIKM. 2568–2577

  37. [2024]

    arXiv preprint arXiv:2401.01055 (2024)

    Llama Beyond English: An Empirical Study on Language Capability Trans- fer. arXiv preprint arXiv:2401.01055 (2024)

    2024 arXiv

  38. [2025]

    In AAAI, Vol

    From Pairwise to Ranking: Climbing the Ladder to Ideal Collaborative Filtering with Pseudo-Ranking. In AAAI, Vol. 39. 13392–13400

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.