Pith. sign in

REVIEW 4 major objections 5 minor 5 cited by

AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AgentRecBench is a new benchmark that reports LLM-agent recommender systems outperform traditional methods and yields design guidelines for building them.

desk verdict The benchmark environment is a solid artifact, but the central comparison is broken: the traditional baselines in the classic/cold-start tables are random-chance constant predictions, not trained models. read the letter →

arxiv 2505.19623 v2 pith:XMOIFRA2 submitted 2025-05-26 cs.IR cs.AI

Yu Shang , Peijie Liu , Yuwei Yan , Zijing Wu , Leheng Sheng , Yuanqing Yu , Chumeng Jiang , An Zhang
show 4 more authors
Fengli Xu Yu Wang Min Zhang Yong Li
This is my paper
classification cs.IRcs.AI
keywords agenticrecommendersystemsLLMagentsrecommendationbenchmarktextualenvironmentsimulatorcold-startevolvinginterestshitrate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AgentRecBench is a proposed benchmark for LLM-based agentic recommender systems, built on a textual interaction simulator over Yelp, GoodReads, and Amazon data. The paper claims to provide the first comprehensive, standardized evaluation that compares eight agentic systems against traditional methods across classic, evolving-interest, and cold-start scenarios. Its central finding is that well-designed agentic systems, especially those using platform-specific item and review feature engineering, outperform simpler agents and classical baselines, and it distills this into actionable design guidelines. A sympathetic reader would care because agentic recommenders are a fast-growing approach that currently lacks a common testbed, so a validated benchmark would let the field measure progress and transfer lessons across methods.

What carries the argument

The load-bearing object is the interactive textual environment simulator, which models a platform as a User-Review-Item network and exposes a standardized query function Query(Type, SortMethod, Formation) that agents call to retrieve structured or textual data. A two-layer dynamic data visibility control (scenario-level filters over time and item properties, task-level target user and ground truth) makes the same environment serve classic, evolving-interest, and cold-start evaluations. This machinery is what makes fair, reproducible comparison possible: every tested agent sees the same queryable world and the same 20-candidate ranking tasks, scored by Hit Rate@N. The modular agent framework (planning, reasoning, tool use, memory) is the accompanying toolkit for building the agents being compared.

What would settle it

Train MF and LightGCN on the full classic-task training data (or a better-tuned subset) and measure HR@N on the same 20-candidate test set; if their scores rise well above the reported 15.0 chance level, the claim of agentic superiority in the classic scenario is empirically wrong as stated.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that AgentRecBench constitutes a reliable and comprehensive benchmark for agentic recommender systems, and that on it the best-performing agents (Baseline666, DummyAgent, and RecHackers) consistently surpass simpler agent designs and classical methods across domains and scenarios. The paper establishes a unified textual interaction environment with dynamic data visibility control, and shows through an open challenge with 295 teams that the benchmark is usable and results are reproducible. From the performance differences it derives design guidelines: effective agent workflows combine user history, candidate item details, and platform-specific features; they extract representative domain-adapted item attributes; and they prioritize information-rich reviews. The paper asserts these findings demonstrate the superiority of agentic systems and provide actionable guidance for building them.

Load-bearing premise

The load-bearing premise is that the traditional baselines, trained on a small subset of classic-task data and unable to learn there, fairly represent what classical recommenders can do in this benchmark.

Editorial extensions

If this is right

  • If AgentRecBench is right, agentic recommender systems can be compared on a common, reproducible testbed, ending the field's reliance on ad-hoc evaluations.
  • The design guidelines, such as platform-aware item feature extraction and review-side engineering, give concrete starting points for building new recommendation agents, not just tuning prompts.
  • The three-scenario design (classic, evolving-interest, cold-start) means an agent's robustness to distribution shift and data sparsity becomes measurable alongside raw accuracy.
  • The benchmark's validation through a 295-team open challenge suggests the environment is usable by outside researchers and its leaderboard can track progress over time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that training MF and LightGCN on the full classic-task dataset rather than a small subset could change the headline comparison, since the paper itself reports these models reach substantive accuracy in the evolving-interest scenario, suggesting the classic-task chance-level results reflect the training subset, not the methods.
  • The winning agents' heavy reliance on platform-specific features implies that 'general' agentic recommendation may actually be domain-customized ranking with LLM glue, a nuance worth stating for practitioners.
  • One could extend the benchmark to include traditional models trained on the full data, as well as more deep-learning baselines, and re-run the same three scenarios to test the generality of the claimed superiority.
  • The benchmark's textual-only environment leaves open whether the design guidelines transfer to multimodal platforms (e.g., image-based recommendations), which the paper itself lists as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces AgentRecBench, a benchmark for evaluating LLM-agent-based personalized recommender systems. It provides an interactive textual environment built from Yelp, GoodReads, and Amazon data, with three evaluation scenarios (classic, evolving-interest, and cold-start) and a modular agent framework. The authors evaluate eight agentic systems plus two conventional baselines (MF and LightGCN), and claim that agentic systems are superior and that the results yield actionable design guidelines for constructing recommendation agents.

Significance. If the empirical claims were valid, AgentRecBench would be a useful community resource: the environment design, dataset integration, scenario taxonomy, and the open challenge are valuable contributions, and the released benchmark with leaderboard could foster reproducible research. However, the central comparative claims are not supported by the evidence presented. The conventional baselines in two of the three scenarios are reported as a constant 15.0 HR@N, which is exactly the random-chance expectation, while the same models attain strong results in the third scenario. The claimed 'superiority of agentic systems' therefore rests on comparison against a random baseline in the classic and cold-start tasks, and is actively contradicted by the evolving-interest results. The design guidelines are also distilled from the winners of a challenge run on this same benchmark, creating an in-sample circularity that undermines their generality.

major comments (4)
  1. [Tables 2 and 3; Section 5.1] The 'Traditional RS' (MF) and 'DL-based RS' (LightGCN) rows in Tables 2 and 3 report a constant value of 15.0 for every dataset and scenario. This value is precisely the expected Hit Rate for a uniformly random ranking over 20 candidate items: HR@1=5%, HR@3=15%, HR@5=25%, and their average is 15%. Section 5.1 confirms that these models 'are unable to effectively learn meaningful patterns' and that 'we report the mean prediction as a reference point for comparison.' Ranking by a constant mean prediction is equivalent to a no-signal random baseline. Consequently, the paper's headline claim that agentic systems are superior to traditional methods in the classic and cold-start scenarios is unsupported: those comparisons pit agents against a dummy baseline, not against trained recommendation models.
  2. [Table 4 vs. Tables 2 and 3; Section 5.1] The same MF and LightGCN models that yield 15.0 in the classic and cold-start tasks achieve HR@N values of 32.3–59.1 on Amazon, 17.7–22.3 on GoodReads, and 31.6–68.9 on Yelp in the evolving-interest scenario (Table 4). This directly contradicts the claim that high data sparsity prevents these models from learning on the benchmark datasets. The correct interpretation is that the classic and cold-start tasks were constructed without a usable training set for collaborative filtering methods, so the chance-level results are an artifact of the evaluation protocol, not evidence of inherent advantages of agentic systems. The authors must either repair the baseline training/evaluation setup for those scenarios or refrain from making comparative superiority claims based on the current tables.
  3. [Section 4.3 and Appendix A.4] The three agents identified as 'superior' (Baseline666, DummyAgent, and RecHackers) are, respectively, the first-, third-, and second-place solutions from the AgentSociety Challenge [13], which was run using this same benchmark and its leaderboard. The design guidelines in Appendix A.4 are distilled from these agents' workflows. Because these agents were selected by competing on this exact benchmark, the guidelines may reflect benchmark-specific overfitting rather than general principles for agentic recommender systems. No out-of-sample validation is provided to show that the guidelines transfer to agents not developed on this benchmark, so the 'actionable design guidelines' claim (Contribution 3) is not established.
  4. [Appendix A.2] The limitations section states that the authors 'plan to incorporate more traditional and deep learning-based baselines for more thorough comparative analysis.' This admission is in tension with the paper's claim in the Introduction and Abstract of being 'the first comprehensive benchmark' that 'systematically evaluates both emerging agentic recommender systems and traditional recommendation methods.' A benchmark whose only two traditional baselines fail to train in two of the three scenarios cannot currently substantiate the 'comprehensive' and 'superiority' claims. The benchmark may still be a useful resource, but the present evaluation is incomplete in a way that directly affects the paper's main conclusions.
minor comments (5)
  1. [Section 4.4] The subsection heading 'Envaluation Metric' contains a typo and should read 'Evaluation Metric.'
  2. [Section 5.1] The phrase 'we report the mean prediction as a reference point' is not defined precisely; the authors should state explicitly what 'mean prediction' means operationally (e.g., ranking all items by the same predicted score, or averaging the rating predictions).
  3. [Tables 5 and 6] The supplementary tables report results for DeepSeek-V3 and GPT-4o-mini, but no statistical significance tests or variance estimates are provided for any of the tables; given the small differences between many agents (e.g., 44.7 vs. 45.6 in Table 3), such tests or repeated-run statistics would be important for the reliability of the ranking conclusions.
  4. [Table 3] In the Yelp cold-start columns, many agentic methods score at or below the chance level of 15.0 (e.g., 0.3–4.3), which may indicate a data or evaluation anomaly in that scenario; the authors should discuss this pattern, as it suggests that the Yelp cold-start task may not be functioning as intended.
  5. [Figures 1 and 2] Figures 1 and 2 are difficult to read in the preprint resolution; the text in the figures is small and the control-flow arrows are hard to follow, reducing the clarity of the environment design described in Section 3.

Circularity Check

1 steps flagged · score 6.0 of 10

The MF/LightGCN rows in Tables 2-3 are a by-construction chance value (15.0 = expected HR@N for a constant mean prediction over 20 candidates), so the paper's headline superiority claim rests on comparing agents to a dummy baseline rather than to trained traditional models.

  1. fitted input called prediction [Section 4.4 (Eq. 5), Section 5.1, Tables 2 and 3]
    "each test instance consists of 20 candidate items: one ground-truth positive item sampled from the user's interaction history and 19 negative items sampled from unobserved interactions. ... due to the high sparsity of the dataset, these models are unable to effectively learn meaningful patterns. As a result, we report the mean prediction as a reference point for comparison."

    Ranking by a constant 'mean prediction' over 20 candidates gives the ground-truth item a uniformly random rank, so the expected HR@N averaged over N=1,3,5 is (5+15+25)/3 = 15.0. Every MF and LightGCN entry in Tables 2 and 3 is exactly this by-construction value, not a trained-model output. The Abstract's 'superiority of agentic systems' and Section 5.1's 'Baseline666 consistently outperforms other methods' are therefore comparisons against a chance baseline. Table 4 shows the same MF and LightGCN models reaching 32.3-68.9 under the evolving-interest protocol, contradicting the sparsity explanation and confirming the classic-task 15.0 values are a protocol artifact rather than evidence about the models.

full rationale

The paper is a benchmark/empirical study, not a mathematical derivation, so most of its construction (environment, tasks, metric, agent framework) is self-contained and not circular. The one load-bearing step that reduces by construction is the traditional-baseline comparison: the paper admits it could not train MF/LightGCN on the classic-task subset and reports the mean prediction instead. With 20 candidates and a constant score, HR@1/3/5 average to exactly 15.0, which is why every Traditional RS and DL-based RS cell in Tables 2 and 3 is 15.0. The abstract's superiority claim is then read off this dummy baseline. The contradiction in Table 4 (the same models train to 32.3-68.9 in the evolving-interest scenario) shows the failure is protocol-induced, not a property of the models. I considered the AgentSociety Challenge self-citation ([13], overlapping authors) as a possible load-bearing self-citation, but the challenge involved 295 external teams and 1,400 submissions, so it is external, falsifiable evidence of benchmark usability rather than a circular premise. The design guidelines distilled from the top agents are post-hoc descriptions of the winning workflows; they are in-sample but not circular in the equation-reduction sense. Hence the score reflects a single, central, by-construction baseline rather than a self-citation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No fitted parameters appear in the mathematical sense; the benchmark is an empirical measurement. The hand-chosen design parameters listed above shape every reported number: the 20-item candidate pool fixes the random-chance baseline at 15.0 average HR@N, the cold-start thresholds and evolving-interest windows control task difficulty, and the averaging over N=1,3,5 changes the apparent performance. The domain assumptions about ground truth and negative sampling are standard for recommender benchmarks but are not validated here. No invented entities are introduced.

free parameters (4)
  • Candidate pool size (20 items per test instance)
    Hand-chosen; sets the scale for HR@N and the random-chance baseline (15.0 average for N=1,3,5).
  • Cold-start thresholds m (users) and n (items)
    Described as dataset-dependent in Section 4.2 but the actual values are never reported; the splits directly determine cold-start difficulty.
  • Evolving-interest time windows (3 months long-term, 1 week short-term)
    Hand-chosen; defines what counts as stable versus emerging preference.
  • HR@N averaging over N=1,3,5
    Metric aggregation choice; can hide low HR@1 performance and is not standard.
assumptions (4)
  • domain assumption Historical interactions are positive ground truth for recommendation quality.
    Each task samples the ground-truth positive from the user's interaction history (Section 4.4), assuming past interactions are desirable items to recommend.
  • domain assumption Unobserved interactions are negative.
    Nineteen negative items are sampled from unobserved interactions (Section 4.4), the standard implicit-feedback assumption.
  • domain assumption The textual simulation and query interface faithfully represent real platform retrieval.
    The environment is designed to simulate social and web platforms (Section 3), but no validation against live platform behavior is provided.
  • domain assumption LLM outputs can be reliably parsed as ranked lists.
    The evaluation depends on exact output-format compliance; prompts instruct the model to output only a list, but parse failures are not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems." pith.science (2026). https://pith.science/paper/XMOIFRA2

@misc{pith2026250519623,
  author       = {Pith},
  title        = {Pith review of: AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XMOIFRA2}},
  note         = {Machine review of arXiv:2505.19623}
}
read the original abstract

The emergence of agentic recommender systems powered by Large Language Models (LLMs) represents a paradigm shift in personalized recommendations, leveraging LLMs' advanced reasoning and role-playing capabilities to enable autonomous, adaptive decision-making. Unlike traditional recommendation approaches, agentic recommender systems can dynamically gather and interpret user-item interactions from complex environments, generating robust recommendation strategies that generalize across diverse scenarios. However, the field currently lacks standardized evaluation protocols to systematically assess these methods. To address this critical gap, we propose: (1) an interactive textual recommendation simulator incorporating rich user and item metadata and three typical evaluation scenarios (classic, evolving-interest, and cold-start recommendation tasks); (2) a unified modular framework for developing and studying agentic recommender systems; and (3) the first comprehensive benchmark comparing 10 classical and agentic recommendation methods. Our findings demonstrate the superiority of agentic systems and establish actionable design guidelines for their core components. The benchmark environment has been rigorously validated through an open challenge and remains publicly available with a continuously maintained leaderboard~\footnote[2]{https://tsinghua-fib-lab.github.io/AgentSocietyChallenge/pages/overview.html}, fostering ongoing community engagement and reproducible research. The benchmark is available at: \hyperlink{https://huggingface.co/datasets/SGJQovo/AgentRecBench}{https://huggingface.co/datasets/SGJQovo/AgentRecBench}.

Figures

Figures reproduced from arXiv: 2505.19623 by the authors.

Figure 1
Figure 1. The overall framework of our interactive textual environment simulator. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the dynamic data visibility control workflow of our textual environment [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Statistical distributions of our aggregated multi-platform dataset showing (a) user/item [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Illustration of the agentic workflow of a superior recommendation agent (Baseline666), [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustration of the agentic workflow of a superior recommendation agent (DummyAgent), [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Illustration of the agentic workflow of a superior recommendation agent (RecHacker), [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. APeB: Benchmarking Personalization Ability of Large Language Model Agents

    cs.AI 2026-07 conditional novelty 6.5 of 10

    LLM agents succeed on refined product queries but fail on early underspecified intents mainly because they underuse noisy histories; APeB measures this gap and VQRA partially closes it.

  2. Attacking and Defending Multi-Agent Collaborative Filtering Systems Through Connectivity

    cs.IR 2026-08 conditional novelty 6.0 of 10

    In agent-based collaborative filtering, attack spread and privacy leakage grow with interaction connectivity, but the effect is asymmetric between user and item agents and differs between early and steady-state phases.

  3. Reason-to-Recommend: Using Interaction-of-Thought Reasoning to Enhance LLM Recommendation

    cs.IR 2025-06 conditional novelty 6.0 of 10

    R2Rec combines graph-sampled interaction chains, masked stepwise reasoning prompts, and SFT plus GRPO training to improve LLM recommendation hit ratios.

  4. Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems

    cs.IR 2026-07 accept novelty 5.0 of 10

    Agentic recommender systems are organized by agent role (assisted, as-recommender, as-simulator) crossed with autonomy levels L2–L5, yielding a roadmap of architectures, evaluation limits, and open challenges.

  5. RecoWorld: Building Simulated Environments for Agentic Recommender Systems

    cs.IR 2025-09 conditional novelty 5.0 of 10

    A design proposal, not a tested system: a dual-view simulation loop in which an LLM-simulated user issues reflective instructions when about to disengage, and an instruction-following recommender adapts to maximize si...

Reference graph

Works this paper leans on

40 extracted references · 14 canonical work pages · cited by 5 Pith papers

  1. [13]

    Agentsociety challenge: Designing llm agents for user modeling and recommendation on web platforms

    Yuwei Yan, Yu Shang, Qingbin Zeng, Yu Li, Keyu Zhao, Zhiheng Zheng, Xuefei Ning, Tianji Wu, Shengen Yan, Yu Wang, et al. Agentsociety challenge: Designing llm agents for user modeling and recommendation on web platforms. arXiv preprint arXiv:2502.18754, 2025

    arXiv 2025

  2. [1]

    Towards next-generation llm-based recommender systems: A survey and beyond

    Qi Wang, Jindong Li, Shiqi Wang, Qianli Xing, Runliang Niu, He Kong, Rui Li, Guodong Long, Yi Chang, and Chengqi Zhang. Towards next-generation llm-based recommender systems: A survey and beyond. arXiv preprint arXiv:2410.19744, 2024

    arXiv

  3. [2]

    A Large-scale Dataset with Behavior, Attributes, and Content of Mobile Short-video Platform

    Yu Shang, Chen Gao, Nian Li, and Yong Li. A large-scale dataset with behavior, attributes, and content of mobile short-video platform. arXiv preprint arXiv:2502.05922, 2025

    work page Pith review arXiv 2025

  4. [3]

    A survey of graph neural networks for recommender systems: Challenges, methods, and directions

    Chen Gao, Yu Zheng, Nian Li, Yinfeng Li, Yingrong Qin, Jinghua Piao, Yuhan Quan, Jianxin Chang, Depeng Jin, Xiangnan He, et al. A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Transactions on Recommender Systems, 1(1):1–51, 2023

    work page 2023

  5. [4]

    Learning fine- grained user interests for micro-video recommendation

    Yu Shang, Chen Gao, Jiansheng Chen, Depeng Jin, Meng Wang, and Yong Li. Learning fine- grained user interests for micro-video recommendation. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval , pages 433–442, 2023

    work page 2023

  6. [5]

    Matrix factorization techniques for recom- mender systems

    Yehuda Koren, Robert Bell, and Chris V olinsky. Matrix factorization techniques for recom- mender systems. Computer, 42(8):30–37, 2009

    2009

  7. [6]

    Collaborative filtering recom- mender systems

    J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. Collaborative filtering recom- mender systems. In The adaptive web: methods and strategies of web personalization , pages 291–324. Springer, 2007

    work page 2007

  8. [7]

    Neural collaborative filtering

    Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web , pages 173–182, 2017

    2017

Show all 40 references
  1. [8]

    Lightgcn: Simplifying and powering graph convolution network for recommendation

    Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages...

    2020

  2. [9]

    Chat-rec: Towards interactive and explainable llms-augmented recommender system

    Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524, 2023

    2023 arXiv

  3. [10]

    A survey on large language models for recommendation

    Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al. A survey on large language models for recommendation. World Wide Web, 27(5):60, 2024

    2024

  4. [11]

    A survey on llm-powered agents for recommender systems

    Qiyao Peng, Hongtao Liu, Hua Huang, Qing Yang, and Minglai Shao. A survey on llm-powered agents for recommender systems. arXiv preprint arXiv:2502.10050, 2025

    2025 arXiv

  5. [12]

    Feature-based recommendation system

    Eui-Hong Han and George Karypis. Feature-based recommendation system. In Proceedings of the 14th ACM international conference on Information and knowledge management , pages 446–452, 2005

    2005

  6. [14]

    Towards agentic recommender systems in the era of multimodal large language models

    Chengkai Huang, Junda Wu, Yu Xia, Zixu Yu, Ruhan Wang, Tong Yu, Ruiyi Zhang, Ryan A Rossi, Branislav Kveton, Dongruo Zhou, et al. Towards agentic recommender systems in the era of multimodal large language models. arXiv preprint arXiv:2503.16734, 2025

    2025 arXiv

  7. [15]

    On generative agents in recommendation

    An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. On generative agents in recommendation. In Proceedings of the 47th international ACM SIGIR conference on research and development in Information Retrieval , pages 1807–1817, 2024

    2024

  8. [16]

    Rah! recsys–assistant–human: A human-centered recommendation framework with llm agents

    Yubo Shu, Haonan Zhang, Hansu Gu, Peng Zhang, Tun Lu, Dongsheng Li, and Ning Gu. Rah! recsys–assistant–human: A human-centered recommendation framework with llm agents. IEEE Transactions on Computational Social Systems , 2024. 10

    2024

  9. [17]

    Agentcf: Collaborative learning with autonomous language agents for recommender systems

    Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. Agentcf: Collaborative learning with autonomous language agents for recommender systems. In Proceedings of the ACM Web Conference 2024, pages 3679–3689, 2024

    2024

  10. [18]

    Understanding the planning of llm agents: A survey

    Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Understanding the planning of llm agents: A survey. arXiv preprint arXiv:2402.02716, 2024

    2024 arXiv

  11. [19]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceed- ings of the 36th annual acm symposium on user interface software and technology , pages 1–22, 2023

    2023

  12. [20]

    Towards large reasoning models: A survey of reinforced reasoning with large language models

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.09686, 2025

    2025 arXiv

  13. [21]

    Synergy-of-thoughts: Eliciting efficient reasoning in hybrid language models

    Yu Shang, Yu Li, Fengli Xu, and Yong Li. Synergy-of-thoughts: Eliciting efficient reasoning in hybrid language models. arXiv preprint arXiv:2402.02563, 2024

    2024 arXiv

  14. [22]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024

    2024

  15. [23]

    The rise and potential of large language model based agents: A survey

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. Science China Information Sciences , 68(2):121101, 2025

    2025

  16. [24]

    Agentsquare: Automatic llm agent search in modular design space

    Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. Agentsquare: Automatic llm agent search in modular design space. arXiv preprint arXiv:2410.06153, 2024

    2024 arXiv

  17. [25]

    Cognitive architec- tures for language agents

    Theodore Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas Griffiths. Cognitive architec- tures for language agents. Transactions on Machine Learning Research, 2023

    2023

  18. [26]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems , 35:24824–24837, 2022

    2022

  19. [27]

    Recmind: Large language model powered agent for recommendation

    Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Xiaojiang Huang, Yanbin Lu, and Yingzhen Yang. Recmind: Large language model powered agent for recommendation. arXiv preprint arXiv:2308.14296, 2023

    2023 arXiv

  20. [28]

    Macrec: A multi-agent collaboration framework for recommendation

    Zhefan Wang, Yuanqing Yu, Wendi Zheng, Weizhi Ma, and Min Zhang. Macrec: A multi-agent collaboration framework for recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages 2760–2764, 2024

    2024

  21. [29]

    Let me do it for you: Towards llm empowered recommendation via tool learning

    Yuyue Zhao, Jiancan Wu, Xiang Wang, Wei Tang, Dingxian Wang, and Maarten De Rijke. Let me do it for you: Towards llm empowered recommendation via tool learning. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, ...

    2024

  22. [30]

    Flow: A feedback loop framework for simultaneously enhancing recommendation and user agents

    Shihao Cai, Jizhi Zhang, Keqin Bao, Chongming Gao, and Fuli Feng. Flow: A feedback loop framework for simultaneously enhancing recommendation and user agents. arXiv preprint arXiv:2410.20027, 2024

    2024 arXiv

  23. [31]

    Recommender ai agent: Integrating large language models for interactive recommendations

    Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. Recommender ai agent: Integrating large language models for interactive recommendations. arXiv preprint arXiv:2308.16505, 2023. 11

    2023 arXiv

  24. [32]

    Leveraging large language models in conversational recommender systems

    Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, et al. Leveraging large language models in conversational recommender systems. arXiv preprint arXiv:2305.07961, 2023

    2023 arXiv

  25. [33]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in Neural Information Processing Systems , 36, 2024

    2024

  26. [34]

    Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents

    Yuan Li, Yixuan Zhang, and Lichao Sun. Metaagents: Simulating interactions of human behaviors for llm-based task-oriented coordination via collaborative generative agents. arXiv preprint arXiv:2310.06500, 2023

    2023 arXiv

  27. [35]

    V oyager: An open-ended embodied agent with large language models

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. V oyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023

    2023 arXiv

  28. [36]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  29. [37]

    Mind2web: Towards a generalist agent for the web

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36:28091–28114, 2023

    2023

  30. [38]

    Large language model based multi-agents: A survey of progress and challenges

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680, 2024

    2024 arXiv

  31. [39]

    Metagpt: Meta programming for multi-agent collaborative framework

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 3(4):6, 2023

    2023 arXiv

  32. [40]

    useful,” “cool,

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848, 2(4):6, 2023. 12 A Ap...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.