Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation

T0 review · 2 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read OMuleT's central claim is that a conversational recommender should hand a large language model a large toolbox—more than ten lookup, linking, retrieval, and filtering tools—and a fixed policy for using them; on real user requests this…

desk verdict A sensible applied multi-tool LLM recommender with a real dataset, whose headline relevance numbers are weakened by an evaluation set built from the same similarity APIs the system uses. read the letter →

arxiv 2411.19352 v2 pith:GY7GX7BI submitted 2024-11-28 cs.AI

Se-eun Yoon , Xiaokai Wei , Yexi Jiang , Rachit Pareek , Frank Ong , Kevin Gao , Julian McAuley , Michelle Gong This is my paper · ORCID
classification cs.AI
keywords conversationalrecommendersystemslargelanguagemodelstool-augmentedLLMstoolorchestrationrecommendationdiversityrealuserrequestsgamefactuality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a conversational recommender must give a large language model many more tools than the one to three used in prior synthetic-query work. The authors build OMuleT, a system that turns a real user's free-form request into a structured intent, runs a fixed list of over ten lookup, linking, retrieval, and formatting tools, and then asks the LLM to write recommendations with the tool output in front of it. On a dataset of 556 real Reddit requests for Roblox games, they report that this approach outperforms prompt-only LLaMA-405B and GPT-4o across factuality, relevance, novelty, and coverage, with near-perfect factuality and much less concentration on popular games. They also report that using the full toolbox beats removing any one tool, and that a fixed human-designed execution policy works better than letting the LLM write its own tool-execution code. The payoff, if the results hold, is a template for deploying LLM recommenders that stay current with a changing item catalog without fine-tuning.

What carries the argument

The load-bearing object is a handcrafted tool-execution policy P (Algorithm 1) that maps a structured 'formatted intent'—a JSON dictionary of liked and disliked genres, game names, properties, devices, and age groups—into an augmented context D_aug by running lookup, fuzzy linking, retrieval, and filtering tools, then formatting the results into natural language. The LLM is never asked to decide which tools to call; it only converts the user request into the intent JSON and later uses the tool output to enumerate recommendations. That division—LLM for language understanding, fixed policy for tool choice—is what the paper claims produces the gains in factuality, novelty, and coverage.

What would settle it

A decisive check is to recompute the comparison using ground-truth items selected by independent human judges who never see OMuleT's outputs or the two similarity APIs; if the fixed-policy toolbox then fails to beat base LLMs on Hit@10 and Precision@10 on the same 208 requests, the central claim would be refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the practical bottleneck in conversational recommendation is not language understanding but access to item knowledge: LLMs already understand messy requests, but left alone they hallucinate, repeat front-page items, and cannot see new or long-tail catalog entries. Giving the same LLM access to over ten simple production tools—fuzzy name linking, device filters, age-group popularity, collaborative and content similarity, search, metadata lookup—and a fixed hand-written policy P converts it into a recommender that is factual, relevant, novel, and diverse. The paper presents this as the first system to combine real user requests, a large toolbox, extensive multi-metric evaluation, and deployment experience, and the headline result is that OMuleT beats zero-shot base LLMs on every metric on the human-annotated subset.

Load-bearing premise

The load-bearing premise is that the human-annotated ground-truth items measure what a user really wanted: those labels were created by expanding upvoted Reddit comments with the same two similarity APIs the toolbox calls, so if those APIs are weak sources of relevant games, the reported head-to-head advantage over plain LLMs may be partly an artifact of grading on the system's own retrieval paths.

Editorial extensions

If this is right

  • Tool augmentation without fine-tuning can keep a deployed recommender current: new items enter the catalog through retrieval APIs, so the LLM does not need retraining to know them.
  • Prompting a base LLM for diversity is not a substitute for tool augmentation, since it sharply reduces factuality and relevance while OMuleT keeps both high.
  • A fixed, human-readable execution policy is preferable to letting the LLM generate code: relevance falls under PLLM, and the fixed policy keeps the system inspectable and controllable.
  • The ablation results imply every tool contributes: removing any one tool hurts the LLaMA-based system, and removing search trades a small relevance gain for noticeably worse novelty and coverage.
  • Across requests, OMuleT's recommendations become much less concentrated on the most popular games, which matters for platforms that depend on surfacing long-tail user-generated content.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: because the ground-truth labels were built by expanding upvoted Reddit recommendations with the same two similarity APIs that the toolbox calls, a portion of the reported advantage over plain LLMs may come from grading on the system's own retrieval sources; an independent human re-annotation study would separate the tool's real value from that overlap.
  • Editorial: the architecture is domain-agnostic in principle—fuzzy linking, metadata lookup, similarity search, and device or age filters map to any content catalog—so the same fixed-policy design is a plausible template for movies, books, or apps, though the paper's own results only cover Roblox games.
  • Editorial: prior one-to-three-tool systems are compared only indirectly; directly re-implementing the strongest such baseline on the same dataset would quantify how much of the gain comes from having many tools rather than from any single tool or from the evaluation setup.
  • Editorial: the deployment section suggests latency, not quality, is the main barrier to production; caching formatted intents or precomputing tool outputs for frequent request patterns would be a natural engineering extension of the paper's design.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes OMuleT, a conversational recommender that augments an LLM with more than ten tools (lookup, linking, retrieval, and formatting functions) governed by a handcrafted execution policy P. A formatted intent is extracted from a raw user request, P executes the relevant tools and filters their outputs, and the LLM generates the final ranked list of game recommendations. The authors collect 556 real recommendation requests from /r/Roblox, construct human-annotated ground truth for 208 requests via expert filtering of Reddit oracle comments plus API-expanded candidates, and compare OMuleT against base LLMs, a diversity-prompted baseline, and LLM-generated policies on factuality, relevance, novelty, and coverage metrics. The paper also reports an ablation study and deployment lessons from an internal prototype.

Significance. If the evaluation were clean, this would be a useful practical contribution: it targets real user utterances rather than synthetic queries, scales tool augmentation to more than ten tools, demonstrates a fixed and inspectable policy P that can outperform LLM-generated policies, and provides deployment insights that are rare in the academic CRS literature. The dataset of real requests is itself a concrete asset for future work. However, the headline claim that tool augmentation improves relevance over base LLMs is currently undercut by the construction of the ground-truth set from the same similarity APIs that OMuleT can call, and by the absence of any uncertainty quantification. These two issues affect the central Q1 result in Section 4.3 and must be addressed before the comparisons can be taken as established.

major comments (2)
  1. [Section 3.1.3, Section 3.2.3, Algorithm 1] The human-annotated ground truth is built by expanding Reddit oracles with get_similar_games_cf and get_similar_games_content (Section 3.1.3, footnote 4), and those two functions are also tools in OMuleT's toolbox (Table 1) that policy P invokes on liked games (Algorithm 1, lines 5-6). A game can therefore enter the label set only if it is produced by one of the APIs that OMuleT is explicitly designed to call, while base LLMs have no access to these APIs. This biases Hit@k and Precision@k in favor of OMuleT and makes the Q1 result in Section 4.3 unverified. Please re-evaluate on a label set constructed without those APIs (e.g., oracle comments only or independently created relevant-item lists), or at minimum quantify the overlap between tool-returned candidates and ground-truth items and show that the headline comparisons survive when overlapping items are removed.
  2. [Section 4.3, Table 2] All metrics in Table 2 are reported as single point estimates with no confidence intervals, standard deviations, or significance tests, and Section 4.2.1 only notes that temperature is set to 0. Several of the reported advantages are very small (e.g., GPT-4o Hit@5 rises from .26 to .27 and Precision@5 from .07 to .08 on the human-annotated set). Without per-request variance estimates or paired significance tests (e.g., bootstrap or Wilcoxon tests over the 208 requests), the claim that OMuleT 'outperforms base LLMs in all metrics' is not supported by the evidence presented. Please add uncertainty estimates or statistical tests.
minor comments (5)
  1. [Section 3.1.3, footnote 4] Footnote 4 refers to 'Table 2' for the two similarity tools, but the tools are listed in Table 1; this cross-reference should be corrected.
  2. [Table 2] The column header 'Precise' should be 'Precision', and the LLaMA-405B OMuleT w/ PLLM row appears to contain '.22 18' where a decimal point is missing (likely '.22 .18').
  3. [Algorithm 1] The indentation of Algorithm 1 is ambiguous: lines 7-8 and 13-14 appear to be outside the for loops, but line 7 uses D_int[liked genres] while the surrounding loop variable is a game; please clarify the intended scope of each step.
  4. [Section 4.3, Q4] There are small typos in this paragraph: 'converage' should be 'coverage' and 'LLaMA-405o' should be 'LLaMA-405B'.
  5. [Section 5] The deployment section is only one paragraph; if 'deployment insights' is one of the four claimed contributions, the paper would be strengthened by more detail on the actual feasibility study, such as observed latency, failure cases, or qualitative user feedback from the internal release.

Circularity Check

1 steps flagged · score 6.0 of 10

Ground-truth candidates are built from the same similarity APIs that OMuleT is designed to call, making the headline Q1 comparison partially circular by construction.

  1. self definitional [Section 3.1.3 (From Oracles to Ground-Truth Items), footnote 4; Table 1; Algorithm 1 line 6]
    "First, for each request, we generate a candidate set of games. This is done by using the oracles: we obtain games similar to the oracle by using two Roblox APIs. Oracles and similar games are added to the candidate set by prioritizing their frequency across all oracles and APIs, with up to 30 candidates generated per request. Second, human experts determine whether each candidate is relevant to the request."

    Footnote 4 identifies the two APIs as get_similar_games_cf and get_similar_games_content, which are exactly the retrieval tools in OMuleT's toolbox (Table 1) and are executed by policy P for every liked game (Algorithm 1 line 6: D_aug <- D_aug U similar(game)). The human-annotated ground-truth set is therefore a filtered subset of the outputs of the same functions OMuleT is built to invoke. A relevant game not returned by these APIs is absent from the ground truth, so Hit@k and Precision@k cannot credit a model that names it; base LLMs have no access to the APIs and can only score on the Reddit-oracle portion or by coincidentally knowing an API-retrieved item.

full rationale

The main derivation chain is not circular: OMuleT's intent formatting, handcrafted policy, and LLM recommendation stage are defined independently of the evaluation labels, and no parameter is fitted to the ground-truth set. The ablation study and the P vs P_LLM comparison are also self-contained experiments. However, one load-bearing step in the evaluation construction is partially circular. Section 3.1.3 creates the human-annotated ground truth by expanding Reddit oracle games with the two Roblox similarity APIs get_similar_games_cf and get_similar_games_content; these are precisely two of the retrieval tools in OMuleT's toolbox (Table 1) and are invoked by the fixed policy P for every liked game (Algorithm 1). Consequently, any relevant game that these APIs do not surface cannot be in the ground truth, so base LLMs are structurally unable to earn credit on Hit@k and Precision@k for such games, while OMuleT is fed the APIs' outputs directly. The expert filtering adds a genuine relevance judgment and prevents the ground truth from being a pure echo of the tools; the Reddit-oracle component also provides independent signal. Hence the Q1 claim 'OMuleT outperforms base LLMs in all metrics' is partially forced by construction but not fully circular. No self-citation chain or imported uniqueness theorem is load-bearing; the authors' cited prior work is used only for background and context. Overall score 6.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim depends on design choices such as policy P, tool count, list length, and candidate caps, and on assumptions about Reddit comments, upvotes, expert labels, and embedding-based similarity. No fitted model parameters or invented physical entities are introduced. The load-bearing hidden assumption is that the similarity APIs used to build ground truth are a legitimate source of true positives even though the same APIs are part of the proposed toolbox.

free parameters (4)
  • Tool execution policy P (Algorithm 1) = Handcrafted rule sequence; default_games(30); top-100 pool; genre and device filters
    The sequence and thresholds are chosen by the authors and are not fitted or validated on a held-out set; the paper's main claim depends on this policy being reasonable for real requests.
  • Recommended list length k = 20 (evaluated at k=5 and k=10)
    The prompt instructs the LLM to enumerate 20 games; this choice sets the ceiling for metrics and is not justified by user needs.
  • Ground-truth candidate cap = Up to 30 candidates per request
    Candidate generation for expert labeling stops at 30 games, so the set of possible true positives is truncated by this design choice.
  • Number of demonstrations for intent parsing = 5
    Authors report that 5 demonstrations make formatting stable; no systematic selection is described.
assumptions (5)
  • domain assumption Reddit r/Roblox posts, filtered by keyphrases and GPT-3.5, are representative of real user recommendation requests on the platform.
    Used in Section 3.1.1 to build the evaluation set; the authors themselves note in Section 7 that the Reddit dataset may not fully represent all user types.
  • domain assumption Net upvoted comments on Reddit are a reliable signal that a recommended game is relevant.
    Section 3.1.2 uses at least one net upvote as the oracle quality cutoff, which assumes community agreement approximates relevance.
  • domain assumption Human experts can judge whether a candidate game is relevant without feedback from the original requester.
    Section 3.1.3 relies on expert play-testing and judgment to define ground-truth items; this assumes experts can infer the requester's preferences from the post.
  • domain assumption SimCSE embeddings of game descriptions measure item similarity for the Similar@k metric.
    Section 4.1.1 defines Similar@k as cosine distance between embedding centroids, assuming description similarity tracks recommendation relevance.
  • ad hoc to paper The two Roblox similarity APIs return candidates that are useful for building ground truth.
    Section 3.1.3 expands oracle games through get_similar_games_cf and get_similar_games_content; the validity of the evaluation partly depends on these APIs surfacing relevant games, and the same APIs are also used as OMuleT tools.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation." pith.science (2026). https://pith.science/paper/GY7GX7BI

@misc{pith2026241119352,
  author       = {Pith},
  title        = {Pith review of: OMuleT: Orchestrating Multiple Tools for Practicable Conversational Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GY7GX7BI}},
  note         = {Machine review of arXiv:2411.19352}
}
read the original abstract

In this paper, we present a systematic effort to design, evaluate, and implement a realistic conversational recommender system (CRS). The objective of our system is to allow users to input free-form text to request recommendations, and then receive a list of relevant and diverse items. While previous work on synthetic queries augments large language models (LLMs) with 1-3 tools, we argue that a more extensive toolbox is necessary to effectively handle real user requests. As such, we propose a novel approach that equips LLMs with over 10 tools, providing them access to the internal knowledge base and API calls used in production. We evaluate our model on a dataset of real users and show that it generates relevant, novel, and diverse recommendations compared to vanilla LLMs. Furthermore, we conduct ablation studies to demonstrate the effectiveness of using the full range of tools in our toolbox. We share our designs and lessons learned from deploying the system for internal alpha release. Our contribution is the addressing of all four key aspects of a practicable CRS: (1) real user requests, (2) augmenting LLMs with a wide variety of tools, (3) extensive evaluation, and (4) deployment insights.

Figures

Figures reproduced from arXiv: 2411.19352 by the authors.

Figure 1
Figure 1. Examples of recommendation requests from users. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our dataset collection process. ◦ Real user requests. We use real user requests, which are more complex and diverse than queries synthesized from templates. ◦ Framework for augmenting LLMs with nontrivial amount of tools. Complex requests require the use of a wide variety of tools. Our tools are simple and generic, and our framework effectively orchestrates the tools to augment LLMs. ◦ Extensive evaluation. We condu… view at source ↗
Figure 3
Figure 3. Overview of OMuleT. Orange boxes are in the user interface (a user inputs a raw request and observes recommended items); blue boxes are where LLMs are used; green boxes are where tools are used. Here, we find that some posts are asking for Roblox game rec￾ommendations. We sample the posts by using the Python Reddit API Wrapper (PRAW),1 using keyphrases such as ‘recommend me games’ and ‘what games to play’. We furthe… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Ablation study. Recommendations are more relevant if we use more tools. One exception is when removing the search [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Screenshot of the deployed UI. We add simple greet [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. RecoWorld: Building Simulated Environments for Agentic Recommender Systems

    cs.IR 2025-09 conditional novelty 5.0 of 10

    A design proposal, not a tested system: a dual-view simulation loop in which an LLM-simulated user issues reflective instructions when about to disengage, and an instruction-following recommender adapts to maximize si...

Reference graph

Works this paper leans on

61 extracted references · 36 canonical work pages · cited by 1 Pith paper

  1. [1]

    Meta AI. 2024. LLaMA 3.1: 405B Parameter Model. https://ai.meta.com/llama. Accessed: 2024-08-09

    work page 2024

  2. [2]

    Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, et al. 2023. Do as I can, not as I say: Grounding language in robotic affordances. In Conference on robot learning. PMLR

    work page 2023

  3. [3]

    Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards conversational recommender systems. In KDD

    work page 2016

  4. [4]

    Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, et al. 2023. Leveraging large language models in conversational recommender systems.arXiv preprint arXiv:2305.07961 (2023)

    arXiv 2023

  5. [5]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021)

    arXiv 2021

  6. [6]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In RecSys

    work page 2022

  7. [7]

    Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2024. Toolkengpt: Aug- menting frozen language models with massive tools via tool embeddings. In NeurIPS

    work page 2024

  8. [8]

    Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023. Leveraging large language models for sequential recommendation. In RecSys

    work page 2023

Show all 61 references
  1. [9]

    Hashicorp. 2024. Nomad and Consul. https://developer.hashicorp.com/consul/ docs/connect/nomad. Accessed: 2024-08-09

    2024

  2. [10]

    Shirley Anugrah Hayati, Dongyeop Kang, Qingxiaoyang Zhu, Weiyan Shi, and Zhou Yu. 2020. Inspired: Toward sociable recommendation dialog systems. arXiv preprint arXiv:2009.14306 (2020)

    2020 arXiv

  3. [11]

    Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian McAuley. 2023. Large language models as zero-shot conversational recommenders. In CIKM

    2023

  4. [12]

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large language models are zero-shot rankers for recommender systems. In ECIR

    2024

  5. [13]

    Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. 2023. Recommender ai agent: Integrating large language models for interactive recom- mendations. arXiv preprint arXiv:2308.16505 (2023)

    2023 arXiv

  6. [14]

    Marius Kaminskas and Derek Bridge. 2016. Diversity, serendipity, novelty, and coverage: a survey and empirical analysis of beyond-accuracy objectives in recommender systems. ACM TiiS 7, 1 (2016), 1–42

    2016

  7. [15]

    Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed Chi, and Derek Zhiyuan Cheng. 2023. Do llms understand user prefer- ences? evaluating llms on user rating prediction. arXiv preprint arXiv:2305.06474 (2023)

    2023 arXiv

  8. [16]

    Sara Kemper, Justin Cui, Kai Dicarlantonio, Kathy Lin, Danjie Tang, Anton Ko- rikov, and Scott Sanner. 2024. Retrieval-Augmented Conversational Recommen- dation with Prompt-based Semi-Structured Natural Language State Tracking. In SIGIR

    2024

  9. [17]

    Sein Kim, Hongseok Kang, Seungyoon Choi, Donghyun Kim, Minchul Yang, and Chanyoung Park. 2024. Large Language Models meet Collaborative Filter- ing: An Efficient All-round LLM-based Recommender System. arXiv preprint arXiv:2404.11343 (2024)

    2024 arXiv

  10. [18]

    Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min- Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In WSDM

    2020

  11. [19]

    Chuang Li, Yang Deng, Hengchang Hu, Min-Yen Kan, and Haizhou Li. 2024. Incor- porating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems. arXiv preprint arXiv:2405.01868 (2024)

    2024 arXiv

  12. [20]

    Lei Li, Yongfeng Zhang, and Li Chen. 2023. Prompt distillation for efficient llm-based recommendation. In CIKM

    2023

  13. [21]

    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , Houda B...

    2023 doi

  14. [22]

    Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In NeurIPS

    2018

  15. [23]

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. 2023. Code as policies: Language model programs for embodied control. In ICRA

    2023

  16. [24]

    Bill Yuchen Lin, Yicheng Fu, Karina Yang, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Prithviraj Ammanabrolu, Yejin Choi, and Xiang Ren. 2024. Swift- sage: A generative agent with fast and slow thinking for complex interactive tasks. In NeurIPS

    2024

  17. [25]

    Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. 2023. Chameleon: Plug-and-play composi- tional reasoning with large language models. In NeurIPS

    2023

  18. [26]

    Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In ACL

    2019

  19. [27]

    Sheshera Mysore, Andrew McCallum, and Hamed Zamani. 2023. Large language model augmented narrative driven recommendations. In RecSys

    2023

  20. [28]

    OpenAI. 2024. GPT-3.5 Turbo. https://platform.openai.com/docs/models/gpt-3- 5-turbo. Accessed: 2024-08-09. Conf, Month DD, YYYY, City, State Se-eun Yoon, Xiaokai Wei, Yexi Jiang, Rachit Pareek, Frank Ong, Kevin Gao, Julian McAuley, and Michelle Gong

    2024

  21. [29]

    OpenAI. 2024. GPT-4o. https://platform.openai.com/docs/models/gpt-4o. Ac- cessed: 2024-08-09

    2024

  22. [30]

    Lijing Qin and Xiaoyan Zhu. 2013. Promoting diversity in recommendation by entropy regularizer. In IJCAI

    2013

  23. [31]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789 (2023)

    2023 arXiv

  24. [32]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  25. [33]

    Scott Sanner, Krisztian Balog, Filip Radlinski, Ben Wedin, and Lucas Dixon

  26. [34]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. Toolformer: Language models can teach themselves to use tools. In NeurIPS

    2024

  27. [35]

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2024. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. In NeurIPS

    2024

  28. [36]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. In NeurIPS

    2024

  29. [37]

    Streamlit. 2024. Streamlit. https://streamlit.io/. Accessed: 2024-08-09

    2024

  30. [38]

    Yueming Sun and Yi Zhang. 2018. Conversational recommender system. InSIGIR

    2018

  31. [39]

    Saúl Vargas and Pablo Castells. 2011. Rank and relevance in novelty and diversity metrics for recommender systems. In RecSys

    2011

  32. [40]

    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023)

    2023 arXiv

  33. [41]

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable code actions elicit better llm agents. arXiv preprint arXiv:2402.01030 (2024)

    2024 arXiv

  34. [42]

    Xinfeng Wang, Jin Cui, Yoshimi Suzuki, and Fumiyo Fukumoto. 2024. RDRec: Rationale Distillation for LLM-based Recommendation. arXiv preprint arXiv:2405.10587 (2024)

    2024 arXiv

  35. [43]

    Xiaolei Wang, Kun Zhou, Ji-Rong Wen, and Wayne Xin Zhao. 2022. Towards unified conversational recommender systems via knowledge-enhanced prompt learning. In KDD

    2022

  36. [44]

    Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xiaojiang Huang, and Yingzhen Yang. 2024. RecMind: Large Language Model Powered Agent For Recommendation. In NAACL (Findings), Kevin Duh, Helena Gomez, and Steven Bethard (Eds.)

    2024

  37. [45]

    Zhiruo Wang, Zhoujun Cheng, Hao Zhu, Daniel Fried, and Graham Neubig. 2024. What are tools anyway? a survey from the language model perspective. arXiv preprint arXiv:2403.15452 (2024)

    2024 arXiv

  38. [46]

    Zhiruo Wang, Daniel Fried, and Graham Neubig. 2024. Trove: Inducing veri- fiable and efficient toolboxes for solving programmatic tasks. arXiv preprint arXiv:2401.12869 (2024)

    2024 arXiv

  39. [47]

    Yunjia Xi, Weiwen Liu, Jianghao Lin, Bo Chen, Ruiming Tang, Weinan Zhang, and Yong Yu. 2024. MemoCRS: Memory-enhanced Sequential Conversational Recom- mender Systems with Large Language Models. arXiv preprint arXiv:2407.04960 (2024)

    2024 arXiv

  40. [48]

    Zhouhang Xie, Junda Wu, Hyunsik Jeon, Zhankui He, Harald Steck, Rahul Jha, Dawen Liang, Nathan Kallus, and Julian McAuley. 2024. Neighborhood-Based Collaborative Filtering for Conversational Recommendation. In RecSys

    2024

  41. [49]

    Li Yang, Anushya Subbiah, Hardik Patel, Judith Yue Li, Yanwei Song, Reza Mirghaderi, and Vikram Aggarwal. 2024. Item-Language Model for Conver- sational Recommendation. arXiv preprint arXiv:2406.02844 (2024)

    2024 arXiv

  42. [50]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022)

    2022 arXiv

  43. [51]

    Se-eun Yoon, Zhankui He, Jessica Echterhoff, and Julian McAuley. 2024. Evaluat- ing Large Language Models as Generative User Simulators for Conversational Recommendation. In NACCL

    2024

  44. [52]

    Se-eun Yoon, Hyunsik Jeon, and Julian McAuley. 2024. Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation. arXiv preprint arXiv:2405.14142 (2024)

    2024 arXiv

  45. [53]

    Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R Fung, Hao Peng, and Heng Ji

  46. [54]

    Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2023. Recommendation as instruction following: A large language model empowered recommendation approach. arXiv preprint arXiv:2305.07001 (2023)

    2023 arXiv

  47. [55]

    arXiv preprint arXiv:2309.17428 (2023)

    Craft: Customizing llms by creating and retrieving from specialized toolsets. arXiv preprint arXiv:2309.17428 (2023)

    2023 arXiv

  48. [56]

    Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In CIKM

    2018

  49. [57]

    Xiaoying Zhang, Hong Xie, Hang Li, and John CS Lui. 2020. Conversational contextual bandit: Algorithm and application. In WWW

    2020

  50. [58]

    Huaixiu Steven Zheng, Swaroop Mishra, Hugh Zhang, Xinyun Chen, Minmin Chen, Azade Nova, Le Hou, Heng-Tze Cheng, Quoc V Le, Ed H Chi, et al. 2024. NATURAL PLAN: Benchmarking LLMs on Natural Language Planning. arXiv preprint arXiv:2406.04520 (2024)

    2024 arXiv

  51. [59]

    Bowen Zheng, Yupeng Hou, Hongyu Lu, Yu Chen, Wayne Xin Zhao, and Ji- Rong Wen. 2023. Adapting large language models by integrating collaborative semantics for recommendation. arXiv preprint arXiv:2311.09049 (2023)

    2023 arXiv

  52. [61]

    Yaochen Zhu, Liang Wu, Qi Guo, Liangjie Hong, and Jundong Li. 2024. Collabo- rative large language model for recommender systems. In WWW

    2024

  53. [2023]

    In RecSys

    Large language models are competitive near cold-start recommenders for language-and item-based preferences. In RecSys

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.