PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

Giacomo Frisoni; Gianluca Moro; Lorenzo Molfetta; Nicol\`o Monaldini

REVIEW 3 major objections 4 minor 16 references

PORTS trains tool retrievers by aligning selection odds with a frozen LLM's gold-call confidence, lifting recall by tens of points over frozen and REPLUG baselines.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 01:36 UTC pith:NROPFPP3

load-bearing objection Clean, practical recipe that actually moves tool-retrieval numbers; the LLM-likelihood proxy is noisy but the paper owns that and still shows consistent gains over REPLUG. the 3 major comments →

arxiv 2607.05441 v1 pith:NROPFPP3 submitted 2026-07-03 cs.IR cs.AI

PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models

Lorenzo Molfetta , Giacomo Frisoni , Nicol\`o Monaldini , Gianluca Moro This is my paper

classification cs.IR cs.AI

keywords tool retrievalpreference optimizationodds-ratio lossLLM-guided retrieverhard-negative samplingtool selectionREPLUG alignmentencoder fine-tuning

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

When large language models must choose among hundreds of external tools, a separate retriever is usually used to shortlist candidates so that the prompt stays short. Existing retrievers are trained only on textual similarity and therefore often surface tools that look relevant but do not help the model produce the correct call. PORTS fixes the misalignment by fine-tuning the retriever with two signals taken from a frozen LLM: a KL term that matches the retriever's ranking probabilities to the LLM's own log-likelihood of the gold tool-call, and an odds-ratio preference loss that explicitly prefers the correct tool over hard semantic near-misses. The resulting encoder raises average Recall@1-3 by as much as 72 points on seen tools and 61 points on unseen tools across six public benchmarks, two base encoders and three different LLMs, while remaining cheap enough to re-run whenever the tool catalogue changes.

Core claim

A dual-loss preference objective that (i) minimises the KL divergence between the retriever's softmax over tool docstrings and a frozen LLM's next-token log-likelihood of the gold call, and (ii) maximises the odds ratio of selecting the positive tool over hard negatives, produces tool rankings that are substantially better aligned with actual downstream success than either frozen encoders or pure REPLUG training.

What carries the argument

The combined PORTS loss L_PORTS = L_replug + λ L_po, where L_replug is the KL between retriever probabilities P_R and LLM gold-call confidences Q_G, and L_po is the sigmoid of the log-odds ratio of positive versus hard-negative tools; hard negatives are refreshed every T steps from the evolving encoder embeddings.

Load-bearing premise

That the average next-token log-likelihood of the gold tool-call string, when the query is paired with only one tool docstring, is a clean enough proxy for whether that tool is actually helpful.

What would settle it

On a held-out suite of tools whose docstrings are deliberately vague or missing parameter types, measure whether the Recall@1 gap between PORTS and a pure contrastive baseline collapses to near zero; if the gap disappears, the LLM-likelihood signal is not carrying the claimed information.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Clean, practical recipe that actually moves tool-retrieval numbers; the LLM-likelihood proxy is noisy but the paper owns that and still shows consistent gains over REPLUG.

read the letter

This is a solid methods paper for the tool-augmented LLM crowd. What is new is the concrete combination: REPLUG-style KL of retriever probs to a frozen LLM’s gold-call log-likelihood, plus an odds-ratio preference term over hard negatives drawn from the same encoder, all aimed at tool docstrings. They ship code, six heterogeneous datasets, two encoders, three LLMs, in- and out-of-domain splits, ablations, seed variance, and full appendix tables. That is more thorough than most retrieval-for-tools work I see.

The gains are real on the metrics they report—large lifts over frozen baselines and consistent edges over REPLUG alone, including on unseen tools. RoBERTa benefits more than BGE, which makes sense given the starting distributions. The contrastive term helps most where tools look similar but differ in parameters, exactly the regime that matters for growing API catalogs. Low compute (single 3090, 4-bit LLMs) is a genuine plus for practitioners who cannot retrain the generator.

The soft spot the stress-test flags is real but already acknowledged by the authors: when docstrings are vague (APIBench is the clearest case), the next-token likelihood of the gold call becomes a noisier teacher, and hard negatives sampled from the same encoder can amplify surface cues. They flag this in the Docstring Impact and Limitations sections and still show gains, so it is a caveat rather than a collapse. Stronger non-REPLUG baselines and a couple of significance tests would have been nice; neither is fatal. Math is standard, citations are appropriate, no circularity.

This is for people building or maintaining tool-selection pipelines who need a cheap way to keep the retriever in step with the calling LLM. I would bring it to reading group, cite the objective and the out-of-domain numbers if I am working in the area, and send it to peer review without hesitation.

Referee Report

3 major / 4 minor

Summary. The paper proposes PORTS, a method for fine-tuning dense retrievers for tool selection that jointly optimizes a REPLUG-style KL loss aligning retriever probabilities P_R (Eq. 1) to a frozen LLM's normalized gold-call log-likelihoods Q_G (Eq. 2) and an odds-ratio preference loss L_po (Eq. 5) over hard-negative tool docstrings. The combined objective is applied to two encoders (RoBERTa-base, BGE-base) under three LLMs on six tool-selection datasets with in-domain and out-of-domain splits. Results (Table 2, Figures 1/3/4) show large gains in Recall@K and NDCG@K over frozen baselines and REPLUG alone (up to +71.66 / +61.24 average points), with ablations, hyper-parameter studies, seed-variance checks, and low compute cost. The authors claim improved alignment of retrieval to LLM tool-calling needs and better generalization to evolving toolsets.

Significance. If the reported gains hold under broader evaluation, PORTS offers a practical, LLM-agnostic, low-overhead recipe for specializing retrievers to tool selection without updating the generator—an increasingly important capability as tool collections and MCP servers proliferate. Strengths that raise confidence include the public code/models/datasets release, systematic multi-encoder/multi-LLM/multi-dataset design, explicit ablation of the preference term versus pure REPLUG (Table 2), OOD progressive-unseen experiments (Figure 4), clustering analysis of tool embeddings (Appendix G), seed-variance tables (Table 10), and qualitative ranking examples (Appendix I). The adaptation of odds-ratio preference optimization to the retrieval setting via frozen-LLM perplexity signals is a clear, reusable contribution relative to prior tool-retrieval work that largely ignored LLM preferences.

major comments (3)

[§4.2 Evaluation Metrics / §5 Results / Table 2] Evaluation (§4.2, Table 2, Appendix H) reports only gold-tool Recall@K and NDCG@K. The central claim is that PORTS produces retrievers better aligned to the needs of a tool-calling LLM. There is no end-to-end measurement of whether the improved top-K actually raise the LLM's tool-call accuracy, parameter correctness, or downstream task success (e.g., the PASS@K suggested in the Conclusion). Because Q_G is computed under independent single-tool prompting, it remains possible that the ranking improvements do not translate when multiple tools appear together in the prompt; this gap is load-bearing for the alignment narrative.
[§5 Results (baseline paragraph) / §2 Related Work] PORTS is compared solely to frozen encoders and REPLUG. Related Work surveys several specialized tool retrievers (ToolLLaMA SentenceBERT, COLT, ProTIP, ToolRerank, CRAFT). Even if their training objectives differ, head-to-head retrieval-metric comparisons on the same six datasets (or at least a supervised contrastive baseline without the LLM signal) are needed to locate the absolute contribution. The stated rationale of 'divergent optimization goals' does not justify omitting such numbers.
[§3.3 / Eq. (2) / Docstring Impact / Limitations] The preference signal rests on the average next-token log-likelihood of the gold call under independent single-tool prompting (Eq. 2). The paper itself notes (Docstring Impact, Limitations) that this proxy weakens for vague or underspecified docstrings (APIBench). Hard negatives are drawn from the same encoder's embedding space and refreshed only every T steps, creating a potential feedback loop that can reinforce surface similarity rather than functional utility. While positive gains remain, the largest improvements coincide with datasets where the proxy is strongest; a quantitative analysis of rank correlation between Q_G and true helpfulness (or multi-tool prompting) would better ground the method.

minor comments (4)

[Figure 1] Figure 1 averages Recall across LLMs and datasets, yet Table 2 reports 'Best LLM' per configuration. The caption and axis label should explicitly state the aggregation rule to avoid ambiguity.
[§3.3 Eqs. (1)–(2)] Temperatures γ (retriever) and β (LLM) appear in Eqs. (1)–(2) with little immediate intuition. A brief clause explaining their softening role would improve readability.
[Throughout] Minor naming and formatting inconsistencies appear throughout (PORTS vs. PORTS-tuned, occasional missing hyphens in 'tool-calling', slight variation in dataset numbering). These are cosmetic but easy to clean.
[Appendix H] Appendix H tables are dense and valuable; a short note clarifying how the 'Best LLM' column is chosen for each encoder–dataset–loss triple would help readers navigate them.

Circularity Check

0 steps flagged

No circularity: empirical loss design and supervised evaluation against gold tools; no fitted quantity is later presented as a prediction, and no load-bearing self-citation chain exists.

full rationale

PORTS defines a composite training objective (L_PORTS = L_replug + λ L_po) that aligns a retriever's softmax over cosine similarities (Eq. 1) to a frozen LLM's next-token log-likelihood of the gold tool-call string (Eq. 2) via KL, while adding an odds-ratio preference term over hard negatives (Eqs. 4-5). The resulting encoder is then evaluated with ordinary Recall@K / NDCG@K against the same gold tools on held-out queries. Nothing in this chain is algebraically forced by construction: the LLM likelihoods are external black-box signals, the hard negatives are periodically re-sampled from the evolving embedding space, and the reported metric gains are measured on independent test splits (including out-of-domain tool partitions). There are no fitted constants later re-labeled as predictions, no uniqueness theorems imported from the authors' prior work, and no ansatz smuggled via self-citation. Ordinary related-work citations (REPLUG, ORPO, etc.) supply background only. The paper is therefore free of the circular patterns enumerated in the analyzer instructions; the derivation is simply a well-specified empirical training recipe.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 0 invented entities

PORTS is an empirical training recipe; its free parameters are ordinary hyper-parameters chosen by grid search, its axioms are standard dense-retrieval and language-model assumptions, and it invents no new physical or mathematical entities.

free parameters (5)

loss weight λ = 0.3
Balances the REPLUG KL term against the odds-ratio preference term; final value 0.3 chosen from {0.1,0.3,0.5,0.7,0.9}.
retriever temperature γ = 0.5
Softmax temperature for retrieval probabilities; final value 0.5 chosen from {0.3,0.5,0.7,1}.
LLM temperature β = 0.5
Softmax temperature for the LLM utility distribution Q_G; final value 0.5 chosen from the same grid.
number of hard negatives n = 3
How many nearest-neighbor tools are treated as negatives; final value 3 chosen from {1,2,3}.
hard-negative refresh interval T = 50
Embeddings and negatives recomputed every T steps; fixed at 50.

axioms (3)

domain assumption Average next-token log-likelihood of the gold tool-call string under a frozen LLM is a useful ranking signal for tool helpfulness.
Core modeling choice stated in §3.3 and Eq. 2; the paper itself notes degradation when docstrings are vague.
domain assumption Cosine similarity of encoder embeddings yields meaningful hard negatives for contrastive preference learning.
Used for asynchronous hard-negative mining (§3.3); standard in dense retrieval but not proven optimal for tool docstrings.
ad hoc to paper Independent prompting of each tool docstring (no multi-tool context) isolates the contribution of that tool to call likelihood.
Explicit design decision following REPLUG; enables clean KL but ignores interactions among simultaneously retrieved tools.

pith-pipeline@v1.1.0-grok45 · 29920 in / 2666 out tokens · 23491 ms · 2026-07-12T01:36:12.421121+00:00 · methodology

0 comments

read the original abstract

Integrating external tools with Large Language Models (LLMs) has emerged as a promising paradigm for accomplishing complex tasks. Since LLMs still struggle to effectively manage large tool collections, researchers have begun exploring retrieval-based methods to pre-select the most relevant options, addressing input length and latency constraints. However, existing retrievers are often misaligned with tool-calling LLMs due to their separate training processes. This paper presents PORTS, a novel odds ratio preference optimization method for training retrievers aimed at tool selection. Using a perplexity-inspired preference signal from a frozen LLM, our approach fine-tunes a retriever to find helpful tools by optimizing the correlation between the selection probabilities and the downstream performances while jointly enforcing a contrastive semantic loss between documentation strings. The versatility of PORTS and its ability to significantly improve tool selection accuracy are demonstrated through extensive experiments on six datasets, two encoder models, and three LLMs with diverse prior knowledge. With low computational demands, our alignment process facilitates generalization to new queries and tools, proving valuable for practical applications with evolving toolsets.

Figures

Figures reproduced from arXiv: 2607.05441 by Giacomo Frisoni, Gianluca Moro, Lorenzo Molfetta, Nicol\`o Monaldini.

**Figure 1.** Figure 1: Results overview. Comparison between frozen, REPLUG-tuned, and PORTS-tuned retrievers. Scores are averaged across Recall@{1,2,3} for three LLMs (if trained) and six datasets (test set). tools2 has garnered significant research attention. This integration not only improves the problemsolving potential of LLMs, but also dramatically expands their functional scope (Yao et al., 2022; Lazaridou et al., 2022). … view at source ↗

**Figure 2.** Figure 2: PORTS training process. Simplified illustration of the PORTS’ training recipe with one positive and two negative tool docstrings from APIBench. Input tool documentation triplets are encoded independently and prompted separately to the frozen LLM. The retriever is fine-tuned to align tool selection probabilities with the correct answer likelihood while maximizing the ratio between the odds of selecting the … view at source ↗

**Figure 3.** Figure 3: Average Recall@K for each dataset (test set). Effectiveness of PORTS-tuned retrievers (LPORTS) against frozen baselines, utilizing guidance from different LLMs. Evaluated in both in-domain and out-of-domain settings. peaks attaining a +13 average recall improvement. LLAMA3-8B demonstrated superior performance when paired with RoBERTa, where–as previously discussed–there was markedly less resistance to adap… view at source ↗

**Figure 4.** Figure 4: Average out-of-domain Recall@K and NDGC@K with a progressive number of train tools (decreasing unseen). Reported results refer to RoBERTa on ToolE with REPLUG and PORTS under LLAMA3-8B supervision. investigated the performance of retrieval models trained with REPLUG and PORTS when exposed to varying proportions of unseen tools from the ToolE dataset. Starting with a 90/10 ratio of seen to unseen tools, we… view at source ↗

**Figure 5.** Figure 5: Prompt template for call generation with [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗

**Figure 7.** Figure 7: Average Recall@K across different numbers of negatives and seeds on the ToolE ❺ dataset using ModernBERT and LLAMA3-8B. 0.3 83.64 84.78 85.22 84.61 0.5 83.78 85.78 85.29 84.52 0.7 83.86 84.42 85.05 84.08 1 84.61 0.3 84.52 0.5 84.30 0.7 84.61 1 γ β [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗

**Figure 8.** Figure 8: Confusion Matrix for Average Recall@K on the ToolE ❺ dataset using ModernBERT and LLAMA3- 8B, with varying β and γ hyperparameters. Dataset AVG Recall Seed σ 2 17.45 0 ❶ 14.38 42ToolBench 16.84 100 1.76 56.93 0 API-Bank ❷ 60.38 42 46.73 100 33.59 26.10 0 APIBench ❸ 27.75 42 27.10 100 1.20 55.45 0 BFCL ❹ 60.51 42 61.98 100 7.82 86.86 0 ToolE ❺ 88.32 42 86.37 100 0.69 95.00 0 Octopus ❻ 96.66 42 86.60 100 19.… view at source ↗

**Figure 9.** Figure 9: Average Silhouette scores across datasets (top–general-domain, bottom–tool-specific) and clustering [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

16 extracted references · 10 linked inside Pith

[1]

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, and 1 others

Perception encoder: The best visual embed- dings are not at the output of the network.CoRR, abs/2504.13181. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, and 1 others. 2022. Improving language models by retrieving from trillions of tokens. InICML, volume 162 ofPMLR, pages 2206–2240. PMLR. Wei Chen and Zhiyuan Li. 2024. Octopus v2: On- device languag...

Pith/arXiv arXiv 2022
[2]

Giacomo Domeniconi, Gianluca Moro, Roberto Pa- solini, and Claudio Sartori

SciTePress. Giacomo Domeniconi, Gianluca Moro, Roberto Pa- solini, and Claudio Sartori. 2014b. Cross-domain text classification through iterative refining of tar- get categories representations. InKDIR 2014, pages 31–42. SciTePress. Giacomo Domeniconi, Gianluca Moro, Roberto Pa- solini, and Claudio Sartori. 2014c. Iterative refin- ing of category profiles...

Pith/arXiv arXiv 2014
[3]

In AAAI, pages 18030–18038

Confucius: Iterative tool learning from intro- spection feedback by easy-to-difficult curriculum. In AAAI, pages 18030–18038. AAAI Press. Kelvin Guu, Kenton Lee, Zora Tung, and 1 others. 2020. Retrieval augmented language model pre-training. InICML, volume 119 ofPMLR, pages 3929–3938. PMLR. Shibo Hao, Tianyang Liu, Zhen Wang, and 1 others

2020
[4]

In NeurIPS

Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings. In NeurIPS. Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: monolithic preference optimization without reference model.CoRR, abs/2403.07691. Yue Huang, Jiawen Shi, Yuan Li, and 1 others. 2024. Metatool benchmark for large language models: De- ciding whether to use to...

Pith/arXiv arXiv 2024
[5]

Gautier Izacard, Patrick S

ACL. Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, and 1 others. 2023. Atlas: Few-shot learning with retrieval augmented language models.JMLR, 24:251:1–251:43. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumu- lated gain-based evaluation of IR techniques.ACM Trans. Inf. Syst., 20(4):422–446. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen...

2023
[6]

Mahmut Kaya and Hasan Sakir Bilge

ACL. Mahmut Kaya and Hasan Sakir Bilge. 2019. Deep metric learning: A survey.Symmetry, 11(9):1066. Yilun Kong, Jingqing Ruan, Yihong Chen, and 1 others

2019
[7]

Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and 1 others

Tptu-v2: Boosting task planning and tool usage of large language model-based agents in real- world systems.CoRR, abs/2311.11315. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and 1 others. 2022. Internet-augmented language models through few-shot prompting for open-domain question answering.CoRR, abs/2203.05115. Mingda Li, Xinyu Li, Yifan Che...

Pith/arXiv arXiv 2022
[8]

InEMNLP, pages 3102– 3116, Singapore

API-bank: A comprehensive benchmark for tool-augmented LLMs. InEMNLP, pages 3102– 3116, Singapore. ACL. Xi Victoria Lin, Xilun Chen, Mingda Chen, and 1 others
[9]

RA-DIT: retrieval-augmented dual instruction tuning. InICLR. OpenReview.net. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran- jape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts.TACL, 12:157–173. Yinhan Liu, Myle Ott, Naman Goyal, and 1 others. 2019. Roberta: A robustly optimized BE...

Pith/arXiv arXiv 2024
[10]

Nexusflow.ai

Australian Computer Society. Nexusflow.ai. 2023. Nexusraven: Surpassing the state- of-the-art in open-source function calling llms. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human gener- ated machine reading comprehension dataset.CoRR, abs/1611.09268. OpenAI. 2023. GPT-4 technical re...

Pith/arXiv arXiv 2023
[11]

Yun Peng, Shuqing Li, Wenwei Gu, and 1 others

Gorilla: Large language model connected with massive apis.CoRR, abs/2305.15334. Yun Peng, Shuqing Li, Wenwei Gu, and 1 others. 2023. Revisiting, benchmarking and exploring API recom- mendation: How far are we?IEEE Trans. Software Eng., 49(4):1876–1897. Shuofei Qiao, Honghao Gui, Huajun Chen, and 1 others

Pith/arXiv arXiv 2023
[12]

Yujia Qin, Shihao Liang, Yining Ye, and 1 others

Making language models better tool learners with execution feedback.CoRR, abs/2305.13068. Yujia Qin, Shihao Liang, Yining Ye, and 1 others. 2024. Toolllm: Facilitating large language models to master 16000+ real-world apis. InICLR. OpenReview.net. Changle Qu, Sunhao Dai, Xiaochi Wei, and 1 oth- ers. 2024a. COLT: towards completeness-oriented tool retrieva...

Pith/arXiv arXiv 2024
[13]

InNeurIPS

Hugginggpt: Solving AI tasks with chatgpt and its friends in hugging face. InNeurIPS. Freda Shi, Xinyun Chen, Kanishka Misra, and 1 others
[14]

InICML, volume 202 ofPMLR, pages 31210–31227

Large language models can be easily distracted by irrelevant context. InICML, volume 202 ofPMLR, pages 31210–31227. PMLR. Weijia Shi, Sewon Min, Michihiro Yasunaga, and 1 others. 2024. REPLUG: Retrieval-augmented black- box language models. InNAACL, pages 8371–8384, Mexico City, Mexico. ACL. Shamane Siriwardhana, Rivindu Weerasekera, Tharindu Kaluarachchi...

2024
[15]

Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yix- uan Yuan

Restgpt: Connecting large language models with real-world applications via restful apis.CoRR, abs/2306.06624. Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yix- uan Yuan. 2025. A survey of llm-based agents in medicine: How far are we from baymax? InACL 2025, pages 10345–10359. Association for Computa- tional Lin...

Pith/arXiv arXiv 2025
[16]

"" Query:

C-pack: Packed resources for general chinese embeddings. InSIGIR, pages 641–649. ACM. Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, and 1 others. 2024. Berkeley function calling leaderboard. https://gorilla.cs.berkeley.edu/blogs/ 8_berkeley_function_calling_leaderboard. html. Rui Yang, Lin Song, Yanwei Li, and 1 others. 2023. Gpt4tools: Teaching large la...

Pith/arXiv arXiv 2024

[1] [1]

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, and 1 others

Perception encoder: The best visual embed- dings are not at the output of the network.CoRR, abs/2504.13181. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, and 1 others. 2022. Improving language models by retrieving from trillions of tokens. InICML, volume 162 ofPMLR, pages 2206–2240. PMLR. Wei Chen and Zhiyuan Li. 2024. Octopus v2: On- device languag...

Pith/arXiv arXiv 2022

[2] [2]

Giacomo Domeniconi, Gianluca Moro, Roberto Pa- solini, and Claudio Sartori

SciTePress. Giacomo Domeniconi, Gianluca Moro, Roberto Pa- solini, and Claudio Sartori. 2014b. Cross-domain text classification through iterative refining of tar- get categories representations. InKDIR 2014, pages 31–42. SciTePress. Giacomo Domeniconi, Gianluca Moro, Roberto Pa- solini, and Claudio Sartori. 2014c. Iterative refin- ing of category profiles...

Pith/arXiv arXiv 2014

[3] [3]

In AAAI, pages 18030–18038

Confucius: Iterative tool learning from intro- spection feedback by easy-to-difficult curriculum. In AAAI, pages 18030–18038. AAAI Press. Kelvin Guu, Kenton Lee, Zora Tung, and 1 others. 2020. Retrieval augmented language model pre-training. InICML, volume 119 ofPMLR, pages 3929–3938. PMLR. Shibo Hao, Tianyang Liu, Zhen Wang, and 1 others

2020

[4] [4]

In NeurIPS

Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings. In NeurIPS. Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: monolithic preference optimization without reference model.CoRR, abs/2403.07691. Yue Huang, Jiawen Shi, Yuan Li, and 1 others. 2024. Metatool benchmark for large language models: De- ciding whether to use to...

Pith/arXiv arXiv 2024

[5] [5]

Gautier Izacard, Patrick S

ACL. Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, and 1 others. 2023. Atlas: Few-shot learning with retrieval augmented language models.JMLR, 24:251:1–251:43. Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumu- lated gain-based evaluation of IR techniques.ACM Trans. Inf. Syst., 20(4):422–446. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen...

2023

[6] [6]

Mahmut Kaya and Hasan Sakir Bilge

ACL. Mahmut Kaya and Hasan Sakir Bilge. 2019. Deep metric learning: A survey.Symmetry, 11(9):1066. Yilun Kong, Jingqing Ruan, Yihong Chen, and 1 others

2019

[7] [7]

Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and 1 others

Tptu-v2: Boosting task planning and tool usage of large language model-based agents in real- world systems.CoRR, abs/2311.11315. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and 1 others. 2022. Internet-augmented language models through few-shot prompting for open-domain question answering.CoRR, abs/2203.05115. Mingda Li, Xinyu Li, Yifan Che...

Pith/arXiv arXiv 2022

[8] [8]

InEMNLP, pages 3102– 3116, Singapore

API-bank: A comprehensive benchmark for tool-augmented LLMs. InEMNLP, pages 3102– 3116, Singapore. ACL. Xi Victoria Lin, Xilun Chen, Mingda Chen, and 1 others

[9] [9]

RA-DIT: retrieval-augmented dual instruction tuning. InICLR. OpenReview.net. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paran- jape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts.TACL, 12:157–173. Yinhan Liu, Myle Ott, Naman Goyal, and 1 others. 2019. Roberta: A robustly optimized BE...

Pith/arXiv arXiv 2024

[10] [10]

Nexusflow.ai

Australian Computer Society. Nexusflow.ai. 2023. Nexusraven: Surpassing the state- of-the-art in open-source function calling llms. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human gener- ated machine reading comprehension dataset.CoRR, abs/1611.09268. OpenAI. 2023. GPT-4 technical re...

Pith/arXiv arXiv 2023

[11] [11]

Yun Peng, Shuqing Li, Wenwei Gu, and 1 others

Gorilla: Large language model connected with massive apis.CoRR, abs/2305.15334. Yun Peng, Shuqing Li, Wenwei Gu, and 1 others. 2023. Revisiting, benchmarking and exploring API recom- mendation: How far are we?IEEE Trans. Software Eng., 49(4):1876–1897. Shuofei Qiao, Honghao Gui, Huajun Chen, and 1 others

Pith/arXiv arXiv 2023

[12] [12]

Yujia Qin, Shihao Liang, Yining Ye, and 1 others

Making language models better tool learners with execution feedback.CoRR, abs/2305.13068. Yujia Qin, Shihao Liang, Yining Ye, and 1 others. 2024. Toolllm: Facilitating large language models to master 16000+ real-world apis. InICLR. OpenReview.net. Changle Qu, Sunhao Dai, Xiaochi Wei, and 1 oth- ers. 2024a. COLT: towards completeness-oriented tool retrieva...

Pith/arXiv arXiv 2024

[13] [13]

InNeurIPS

Hugginggpt: Solving AI tasks with chatgpt and its friends in hugging face. InNeurIPS. Freda Shi, Xinyun Chen, Kanishka Misra, and 1 others

[14] [14]

InICML, volume 202 ofPMLR, pages 31210–31227

Large language models can be easily distracted by irrelevant context. InICML, volume 202 ofPMLR, pages 31210–31227. PMLR. Weijia Shi, Sewon Min, Michihiro Yasunaga, and 1 others. 2024. REPLUG: Retrieval-augmented black- box language models. InNAACL, pages 8371–8384, Mexico City, Mexico. ACL. Shamane Siriwardhana, Rivindu Weerasekera, Tharindu Kaluarachchi...

2024

[15] [15]

Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yix- uan Yuan

Restgpt: Connecting large language models with real-world applications via restful apis.CoRR, abs/2306.06624. Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yix- uan Yuan. 2025. A survey of llm-based agents in medicine: How far are we from baymax? InACL 2025, pages 10345–10359. Association for Computa- tional Lin...

Pith/arXiv arXiv 2025

[16] [16]

"" Query:

C-pack: Packed resources for general chinese embeddings. InSIGIR, pages 641–649. ACM. Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, and 1 others. 2024. Berkeley function calling leaderboard. https://gorilla.cs.berkeley.edu/blogs/ 8_berkeley_function_calling_leaderboard. html. Rui Yang, Lin Song, Yanwei Li, and 1 others. 2023. Gpt4tools: Teaching large la...

Pith/arXiv arXiv 2024