{"work":{"id":"3c555b48-a4d9-42dd-9fdd-0f6018fbe9cb","openalex_id":"https://openalex.org/W4385474529","doi":"10.48550/arxiv.2307.16789","arxiv_id":"2307.16789","raw_key":null,"title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs","authors":null,"authors_text":"Y","year":2023,"venue":"cs.AI","abstract":"Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain. This is in contrast to the excellent tool-use capabilities of state-of-the-art (SOTA) closed-source LLMs, e.g., ChatGPT. To bridge this gap, we introduce ToolLLM, a general tool-use framework encompassing data construction, model training, and evaluation. We first present ToolBench, an instruction-tuning dataset for tool use, which is constructed automatically using ChatGPT. Specifically, the construction can be divided into three stages: (i) API collection: we collect 16,464 real-world RESTful APIs spanning 49 categories from RapidAPI Hub; (ii) instruction generation: we prompt ChatGPT to generate diverse instructions involving these APIs, covering both single-tool and multi-tool scenarios; (iii) solution path annotation: we use ChatGPT to search for a valid solution path (chain of API calls) for each instruction. To enhance the reasoning capabilities of LLMs, we develop a novel depth-first search-based decision tree algorithm. It enables LLMs to evaluate multiple reasoning traces and expand the search space. Moreover, to evaluate the tool-use capabilities of LLMs, we develop an automatic evaluator: ToolEval. Based on ToolBench, we fine-tune LLaMA to obtain an LLM ToolLLaMA, and equip it with a neural API retriever to recommend appropriate APIs for each instruction. Experiments show that ToolLLaMA demonstrates a remarkable ability to execute complex instructions and generalize to unseen APIs, and exhibits comparable performance to ChatGPT. Our ToolLLaMA also demonstrates strong zero-shot generalization ability in an out-of-distribution tool-use dataset: APIBench.","external_url":"https://arxiv.org/abs/2307.16789","cited_by_count":69,"metadata_source":"pith","metadata_fetched_at":"2026-08-05T02:28:24.338817+00:00","pith_arxiv_id":"2307.16789","created_at":"2026-05-09T00:14:27.020535+00:00","updated_at":"2026-08-05T02:28:24.338817+00:00","title_quality_ok":true,"display_title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs","render_title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs"},"hub":{"state":{"work_id":"3c555b48-a4d9-42dd-9fdd-0f6018fbe9cb","tier":"super_hub","tier_reason":"100+ Pith inbound or 10,000+ external citations","pith_inbound_count":199,"external_cited_by_count":69,"distinct_field_count":15,"first_pith_cited_at":"2023-07-12T20:01:52+00:00","last_pith_cited_at":"2026-07-09T17:59:32+00:00","author_build_status":"needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-21T17:59:23.659165+00:00","tier_text":"super_hub"},"tier":"super_hub","role_counts":[{"context_role":"background","n":29},{"context_role":"dataset","n":2}],"polarity_counts":[{"context_polarity":"background","n":28},{"context_polarity":"use_dataset","n":2},{"context_polarity":"support","n":1}],"runs":{"ask_index":{"job_type":"ask_index","status":"succeeded","result":{"title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs","claims":[{"claim_text":"Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain. This is in contrast to the excellent tool-use capabilities of state-of-the-art (SOTA) closed-source LLMs, e.g., ChatGPT. To bridge this gap, we introduce ToolLLM, a general tool-use framework encompassing data construction, model training, and evaluation. We first present ","claim_type":"abstract","evidence_strength":"source_metadata"},{"claim_text":"Llm+ p: Empowering large language models with optimal planning proficiency.arXiv preprint arXiv:2304.11477, 2023. 16 [119] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154-38180, 2023. [120] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating","claim_type":"background","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Interactive-agent evaluation and outcome checks.Interactive-agent benchmarks evaluate agents that navigate interfaces, call tools, and alter environment state. Web-agent benchmarks include MINI- WOB [10], WebShop [28], Mind2Web [3], VisualWebArena [9], WebArena [31], and BrowserGym [1]. Tool-use and stateful-interaction benchmarks include ToolBench-style tool-use benchmarks [19], τ-bench [29], τ 3-bench retail [22], and ToolSandbox [11]. Broader agent environments include AGENTDOJO[ 2], ANDROIDW","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"We evaluate UNO-ORCHESTRAand competing routers on 13 bench- marks spanning five capability domains:mathematical reasoning(MATH-500 [ 36], AIME [ 3]); code & software engineering(HumanEval [ 8], MBPP [5], LiveCodeBench [28], SWE-bench [30]); knowledge & scientific reasoning(MMLU [ 24], GPQA [49]);reading & long-context(DROP [ 15], MRCR [61]);agentic & tool use(GAIA [ 40], Terminal-Bench [63], ToolBench [46]); plus LLM- RouterBench [33] as a routing-specialised diagnostic that isnotcounted in the ","claim_type":"dataset","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"https : / / huggingface.co/ProtectAI/deberta-v3-base-prompt-injection-v2 . 2024. [46] Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. \"Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI\". In: 2022 ACM Conference on Fairness, Accountability, and Transparency. FAccT '22. ACM, 2022.DOI: 10.1145/3531146. 3533231. [47] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. \"ToolLLM: Facilitating la","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"face: software engineering (SWE-bench [ 17], Terminal-Bench [24], LiveCodeBench [ 16]), web and GUI con- trol (WebArena [59], WebShop [48], VisualWebArena [20]), OS and mobile control (OSWorld [45], Windows Agent Arena [ 5], AndroidWorld [ 33]), enterprise knowledge work (WorkArena [ 11], OdysseyBench [ 38]), interactive coding (AppWorld [ 37]), browsing-centric research (BrowseComp [ 40]), and tool orchestration (ToolBench [31], τ -bench [ 50]). Broader suites such as GAIA [ 25] and TheAgentCom","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"Gorilla: Large language model connected with massive apis.Advances in Neural Information Processing Systems, 37:126544-126565, 2024. [36] Ruoyu Qin, Weiran He, Weixiao Huang, Yangkun Zhang, Yikai Zhao, Bo Pang, Xinran Xu, Yingdi Shan, Yongwei Wu, and Mingxing Zhang. Seer: Online con- text learning for fast synchronous llm reinforcement learning.arXiv preprint arXiv:2511.14617, 2025. [37] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qi","claim_type":"background","confidence":0.85,"evidence_strength":"citation_context"}],"why_cited":"Pith tracks ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs because it crossed a citation-hub threshold. Current citing contexts most often use it as background evidence (16 contexts).","role_counts":[{"n":16,"context_role":"background"},{"n":2,"context_role":"dataset"}]},"error":null,"updated_at":"2026-05-18T11:10:49.049730+00:00"},"author_expand":{"job_type":"author_expand","status":"succeeded","result":{"authors_linked":[]},"error":null,"updated_at":"2026-05-18T11:10:49.052165+00:00"},"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T08:18:00.536717+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"WebArena: A Realistic Web Environment for Building Autonomous Agents","work_id":"7058ffd2-a339-4102-89eb-248eeb074652","shared_citers":17},{"title":"Gorilla: Large Language Model Connected with Massive APIs","work_id":"126a464a-4a73-495f-b669-de1e44aa8f09","shared_citers":16},{"title":"SWE-bench: Can Language Models Resolve Real-World GitHub Issues?","work_id":"d0effe15-a689-441a-8e3f-ea35f1c4e4b1","shared_citers":13},{"title":"Voyager: An Open-Ended Embodied Agent with Large Language Models","work_id":"ffe0d207-86cf-4742-a100-e988ac8b9676","shared_citers":13},{"title":"AgentBench: Evaluating LLMs as Agents","work_id":"a37549b4-4c94-412d-acc4-4efeb08509be","shared_citers":12},{"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","shared_citers":12},{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":12},{"title":"ReAct: Synergizing Reasoning and Acting in Language Models","work_id":"407a2351-25f1-497d-b611-f77d0292a8e6","shared_citers":11},{"title":"Toolformer: Language Models Can Teach Themselves to Use Tools","work_id":"9bce40c8-cfd7-4983-80e0-c3bd4402322a","shared_citers":10},{"title":"$\\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains","work_id":"6a8d8dc4-0cc0-4052-8109-abbcdcd4a962","shared_citers":9},{"title":"DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models","work_id":"c5006563-f3ec-438a-9e35-b7b484f34828","shared_citers":9},{"title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","work_id":"008df105-2fdd-45d8-857a-8e35868aecb6","shared_citers":9},{"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","shared_citers":9},{"title":"Identifying the Risks of LM Agents with an LM-Emulated Sandbox","work_id":"3d4c3b66-d749-4939-b1bc-62b10b2ebbb6","shared_citers":9},{"title":"DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning","work_id":"e6b75ad5-2877-4168-97c8-710407094d20","shared_citers":8},{"title":"AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation","work_id":"92b7eb9c-c3d8-4518-a376-06fa15dd895b","shared_citers":6},{"title":"DeepSeek-V3 Technical Report","work_id":"57d2791d-2219-4c31-a077-afc04b12a75c","shared_citers":6},{"title":"LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code","work_id":"ea9e51ce-1e75-4182-92d8-4d25f70d2ee4","shared_citers":6},{"title":"Qwen2.5 Technical Report","work_id":"d8432992-4980-4a81-85c7-9fa2c2b87f85","shared_citers":6},{"title":"Retrieval-Augmented Generation for Large Language Models: A Survey","work_id":"b80d2790-6cd9-4c87-b3c4-de404f99a80e","shared_citers":6},{"title":"Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning","work_id":"0e0b7549-2bc4-4574-aa7f-588ffa16eaae","shared_citers":6},{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":6},{"title":"Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback","work_id":"a1f2574b-a899-4713-be60-c87ba332656c","shared_citers":6},{"title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","work_id":"3322fa86-1768-4677-8425-dd326b45e078","shared_citers":6}],"time_series":[{"n":2,"year":2023},{"n":2,"year":2024},{"n":1,"year":2025},{"n":67,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T08:18:09.055767+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T08:18:08.954446+00:00"},"role_polarity":{"job_type":"role_polarity","status":"succeeded","result":{"title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs","claims":[{"claim_text":"Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain. This is in contrast to the excellent tool-use capabilities of state-of-the-art (SOTA) closed-source LLMs, e.g., ChatGPT. To bridge this gap, we introduce ToolLLM, a general tool-use framework encompassing data construction, model training, and evaluation. We first present ","claim_type":"abstract","evidence_strength":"source_metadata"},{"claim_text":"Llm+ p: Empowering large language models with optimal planning proficiency.arXiv preprint arXiv:2304.11477, 2023. 16 [119] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.Advances in Neural Information Processing Systems, 36:38154-38180, 2023. [120] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating","claim_type":"background","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Interactive-agent evaluation and outcome checks.Interactive-agent benchmarks evaluate agents that navigate interfaces, call tools, and alter environment state. Web-agent benchmarks include MINI- WOB [10], WebShop [28], Mind2Web [3], VisualWebArena [9], WebArena [31], and BrowserGym [1]. Tool-use and stateful-interaction benchmarks include ToolBench-style tool-use benchmarks [19], τ-bench [29], τ 3-bench retail [22], and ToolSandbox [11]. Broader agent environments include AGENTDOJO[ 2], ANDROIDW","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"We evaluate UNO-ORCHESTRAand competing routers on 13 bench- marks spanning five capability domains:mathematical reasoning(MATH-500 [ 36], AIME [ 3]); code & software engineering(HumanEval [ 8], MBPP [5], LiveCodeBench [28], SWE-bench [30]); knowledge & scientific reasoning(MMLU [ 24], GPQA [49]);reading & long-context(DROP [ 15], MRCR [61]);agentic & tool use(GAIA [ 40], Terminal-Bench [63], ToolBench [46]); plus LLM- RouterBench [33] as a routing-specialised diagnostic that isnotcounted in the ","claim_type":"dataset","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"https : / / huggingface.co/ProtectAI/deberta-v3-base-prompt-injection-v2 . 2024. [46] Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. \"Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI\". In: 2022 ACM Conference on Fairness, Accountability, and Transparency. FAccT '22. ACM, 2022.DOI: 10.1145/3531146. 3533231. [47] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. \"ToolLLM: Facilitating la","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"face: software engineering (SWE-bench [ 17], Terminal-Bench [24], LiveCodeBench [ 16]), web and GUI con- trol (WebArena [59], WebShop [48], VisualWebArena [20]), OS and mobile control (OSWorld [45], Windows Agent Arena [ 5], AndroidWorld [ 33]), enterprise knowledge work (WorkArena [ 11], OdysseyBench [ 38]), interactive coding (AppWorld [ 37]), browsing-centric research (BrowseComp [ 40]), and tool orchestration (ToolBench [31], τ -bench [ 50]). Broader suites such as GAIA [ 25] and TheAgentCom","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"Gorilla: Large language model connected with massive apis.Advances in Neural Information Processing Systems, 37:126544-126565, 2024. [36] Ruoyu Qin, Weiran He, Weixiao Huang, Yangkun Zhang, Yikai Zhao, Bo Pang, Xinran Xu, Yingdi Shan, Yongwei Wu, and Mingxing Zhang. Seer: Online con- text learning for fast synchronous llm reinforcement learning.arXiv preprint arXiv:2511.14617, 2025. [37] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qi","claim_type":"background","confidence":0.85,"evidence_strength":"citation_context"}],"why_cited":"Pith tracks ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs because it crossed a citation-hub threshold. Current citing contexts most often use it as background evidence (16 contexts).","role_counts":[{"n":16,"context_role":"background"},{"n":2,"context_role":"dataset"}]},"error":null,"updated_at":"2026-05-18T11:10:49.045928+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs","claims":[{"claim_text":"Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain. This is in contrast to the excellent tool-use capabilities of state-of-the-art (SOTA) closed-source LLMs, e.g., ChatGPT. To bridge this gap, we introduce ToolLLM, a general tool-use framework encompassing data construction, model training, and evaluation. We first present ","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T08:18:20.930982+00:00"}},"summary":{"title":"ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs","claims":[{"claim_text":"Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain. This is in contrast to the excellent tool-use capabilities of state-of-the-art (SOTA) closed-source LLMs, e.g., ChatGPT. To bridge this gap, we introduce ToolLLM, a general tool-use framework encompassing data construction, model training, and evaluation. We first present ","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"WebArena: A Realistic Web Environment for Building Autonomous Agents","work_id":"7058ffd2-a339-4102-89eb-248eeb074652","shared_citers":17},{"title":"Gorilla: Large Language Model Connected with Massive APIs","work_id":"126a464a-4a73-495f-b669-de1e44aa8f09","shared_citers":16},{"title":"SWE-bench: Can Language Models Resolve Real-World GitHub Issues?","work_id":"d0effe15-a689-441a-8e3f-ea35f1c4e4b1","shared_citers":13},{"title":"Voyager: An Open-Ended Embodied Agent with Large Language Models","work_id":"ffe0d207-86cf-4742-a100-e988ac8b9676","shared_citers":13},{"title":"AgentBench: Evaluating LLMs as Agents","work_id":"a37549b4-4c94-412d-acc4-4efeb08509be","shared_citers":12},{"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","shared_citers":12},{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":12},{"title":"ReAct: Synergizing Reasoning and Acting in Language Models","work_id":"407a2351-25f1-497d-b611-f77d0292a8e6","shared_citers":11},{"title":"Toolformer: Language Models Can Teach Themselves to Use Tools","work_id":"9bce40c8-cfd7-4983-80e0-c3bd4402322a","shared_citers":10},{"title":"$\\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains","work_id":"6a8d8dc4-0cc0-4052-8109-abbcdcd4a962","shared_citers":9},{"title":"DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models","work_id":"c5006563-f3ec-438a-9e35-b7b484f34828","shared_citers":9},{"title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","work_id":"008df105-2fdd-45d8-857a-8e35868aecb6","shared_citers":9},{"title":"GPT-4 Technical Report","work_id":"b928e041-6991-4c08-8c81-0359e4097c7b","shared_citers":9},{"title":"Identifying the Risks of LM Agents with an LM-Emulated Sandbox","work_id":"3d4c3b66-d749-4939-b1bc-62b10b2ebbb6","shared_citers":9},{"title":"DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning","work_id":"e6b75ad5-2877-4168-97c8-710407094d20","shared_citers":8},{"title":"AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation","work_id":"92b7eb9c-c3d8-4518-a376-06fa15dd895b","shared_citers":6},{"title":"DeepSeek-V3 Technical Report","work_id":"57d2791d-2219-4c31-a077-afc04b12a75c","shared_citers":6},{"title":"LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code","work_id":"ea9e51ce-1e75-4182-92d8-4d25f70d2ee4","shared_citers":6},{"title":"Qwen2.5 Technical Report","work_id":"d8432992-4980-4a81-85c7-9fa2c2b87f85","shared_citers":6},{"title":"Retrieval-Augmented Generation for Large Language Models: A Survey","work_id":"b80d2790-6cd9-4c87-b3c4-de404f99a80e","shared_citers":6},{"title":"Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning","work_id":"0e0b7549-2bc4-4574-aa7f-588ffa16eaae","shared_citers":6},{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":6},{"title":"Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback","work_id":"a1f2574b-a899-4713-be60-c87ba332656c","shared_citers":6},{"title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","work_id":"3322fa86-1768-4677-8425-dd326b45e078","shared_citers":6}],"time_series":[{"n":2,"year":2023},{"n":2,"year":2024},{"n":1,"year":2025},{"n":67,"year":2026}],"dependency_candidates":[]},"authors":[]}}