REVIEW 6 cited by
LLM Agents Making Agent Tools
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Tool use has turned large language models (LLMs) into powerful agents that can perform complex multi-step tasks by dynamically utilising external software components. However, these tools must be implemented in advance by human developers, hindering the applicability of LLM agents in domains demanding large numbers of highly specialised tools, like in life sciences and medicine. Motivated by the growing trend of scientific studies accompanied by public code repositories, we propose ToolMaker, an agentic framework that autonomously transforms papers with code into LLM-compatible tools. Given a GitHub URL and short task description, ToolMaker autonomously installs dependencies and generates code to perform the task, using a closed-loop self-correction mechanism for debugging. To evaluate our approach, we introduce a benchmark comprising 15 complex computational tasks spanning various domains with over 100 unit tests to assess correctness and robustness. Our method correctly implements 80% of the tasks, substantially outperforming current state-of-the-art software engineering agents. ToolMaker therefore is a step towards fully autonomous agent-based scientific workflows. Our code and benchmark are publicly available at https://github.com/KatherLab/ToolMaker.
Forward citations
Cited by 6 Pith papers
-
Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents
Synthesized tools from tool-evolving agents pass in-session checks but 96.8% of 222 tools score C=0.00 on held-out conformance suites that hand-written references pass perfectly.
-
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
A constrained, feedback-augmented, component-wise harness evolution loop improves coding agent Pass@1 on SWE-bench Verified and transfers to held-out tasks and another base model.
-
When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agents
The paper claims prompt compression is a new attack surface, but the abstract's COMA attack never appears in the body and the body's SoftCom requires white-box access.
-
A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomy
A position paper arguing that LLM-based human-agent systems, not fully autonomous agents, should be the immediate goal for AI development.
-
LLM-Agent-Controller: A Universal Multi-Agent Large Language Model System as a Control Engineer
A nine-agent LLM system with RAG and 140+ Python Control Library tools solves textbook control-theory tasks with a reported 83% completion rate.
-
Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI
A qualitative taxonomy positions vibe coding and agentic coding as complementary paradigms rather than rivals in AI-assisted software development.
Discussion (0). Continue with ORCID to comment.