REVIEW 5 cited by
An LLM-Tool Compiler for Fused Parallel Function Calling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
State-of-the-art sequential reasoning in Large Language Models (LLMs) has expanded the capabilities of Copilots beyond conversational tasks to complex function calling, managing thousands of API calls. However, the tendency of compositional prompting to segment tasks into multiple steps, each requiring a round-trip to the GPT APIs, leads to increased system latency and costs. Although recent advancements in parallel function calling have improved tool execution per API call, they may necessitate more detailed in-context instructions and task breakdown at the prompt level, resulting in higher engineering and production costs. Inspired by the hardware design principles of multiply-add (MAD) operations, which fuse multiple arithmetic operations into a single task from the compiler's perspective, we propose LLM-Tool Compiler, which selectively fuses similar types of tool operations under a single function at runtime, presenting them as a unified task to the LLM. This selective fusion inherently enhances parallelization and efficiency. Benchmarked on a large-scale Copilot platform, LLM-Tool Compiler achieves up to four times more parallel calls than existing methods, reducing token costs and latency by up to 40% and 12%, respectively.
Forward citations
Cited by 5 Pith papers
-
GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction
Decomposing graph reasoning into graph extraction, tool name identification, and tool parameter extraction subtasks yields large accuracy gains on small LLMs and a fine-tuned 8B model that competes with GPT-4o.
-
Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference
EHPC uses a handful of attention heads, selected via a synthetic needle-in-a-haystack test, to compress long prompts and speed up LLM inference while retaining task performance.
-
Less is More: Optimizing Function Calling for LLM Execution on Edge Devices
Selectively reducing the number of tools presented to an LLM, using embedding similarity over individual tools or clusters, improves function-calling success and efficiency on edge devices.
-
LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios
A survey that classifies LLM-based agent reasoning frameworks into single-agent, tool-based, and multi-agent types and reviews their use across scientific, medical, software, and social domains.
-
Toward Efficient Agents: Memory, Tool learning, and Planning
A survey that organizes efficiency techniques for LLM agents into memory, tool learning, and planning, and consolidates benchmarks and metrics for measuring cost-performance trade-offs.
Discussion (0). Continue with ORCID to comment.