Pith. sign in

REVIEW 6 cited by

Asynchronous LLM Function Calling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2412.07017 v1 pith:KMZR4RHI submitted 2024-12-09 cs.CL cs.AI

In Gim , Seung-seob Lee , Lin Zhong This is my paper
classification cs.CLcs.AI
keywords functioncallingasynclmcallsinterruptllmsasynchronouscall
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) use function calls to interface with external tools and data source. However, the current approach to LLM function calling is inherently synchronous, where each call blocks LLM inference, limiting LLM operation and concurrent function execution. In this work, we propose AsyncLM, a system for asynchronous LLM function calling. AsyncLM improves LLM's operational efficiency by enabling LLMs to generate and execute function calls concurrently. Instead of waiting for each call's completion, AsyncLM introduces an interrupt mechanism to asynchronously notify the LLM in-flight when function calls return. We design an in-context protocol for function calls and interrupts, provide fine-tuning strategy to adapt LLMs to the interrupt semantics, and implement these mechanisms efficiently on LLM inference process. We demonstrate that AsyncLM can reduce end-to-end task completion latency from 1.6x-5.4x compared to synchronous function calling on a set of benchmark tasks in the Berkeley function calling leaderboard (BFCL). Furthermore, we discuss how interrupt mechanisms can be extended to enable novel human-LLM or LLM-LLM interactions.

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows

    cs.LG 2026-02 conditional novelty 6.0 of 10

    A runtime that adds epoch-based, frontier-gated commits and compensation-based aborts to LLM-agent tool calls, improving fault recovery and preventing side effects from losing speculative branches.

  2. Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live

    cs.OS 2025-11 unverdicted novelty 6.0 of 10

    TTL-based KV-cache pinning that uses predicted tool-call durations and queueing-delay costs cuts agent job completion time by up to 8x.

  3. Neural Object Detection for 4D STEM: High-Throughput Sub-Pixel Electron Diffraction Pattern Recognition

    cond-mat.mtrl-sci 2025-06 conditional novelty 5.0 of 10

    A YOLOv8-based object detector localizes Bragg discs in 4D-STEM diffraction patterns with reported strain precision of 5x10^-4 and throughput above 100 fps, demonstrated on Si/SiGe and Ti-Nb samples.

  4. AI Agent for Reverse-Engineering Legacy Finite-Difference Code and Translating to Devito

    cs.AI 2026-01 conditional novelty 4.0 of 10

    An AI agent combining GraphRAG, static Fortran analysis, and LLM code generation is reported to translate legacy Fortran finite-difference code into Devito, with Grade-A results claimed on roughly three-quarters of 13...

  5. AugServe: Adaptive Request Scheduling for Augmented Large Language Model Inference Serving

    cs.CL 2025-12 conditional novelty 4.0 of 10

    An adaptive two-stage scheduler plus dynamic token batching improves SLO-satisfying throughput for tool-augmented LLM inference versus vLLM and InferCept in the reported experiments.

  6. ODIA: Oriented Distillation for Inline Acceleration of LLM-based Function Calling

    cs.LG 2025-07 conditional novelty 3.0 of 10

    ODIA routes 60% of function-calling traffic in a music app to a small 1.3B model, reducing expected latency by 45% and median latency by 78%.

Pith tools