REVIEW 13 cited by
If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The prominent large language models (LLMs) of today differ from past language models not only in size, but also in the fact that they are trained on a combination of natural language and formal language (code). As a medium between humans and computers, code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity. In this survey, we present an overview of the various benefits of integrating code into LLMs' training data. Specifically, beyond enhancing LLMs in code generation, we observe that these unique properties of code help (i) unlock the reasoning ability of LLMs, enabling their applications to a range of more complex natural language tasks; (ii) steer LLMs to produce structured and precise intermediate steps, which can then be connected to external execution ends through function calls; and (iii) take advantage of code compilation and execution environment, which also provides diverse feedback for model improvement. In addition, we trace how these profound capabilities of LLMs, brought by code, have led to their emergence as intelligent agents (IAs) in situations where the ability to understand instructions, decompose goals, plan and execute actions, and refine from feedback are crucial to their success on downstream tasks. Finally, we present several key challenges and future directions of empowering LLMs with code.
Forward citations
Cited by 13 Pith papers
-
Back to the Future: A workbook time machine for spread sheet creation benchmarks
The workbook time machine reverse-engineers real workbooks into creation tasks, and the resulting benchmark shows pivot tables remain nearly unsolved by all tested frontier models.
-
Interactive and Expressive Code-Augmented Planning with Large Language Models
REPL-Plan uses an interactive code REPL with recursive sub-REPLs to improve LLM performance on long-horizon text-based planning tasks.
-
OGD4All: A Framework for Accessible Interaction with Geospatial Open Government Data Based on Large Language Models
A new LLM framework retrieves relevant geospatial open-data datasets, generates and executes analysis code in a sandbox, and rejects unanswerable queries, reaching 98% analytical correctness with GPT-4.1 on a 199-ques...
-
Automating Financial Statement Audits with Large Language Models
A five-stage benchmark shows LLMs detect financial statement errors well but fail at explanation, standards citation, and statement revision.
-
Self-Challenging Language Model Agents
A language model agent can generate its own verifiable training tasks and improve its tool-use success rate by about 2x without human-annotated data.
-
Branch-and-Browse: Efficient and Controllable Web Exploration with Tree-Structured Reasoning and Action Memory
Branch-and-Browse, a tree-structured web agent with page-level action memory, reports 35.8% success and up to 40.4% less time than the cited Tree Search baseline on WebArena.
-
PresentAgent: Multimodal Agent for Presentation Video Generation
PresentAgent chains LLM segmentation, slide rendering, TTS, and ffmpeg to turn documents into narrated presentation videos, but the human-level claim rests on five documents and an unvalidated VLM judge.
-
SimuGen: Multi-modal Agentic Framework for Constructing Block Diagram-Based Simulation Models
SimuGen uses specialized LLM agents and a Simulink block database to convert diagram images into executable MATLAB/Simulink code, reporting 94.5% average accuracy on nine examples.
-
Reasoning-OCR: Can Large Multimodal Models Solve Complex Logical Reasoning Problems from OCR Cues?
Reasoning-OCR is a new bilingual benchmark of 150 image-based logical reasoning questions; the best tested model, GPT-4o, scores 68.1% and open-source models stay below 63%.
-
Mining Hidden Thoughts from Texts: Evaluating Continual Pretraining with Synthetic Data for LLM Reasoning
Continual pretraining on texts augmented with LLM-generated "hidden thoughts" improves Gemma2-9B's MMLU accuracy more than standard continual pretraining, with the largest gains on harder questions and across domains.
-
Salamandra Technical Report
Salamandra is an open, from-scratch multilingual LLM family with 2B, 7B, and 40B checkpoints, instruction-tuned variants, a vision proof-of-concept, and detailed evaluations across Iberian and European languages.
-
TUMS: Enhancing Tool-use Abilities of LLMs with Multi-structure Handlers
A prompt-based framework improves LLM tool-use accuracy on ToolQA by generating tool parameters with tool-specific handler structures instead of one generic structure.
-
Practical Considerations for Agentic LLM Systems
This paper is a practical survey that organizes research on LLM-based agents into design considerations for planning, memory, tools, and control flow.
Discussion (0). Continue with ORCID to comment.