REVIEW 20 cited by
FireAct: Toward Language Agent Fine-tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent efforts have augmented language models (LMs) with external tools or environments, leading to the development of language agents that can reason and act. However, most of these agents rely on few-shot prompting techniques with off-the-shelf LMs. In this paper, we investigate and argue for the overlooked direction of fine-tuning LMs to obtain language agents. Using a setup of question answering (QA) with a Google search API, we explore a variety of base LMs, prompting methods, fine-tuning data, and QA tasks, and find language agents are consistently improved after fine-tuning their backbone LMs. For example, fine-tuning Llama2-7B with 500 agent trajectories generated by GPT-4 leads to a 77% HotpotQA performance increase. Furthermore, we propose FireAct, a novel approach to fine-tuning LMs with trajectories from multiple tasks and prompting methods, and show having more diverse fine-tuning data can further improve agents. Along with other findings regarding scaling effects, robustness, generalization, efficiency and cost, our work establishes comprehensive benefits of fine-tuning LMs for agents, and provides an initial set of experimental designs, insights, as well as open questions toward language agent fine-tuning.
Forward citations
Cited by 20 Pith papers
-
AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents
A trajectory-adaptive honeypot system, AgentSnare, achieves a 0/45 verified exploit rate against LLM-based penetration testers across 15 vulnerable web apps and three attacker models.
-
MemHarness: Memory Is Reconstructed, Not Replayed
End-to-end GRPO training teaches a unified LLM policy to reconstruct retrieved experiences into state-aligned guidance, outperforming verbatim memory replay on ALFWorld and WebShop.
-
Leveraging Trajectory Graphs for Pre-Execution Error Diagnosis in Agentic LLM Systems
Modeling agent trajectories as action-centric probabilistic graphs lets a GNN warn LLM agents of likely step-level errors before execution, improving pass ratio ~14.7% across four benchmarks.
-
Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories
Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.
-
Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution
A graph-guided multi-agent framework plus asymmetric episodic distillation lets fine-tuned 32B models match larger teachers on warehouse SOP execution and hit 94% expert agreement on ticket processing.
-
STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Training
Normalized entropy locates trajectory-neglect steps; STAPO selectively optimizes them with a trajectory-aware reward and trajectory-independent penalty, beating group-based RL baselines on agent tasks.
-
From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Training
HPO learns language-agent policies by using the Wasserstein distance between the current policy and a hindsight distribution in an intent embedding space, producing low-variance step-level advantages.
-
Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedures
On multi-step procedural tasks, LoRA fine-tuning underperforms full fine-tuning at every rank tested because procedural knowledge requires high-rank weight updates.
-
SOD: Step-wise On-policy Distillation for Small Language Model Agents
SOD reweights on-policy distillation strength step-by-step using divergence to stabilize tool use in small language model agents, yielding up to 20.86% gains and 26.13% on AIME 2025 for a 0.6B model.
-
Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models
A new benchmark shows multilingual tool-calling errors in LLMs are mostly parameter-language mismatches at the execution boundary, not failures of intent understanding.
-
Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems
MOAT alternately aligns a planning and a grounding LLM agent via perplexity-based DPO and self-generated SFT, reporting small but consistent gains over independently tuned baselines.
-
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
A new one-million-sample multimodal agent tuning dataset with GPT-4o-generated rationales, reflection, and tool/RAG calls is shown to improve fine-tuned models, though training/eval benchmark overlap is not addressed.
-
SAND: Boosting LLM Agents with Self-Taught Action Deliberation
A self-training framework that uses self-consistency action sampling and execution-guided critiques to teach LLM agents to explicitly deliberate over candidate actions, improving average rewards by about 20% over supe...
-
Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning
A 7B LLM trained with sparse completion rewards and a GRPO-style algorithm reaches state-of-the-art on ALFWorld and ScienceWorld.
-
Reason Before You Retrieve: Agentic Planning for Multi-modal RAG
MM-R2 claims SOTA multimodal RAG accuracy on InfoSeek and Encyclopedic VQA via intent grounding plus a 10-topic KnowledgeMap, but its teacher trajectories leak the gold Wikipedia page and omit the image.
-
Initial Steps in Integrating Large Reasoning and Action Models for Service Composition
A conceptual framework integrating LRMs and LAMs for end-to-end automated service composition, with no empirical validation.
-
Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents
A new step-level reward-modeling benchmark for multimodal agents shows current MLLMs reach at most 61.6 percent accuracy, and benchmark score correlates strongly (r=0.981 across five models) with downstream A* search ...
-
ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning
An episode-level variant of GRPO (ESPO) improves personalized GUI-agent reasoning on the 102-episode SmartSpot benchmark, outperforming step-wise and outcome-only training baselines.
-
MemVerse: Multimodal Memory for Lifelong Learning Agents
MemVerse reports large gains on multimodal benchmarks by adding a hierarchical knowledge-graph memory plus fine-tuned parametric recall, but its strongest video-retrieval result uses ground-truth caption-video pairs i...
-
Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems
On MobileCS2, a GPT-2 finetuned with retrieved knowledge outperforms a prompted GPT-3.5 in both RAG and agent dialog systems, though the comparison is confounded.
Discussion (0). Sign in to comment.