REVIEW 33 cited by
FireAct: Toward Language Agent Fine-tuning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent efforts have augmented language models (LMs) with external tools or environments, leading to the development of language agents that can reason and act. However, most of these agents rely on few-shot prompting techniques with off-the-shelf LMs. In this paper, we investigate and argue for the overlooked direction of fine-tuning LMs to obtain language agents. Using a setup of question answering (QA) with a Google search API, we explore a variety of base LMs, prompting methods, fine-tuning data, and QA tasks, and find language agents are consistently improved after fine-tuning their backbone LMs. For example, fine-tuning Llama2-7B with 500 agent trajectories generated by GPT-4 leads to a 77% HotpotQA performance increase. Furthermore, we propose FireAct, a novel approach to fine-tuning LMs with trajectories from multiple tasks and prompting methods, and show having more diverse fine-tuning data can further improve agents. Along with other findings regarding scaling effects, robustness, generalization, efficiency and cost, our work establishes comprehensive benefits of fine-tuning LMs for agents, and provides an initial set of experimental designs, insights, as well as open questions toward language agent fine-tuning.
Forward citations
Cited by 33 Pith papers
-
AgentSnare: Learning to Delay, Divert, and Defuse Autonomous Penetration Agents
A trajectory-adaptive honeypot system, AgentSnare, achieves a 0/45 verified exploit rate against LLM-based penetration testers across 15 vulnerable web apps and three attacker models.
-
MemHarness: Memory Is Reconstructed, Not Replayed
End-to-end GRPO training teaches a unified LLM policy to reconstruct retrieved experiences into state-aligned guidance, outperforming verbatim memory replay on ALFWorld and WebShop.
-
Leveraging Trajectory Graphs for Pre-Execution Error Diagnosis in Agentic LLM Systems
Modeling agent trajectories as action-centric probabilistic graphs lets a GNN warn LLM agents of likely step-level errors before execution, improving pass ratio ~14.7% across four benchmarks.
-
Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories
Offline DPO on expert states with one-step student negatives and schema-preserving augmentation improves LLM agents beyond SFT and can match online GRPO without environment interaction.
-
Eluna: An Agentic LLM System for Automating Warehouse Operations with Reasoning and Task Execution
A graph-guided multi-agent framework plus asymmetric episodic distillation lets fine-tuned 32B models match larger teachers on warehouse SOP execution and hit 94% expert agreement on ticket processing.
-
STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Training
Normalized entropy locates trajectory-neglect steps; STAPO selectively optimizes them with a trajectory-aware reward and trajectory-independent penalty, beating group-based RL baselines on agent tasks.
-
From Outcomes to Actions: Leveraging Hindsight for Long-Horizon Language Agent Training
HPO learns language-agent policies by using the Wasserstein distance between the current policy and a hindsight distribution in an intent embedding space, producing low-variance step-level advantages.
-
Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedures
On multi-step procedural tasks, LoRA fine-tuning underperforms full fine-tuning at every rank tested because procedural knowledge requires high-rank weight updates.
-
SOD: Step-wise On-policy Distillation for Small Language Model Agents
SOD reweights on-policy distillation strength step-by-step using divergence to stabilize tool use in small language model agents, yielding up to 20.86% gains and 26.13% on AIME 2025 for a 0.6B model.
-
Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models
A new benchmark shows multilingual tool-calling errors in LLMs are mostly parameter-language mismatches at the execution boundary, not failures of intent understanding.
-
Bridging the Capability Gap: Joint Alignment Tuning for Harmonizing LLM-based Multi-Agent Systems
MOAT alternately aligns a planning and a grounding LLM agent via perplexity-based DPO and self-generated SFT, reporting small but consistent gains over independently tuned baselines.
-
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
A new one-million-sample multimodal agent tuning dataset with GPT-4o-generated rationales, reflection, and tool/RAG calls is shown to improve fine-tuned models, though training/eval benchmark overlap is not addressed.
-
SAND: Boosting LLM Agents with Self-Taught Action Deliberation
A self-training framework that uses self-consistency action sampling and execution-guided critiques to teach LLM agents to explicitly deliberate over candidate actions, improving average rewards by about 20% over supe...
-
Unleashing Embodied Task Planning Ability in LLMs via Reinforcement Learning
A 7B LLM trained with sparse completion rewards and a GRPO-style algorithm reaches state-of-the-art on ALFWorld and ScienceWorld.
-
LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback
Self-generated agent trajectories from LAM SIMULATOR improved fine-tuned model pass rates by up to 49.3% on ToolBench and CRMArena.
-
PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization
Pseudocode-structured plans paired with preference optimization improve LLM agent success rates and generalization across interactive benchmarks.
-
Agent-Environment Alignment via Automated Interface Generation
ALIGN automatically generates and verifies an interface wrapper that enriches environment observations, improving LLM agent success rates by up to 45.67 percentage points on ALFWorld without modifying the agent or env...
-
Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking
A new agent-training method combining teacher-generated self-reflection corrections with partial masking of error steps improves open-source LLM agents on ALFWorld, WebShop, and SciWorld.
-
Divide and Conquer: Grounding LLMs as Efficient Decision-Making Agents via Offline Hierarchical Reinforcement Learning
A prompt-shared hierarchical offline RL pipeline for LLM agents improves long-horizon task scores on ScienceWorld and ALFWorld over non-hierarchical baselines.
-
Effective Reinforcement Learning for Reasoning in Language Models
DASH uses large-batch sampling plus filtering of low-advantage examples to cut RL training time for small reasoning models by about 83% with no measured accuracy loss.
-
Reason Before You Retrieve: Agentic Planning for Multi-modal RAG
MM-R2 claims SOTA multimodal RAG accuracy on InfoSeek and Encyclopedic VQA via intent grounding plus a 10-topic KnowledgeMap, but its teacher trajectories leak the gold Wikipedia page and omit the image.
-
Initial Steps in Integrating Large Reasoning and Action Models for Service Composition
A conceptual framework integrating LRMs and LAMs for end-to-end automated service composition, with no empirical validation.
-
Agent-RewardBench: Towards a Unified Benchmark for Reward Modeling across Perception, Planning, and Safety in Real-World Multimodal Agents
A new step-level reward-modeling benchmark for multimodal agents shows current MLLMs reach at most 61.6 percent accuracy, and benchmark score correlates strongly (r=0.981 across five models) with downstream A* search ...
-
Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback
EXIF repeatedly has a teacher agent explore an environment, relabel the exploration as tasks, train a student agent on it, and use the student's failures to guide the next round, improving 7B-8B agents in Webshop and Crafter.
-
OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation
A hierarchical multi-agent framework with a domain-agnostic planner trained by SFT and DPO achieves 69.70% on GAIA and lifts a Qwen2.5-32B planner by 16.37 points.
-
WebDancer: Towards Autonomous Information Seeking Agency
A four-stage data-and-training pipeline (synthetic QA, trajectory rejection sampling, SFT, DAPO reinforcement learning) produces WebDancer, a web agent that outperforms vanilla ReAct and several open-source agentic fr...
-
RRO: LLM Agent Optimization Through Rising Reward Trajectories
RRO samples LLM agent actions until a step shows a rising process reward, converts those steps into DPO preference pairs, and reports modest gains on WebShop and InterCode-SQL with fewer samples.
-
Digi-Q: Learning Q-Value Functions for Training Device-Control Agents
An offline RL method learns a Q-function from frozen VLM features and extracts a device-control policy by imitating the best of several actions ranked by that Q-function.
-
ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning
An episode-level variant of GRPO (ESPO) improves personalized GUI-agent reasoning on the 102-episode SmartSpot benchmark, outperforming step-wise and outcome-only training baselines.
-
MemVerse: Multimodal Memory for Lifelong Learning Agents
MemVerse reports large gains on multimodal benchmarks by adding a hierarchical knowledge-graph memory plus fine-tuned parametric recall, but its strongest video-retrieval result uses ground-truth caption-video pairs i...
-
Knowledge Augmented Finetuning Matters in both RAG and Agent Based Dialog Systems
On MobileCS2, a GPT-2 finetuned with retrieved knowledge outperforms a prompted GPT-3.5 in both RAG and agent dialog systems, though the comparison is confounded.
-
SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution
SPA-RL attributes a single final task reward to individual agent steps and uses the attributions plus an executability signal as dense rewards for PPO, yielding modest success-rate gains on WebShop, ALFWorld, and VirtualHome.
-
Large Language Models for Planning: A Comprehensive and Systematic Survey
A structured survey of LLM planning methods, benchmarks, and interpretability work, organized around a three-way taxonomy.
Discussion (0). Sign in to comment.