pith. sign in

arxiv: 2605.02168 · v1 · submitted 2026-05-04 · 💻 cs.AI · cs.LG· cs.MA

Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning

Wenyi Wu , Sibo Zhu , Kun Zhou , Biwei Huang This is my paper

Pith reviewed 2026-05-09 16:46 UTC · model grok-4.3

classification 💻 cs.AI cs.LGcs.MA
keywords multi-agent collaborationlong-horizon planninglanguage model agentsreinforcement learningcompute allocationplanner optimizationVLM-as-judge
0
0 comments X

The pith

Allocating most compute and learning to a dedicated planner improves long-horizon agent tasks while freezing execution and memory components.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Language model agents struggle with long sequences of decisions in tasks such as web navigation or operating system control. This paper decomposes the system into three roles: a planner for high-level choices, an actor for carrying out steps, and a memory manager for context. Analysis shows that planning accounts for the largest share of performance, so the authors train only the planner with reinforcement learning that uses trajectory-level rewards judged by a vision-language model, leaving the actor and memory parts unchanged. The result is higher success rates across benchmarks at lower total compute cost than balanced training approaches.

Core claim

The central claim is that planning is the dominant factor in long-horizon performance. A systematic compute-allocation study shows execution and memory management need far less model capacity. The planner-centric reinforcement learning method therefore optimizes only the planner using VLM-as-judge trajectory rewards while freezing the remaining components, producing robust gains on web navigation, OS control, and tool-use benchmarks with clear efficiency advantages.

What carries the argument

The planner-centric reinforcement learning approach, which trains solely the high-level decision component with external trajectory rewards while the actor and memory manager stay fixed.

If this is right

  • Success rates rise on web navigation, OS control, and tool-use benchmarks without enlarging the execution or memory models.
  • Overall compute drops because only the planner receives significant training and capacity.
  • Competitive results remain possible even when actor and memory components are never updated during learning.
  • The planning-dominant pattern appears across the three tested domains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Future agent designs could assign larger models and more training exclusively to planning modules while using lightweight frozen executors.
  • The unbalanced allocation may reduce the need for full end-to-end fine-tuning in multi-step agent systems.
  • Replacing the VLM judge with other reward sources would test how much the gains depend on that particular evaluator.

Load-bearing premise

The vision-language model judge supplies unbiased, reliable rewards for full trajectories, and the dominance of planning holds beyond the specific benchmarks and model sizes tested.

What would settle it

Replace the VLM judge with human ratings or a different automated scorer on the same benchmarks and check whether the performance edge over balanced training disappears; or run the method on an untested long-horizon domain such as multi-step physical assembly.

Figures

Figures reproduced from arXiv: 2605.02168 by Biwei Huang, Kun Zhou, Sibo Zhu, Wenyi Wu.

Figure 1
Figure 1. Figure 1: Performance on WebVoyager with planner-centered multi-agent setup across model scales. Using a strong planner with smaller actor and memory manager yields significant gains. Driven by scaling laws, large vi￾sion–language models (VLMs) (Bor￾des et al., 2024; Zhang et al., 2024) have achieved remarkable perfor￾mance across a wide range of multi￾modal reasoning and planning tasks. Building on this capability,… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the planner-centric multi-agent framework. The system decomposes tasks into three agents: a central Planner that reasons over long-horizon goals; a Memory Manager that retrieves relevant discrete and continuous memories; and an Actor that executes precise actions. The pipeline supports optimization via reinforcement learning. Actions for GUI tasks are executed using Playwright2 or PyAutoGUI3 , … view at source ↗
Figure 3
Figure 3. Figure 3: Performance when scaling individual agents (Actor, Planner, Memory Manager), view at source ↗
Figure 4
Figure 4. Figure 4: Reward curves under dif￾ferent RL training strategies. RL training strategy ablation. We ablate which components are optimized during training: (1) Planner-only (ours), (2) Actor-only with frozen Plan￾ner, and (3) joint training of Planner and Actor with shared parameters. As shown in view at source ↗
Figure 5
Figure 5. Figure 5: Success and failure cases of the training-free planner. view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison on Amazon. Top: The multi-agent framework successfully decomposes the task with the Planner instructing the Actor to apply price filters. Bottom: The single-agent baseline fails to utilize UI filters, repeating search queries until reaching the step limit. 20 view at source ↗
read the original abstract

Language model (LM)-based agents have demonstrated promising capabilities in automating complex tasks from natural language instructions, yet they continue to struggle with long-horizon planning and reasoning. To address this, we propose an enhanced multi-agent framework that decomposes automation into three roles: a planner for high-level decision-making, an actor for task execution, and a memory manager for contextual reasoning. While this modular decomposition aligns with established design patterns, our core contribution lies in a systematic compute-allocation analysis, revealing that planning is the dominant factor influencing task performance. Execution and memory management require significantly less compute and model capacity to achieve competitive results. Building on these insights, we introduce a planner-centric reinforcement learning approach, which exclusively optimizes the planner using trajectory-level rewards from a VLM-as-judge, while freezing the other components. Extensive experiments on benchmarks spanning web navigation, OS control, and tool use demonstrate that concentrating model capacity and learning on high-level planning yields robust and compute-efficient improvements in long-horizon agent automation. Our code is publicly released.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper introduces an unbalanced multi-agent framework for LM agents with planner, actor, and memory manager. A compute-allocation analysis identifies planning as the key bottleneck and performance driver. Based on this, they develop a planner-centric RL approach that optimizes only the planner using VLM-provided trajectory rewards while keeping actor and memory frozen. Experiments on web navigation, OS control, and tool-use benchmarks are reported to show improved performance and efficiency in long-horizon tasks.

Significance. Should the findings prove robust, this work offers a practical and efficient paradigm for scaling LM agents by focusing learning and capacity on high-level planning rather than uniform optimization. The release of code is a positive contribution to the field, enabling reproducibility and extension. It challenges the default of balanced multi-agent designs and provides empirical support for planner-centric approaches.

major comments (2)
  1. Experiments section: The manuscript reports positive experimental outcomes on three benchmark categories but does not include quantitative results, specific baselines, error bars, or details on the compute-allocation analysis methodology. This makes it challenging to verify the claimed dominance of planning and the improvements from the RL approach.
  2. Method section (VLM-as-Judge): The planner-centric RL relies on trajectory-level rewards from a VLM-as-judge, but there is no validation of the judge's reliability, such as correlation with human judgments, inter-rater agreement, or comparison to oracle rewards. This is a load-bearing issue for the central claim, as biases in the judge could lead to the planner exploiting artifacts rather than achieving genuine task improvements.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and the positive assessment of the work's potential significance. We agree that strengthening the experimental reporting and validating the VLM-as-judge are important for robustness. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: Experiments section: The manuscript reports positive experimental outcomes on three benchmark categories but does not include quantitative results, specific baselines, error bars, or details on the compute-allocation analysis methodology. This makes it challenging to verify the claimed dominance of planning and the improvements from the RL approach.

    Authors: We acknowledge that the current presentation of results could be more detailed for independent verification. The manuscript does report outcomes across web navigation, OS control, and tool-use benchmarks with comparisons to baselines, but we will expand the Experiments section in revision to include explicit quantitative tables (e.g., success rates, efficiency metrics), error bars from repeated runs, and a dedicated subsection detailing the compute-allocation analysis methodology (including how per-component token/FLOP usage was measured and how planning was identified as the bottleneck). This will directly support the claims of planner dominance and RL gains. revision: yes

  2. Referee: Method section (VLM-as-Judge): The planner-centric RL relies on trajectory-level rewards from a VLM-as-judge, but there is no validation of the judge's reliability, such as correlation with human judgments, inter-rater agreement, or comparison to oracle rewards. This is a load-bearing issue for the central claim, as biases in the judge could lead to the planner exploiting artifacts rather than achieving genuine task improvements.

    Authors: We agree this is a critical point. The VLM-as-judge enables scalable trajectory rewards for long-horizon tasks where dense human feedback is impractical, and the unbalanced design (freezing actor/memory) limits the planner's ability to exploit low-level artifacts. In the revised manuscript we will add an appendix with a validation study on a sampled subset of trajectories, reporting Pearson/Spearman correlations with human expert judgments, inter-rater agreement (e.g., Cohen's kappa), and, where benchmark oracles exist, performance comparisons against oracle-reward training. We will also discuss observed biases and mitigation strategies. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical RL against external judge

full rationale

The paper presents a modular multi-agent framework (planner/actor/memory) whose central contribution is an empirical compute-allocation study followed by planner-centric RL that optimizes only the planner against trajectory rewards supplied by an external VLM-as-judge while freezing the remaining components. No equations, derivations, or predictions are supplied that reduce by construction to fitted parameters, self-definitions, or self-citations. The method is therefore self-contained: performance claims rest on benchmark experiments rather than any internal re-labeling of inputs as outputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central claim rests on the empirical finding that planning dominates performance and that freezing actor/memory while training the planner is sufficient; no explicit free parameters, axioms, or invented entities are stated in the abstract.

pith-pipeline@v0.9.0 · 5488 in / 1023 out tokens · 42181 ms · 2026-05-09T16:46:45.819269+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    Miller and Jonathan D

    URLhttps://arxiv.org/abs/2505.05470. Run Luo, Lu Wang, Wanwei He, Longze Chen, Jiaming Li, and Xiaobo Xia. Gui-r1: A gener- alist r1-style vision-language action model for gui agents.arXiv preprint arXiv:2504.10458, 2025. Earl K. Miller and Jonathan D. Cohen. An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24:167–202, 2...

    work page doi:10.1146/annurev.neuro.24.1.167 2025

  2. [2]

    Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong

    URLhttps://arxiv.org/abs/2408.07199. Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, Tianjie Zhang, Wei Xu, Jie Tang, and Yuxiao Dong. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning, 2025. URLhttps://arxiv.org/abs/2411.02337. Chen Qian, Wei Li...

    work page doi:10.1016/j.neuron.2014.08.011 2025

  3. [3]

    Tasks that were valid months ago may no longer be feasible in their original form

    Dynamic web environments: Real-world websites change continuously, such as page layouts, product listings, UI elements, and even entire website structures are updated frequently. Tasks that were valid months ago may no longer be feasible in their original form. Our evaluation is conducted on live websites at the time of our experiments, meaning the diffic...

    work page

  4. [4]

    Step limit: We enforce a strict 15-step limit for all methods and models, which is more restrictive than some prior work that allows 50-100 steps

    work page

  5. [5]

    Our framework uses the same configuration across all evaluation domains

    No website-specific optimization: We do not fine-tune prompts, action spaces, or mem- ory banks to specific websites. Our framework uses the same configuration across all evaluation domains

    work page

  6. [6]

    This ensures that performance differences reflect genuine method improvements rather than evaluation setup advan- tages

    Fair comparison environment: All methods and models in our evaluation, including closed-source baselines, open-source models, memory-augmented variants, and our multi-agent framework, are evaluated under identical conditions (same websites, same tasks, same step limits, same evaluation time window). This ensures that performance differences reflect genuin...

    work page 2023

  7. [7]

    Is concise and actionable

    work page

  8. [8]

    Breaks down the task into clear, sequential steps

    work page

  9. [9]

    Plan update instruction You are a task planning assistant

    Uses insights from similar experiences when relevant Format your response as a numbered list of steps. Plan update instruction You are a task planning assistant. Update the original plan based on recent observations and actions. Experience Memory: {DISCRETE MEMORY} Original Plan: {PLAN} Recent Screenshot Observations: {SCREENSHOTS} Recent Actions: {ACTION...

    work page

  10. [10]

    Reflect what has been accomplished

    work page

  11. [11]

    Adjust remaining steps based on current progress

    work page

  12. [12]

    Keep it concise (3-5 steps total)

    work page

  13. [13]

    If the task is finished, yield STOP

    Clearly state the next single step. If the task is finished, yield STOP

    work page

  14. [14]

    Output format: <plan>Your updated plan here</plan> <subgoal>next single step</subgoal> 21 Preprint

    Do NOT repeat actions that have already failed. Output format: <plan>Your updated plan here</plan> <subgoal>next single step</subgoal> 21 Preprint. Under review. Action generation prompt System instruction You are a helpful execution agent following the given plan. Task: {QUERY} Plan: {PLAN} Available Actions: {ACTION_SPACE} Constraints:

    work page

  15. [15]

    Follow the plan step by step

    work page

  16. [16]

    Specify the element number to interact with

    work page

  17. [17]

    Don't repeat failed actions

    work page

  18. [18]

    Provide an answer within the remaining steps

    work page

  19. [19]

    Memory check and update prompt Input prompt You are helping a GUI agent decide if it should look for different reference experiences

    Output one action at a time. Memory check and update prompt Input prompt You are helping a GUI agent decide if it should look for different reference experiences. Original Task: {QUERY} Recent Observations: {SCREENSHOTS} Recent Actions: {ACTIONS} Current Memories: {DISCRETE MEMORY} Rules:

    work page

  20. [20]

    Memories are loose references - they don't need to match perfectly

    work page

  21. [21]

    If current memories are related, output NO_UPDATE

    work page

  22. [22]

    NO_UPDATE

    Only output NEEDS_UPDATE if the agent moved to a completely different activity type. Output: "NO_UPDATE" or "NEEDS_UPDATE: <2-5 keywords>" 22 Preprint. Under review. VLM-as-Judge Evaluation Prompt System instruction You are an expert at analyzing web browsing task completion from screenshots. You will be given a task instruction and a series of screenshot...

    work page

  23. [23]

    Describe what successful task completion should look like (i.e., what the final screenshot should show)

    First, understand the task instruction. Describe what successful task completion should look like (i.e., what the final screenshot should show)

    work page

  24. [24]

    For each screenshot, analyze: - What is visible on the page (URL, elements, forms, buttons, messages, etc.) - What action the user performed (click, type, scroll, etc.) - Whether and how the action contributed to progress (or caused mistakes)

    work page

  25. [25]

    Carefully observe URL bars, form fields, search boxes, buttons, error messages, results, and confirmation screens

    work page

  26. [26]

    After analyzing all screenshots, provide your reasoning about: - Whether the task was completed - How clear, logical, and efficient the interaction sequence was - Whether actions were redundant or irrelevant Then, assign a score using this 3-level scale: SCORING SCALE: - 5 -- Task Fully Completed and Efficiently Done - The final screenshot shows the task ...

    work page