AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

Choonghan Kim; Hangeol Chang; Hyunmin Hwang; Jaemin Kim; Jong Chul Ye

arxiv: 2605.08704 · v2 · pith:25WBGP2Znew · submitted 2026-05-09 · 💻 cs.AI

AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

Hyunmin Hwang , Jaemin Kim , Choonghan Kim , Hangeol Chang , Jong Chul Ye This is my paper

Pith reviewed 2026-05-12 00:51 UTC · model grok-4.3

classification 💻 cs.AI

keywords multi-agent reasoningparticle swarm optimizationlarge language modelsreasoning skill evolutiontransferable skillsbenchmark performancesemantic skill updates

0 comments

The pith

AgentPSO evolves multi-agent reasoning skills by treating each agent's natural-language description as a particle state that updates toward better performance.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces AgentPSO to evolve reasoning skills in multi-agent setups for large language models. Each agent acts like a particle, updating its skill description by combining its previous semantic direction, personal best skill, global best skill, and reflections drawn from peer trajectories. This happens iteratively during a training phase to strengthen both individual and collective problem solving on mathematical and general reasoning benchmarks. The resulting skills outperform static single-agent baselines and test-time multi-agent debate methods, and they transfer to new benchmarks and different backbone models.

Core claim

AgentPSO models agents as particles whose states are natural-language skill descriptions. In each iteration, an agent revises its skill by blending its prior velocity, personal-best skill, global-best skill, and a self-reflective direction derived from peer reasoning trajectories. The process improves reasoning performance across the population without updating any parameters of the underlying language model and produces skills that generalize beyond the training tasks.

What carries the argument

The semantic velocity update that fuses an agent's previous direction, personal-best skill, global-best skill, and self-reflection from collective trajectories to refine natural-language reasoning skills.

If this is right

Agents achieve higher accuracy on mathematical and general reasoning benchmarks than static single-agent skills or test-time-only multi-agent methods.
Skills learned during evolution transfer successfully to different benchmarks.
The same evolved skills retain their benefits when deployed on a different backbone language model.
Reasoning gains arise from population-level discovery of reusable procedures without access to model gradients or internals.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The approach could be applied to evolve skills for non-reasoning tasks such as collaborative code generation or multi-step planning.
Focusing on skill evolution rather than inference-time aggregation may reduce problems like biased consensus in debate systems.
Evolved skills could be archived as a reusable library for deployment across multiple models and problem domains.
Because updates require only text outputs, the method works with black-box API models that provide no internal access.

Load-bearing premise

Combining previous directions with personal and global best skills through natural-language updates produces genuine reasoning gains rather than superficial changes in prompt wording.

What would settle it

If the evolved skills show no accuracy gain on held-out benchmarks compared with random semantic perturbations or fail to transfer when applied to new tasks or models, the claim of reusable reasoning procedures would be falsified.

Figures

Figures reproduced from arXiv: 2605.08704 by Choonghan Kim, Hangeol Chang, Hyunmin Hwang, Jaemin Kim, Jong Chul Ye.

**Figure 1.** Figure 1: Overview of AgentPSO. Each agent independently solves the training batch with its current skill, producing answers and reasoning traces. Peer observations summarize other agents’ reasoning traces and correctness, from which each agent derives a self-reflective direction for skill improvement. This direction is combined with guidance from the personal-best and global-best skills through a PSO-like update. l… view at source ↗

**Figure 2.** Figure 2: Progressive improvement of AgentPSO-evolved skills. (Left) Evolved personal-best and global-best skills outperform the initial skills on average. (Right) DeepMath performance improves over iterations, showing gradual refinement of the agent population. and F. At each validation step, we evaluate the current agent skills on 20 validation samples. To reduce overfitting to a single small validation subset, we… view at source ↗

**Figure 3.** Figure 3: Validation accuracy of the global-best and average accuracy over iterations. The global-best trajectory improves steadily, while the variant without it shows only limited progress [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗

**Figure 4.** Figure 4: Example of velocity across AgentPSO training iterations. The velocity is evolved from “”. [PB] denotes influence from Agent’s personal-best skill, [GB] denotes influence from the global-best skill, [di] denotes the self-reflective direction at the corresponding iteration, [si] denotes influence from Agent’s current skill, and [prev_v] denotes accumulated influence from the previous velocity. When the perso… view at source ↗

**Figure 5.** Figure 5: Example of skill evolution across AgentPSO training iterations. The skill is evolved from “First solve the problem, then review and improve the solution.”. [PB] denotes instructions inherited from the previous personal-best skill, [GB] denotes instructions influenced by the globalbest skill, and [di] denotes instructions introduced by the feedback-guided reflective direction. 21 [PITH_FULL_IMAGE:figures/… view at source ↗

**Figure 6.** Figure 6: Problem-solving prompt used by each agent. Given the current skill file, each agent independently solves the input problem and returns its reasoning trace and final answer in JSON format. Generate Self-reflective Direction Eq.(1) Current agent skill: {current_skill} Agent’s own reasoning traces and answers: {own_outputs_json} Other agents’ reasoning traces and answers, including correctness: {peer_outputs_… view at source ↗

**Figure 7.** Figure 7: Self-reflective direction generation prompt. This prompt generates an update direction by comparing the current agent’s performance with peer agents, aiming to identify general reasoning improvements. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗

**Figure 8.** Figure 8: Prompt for generating the PSO-guided velocity v t+1 i . Given the previous velocity, the self-reflective direction, the personal-best skill, and the global-best skill, the model generates a concise natural-language velocity that captures generalizable improvements while preserving the agent’s identity. Skill Update from Velocity Eq.(3) Agent identity to preserve: {agent_identity} Current skill s_i: {curren… view at source ↗

**Figure 9.** Figure 9: Prompt for applying the velocity to the skill. Given the current skill and the PSO-guided velocity, the model rewrites the skill into an updated form that preserves the agent’s identity while incorporating concise and generalizable improvements. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗

read the original abstract

Multi-agent reasoning has shown promise for improving the problem-solving ability of large language models by allowing multiple agents to explore diverse reasoning paths. However, most existing multi-agent methods rely on inference-time debate or aggregation, which can be vulnerable to incorrect peer influence and biased consensus. Moreover, the agents themselves remain static, as their underlying reasoning skills do not evolve across tasks. In this paper, we introduce \textbf{AgentPSO}, a particle-swarm-inspired framework for evolving multi-agent reasoning skills. AgentPSO treats each agent as a particle-like reasoner whose state is a natural-language skill and whose velocity is a semantic update direction, iteratively guiding agents toward higher-performing skill configurations. Across training iterations, each agent updates its skill by combining its previous velocity, personal-best skill, global-best skill, and a self-reflective direction derived from peer reasoning trajectories. This enables agents to learn reusable reasoning behaviors by drawing on their own experience and on the strongest skills found by the population, without updating the parameters of the backbone language model. Experiments on mathematical and general reasoning benchmarks show that AgentPSO improves over static single-agent skills and test-time-only multi-agent reasoning baselines. The evolved skills further transfer across benchmarks and to another backbone model, suggesting that AgentPSO captures reusable reasoning procedures rather than merely optimizing benchmark-specific prompts. Code is publicly available at https://github.com/HYUNMIN-HWANG/AgentPSO/.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

AgentPSO maps PSO velocity and position updates onto natural-language skill descriptions for multi-agent LLMs, with reported benchmark gains and cross-task transfer that still need stronger evidence they are more than prompt refinement.

read the letter

The main point on this paper is that AgentPSO borrows the particle swarm update rule and applies it directly to text: each agent's reasoning skill is treated as a particle position, and the update direction combines inertia from prior velocity, personal best, global best, and a self-reflective component derived from peer trajectories. All of this happens through additional LLM calls on the skill strings themselves, without any model parameter changes. The experiments claim improvements over static single-agent prompting and over test-time-only multi-agent debate or aggregation on math and general reasoning benchmarks, plus some transfer to held-out tasks and a second backbone model. Code is released, which is useful for checking the implementation details. That combination of a concrete algorithmic mapping and open resources is the clearest contribution here. The transfer results are the part that would matter most if they hold up, because they suggest the evolved skills are not just benchmark-specific prompt fragments. The soft spots are around the mechanism. Every update still relies on the same LLM generating new text from old text, so the performance lift could come from the extra inference budget spent during the evolution loop rather than from discovering deeper reusable procedures. The paper contrasts against static and test-time baselines, but without component ablations that isolate the velocity terms or direct inspection of the skill strings before and after evolution, it is difficult to separate genuine skill improvement from iterative prompt tuning. The abstract and experiments do not appear to include statistical tests or variance estimates across multiple runs, which leaves the magnitude of the gains harder to assess. This work is aimed at people already working on multi-agent LLM systems who want a population-based optimization method that stays in text space. A reader focused on test-time scaling or prompt engineering would get the most out of it. The idea is coherent enough and the empirical claims are concrete enough that it deserves a serious referee to check the experimental controls and the transfer evidence. I would send it to peer review.

Referee Report

3 major / 2 minor

Summary. The paper introduces AgentPSO, a multi-agent framework that adapts particle swarm optimization to evolve natural-language reasoning skills for LLMs. Each agent is treated as a particle whose position is a skill description and velocity is a semantic update direction; updates combine prior velocity, personal-best skill, global-best skill, and self-reflective direction derived from peer trajectories. The method runs without gradient updates to the backbone model. Experiments on mathematical and general reasoning benchmarks report gains over static single-agent and test-time multi-agent baselines, with claimed transfer of evolved skills across benchmarks and to a different backbone model.

Significance. If the empirical claims hold under rigorous controls, the work would be significant for demonstrating a gradient-free mechanism to discover reusable, transferable reasoning procedures in multi-agent LLM systems. It directly addresses the static-agent limitation of prior multi-agent reasoning methods and provides an open-source implementation, which supports reproducibility.

major comments (3)

[Experiments] Experiments section: the reported improvements over baselines lack any mention of the number of independent runs, variance across seeds, or statistical significance tests. Without these, it is impossible to determine whether the gains exceed what would be expected from additional inference budget or random prompt variation.
[Method and Experiments] Method and Experiments: no component ablations are presented for the four-term update rule (inertia, cognitive/personal-best, social/global-best, self-reflective). This is load-bearing for the central claim that the PSO-style combination produces genuine skill evolution rather than iterative prompt refinement.
[Experiments] Experiments: the paper provides no examples or qualitative analysis of the evolved skill strings before and after optimization. Without inspection of the actual natural-language content, it remains possible that transfer results reflect accumulation of benchmark-specific fragments rather than reusable procedures.

minor comments (2)

[Abstract and Introduction] The abstract and introduction use the term 'parameter-free' for the evolution process, yet the update rule includes tunable PSO-style coefficients (inertia, cognitive, social weights) whose values are not specified or ablated.
[Method] Figure captions and algorithm pseudocode would benefit from explicit notation for how natural-language concatenation is performed during velocity and position updates.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and insightful comments, which help strengthen the empirical rigor and clarity of our work on AgentPSO. We address each major point below and commit to revisions that directly incorporate the suggested improvements.

read point-by-point responses

Referee: Experiments section: the reported improvements over baselines lack any mention of the number of independent runs, variance across seeds, or statistical significance tests. Without these, it is impossible to determine whether the gains exceed what would be expected from additional inference budget or random prompt variation.

Authors: We agree that reporting variance and statistical tests is necessary to rule out effects from random prompt variation or extra inference budget. In the revised manuscript, we will rerun all main experiments across 5 independent runs with distinct random seeds, report mean accuracies with standard deviations, and include paired t-tests (with p-values) comparing AgentPSO against each baseline to establish statistical significance. revision: yes
Referee: Method and Experiments: no component ablations are presented for the four-term update rule (inertia, cognitive/personal-best, social/global-best, self-reflective). This is load-bearing for the central claim that the PSO-style combination produces genuine skill evolution rather than iterative prompt refinement.

Authors: We acknowledge that ablations are required to isolate the contribution of the PSO-style four-term rule versus simpler iterative refinement. The revised version will add a dedicated ablation study that evaluates variants with individual terms removed or replaced (e.g., no inertia, no self-reflective component) on the primary math and general reasoning benchmarks, demonstrating that the full combination yields superior skill evolution. revision: yes
Referee: Experiments: the paper provides no examples or qualitative analysis of the evolved skill strings before and after optimization. Without inspection of the actual natural-language content, it remains possible that transfer results reflect accumulation of benchmark-specific fragments rather than reusable procedures.

Authors: We agree that qualitative inspection is important to substantiate claims of reusable procedures. The revision will include concrete examples of initial versus evolved skill strings for representative agents, accompanied by a qualitative analysis section that highlights recurring reasoning patterns (e.g., decomposition strategies) that persist across benchmarks and support the transfer results. revision: yes

Circularity Check

0 steps flagged

No significant circularity in claimed results

full rationale

The paper introduces an empirical PSO-inspired framework for updating natural-language agent skills via LLM calls, with central claims resting on benchmark experiments showing performance gains and cross-benchmark/model transfer. No equations, derivations, or first-principles results are presented that reduce outputs to inputs by construction, fitted parameters renamed as predictions, or load-bearing self-citations. The method is self-contained as an experimental procedure evaluated against external baselines, with no self-definitional loops or ansatz smuggling identified in the provided text.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 2 invented entities

The framework rests on the assumption that natural-language skill descriptions can be meaningfully updated via semantic analogs of PSO operations and that these updates improve downstream reasoning without model parameter changes.

free parameters (1)

PSO-style coefficients for inertia, cognitive, and social components
The abstract implies tunable weights for combining previous velocity, personal best, global best, and reflection, though exact values are not stated.

axioms (1)

domain assumption Semantic combinations of natural-language skill descriptions can serve as effective velocity and position updates that improve reasoning performance.
Invoked in the description of how agents update their skills iteratively.

invented entities (2)

Agent skill state as particle position no independent evidence
purpose: Represent reasoning capability in natural language for swarm-style optimization
Core modeling choice that maps PSO concepts to LLM agents.
Semantic update direction as particle velocity no independent evidence
purpose: Enable iterative movement toward stronger reasoning skills
Invented mapping from numerical PSO to text-based updates.

pith-pipeline@v0.9.0 · 5564 in / 1399 out tokens · 53050 ms · 2026-05-12T00:51:04.278115+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

"Skill issues'': data-centric optimization of lakehouse agents
cs.AI 2026-05 unverdicted novelty 3.0

Data-centric optimization of skills for agents on a branching lakehouse improves accuracy by 31.9% on 25 tasks via state-verification evaluation.