Pith. sign in

REVIEW 3 major objections 6 minor 21 references

Evolution of Collective AI Beyond Individual Optimization

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A lone agent's over-optimization for chemotaxis is followed by a drop in the cloned group's collective fitness as sensor-motor coupling fades.

desk verdict A promising but under-supported empirical caution that over-optimized individual chemotaxis agents can lose collective effectiveness; the MI evidence is confounded by stationary sitting. read the letter →

arxiv 2412.02085 v1 pith:GBQST56K submitted 2024-12-03 cs.MA cs.AI

Ryosuke Takata , Yujin Tang , Yingtao Tian , Norihiro Maruyama , Hiroki Kojima , Takashi Ikegami This is my paper · ORCID
classification cs.MAcs.AI
keywords collectivebehaviorneuroevolutionchemotaxismutualinformationsensor-motorcouplingpheromonecommunicationroledifferentiationswarmintelligence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what happens when a group is made of clones of an agent that was optimized alone for a single skill, chemotaxis. It reports that the 1024-agent collective's total pheromone gain peaks near generation 500, then declines in later generations even while the single agent's own fitness stays flat and its neural network stays small. Over the same later period the mutual information between sensor inputs and motor outputs, $MI(I;O)$, drops, which the authors interpret as a loss of sensor-motor coupling. The conclusion is that over-optimization of the individual almost always yields less effective group behavior, and that what helps the group is not individual superiority but differentiation in how much pheromone each identical agent gathers under a uniform movement pattern. A sympathetic reader would care because it names a concrete failure mode for building collective AI: optimizing components in isolation can quietly break the interactions the collective depends on.

What carries the argument

The machinery is a minimal recurrent neural network controller with six sensors, one central sensor, six hidden units, two context units, two motor outputs, and 82 weights total, evolved with CMA-ES. The load-bearing diagnostic is $MI(I;O)$, the estimated mutual information between sensor and motor time series, together with the conditional entropy $H(O|I)$ of outputs given inputs. What $MI(I;O)$ does for the argument is to turn a hidden network property, how strongly inputs drive outputs, into a single curve that the paper then correlates with collective fitness and movement uniformity; its decline in later generations is the evidence that individual over-optimization reshapes behavior in a way that hurts the group.

What would settle it

A single observation that would settle it: artificially hold the sensor inputs of the cloned agents constant while leaving motor outputs free, and compare collective pheromone gain against the unclamped control; if collective fitness does not fall, the $MI(I;O)$ decline is not the causal driver.

Watch

Extended reading notes

Core claim

The central discovery is a decoupling of individual and collective fitness in a minimal neuroevolution setup. A single recurrent neural network agent is evolved to climb evaporating pheromone gradients; within roughly a hundred generations it approximates hand-designed rule-based chemotaxis and its individual fitness saturates. When the same network is cloned 1024 times into a shared pheromone field, the group first becomes more effective, with collective fitness peaking around generation 500 while agents split into high-gain and low-gain roles despite similar movement patterns, and then declines. The paper attributes the decline to reduced sensor-motor coupling: $MI(I;O)$ between six sensor traces and two motor outputs falls across generations, and in later generations there are negative correlations between this mutual information and both collective fitness and movement-pattern diversity across ten seeds. The authors state the general result as 'over-optimization of individual agents almost always lead to less effective group behavior,' and they tie the high-fitness epoch to maximal variance in pheromone gain under minimal variance in movement, supporting their 'Community First Hypothesis.'

Load-bearing premise

The load-bearing premise is that the mutual information between sensor inputs and motor outputs measures real sensor-motor coupling, so its decline is a cause of the collective fitness drop rather than an artifact of an optimal agent that sits still on a pheromone peak and therefore has nearly constant inputs.

Editorial extensions

If this is right

  • Collective performance cannot be read off individual performance: even after individual fitness converges, group fitness continues to rise, fall, and restabilize through later generations.
  • The optimal group state is not the optimal individual state: peak collective fitness occurs when agents are nearly stationary and uniform in movement, not when each agent is actively gradient-climbing.
  • Heterogeneity can arise from homogeneity: identical cloned networks differentiate into high- and low-pheromone-gain agents, and this variance, not individual superiority, marks the high-fitness epoch.
  • Later evolutionary drift is not neutral for the group, because reduced sensor-motor coupling accompanies the shift from peaked to declining collective fitness even with a flat individual fitness curve.
  • For collective AI design, selection pressure on the individual alone leaves collective outcomes uncontrolled; the paper's mechanism predicts collective fitness can decline even with well-performing individuals.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is causal, adding a group-level selection term to fitness should prevent or reverse the later decline in collective fitness; this is the paper's natural next experiment, though it is not performed here.
  • In deployed settings, the curve of $MI(I;O)$ across training could serve as an early-warning indicator that individual optimization is pushing a multi-agent system away from cooperative regimes, even where individual benchmarks look flat.
  • The paper's rule-based comparison implies a sharper control: cloned agents with no internal state do not differentiate, so the role-splitting seen in evolved networks depends on internal dynamics rather than on the pheromone environment alone; an explicit ablation that freezes context neurons would isolate this dependence.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper evolves single-agent neural-network controllers for chemotaxis using CMA-ES and then clones the evolved controller into 1024 agents placed in a shared pheromone field. It reports that, after individual fitness converges, collective fitness (average pheromone gain in the multi-agent setting) first rises and then declines, and that the mutual information MI(I;O) between sensor inputs and motor outputs decreases in later generations. The authors interpret this decline as reduced sensor-motor coupling that causally drives less effective group behavior, and they use the results to argue that over-optimization of individuals can undermine collective performance.

Significance. If the causal claim were established, the paper would be a valuable contribution to collective AI and evolutionary swarm design: it demonstrates a concrete simulated system in which individually optimized, homogeneous agents lose collective efficacy, and it proposes an information-theoretic diagnostic (MI between sensors and motors) for tracking this transition. The study is also reproducible in principle: the model, parameters, and evolutionary algorithm are described in detail, and the appendix includes a rule-based chemotaxis baseline and a multiple-seed analysis. However, the load-bearing evidence for reduced sensor-motor coupling is currently confounded, and the cross-seed correlations are too weak and variable to support the headline generalization. The central phenomenon deserves further study, but the paper needs a more direct, input-distribution-independent measure of coupling and a causal test before its main conclusion can be accepted.

major comments (3)
  1. [Section 3.2, Eq. (6), Figure 4(D)] The decline in MI(I;O) is not an independent measure of reduced sensor-motor coupling because MI is estimated from the naturalistic behavioral time series. At generation 500, most agents sit still at pheromone peaks, so their sensor inputs and motor outputs are nearly constant; the empirical mutual information then tends to zero regardless of the network's actual input-output gain. The later decrease in MI can therefore be a byproduct of the stationary strategy rather than evidence of reduced sensory sensitivity. The causal phrasing in Section 4, 'This reduction in sensory sensitivity appears to lead to uniform movement patterns,' is not supported by the current analysis. The authors should compute a perturbation-based or probe-based measure of coupling, for example by injecting controlled input noise or by measuring the input-output gain over a fixed, non-degenerate input ensemble.
  2. [Section 3.4, Figure 6, Appendix 4] The cross-seed correlations do not support the 'almost always' claim in the abstract. Across 10 seeds, the correlation between MI(I;O) and collective fitness ranges from -0.91 to 0.03 with mean -0.41 and standard deviation 0.26, meaning several seeds show essentially no negative relationship. The main text's Figure 4 is labeled 'representative evolutionary seed,' so the reader cannot tell how representative it is. The authors should report the per-seed correlations with confidence intervals, state how many of the 10 seeds have significantly negative correlations, and avoid generalizing from one favorable seed.
  3. [Section 2.3, Eq. (4), Table 1] The definition of collective fitness in the multi-agent test phase is ambiguous and potentially self-referential. In the evolution phase, Eq. (4) explicitly excludes the agent's own pheromone release, but the multi-agent description says only that 'how much pheromones collected by the agents are computed' after agents deposit pheromones in a 3x3 area centered on their previous position. If agents can sense pheromones they deposited themselves, then a stationary agent at a pheromone peak may be collecting its own recent deposits, and the 'collective fitness' measure would partly reflect local self-deposition rather than group benefit. The authors should clarify whether own deposits are excluded in the multi-agent fitness calculation and, if they are not, quantify the contribution of self-deposited pheromone to the reported fitness values.
minor comments (6)
  1. [Abstract and Section 4] The phrase 'almost always lead to less effective group behavior' is grammatically incorrect ('lead' should be 'leads') and, more importantly, overstates the evidence; revise to reflect the variable correlations reported across seeds.
  2. [Section 4] The sentence 'We hypothesis that it is due to adaptability...' should be 'We hypothesize...'.
  3. [Section 4] The term 'Community First Hypothesis' is introduced without a formal definition or citation; please state explicitly what the hypothesis predicts and how the presented data distinguish it from alternative explanations.
  4. [Section 3.2, Eqs. (6)-(8)] Please state explicitly that H(I), H(O), and H(I,O) are empirical entropies computed from discretized time series and specify the binning procedure in the main text rather than only in the paragraph below Eq. (7).
  5. [Figure 4] The caption says 'Moving averages,' but no window size is given; reporting raw traces or confidence bands would help the reader assess whether the phase transition is robust rather than an artifact of smoothing.
  6. [Section 2.2 and Table 1] The statement that the collective declines at a 'constant rate' in Section 4 is not quantified; either specify the rate or rephrase as a gradual decline.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the collective-fitness decline is an emergent simulation result, and the MI measure, while confounded, is not fitted to the outcome.

full rationale

The paper's central claim is that over-optimized individual chemotaxis agents show reduced collective pheromone gain when cloned into groups. This emerges from a forward simulation, not from a quantity fitted or defined in terms of the conclusion. Individual neural-network weights are evolved de novo with CMA-ES against a single-agent pheromone-gain objective; the evolved snapshots are then cloned into a 1024-agent simulation and collective fitness, movement variance, and mutual information are measured from the resulting trajectories. None of these measurements is adjusted to reproduce another: collective fitness is not a training target, and MI(I;O) is computed from the behavioral time series rather than being fit to collective fitness. The paper's self-references ([12], [13], [15]) are background and software citations; no load-bearing uniqueness theorem or ansatz is imported from prior same-author work. The main weakness is that a stationary agent at a pheromone peak has near-zero MI because its sensor inputs are constant; this is a construct-validity confound in interpreting MI as sensor-motor coupling, not a circular derivation. Eq. 6 does not incorporate collective fitness, and the negative MI-fitness correlation is empirical, not an identity. The Discussion's causal phrasing that reduced sensory sensitivity 'appears to lead to uniform movement patterns' overstates the correlational evidence, but that is a correctness risk, not a circularity. The derivation chain is self-contained with respect to the paper's stated inputs and measurements.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper is an empirical simulation study; its conclusions rest on the chosen simulation parameters, the neural network model, and the interpretation of information-theoretic measures. The table above lists the hand-set parameters and domain assumptions that the central claim depends on.

free parameters (4)
  • pheromone decay rate = 0.001
    Set by hand; controls how long pheromone trails persist and thus the timescale of collective aggregation. The paper does not explore sensitivity to this parameter.
  • number of initial pheromone spots = 5
    Set by hand; defines the single-agent chemotaxis task. The paper does not vary this parameter.
  • pheromone deposit value = 1.0
    Fixed deposit amount in the multi-agent phase; determines the strength of the communication signal. The paper does not vary this parameter.
  • maximum steps (multi-agent) = 5000
    Set by hand; determines the window over which collective fitness is averaged. The paper does not test the sensitivity of the result to this parameter.
assumptions (6)
  • domain assumption The neural network architecture (6 sensors, 2 context neurons, 6 hidden neurons, 2 motors) is expressive enough to represent both chemotaxis and the collective behaviors observed.
    The paper never ablates the architecture; all results use this fixed network. Section 2.1.
  • domain assumption The fitness function F = ∫ ψ dt, rewarding time spent in high pheromone concentration, is a valid objective for chemotaxis and for collective performance.
    The definition treats sitting at a pheromone peak as optimal, which shapes the evolved strategy. Equations 4 and 5, Section 2.3.
  • domain assumption The pheromone decay (0.001/step) and fixed deposit (1.0 over a 3x3 area) rules approximate biological pheromone communication closely enough for the conclusions to generalize.
    No sensitivity analysis is performed; the qualitative results could depend on these timescales. Sections 2.2 and 2.3.
  • domain assumption CMA-ES with population 100 over 2000 generations reliably finds near-optimal chemotaxis policies.
    The paper does not verify convergence across replicates beyond 10 seeds; selection pressure in later generations may be weak. Section 2.1 and Table 1.
  • domain assumption Mutual information computed from discretized (bin width 0.01) sensor and motor time series is a fair measure of information flow.
    The discretization and finite sampling may bias the estimate; no alternative measures are reported. Section 3.2, Equations 6 to 8.
  • domain assumption The test-phase environment (no initial pheromones, only agent deposits) is comparable to the evolution-phase environment (five random pheromone bells) despite the difference in initial conditions.
    This distribution shift is never discussed; the evolved agents never experience an empty pheromone field during training, which may explain the collective fitness decline independently of over-optimization. Sections 2.2 and 2.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evolution of Collective AI Beyond Individual Optimization." pith.science (2026). https://pith.science/paper/GBQST56K

@misc{pith2026241202085,
  author       = {Pith},
  title        = {Pith review of: Evolution of Collective AI Beyond Individual Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GBQST56K}},
  note         = {Machine review of arXiv:2412.02085}
}
read the original abstract

This study investigates collective behaviors that emerge from a group of homogeneous individuals optimized for a specific capability. We created a group of simple, identical neural network based agents modeled after chemotaxis-driven vehicles that follow pheromone trails and examined multi-agent simulations using clones of these evolved individuals. Our results show that the evolution of individuals led to population differentiation. Surprisingly, we observed that collective fitness significantly changed during later evolutionary stages, despite maintained high individual performance and simplified neural architectures. This decline occurred when agents developed reduced sensor-motor coupling, suggesting that over-optimization of individual agents almost always lead to less effective group behavior. Our research investigates how individual differentiation can evolve through what evolutionary pathways.

Figures

Figures reproduced from arXiv: 2412.02085 by the authors.

Figure 1
Figure 1. Neural network model of an agent. The agent receives input from six sensors (five on the periphery of its [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Single agent evolutionary simulation environment. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Representative agent trajectories at generations [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Moving averages of statistical metrics for [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Kinetic energy distribution among 1024 agents up to generation 500, calculated over 1000 sampled steps per generation. The colors of the distributions represent different generations. As evolution progresses, the distribution becomes bimodal with values concentrated at…
Figure 6
Figure 6. Figure 6: Results from evolutionary simulations conducted with [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Time series of pheromone gain at generations [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Evolution of single agent pheromone gain up to generation [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Mapping of two context neurons’ outputs from the agent’s neural network, corresponding to the pheromone [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Evolution of collective behavior distributions at generations [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Correlations between mutual information MI(I; O) and corresponding metrics in later generations across 10 different evolutionary seeds. Colors indicate generations. (A) Negative correlation between mutual information and collective fitness. (B) Negative correlation be…
Figure 12
Figure 12. Figure 12: Trajectory of rule-based agent. The agent exhibits chemotaxis in response to pheromone gradients. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 14
Figure 14. Figure 14: Distribution of behaviors in multi-agent simulation with [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 14 canonical work pages

  1. [1]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  2. [2]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022

    arXiv 2022

  3. [3]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022

    arXiv 2022

  4. [4]

    Scaling autoregressive models for content-rich text-to-image generation

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2022

    arXiv 2022

  5. [5]

    Swarm intelligence: from natural to artificial systems

    Eric Bonabeau, Marco Dorigo, and Guy Theraulaz. Swarm intelligence: from natural to artificial systems. Oxford university press, 1999

    work page 1999

  6. [6]

    Environment induced emergence of collective behavior in evolving swarms with limited sensing

    Fuda Van Diggelen, Jie Luo, Tugay Alperen Karagüzel, Nicolas Cambier, Eliseo Ferrante, and AE Eiben. Environment induced emergence of collective behavior in evolving swarms with limited sensing. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 31–39, 2022

    work page 2022

  7. [7]

    Collective gradient perception with a flying robot swarm

    Tugay Alperen Karagüzel, Ali Emre Turgut, AE Eiben, and Eliseo Ferrante. Collective gradient perception with a flying robot swarm. Swarm Intelligence, 17(1):117–146, 2023

    work page 2023

  8. [8]

    Foraging in mind.Current Directions in Psychological Science, 29(3):309–315, 2020

    Peter M Todd and Thomas T Hills. Foraging in mind.Current Directions in Psychological Science, 29(3):309–315, 2020

    work page 2020

Show all 21 references
  1. [9]

    Evolution of explorative and exploitative search strategies in collective foraging

    Ketika Garg, Paul E Smaldino, and Christopher T Kello. Evolution of explorative and exploitative search strategies in collective foraging. Collective Intelligence, 3(1):26339137241228858, 2024

    2024

  2. [10]

    Social insects and beyond: The physics of soft, dense invertebrate aggregations

    Olga Shishkov and Orit Peleg. Social insects and beyond: The physics of soft, dense invertebrate aggregations. Collective Intelligence, 1(2):26339137221123758, 2022

    2022

  3. [11]

    The collective intelligence of evolution and development

    Richard Watson and Michael Levin. The collective intelligence of evolution and development. Collective Intelligence, 2(2):26339137231168355, 2023

    2023

  4. [12]

    Collective intelligence for deep learning: A survey of recent developments

    David Ha and Yujin Tang. Collective intelligence for deep learning: A survey of recent developments. Collective Intelligence, 1(1):26339137221114874, 2022

    2022

  5. [13]

    Evolving collective ai: Simulation of ants communicating via chemicals

    Ryosuke Takata, Yujin Tang, Yingtao Tian, Norihiro Maruyama, Hiroki Kojima, and Takashi Ikegami. Evolving collective ai: Simulation of ants communicating via chemicals. InALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press, 2023

    2023

  6. [14]

    The cma evolution strategy: A tutorial

    Nikolaus Hansen. The cma evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772, 2016

    2016 arXiv

  7. [15]

    Evojax: Hardware-accelerated neuroevolution

    Yujin Tang, Yingtao Tian, and David Ha. Evojax: Hardware-accelerated neuroevolution. arXiv preprint arXiv:2202.05008, 2022

    2022 arXiv

  8. [16]

    Self-organization in biological systems

    Scott Camazine, Jean-Louis Deneubourg, Nigel R Franks, James Sneyd, Guy Theraula, and Eric Bonabeau. Self-organization in biological systems. In Self-Organization in Biological Systems. Princeton university press, 2020

    2020

  9. [17]

    Emergence of swarming behavior: foraging agents evolve collective motion based on signaling

    Olaf Witkowski and Takashi Ikegami. Emergence of swarming behavior: foraging agents evolve collective motion based on signaling. PloS one, 11(4):e0152756, 2016. 14

    2016

  10. [18]

    Adaptability and diversity in simulated turn-taking behavior

    Hiroyuki Iizuka and Takashi Ikegami. Adaptability and diversity in simulated turn-taking behavior. Artificial Life, 10(4):361–378, 2004

    2004

  11. [19]

    Division of labor in insect societies

    Gene E Robinson. Division of labor in insect societies. Encyclopedia of Insects, pages 297–299, 2009

    2009

  12. [20]

    From division of labor to the collective behavior of social insects

    Deborah M Gordon. From division of labor to the collective behavior of social insects. Behavioral ecology and sociobiology, 70:1101–1108, 2016

    2016

  13. [21]

    The wisdom of crowds

    James Surowiecki. The wisdom of crowds. Anchor, 2005. 15

    2005

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.