Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Ant swarm functional control via stigmergic Reinforcement Learning agents

T0 review · 4 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A small population of RL-trained stigmergic agents can shift the phase transition line of the ant swarm model, creating trails where randomness previously prevailed.

desk verdict Plausible effect, but the 'phase transition shift' rests on a circular metric and a missing control: the learned-policy contribution is not yet pinned down. read the letter →

arxiv 2607.17709 v1 pith:VWCYUCHS submitted 2026-07-20 physics.soc-ph nlin.AOnlin.PS

Alessio Pitteri , Andrea Guizzo , Laura Ferrarotti , Bruno Lepri , Riccardo Gallotti This is my paper · ORCID
classification physics.soc-phnlin.AOnlin.PS
keywords antswarmmodelstigmergyreinforcementlearningphasetransitionfunctionalcontrollabilitytrailformationmulti-agentcontrolpheromonefield
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a small population of reinforcement-learned 'smart agents' can functionally control a stochastic ant swarm model by acting only on the shared pheromone field, shifting the order–disorder phase transition so that trail-like ordered states emerge in parameter regimes that would otherwise be random. The central result is quantitative: with 30 trained agents added to 300 model ants, the average trail metric ⟨TS⟩50 rises from −5.99 in the baseline to −1.43, crossing the empirical threshold of −1.6 that the authors associate with genuine trail formation; merely adding ants that release the same extra pheromone without learning gives only −3.63 and no trail networks. The study matters because it suggests macroscopic collective behaviour can be steered from inside the system via local, indirect, stigmergic interventions, without knowledge of the dynamics or control of individual agents.

What carries the argument

The mechanism is stigmergic control through the shared pheromone field: 30 smart-agents move according to a PPO-trained policy and deposit pheromone at rate η=1, altering the same field that guides the 300 ABM-ants. The reward is a local graph-based construction: from the agent's 9×9 neighbourhood the policy masks above-average pheromone, builds a spatial graph of connected components, and rewards components that are thin, weakly branched, populated by ants, and surrounded by a bimodal pheromone distribution (Eq. 19), mirroring the global trail metric TS (Eq. 9). This allows the central learner to shape macroscopic trail formation without explicit modelling of the ants' dynamics.

What would settle it

Take a disordered configuration such as [δ=0.4, β=4.5], run the baseline and the smart setup, and score the resulting pheromone fields with a metric independent of TS—for example the diameter/longest path of the high-pheromone mask or human labelling of trail versus cluster. If the smart setup no longer separates from the baseline, or if a random-movement controller with the same pheromone budget also crosses the −1.6 threshold, the paper's central claim would be unsupported.

Watch

Extended reading notes

Core claim

The central claim is that learned stigmergic control shifts the phase transition line of the ant swarm model: trained smart-agents, interacting with ABM-ants only by depositing pheromone, turn disordered configurations into trail-forming ordered ones across most of the phase diagram. The effect is not explainable by the extra pheromone release alone (enhanced setup yields clustered, poorly connected structures), and it is robust: 28 of 42 [δ, β] configurations exceed the 0.8 order-parameter threshold vs 13 in baseline, and a policy trained in one fixed configuration generalizes to the whole phase diagram with comparable performance (⟨TS⟩50 = −1.44 vs −1.43).

Load-bearing premise

The load-bearing premise is that the trail scenario metric ⟨TS⟩50 with the empirical threshold of −1.6 (and the 0.8 thresholds on m and M) measures what matters about trail formation; if those thresholds were artefacts of the metric, the claimed phase-transition shift could be a metric effect rather than a change in swarm dynamics.

Editorial extensions

If this is right

  • Ten percent of trained agents is sufficient: N′=15 still produces trails, N′=10 approaches the threshold, and performance saturates near N′=30, so full control of the population is not needed.
  • The phase-transition shift is not merely a pheromone dose effect: the enhanced setup with equal pheromone but unlearned movement forms clusters and remains below the trail threshold.
  • A policy trained in a single fixed parameter regime transfers across the whole phase diagram, indicating the learned navigation principles are general rather than memorised for one condition.
  • The approach does not achieve order in the high-noise, low-sensitivity corner of parameter space: pheromone becomes bimodal but ants cannot follow traces, marking an intrinsic controllability limit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test the authors do not run: replacing the learned policy with a simple heuristic—move to the neighbouring site with the most pheromone while depositing at the same rate—would reveal whether the phase shift requires RL at all. If the heuristic matches ⟨TS⟩50≈−1.4, the learning algorithm is not essential.
  • Because the reward and the evaluation metric are built from the same thresholded-mask construction, an independent trail-quality measure (e.g., persistence of the trail over time, or longest connected path length) would make the claimed shift more robust. This is an editorial caution, not a claim in the paper.
  • The framework suggests a general design principle for functional control of stigmergic systems: train a minority to sculpt the interaction field rather than to instruct individuals. One could test this in modified environments with different decay rates, lattice sizes, or field-mediated dynamics, where the authors' own limitation discussion invites such exploration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a stigmergic control framework for the ant swarm ABM: a small population of RL-trained 'smart-agents' deposits pheromone at a higher rate and moves according to a learned local policy, with the aim of shifting the order-disorder phase transition and producing trail scenarios in parameter regimes where the baseline model is disordered. The central quantitative claim is that the smart setup achieves avg<TS>_50 = -1.43 versus -5.99 (baseline) and -3.63 (enhanced), and is the only setup crossing the empirical trail threshold of -1.6 (Section 2.1, Fig. 5). The paper also reports a single-configuration training scenario with similar generalization (Section 2.2), order-parameter improvements over the phase diagram (Table 1), and an ablation over the number of smart-agents (Fig. 8).

Significance. If the central claim holds, the paper makes a useful contribution to RL-based functional control of complex systems: a relatively small set of locally acting agents, interacting only through the stigmergic field, would be able to move a stochastic swarm into a desired macroscopic regime. The study has genuine strengths: n_pi = 10 independent training runs, systematic evaluation across the full [delta, beta] phase diagram, an ablation over N', and an explicit comparison with an enhanced-pheromone control. These practices go beyond many single-run RL demonstrations. The main risk is that success is defined by an author-constructed metric whose threshold is selected from baseline data and whose local version is used as the RL reward, so the headline 'phase transition shift' needs stronger external validation and additional controls.

major comments (4)
  1. [Section 2.1; Eqs. (9) and (19); Section 4.1.3] The reward function in Eq. (19) is a local copy of the evaluation metric TS in Eq. (9): the penalty P and the local order parameters m(s_{t+1}), M(s_{t+1}) mirror the global quantities used to declare success. The empirical threshold -1.6 is also chosen from baseline snapshots (Fig. 10). This does not invalidate the result, but it makes the 'phase transition shift' claim partly self-referential. Please provide an external validation that TS >= -1.6 corresponds to trail formation in an independent sense (e.g., human-annotated snapshots or separate geometric criteria such as path elongation and connectivity), and report sensitivity of the conclusion to the chosen threshold. Without this, crossing -1.6 may be a property of the metric rather than of the swarm dynamics.
  2. [Section 2.1; Fig. 4] The enhanced setup is the only control for 'extra pheromone without a learned movement policy', but it is not a neutral movement baseline: the N_eta=1 enhanced agents follow the ABM dynamics with the same attraction to the pheromone field, which is precisely why they form clustered structures rather than trails. The comparison therefore shows only that one particular non-learned movement rule fails, not that any non-learned rule with the same pheromone output would fail. Because the RL policy is explicitly optimized to maximize the evaluation metric, a random-walk or persistent-motion policy depositing eta'=1 is a necessary control. If such a policy also crosses the -1.6 threshold in a comparable number of configurations, the claim that the learned movement policy drives the phase shift collapses. Please add this experiment or provide a concrete argument why no random/stochastic non-lear
  3. [Section 2.1; Table 1; SM3.1] The paper states that the smart setup provides 'statistically robust evidence' and the SM describes the distributions as 'statistically meaningful', but no hypothesis tests, confidence intervals, or effect-size statistics are reported. The aggregated values avg<TS>_50 = -5.99 +/- 4.44, -3.63 +/- 2.71, -1.43 +/- 1.42 pool over configurations and policies; because the metric is strongly configuration-dependent (SM9), the pooled standard deviation does not establish a systematic improvement. Please report per-configuration paired comparisons (using the same seeds), bootstrap confidence intervals or a mixed-effects model, and the number of configurations in which each setup crosses the threshold. This is load-bearing for the global claim.
  4. [Definition 1; Table 1; Section 4.1.2] The order-parameter threshold of 0.8 is presented as evidence for trail formation, but m and M above 0.8 also occur in the baseline clustered regime (top-left of Fig. 2). Table 1 therefore measures 'ordered versus disordered' rather than 'trails versus clusters'. The counts 28/42 vs 13/42 support a claim about increased order, not specifically about trails. Since the abstract's central claim is about trail emergence, the paper should either restrict the order-parameter analysis to configurations where the trail-geometry condition is satisfied, or explicitly reframe Table 1 as evidence for order rather than trail formation.
minor comments (5)
  1. [Section 2.1] The sentence 'The average value of <TS>_50 ... trespasses the empirical threshold' should be 'crosses' or 'exceeds' the threshold.
  2. [Fig. 5 and Fig. 8] Please add axis labels and units, and clarify the aggregation: is each point one policy-configuration-seed combination or a pooled distribution?
  3. [Eq. (9)] The notation avg_k |N_t^k| should be defined explicitly (presumably the mean number of nodes over the connected components in C2). Also clarify the role of the |C2(G_t)|/2 factor, which is not immediately transparent.
  4. [References] Reference [1] is incomplete (no journal/publisher details); some arXiv and SM cross-references are internally inconsistent (e.g., 'Section 4.4.1 - 4.1.2' appears twice with different numbering).
  5. [Section 4.2] The reward function is defined as r: S x A x S -> R_0^+, but Eq. (19) takes negative values; the codomain should be R or a bounded interval.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reward is a transparent local proxy for the evaluation metric, and the phase-transition claim is supported by independent order parameters and comparisons.

full rationale

The paper's derivation chain is self-contained. The RL reward (Eq. 19) is explicitly constructed as a local version of the trails-scenario metric (Eq. 9), and the paper states this openly ('we propose a step-wise reward function that locally captures the same properties encoded in the trails scenario metric of Eq. 9'). This is a design choice, not a hidden circularity: the global evaluation metric is not identical to the local reward, and the central claim about shifting the phase transition is additionally supported by the order parameters m(T) and M(T) (Eqs. 6-7), which are taken from the external ants-swarm literature [3], and by qualitative phase-diagram snapshots. The empirical thresholds (−1.6 for ⟨TS⟩50 and 0.8 for the order parameters) are descriptive criteria selected from baseline simulations, not parameters fitted to the controlled setup; applying them to the smart setup is a fair comparison. No load-bearing self-citations appear: the cited analytical phase-transition criterion and order parameters come from prior work by other authors ([1-3]), not by the present authors. The absence of a random-policy control for the enhanced setup is an experimental-design limitation that bears on causal attribution, but it is not a circularity: the comparison still tests a non-learned movement policy against the learned one. The paper does not rename a known result or smuggle in an ansatz via self-citation. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on a chain of modeling and metric choices: the discretized ABM, the inherited Eq. 4 phase boundary, the empirical TS/order thresholds, and the reward–evaluation metric alignment. No parameters are fit to external data; all are either inherited from prior work or chosen by the authors.

free parameters (4)
  • Empirical trail threshold ⟨TS⟩50 ≥ −1.6 = -1.6
    Chosen after inspecting baseline phase diagrams (Fig. 10); used to classify trail formation and to support the smart setup's improvement claim.
  • Empirical order-parameter threshold 0.8 = 0.8
    Selected to indicate an ordered phase in the baseline (Section 4.1.2); used to count configurations above/below the phase line in Table 1.
  • Smart-agent pheromone release rate η' = 1.0
    Designed as 100× the ABM rate η=0.01; central to the control mechanism and to the enhanced-setup comparison.
  • Number of smart-agents N' = 30
    Chosen for training; ablation shows 15 suffices and 10 is marginal. It is a design choice rather than a fit to external data.
assumptions (5)
  • domain assumption The analytical phase-transition criterion Eq. 4 from Rauch/Millonas/Chialvo is valid for the discretized ABM and marks the true baseline transition line.
    Used to define the original phase line and to argue that smart agents shift it; not re-derived for the discrete simulation or the controlled setup.
  • domain assumption The discretized ABM faithfully represents the continuous ant swarm model of Eqs. 1–3.
    All results come from the lattice simulation in Section 4.1.1; no convergence or discretization-error analysis is provided.
  • ad hoc to paper The TS metric and its empirical threshold −1.6 correctly identify trail scenarios.
    The metric is newly constructed and calibrated empirically; it is the primary evidence for trail emergence.
  • ad hoc to paper The order parameters m and M with threshold 0.8 reliably indicate ordered phases.
    Authors state 0.8 is an empirical threshold and note it cannot distinguish trails from clusters; it is nevertheless used to count ordered configurations.
  • standard math PPO training converges to a policy that approximates the maximum of the local reward.
    No convergence guarantee is provided; the paper relies on standard RL practice and 10 random seeds.
invented entities (1)
  • Smart-agents (stigmergic RL control agents)
    purpose: Additional agents that release pheromone at η'=1 and move according to a learned policy, aiming to steer ABM-ants into trail configurations.
    They exist only inside the simulation; no external falsifiable prediction or independent experiment is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ant swarm functional control via stigmergic Reinforcement Learning agents." pith.science (2026). https://pith.science/paper/VWCYUCHS

@misc{pith2026260717709,
  author       = {Pith},
  title        = {Pith review of: Ant swarm functional control via stigmergic Reinforcement Learning agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VWCYUCHS}},
  note         = {Machine review of arXiv:2607.17709}
}
read the original abstract

In this work, we propose a novel framework for the functional controllability of the ant swarm model, a well-known and relevant model of collective behaviour. Our approach introduces a population of controlling stigmergic agents, trained via Reinforcement Learning (RL), that act on the environment to influence the system dynamics and promote the emergence of ordered behaviour. Stigmergic agents are optimized in a centralized-training decentralized-execution setting, interacting with ants only through the shared pheromone field. The reward design promotes trail pheromone structures and alignment of ant positions with high-pheromone paths, without requiring control of specific microscopic configurations. Our results demonstrate that the learned policies effectively shift the phase transition line that characterizes the global behaviour of the system, enabling the emergence of trails scenarios in regimes that are typically dominated by randomness. This study provides insights into the potential of RL based control strategies for complex systems, contributing to the general understanding of functional controllability in this field.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 10 canonical work pages

  1. [1]

    Chialvo, D.R., Millonas, M.M.: How swarms build cognitive maps (1995)

    1995

  2. [2]

    Physics Letters A207(3), 185–193 (1995) https://doi.org/10

    Rauch, E.M., Millonas, M.M., Chialvo, D.R.: Pattern formation and functionality in swarm models. Physics Letters A207(3), 185–193 (1995) https://doi.org/10. 1016/0375-9601(95)00624-C

    1995

  3. [3]

    Millonas, M.M.: Swarms, phase transitions, and collective intelligence (paper 1); and a nonequilibrium statistical field theory of swarms and other spatially extended complex systems (paper 2) (1993)

    1993

  4. [4]

    The MIT Press, Cambridge, MA (2018).http://incompleteideas.net/book/the- book-2nd.html

    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge, MA (2018).http://incompleteideas.net/book/the- book-2nd.html

    2018

  5. [5]

    Ladyman, J., Lambert, J., Wiesner, K.: What is a complex system? Euro- pean Journal for Philosophy of Science3(2013) https://doi.org/10.1007/ s13194-012-0056-8 30

    2013

  6. [6]

    Nature453, 779–82 (2008) https://doi.org/10.1038/nature06958

    Gonzalez, M.C., Barabasi, A.-L.: Understanding individual human mobility patterns. Nature453, 779–82 (2008) https://doi.org/10.1038/nature06958

    doi:10.1038/nature06958 2008

  7. [7]

    Physics Reports734, 1–74 (2018) https://doi.org/10.1016/ j.physrep.2018.01.001

    Barbosa, H., Barthelemy, M., Ghoshal, G., James, C.R., Lenormand, M., Louail, T., Menezes, R., Ramasco, J.J., Simini, F., Tomasini, M.: Human mobility: Mod- els and applications. Physics Reports734, 1–74 (2018) https://doi.org/10.1016/ j.physrep.2018.01.001 . Human mobility: Models and applications

    2018

  8. [8]

    Physics Reports948, 1–148 (2022) https://doi.org/ 10.1016/j.physrep.2021.10.005

    Jusup, M., Holme, P., Kanazawa, K., Takayasu, M., Romi´ c, I., Wang, Z., Geˇ cek, S., Lipi´ c, T., Podobnik, B., Wang, L., Luo, W., Klanjˇ sˇ cek, T., Fan, J., Boccaletti, S., Perc, M.: Social physics. Physics Reports948, 1–148 (2022) https://doi.org/ 10.1016/j.physrep.2021.10.005 . Social physics

    doi:10.1016/j.physrep.2021.10.005 2022

Show all 46 references
  1. [9]

    The European Physical Journal Special Topics214(2012) https://doi.org/10.1140/ epjst/e2012-01694-y

    Miguel, M., Johnson, J., Kert´ esz, J., Kaski, K., Diaz-Guilera, A., Mackay, R., Loreto, V., Erdi, P., Helbing, D.: Challenges in complex systems science. The European Physical Journal Special Topics214(2012) https://doi.org/10.1140/ epjst/e2012-01694-y

    2012

  2. [10]

    European Review17, 237–248 (2009) https://doi.org/10.1017/S1062798709000738

    Nicolis, G., Nicolis, C.: Foundations of complex systems. European Review17, 237–248 (2009) https://doi.org/10.1017/S1062798709000738

    2009 doi

  3. [11]

    Copernicus / Springer-Verlag, New York, NY, USA (1996)

    Bak, P.: How Nature Works: The Science of Self-Organized Criticality. Copernicus / Springer-Verlag, New York, NY, USA (1996)

    1996

  4. [12]

    Proceedings of the National Academy of Sciences of the United States of America99 Suppl 3, 7280–7 (2002) https://doi.org/10.1073/pnas

    Bonabeau, E.: Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences of the United States of America99 Suppl 3, 7280–7 (2002) https://doi.org/10.1073/pnas. 082080899

    2002 doi

  5. [13]

    25–70 (2012)

    Helbing, D., Balietti, S.: Social Self-Organization: Agent-Based Simulations and Experiments to Study Emergent Social Behavior, pp. 25–70 (2012). https://doi. org/10.1007/978-3-642-24004-1 2

    2012 doi

  6. [14]

    Liu, Y.-Y., Barab´ asi, A.-L.: Control principles of complex systems. Rev. Mod. Phys.88, 035006 (2016) https://doi.org/10.1103/RevModPhys.88.035006

    2016 doi

  7. [15]

    Nature473, 167–73 (2011) https://doi.org/10.1038/nature10011

    Liu, Y.-Y., Slotine, J.-J., Barabasi, A.-L.: Controllability of complex networks. Nature473, 167–73 (2011) https://doi.org/10.1038/nature10011

    2011 doi

  8. [16]

    Nature497, 51–9 (2013) https://doi.org/10.1038/nature12047

    Helbing, D.: Globally networked risks and how to respond. Nature497, 51–9 (2013) https://doi.org/10.1038/nature12047

    2013 doi

  9. [17]

    https://doi.org/10.48550/arXiv.2504.07579

    Coraggio, M., Salzano, D., Bernardo, M.: Controlling Complex Systems (2025). https://doi.org/10.48550/arXiv.2504.07579

    2025 doi arXiv

  10. [18]

    CoRRabs/1707.06347(2017) 31

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. CoRRabs/1707.06347(2017) 31

    2017 arXiv

  11. [19]

    Royal Society Open Science11(9), 240845 (2024) https://doi.org/10

    Boldini, A., Civitella, M., Porfiri, M.: Stigmergy: from mathematical modelling to control. Royal Society Open Science11(9), 240845 (2024) https://doi.org/10. 1098/rsos.240845

    2024

  12. [20]

    The MIT Press, Cambridge, MA (2004)

    Dorigo, M., St¨ utzle, T.: Ant Colony Optimization. The MIT Press, Cambridge, MA (2004). https://doi.org/10.7551/mitpress/1290.001.0001

    2004 doi

  13. [21]

    Preprint at https://doi.org/10.48550/arXiv.2503.23308 (2025)

    Cai, W., Wang, G., Zhang, Y., Qu, X., Huang, Z.: Reinforcement Learning for Active Matter. Preprint at https://doi.org/10.48550/arXiv.2503.23308 (2025)

    2025 doi arXiv

  14. [22]

    Frontiers in Physics8, 200 (2020) https://doi.org/10.3389/fphy.2020.00200

    Costa, T., Laan, A., Heras, F., Polavieja, G.: Automated discovery of local rules for desired collective-level behavior through reinforcement learning. Frontiers in Physics8, 200 (2020) https://doi.org/10.3389/fphy.2020.00200

    2020

  15. [23]

    Durve, M., Peruani, F., Celani, A.: Learning to flock through reinforcement. Phys. Rev. E102, 012601 (2020) https://doi.org/10.1103/PhysRevE.102.012601

    2020 doi

  16. [24]

    Vicsek, T., Czir´ ok, A., Ben-Jacob, E., Cohen, I., Shochet, O.: Novel type of phase transition in a system of self-driven particles. Phys. Rev. Lett.75, 1226–1229 (1995) https://doi.org/10.1103/PhysRevLett.75.1226

    1995 doi

  17. [25]

    Cognitive Robotics3, 226– 256 (2023) https://doi.org/10.1016/j.cogr.2023.07.004

    Blais, M.-A., Akhloufi, M.A.: Reinforcement learning for swarm robotics: An overview of applications, algorithms and simulators. Cognitive Robotics3, 226– 256 (2023) https://doi.org/10.1016/j.cogr.2023.07.004

    2023 doi

  18. [26]

    Drones7(11) (2023) https://doi.org/10.3390/ drones7110673

    Dong, Z., Wu, Q., Chen, L.: Reinforcement learning-based formation pinning and shape transformation for swarms. Drones7(11) (2023) https://doi.org/10.3390/ drones7110673

    2023

  19. [27]

    Adaptive Behavior32(3), 265–281 (2024)

    Sadeghi Amjadi, A., Bilalo˘ glu, C., Turgut, A.E., Na, S., S ¸ahin, E., Krajn ´ ık, T., Arvin, F.: Reinforcement learning-based aggregation for robot swarms. Adaptive Behavior32(3), 265–281 (2024)

    2024

  20. [28]

    IEEE Transactions on cognitive and developmental systems15(4), 2122–2131 (2023)

    Na, S., Rouˇ cek, T., Ulrich, J., Pikman, J., Krajn ´ ık, T., Lennox, B., Arvin, F.: Fed- erated reinforcement learning for collective navigation of robotic swarms. IEEE Transactions on cognitive and developmental systems15(4), 2122–2131 (2023)

    2023

  21. [29]

    Nature Human Behaviour7(10), 1787–1796 (2023) https://doi.org/10.1038/s41562-023-01686-7

    McKee, K.R., Tacchetti, A., Bakker, M.A., Balaguer, J., Campbell-Gillingham, L., Everett, R., Botvinick, M.: Scaffolding cooperation in human groups with deep reinforcement learning. Nature Human Behaviour7(10), 1787–1796 (2023) https://doi.org/10.1038/s41562-023-01686-7

    2023 doi

  22. [30]

    Nature Human Behaviour6(10), 1372–1380 (2022) https://doi

    Bak-Coleman, J.B., Kennedy, I., Wack, M., Beers, A., Schafer, J.S., Spiro, E.S., Starbird, K., West, J.D.: Combining interventions to reduce the spread of viral misinformation. Nature Human Behaviour6(10), 1372–1380 (2022) https://doi. org/10.1038/s41562-022-01388-6 32

    2022 doi

  23. [31]

    Nature Human Behaviour3(2), 183– 193 (2019) https://doi.org/10.1038/s41562-018-0518-x

    Toyokawa, W., Whalen, A., Laland, K.N.: Social learning strategies regulate the wisdom and madness of interactive crowds. Nature Human Behaviour3(2), 183– 193 (2019) https://doi.org/10.1038/s41562-018-0518-x

    2019 doi

  24. [32]

    https://arxiv.org/abs/2404.16388

    Tovey, S., Lohrmann, C., Merkt, T., Zimmer, D., Nikolaou, K., Koppenh¨ ofer, S., Bushmakina, A., Scheunemann, J., Holm, C.: SwarmRL: Building the Future of Smart Active Systems (2024). https://arxiv.org/abs/2404.16388

    2024 arXiv

  25. [33]

    https://arxiv.org/abs/1709.02379

    Palmer, G., Yaida, S.: Optimizing collective fieldtaxis of swarming agents through reinforcement learning (2017). https://arxiv.org/abs/1709.02379

    2017 arXiv

  26. [34]

    Swarm and Evolutionary Computation72, 101085 (2022) https://doi.org/ 10.1016/j.swevo.2022.101085

    Abpeikar, S., Kasmarik, K., Garratt, M., Hunjet, R., Khan, M.M., Qiu, H.: Automatic collective motion tuning using actor-critic deep reinforcement learn- ing. Swarm and Evolutionary Computation72, 101085 (2022) https://doi.org/ 10.1016/j.swevo.2022.101085

    2022

  27. [35]

    Scientific Reports13(2023) https://doi.org/ 10.1038/s41598-023-44268-3

    L¨ offler, R.C., Panizon, E., Bechinger, C.: Collective foraging of active particles trained by reinforcement learning. Scientific Reports13(2023) https://doi.org/ 10.1038/s41598-023-44268-3

    2023 doi

  28. [36]

    Evolving Systems16(2025) https: //doi.org/10.1007/s12530-024-09651-z

    Sendra-Arranz, R., Guti´ errez, A.: Emergence of flocking behaviors transferring previously evolved alignment robot controllers. Evolving Systems16(2025) https: //doi.org/10.1007/s12530-024-09651-z

    2025 doi

  29. [37]

    Nature Communications15(2024) https://doi.org/10.1038/ s41467-024-52878-2

    Casert, C., Whitelam, S.: Learning protocols for the fast and efficient control of active matter. Nature Communications15(2024) https://doi.org/10.1038/ s41467-024-52878-2

    2024

  30. [38]

    1703.03864

    Salimans, T., Ho, J., Chen, X., Sutskever, I.: Evolution strategies as a scal- able alternative to reinforcement learning (2017) https://doi.org/10.48550/arXiv. 1703.03864

    2017 doi arXiv

  31. [39]

    Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences475, 20190351 (2019) https://doi.org/10.1098/rspa.2019.0351

    Bucci, M., Semeraro, O., Allauzen, A., Wisniewski, G., Cordier, L., Mathelin, L.: Control of chaotic systems by deep reinforcement learning. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences475, 20190351 (2019) https://doi.org/10.1098/rspa.2019.0351

    2019

  32. [40]

    Gronauer, S., Diepold, K.: Multi-agent deep reinforcement learning: a survey. Artif. Intell. Rev.55(2) (2022) https://doi.org/10.1007/s10462-021-09996-w

    2022 doi

  33. [41]

    Lama, A., Bernardo, M.: Shepherding and herdability in complex multi- agent systems. Phys. Rev. Res.6, 032012 (2024) https://doi.org/10.1103/ PhysRevResearch.6.L032012

    2024

  34. [42]

    Journal of Theoretical Biology26(3), 399–415 (1970) https://doi.org/10

    Keller, E.F., Segel, L.A.: Initiation of slime mold aggregation viewed as an insta- bility. Journal of Theoretical Biology26(3), 399–415 (1970) https://doi.org/10. 1016/0022-5193(70)90092-5 33

    1970

  35. [43]

    Journal of Management Infor- mation Systems40(3), 983–1008 (2023) https://doi.org/10.1080/07421222.2023

    Zheng, L.N., Mai, F., Yan, B., Nickerson, J.V.: Stigmergy in open collaboration: An empirical investigation based on wikipedia. Journal of Management Infor- mation Systems40(3), 983–1008 (2023) https://doi.org/10.1080/07421222.2023. 2229119

    2023

  36. [44]

    John Wiley and Sons, Ltd, Chichester, UK (2014)

    White, D.J.: Markov Decision Processes. John Wiley and Sons, Ltd, Chichester, UK (2014). https://doi.org/10.1002/9781118445112.stat02936

    2014 doi

  37. [45]

    CoRRabs/1504.08083(2015) 1504.08083

    Girshick, R.B.: Fast R-CNN. CoRRabs/1504.08083(2015) 1504.08083

    2015 arXiv

  38. [46]

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980(2014) 34 Supplementary Materials SM1 RL algorithm: Proximal Policy Optimization Proximal Policy Optimization (PPO) [18] is a policy gradient actor-critic algorithm for solving RL problems in a...

    2014 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.