Pith. sign in

REVIEW 3 major objections 5 minor 23 references

In a minimal survival economy where LLM agents pay energy for every token, larger models consistently spend more energy than they gain—even when the explicit per-token size penalty is removed—and cooperative objectives push agents to donate

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:49 UTC pith:5RF3C5ES

load-bearing objection A useful new testbed for token-cost-sensitive multi-agent LLM simulation, but the abstract overstates a headline result that the paper's own Table 4 contradicts. the 3 major comments →

arxiv 2607.14865 v1 pith:5RF3C5ES submitted 2026-07-16 cs.MA cs.CL

The Energy Society: A Simulation Environment for Studying Agent Cooperation under Survival Pressure

Lucas Bergholdt Hansen , Federico Torrielli , Filippo Tonini , Lukas Galke Poech This is my paper
classification cs.MA cs.CL
keywords multi-agent systemsLLM agentscooperationcompetitionsurvival pressureenergy efficiencytoken costemergent behavior
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces The Energy Society, a minimal multi-agent simulation in which language-model agents must spend a shared energy resource to think and act: each generated token costs energy scaled by model size, and agents earn energy by solving knowledge questions. The central claim is that when token cost is tied directly to survival, larger models are persistently less energy-efficient than smaller ones, even in a control condition that removes the size penalty from the cost function. The paper also claims that switching from a competitive to a cooperative objective changes behavior markedly: agents begin donating energy to reactivate deactivated peers, sometimes at the cost of their own survival, and task allocation shifts so that smaller agents take on harder jobs. Ablations show that a discussion phase reduces job collisions and supports coordination, while memory helps agents calibrate risk. The authors offer the environment as a compact testbed for studying how inference cost and group incentives shape emergent agent behavior.

Core claim

The paper's central discovery is a recurring efficiency asymmetry and a behavioral shift under cooperation. Across all experimental variants, the two largest models (8B and 9B) consume more energy per round and return less energy from completed jobs than the three 4B models, and this pattern persists when the cost formula is set to charge all models equally per token. In the cooperative condition, smaller agents donate a substantial share of their actions to reactivate the larger, less efficient agents, and these donations—not general redistribution—are the primary driver of the larger agents' extended survival and of the smaller agents' own occasional deactivation. The paper treats these as

What carries the argument

The load-bearing mechanism is the energy cost function C = k·T·S^α, which makes every token an agent generates consume energy proportional to a power of its model size, and the three-phase round structure (discussion, decision, action) in which agents exchange non-binding recommendations before committing to actions. Jobs come in three difficulty tiers with rewards 200/500/800 energy, and successful attempts split the reward, so coordination and risk-taking are both incentivized. Donations are the only way to reactivate a deactivated agent, which makes the energy ledger the direct channel through which cooperative and competitive objectives exert their effects.

Load-bearing premise

The entire size-efficiency finding assumes that the observed differences between the 4B and 8B/9B agents are caused by model size rather than by the different architectures and training recipes of the three model families represented in the five agents.

What would settle it

Run the identical simulation with a single model family that offers multiple sizes (for example, a 1B, 3B, 7B, and 12B variant) and check whether the largest members still spend more energy than they gain. If the pattern disappears or reverses when family is held fixed, the paper's size-efficiency claim is an artifact of comparing different model families rather than a genuine property of scale.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the size-efficiency pattern holds, deploying larger models in resource-constrained, long-horizon agent settings will require either tighter reasoning budgets or smaller models, because the inefficiency persists even when the per-token price is equalized.
  • Cooperative objectives can reduce group efficiency: donations that revive inefficient agents and a shift of hard jobs onto smaller agents lower the overall energy return per unit spent.
  • Communication channels (the discussion phase) measurably reduce redundant job attempts, so agent architectures that want coordination should include a low-cost recommendation stage.
  • Memory of past outcomes changes job selection: without memory, agents chase hard jobs and show higher variance and more deactivations, implying risk calibration depends on history.
  • Halving the available jobs did not collapse the society; smaller agents remained efficient, suggesting decision complexity—not just reward scarcity—shapes energy economics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the five agents come from three model families, the simplest reading of the size-efficiency result—that larger models are intrinsically less efficient—is confounded with family-specific tendencies; a within-family size sweep would resolve whether the asymmetry is about scale or about architecture and training.
  • The donation pattern hints at a design principle for open multi-agent economies: if deactivated agents can only return via donations, cooperative framings create a natural channel for subsidizing inefficient participants, which may be undesirable in systems that must remain energy-positive.
  • The energy-ledger formulation could be applied to test other social mechanisms, such as reputation, punishment, or insurance pooling, by adding actions to the same cost structure; the paper itself only scratches the surface of communication-based selfishness.
  • A sharper test of the 'intrinsic verbosity' hypothesis is to log per-decision token counts across models of different sizes in the same family; if larger models generate more tokens even for the same job, the effect is about generation behavior, not the cost formula.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces The Energy Society, a multi-agent LLM simulation environment in which token generation costs energy scaled by model size, jobs replenish energy, and energy exhaustion deactivates agents. Five locally run models (4B to 9B, mapped to agents 1–5) are compared under competitive and cooperative objectives across a baseline and five controlled variants: no size penalty, no discussion, no memory, scarcity, and sabotage. The reported patterns are that larger models consume more energy and are less efficient; cooperation induces donations, especially from smaller to larger agents, and shifts job allocation; discussion reduces collisions; memory moderates risk-taking; and agents rarely sabotage directly but sometimes use recommendations self-servingly. The authors position the results as an initial empirical characterization and provide source code.

Significance. The environment is simple, reproducible, and targets a realistic and underexplored design: making inference cost consequential for agent survival. If the reported patterns replicate, the testbed offers a compact way to study efficiency incentives, cooperation, and communication in LLM-based multi-agent systems. The paper is appropriately cautious in its Discussion and Limitations sections, and the release of source code is a concrete asset. However, the headline claim as stated in the abstract is contradicted by the paper's own Table 4, and the efficiency metric mixes token consumption with donation transfers in the cooperative condition. These issues require correction, but they do not invalidate the overall empirical characterization.

major comments (3)
  1. [Abstract and §1 (contribution list)] The claim that larger models "spend more energy than they gain, even in those settings where token cost is not size-dependent" is contradicted by Experiment 2 (Table 4, α=0). In the competitive condition, agents 4 and 5 have efficiency ratios of 1.74±0.49 and 1.51±0.50; in the cooperative condition, 1.96±0.45 and 1.03±0.14. All four values exceed 1, meaning these agents gain more job energy than they spend in that setting. The body's conclusion in §6.1 and §8 correctly states the weaker claim "larger models are the least efficient" and "spend the most energy." The abstract and the corresponding bullet in §1 should be rewritten to match the data, e.g., "larger models are consistently the least energy-efficient" rather than "spend more energy than they gain."
  2. [§4.3, Efficiency metric] The efficiency ratio is defined as G_s/C_s, where G_s counts only job gains and C_s is "total energy spent." If C_s includes donated energy, as the tables' "Energy spent / round" values suggest for the cooperative condition, then the ratio penalizes donors for transfers that are not token consumption while excluding donations received from the numerator. In the baseline cooperative condition, agent 1 donates in 29% of actions; its reported efficiency drop from 1.39 to 0.71 may therefore be partly an accounting artifact of energy redistribution rather than a change in token-generation efficiency. Please state explicitly whether C_s includes donation transfers and, if so, report a supplemental efficiency metric based only on token-generation cost, or justify the accounting convention in the text.
  3. [§4.2 and §6.1] The "model size" comparison confounds size with model family: agents 1–3 are Gemma4-E4B, Nemotron-3-Nano-4B, and Qwen3.5-4B, while agents 4–5 are Qwen3-8B and Qwen3.5-9B. The within-family Qwen3.5 comparison (4B vs 9B) does support the efficiency pattern in Tables 3, 4, 5, and 7, so this is not fatal. However, §6.1 should explicitly identify the within-family comparison and avoid causal "model size" language beyond that comparison. Limitations (2) already acknowledges the issue; moving this caveat into the main analysis would make the primary claim more precise.
minor comments (5)
  1. [§3.2] Reference [17] for MMLU-Pro-Stratified appears to point to an unrelated ACL paper, "EducationQ." Please cite the correct source for the MMLU-Pro-Stratified dataset.
  2. [§4.3] Add one sentence clarifying whether C_s includes donation transfers and whether the reported "Energy spent / round" values include the flat 10-energy idle cost. This will remove the ambiguity that affects the interpretation of the cooperative condition.
  3. [Figures 3 and 5] The shaded bands are standard errors over only five seeds. Showing individual runs or seed-level point clouds would better convey the variability that the authors correctly acknowledge in Limitations (3).
  4. [§5.8 and Table 9] The claims about pooled hard-job share deltas in §5.8 are supported by Table 9 in the appendix, but the table is not referenced there. Add a pointer to Table 9.
  5. [§8] Minor grammar: in "the discussion phase and memory seems to act as coordination and calibration mechanisms," the verb should be "seem" to agree with the compound subject.

Circularity Check

0 steps flagged

No significant circularity; empirical study with only a non-load-bearing self-citation.

full rationale

This paper is an empirical simulation study rather than a mathematical derivation, so there is no derivation chain whose conclusions reduce to its inputs. The central claims—larger models spend more energy and are less efficient—are measured outcomes, not fitted predictions. The no-size-penalty condition (alpha=0.0) explicitly removes the size-dependent cost from the energy function, so the observation that larger models still spend more energy per round is an independent empirical finding about token-generation tendencies, not a tautology of the cost equations. The efficiency ratio (G_s/C_s) is defined post-hoc from measured gains and costs, and no parameters are back-fitted to make predictions match. The only self-citation is Tonini & Galke [19], used as background motivation in Related Work; it is not load-bearing for the paper's own results. The abstract's phrasing 'spend more energy than they gain, even in those settings where token cost is not size-dependent' is internally inconsistent with Table 4, where the two largest agents have efficiency ratios greater than 1 (e.g., 1.74 and 1.96 in the no-size-penalty coop condition), but this is a reporting/correctness concern, not circularity. The paper also candidly acknowledges model-family confounding and limited statistical power in its Limitations section, further supporting that its claims are empirical rather than definitionally forced.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The paper introduces the Energy Society as a testbed but no new physical or conceptual entities (particles, forces, dimensions). The main free parameters are the token-cost function constants (k, alpha) and the reward/idle-cost values, all chosen by the authors. The key domain assumptions are that prompt objectives shape behavior and that the difficulty labels are valid.

free parameters (4)
  • k (token-cost scaling constant) = 0.015
    Multiplies token count and model size in C = k*T*S^alpha; chosen by hand, not fitted to data. Its value sets the absolute energy economy and influences survival duration.
  • alpha (size-penalty exponent) = 0.5 (baseline), 0.0 (no-size-penalty control)
    Controls how steeply model size increases per-token cost. The baseline's survival asymmetry is partly built into this choice; the alpha=0 condition attempts to control for it.
  • job reward tiers = easy 200, medium 500, hard 800 energy
    Chosen to create a risk-reward tradeoff; these absolute values set efficiency thresholds and affect which agents are 'efficient'.
  • idle cost = 10 energy
    Flat cost for idling; hand-chosen, affects survival when agents choose inaction.
axioms (3)
  • domain assumption LLM agents interpret and follow the instructed objective (maximize own energy vs total energy)
    The entire competitive/cooperative comparison assumes the prompt changes agent utility. Discussed in Section 4.1; if prompts are not reliably followed, the distinction collapses.
  • domain assumption MMLU-Pro-Stratified difficulty labels are a valid proxy for question difficulty for these models
    Jobs are tiered by difficulty using Top-10 Model Average Accuracy (Section 3.2); if the difficulty mapping is miscalibrated for the five chosen models, success-rate differences are confounded.
  • domain assumption Tool-calling and structured output work as expected for all chosen models
    The simulation relies on structured outputs; Section 7(4) reports 3.27 and 2.80 fallback errors per run, which add noise but are treated as minor.

pith-pipeline@v1.3.0-alltime-deepseek · 15878 in / 10743 out tokens · 94879 ms · 2026-08-02T00:49:33.910320+00:00 · methodology

0 comments
read the original abstract

LLM-based agents are increasingly deployed in multi-agent environments whose incentives can shape their behavior. We introduce The Energy Society, a minimal survival economy for studying how competitive and cooperative incentives affect emergent behavior when inference cost is directly tied to survival: Agents spend energy based on model size when generating tokens, regain energy by completing jobs or receiving donations, and deactivate if their energy reaches zero. We compare competitive and cooperative objectives against a baseline setting and several control variants. Across experiments, larger models consistently consume the most energy and spend more energy than they gain, even in those settings where token cost is not size-dependent. Cooperative incentives substantially alter behavior: agents donate to reactivate others, sometimes at the cost of their own survival, and job allocation changes. Ablations reveal that allowing agents to recommend actions to each other supports coordination and ambitious job selection, while memory helps agents calibrate risk from past outcomes. Agents rarely choose direct sabotage, but show more subtle signs of self-serving behavior in the competitive setting. The Energy Society is a compact testbed for studying the interaction between token costs and group incentives under a survival pressure. Source code is available at https://github.com/LucasBergholdt/EnergySociety

Figures

Figures reproduced from arXiv: 2607.14865 by Federico Torrielli, Filippo Tonini, Lucas Bergholdt Hansen, Lukas Galke Poech.

Figure 1
Figure 1. Figure 1: Depiction of the dataset MMLU-Pro-Stratified from [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Depiction of the directed graph controlling the main [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mean energy over time per agent for the baseline. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of difficulty of chosen jobs in the base [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Mean energy over time per agent for the no size [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of difficulty of chosen jobs for the no [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Metrics related to the discussion phase across ex [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of difficulty of chosen jobs for the [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 2 canonical work pages

  1. [1]

    Joseph Bates. 1994. The role of emotion in believable agents.Commun. ACM37, 7 (July 1994), 122–125. doi:10.1145/176789.176803

    arXiv 1994

  2. [2]

    Marcel Binz and Eric Schulz. 2023. Using cognitive psychology to understand GPT-3.Proceedings of the National Academy of Sciences120, 6 (Feb. 2023). doi:10. 1073/pnas.2218523120

    2023

  3. [3]

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2023. AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. arXiv:2308.10848 [cs.CL] https://arxiv.org/abs/2308.10848

    Pith/arXiv arXiv 2023

  4. [4]

    Gordon Dai, Weijia Zhang, Jinhan Li, Siqi Yang, Chidera Onochie lbe, Srihas Rao, Arthur Caetano, and Misha Sra. 2024. Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory. arXiv:2406.14373 [cs.AI] https://arxiv.org/abs/2406.14373

    Pith/arXiv arXiv 2024

  5. [5]

    Kevin Dill and Lockheed Martin. 2011. A Game AI Approach to Autonomous Control of Virtual Characters

    2011

  6. [6]

    Nicoló Fontana, Francesco Pierri, and Luca Maria Aiello. 2024. Nicer Than Humans: How do Large Language Models Behave in the Prisoner’s Dilemma? arXiv:2406.13605 [cs.CY] https://arxiv.org/abs/2406.13605

    Pith/arXiv arXiv 2024

  7. [7]

    Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2023. Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives. arXiv:2312.11970 [cs.AI] https://arxiv.org/abs/2312.11970

    Pith/arXiv arXiv 2023

  8. [8]

    Hollan, Edwin L

    James D. Hollan, Edwin L. Hutchins, and Louis Weitzman. 1984. STEAMER: An Interactive Inspectable Simulation-Based Training System.AI Mag.5, 2 (June 1984), 15–27. doi:10.1609/aimag.v5i2.434

    work page doi:10.1609/aimag.v5i2.434 1984

  9. [9]

    Horton, Apostolos Filippas, and Benjamin S

    John J. Horton, Apostolos Filippas, and Benjamin S. Manning. 2026. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? arXiv:2301.07543 [econ.GN] https://arxiv.org/abs/2301.07543

    arXiv 2026

  10. [10]

    Laird and Michael van Lent

    John E. Laird and Michael van Lent. 2001. Human-Level AI’s Killer Application. AI Mag.22, 2 (June 2001), 15–25. doi:10.1609/aimag.v22i2.1558

    work page doi:10.1609/aimag.v22i2.1558 2001

  11. [11]

    Xinbei Ma, Ruotian Ma, Xingyu Chen, Zhengliang Shi, Mengru Wang, Jen tse Huang, Qu Yang, Wenxuan Wang, Fanghua Ye, Qingxuan Jiang, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Hai Zhao, Zhaopeng Tu, Xiaolong Li, and Linus

  12. [12]

    Alexander Meinke, Bronson Schoen, Jérémy Scheurer, Mikita Balesni, Rusheb Shah, and Marius Hobbhahn. 2025. Frontier Models are Capable of In-context Scheming. arXiv:2412.04984 [cs.AI] https://arxiv.org/abs/2412.04984

    Pith/arXiv arXiv 2025

  13. [13]

    O’Brien, Carrie J

    Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. arXiv:2304.03442 [cs.HC] https://arxiv.org/abs/2304.03442

    Pith/arXiv arXiv 2023

  14. [14]

    Bernstein

    Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. InProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology(Bend, OR, USA)(UIST ’22). Association for Computing Machinery, New York, NY...

    arXiv 2022

  15. [15]

    Jeff Rickel. 2001. Intelligent Virtual Agents for Education and Training: Oppor- tunities and Challenges. InProceedings of the Third International Workshop on Intelligent Virtual Agents (IV A ’01). Springer-Verlag, Berlin, Heidelberg, 15–22

    2001

  16. [16]

    Mark O. Riedl. 2012. Interactive narrative: a novel application of artificial intelli- gence for computer games. InProceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI’12). AAAI Press, 2160–2165

    2012

  17. [17]

    Yao Shi, Rongkeng Liang, and Yong Xu. 2025. EducationQ: Evaluating LLMs’ Teaching Capabilities Through Multi-Agent Dialogue Framework. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). Association for Computational Linguistics, 32799–32828. doi:10.18653/v1/2025.acl-long.1576

    doi:10.18653/v1/2025.acl-long.1576 2025

  18. [18]

    Lewis Johnson, Randolph M

    Milind Tambe, W. Lewis Johnson, Randolph M. Jones, Frank Koss, John E. Laird, Paul S. Rosenbloom, and Karl Schwamb. 1995. Intelligent Agents for Interactive Simulation Environments.AI Mag.16, 1 (March 1995), 15–39. doi:10.1609/aimag. v16i1.1121

    doi:10.1609/aimag 1995

  19. [19]

    Filippo Tonini and Lukas Galke. 2025. Super-additive Cooperation in Language Model Agents.arXiv preprint arXiv:2508.15510(2025)

    Pith/arXiv arXiv 2025

  20. [20]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. 2024. A survey on large language model based autonomous agents.Frontiers of Computer Science18, 6 (March 2024). doi:10.1007/s11704- 024-40231-1

    doi:10.1007/s11704- 2024

  21. [21]

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. 2024. MMLU-Pro: A More Robust and Challenging Multi-Task Language Understand- ing Benchmark.arXiv preprint arXiv:2406.01574(2024). 12 The Energy Society: A Simulation Environment for Studying Agent Cooperation under Surv...

    Pith/arXiv arXiv 2024

  22. [22]

    Zhaoqian Xue, Beichen Wang, Suiyuan Zhu, Kai Mei, Hua Tang, Wenyue Hua, Mengnan Du, and Yongfeng Zhang. 2025. What if LLMs Have Dif- ferent World Views: Simulating Alien Civilizations with LLM-based Agents. arXiv:2402.13184 [cs.CL] https://arxiv.org/abs/2402.13184 A Supplementary Material Figure 8 shows the distribution of difficulty of chosen jobs for th...

    Pith/arXiv arXiv 2025

  23. [2025]

    arXiv:2509.26126 [cs.CL] https://arxiv.org/abs/2509.26126

    The Hunger Game Debate: On the Emergence of Over-Competition in Multi-Agent Systems. arXiv:2509.26126 [cs.CL] https://arxiv.org/abs/2509.26126

    arXiv