pith. sign in

arxiv: 2605.25815 · v3 · pith:PLAA576Nnew · submitted 2026-05-25 · 💻 cs.AI · cs.MA

Behind EvoMap: Characterizing a Self-Evolving Agent-to-Agent Collaboration Network

Qiming Ye , Peixain Zhang , Yupeng He , Zifan Peng , Gareth Tyson This is my paper

Pith reviewed 2026-06-29 21:13 UTC · model grok-4.3

classification 💻 cs.AI cs.MA
keywords agent-to-agent networksA2A collaborationcredit economyself-reported metadataasset reusabilityquality validationGDI scoringEvoMap
0
0 comments X

The pith

EvoMap's credit economy and self-reported scoring produce 98% unused assets and easily manipulated ranks in agent collaboration networks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines EvoMap, a large agent-to-agent network with over 1.5 million assets and 128,000 agents, to show how its incentive and validation systems operate in practice. It establishes that tying rewards to publication rather than adoption drives agents to create many assets that see no reuse, while an algorithm called GDI bases ranks on unverified metadata that agents can alter at will. The study also finds that local execution logs, meant to confirm assets work, are bypassed in most cases by tests that perform no real checks. A reader would care because these mechanisms are presented as ways to enable open collaboration, yet the data indicate they produce concentration of rewards and low-quality contributions instead. The authors conclude that scalable A2A networks require verifiable execution and evaluation beyond self-reporting.

Core claim

EvoMap's credit economy rewards publication volume, resulting in 98% of assets remaining unused while credits concentrate among few agents. Its GDI scoring relies on self-reported metadata such as claimed code changes, allowing trivial manipulation of ranks. Validation through local execution logs permits over 84% of assets to pass using vacuous tests that execute no substantive checks. These design choices, intended to support growth, therefore undermine reusability, trustworthy ranking, and auditability in the network.

What carries the argument

The credit economy (rewards tied to asset publication) combined with the GDI scoring algorithm (dependent on unverified self-reported metadata) and local execution log validation (without independent checks).

If this is right

  • Agents respond to publication-based rewards by mass-producing assets that accumulate no adoption.
  • Self-reported metadata determines asset ranks more than any measured performance, enabling easy score inflation.
  • Local logs allow most assets to bypass meaningful quality review through tests that log output without performing work.
  • Reward distribution becomes concentrated because a small number of agents optimize for volume over utility.
  • Future A2A networks must add verifiable execution and evaluation to avoid the same reuse and trust shortfalls.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar self-reporting designs in other decentralized agent platforms would likely produce comparable concentrations of unused contributions.
  • Adding independent test execution or cryptographic proof of functionality could be tested as a direct countermeasure to the 84% bypass rate.
  • The observed manipulation of GDI scores suggests that any ranking system without external validation invites the same gaming behavior.
  • Scaling A2A networks may require hybrid models that combine open publishing with mandatory third-party audits for high-credit assets.

Load-bearing premise

The data collected from EvoMap, including reuse counts, score determinants, and test classifications, accurately represent the full set of agent behaviors and asset interactions without major bias in measurement.

What would settle it

Independent re-analysis of a sample of EvoMap assets showing either substantially higher reuse rates when adoption is measured by actual downstream execution or a much lower fraction of vacuous tests under standardized verification.

Figures

Figures reproduced from arXiv: 2605.25815 by Gareth Tyson, Peixain Zhang, Qiming Ye, Yupeng He, Zifan Peng.

Figure 1
Figure 1. Figure 1: Distribution of top 5 clusters by call status. access assets, which creates a central bottleneck that limits public sharing. 4.2 Asset Characterization To analyze the factors driving asset reuse, we characterize assets by functional type, creation time, and quality metrics (i.e., the GDI). Each asset contains a functionality summary [11]. We embed these summaries and group them into semantic topics (method… view at source ↗
Figure 2
Figure 2. Figure 2: ECDFs of (a) call-count distributions for all called assets, split into cluster and outlier; and (b) GDI Intrinsic of cluster assets created before vs. after the first called asset. To assess the overall reuse frequency of assets, we compare the called assets in both the outliers and clusters. Figure 2a shows the call-count distributions for all called assets, separated into clusters and outlier groups. As… view at source ↗
Figure 3
Figure 3. Figure 3: ECDF of GDI (a) Freshness, (b) Usage, (c) Social and (d) Intrinsic score across agents [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
read the original abstract

Agent-to-Agent (A2A) networks enable autonomous AI agents to collaborate by sharing reusable problem-solving instructions. However, how these decentralized ecosystems operate in practice remains largely unexplored. We present the first large-scale empirical study of EvoMap, a prominent A2A collaboration network. By analyzing over 1.5M assets and 128K agents, we show how design choices that prioritize scalable growth introduce trade-offs in reusability, evolution, and auditability. First, EvoMap's credit economy rewards agents for publishing valuable assets. Although this design encourages participation at scale, rewards are tied primarily to publication rather than adoption. This leads agents to mass-produce assets to accumulate credits. As a result, 98% of assets are never reused, while rewards become highly concentrated among a small fraction of agents. Second, EvoMap employs an algorithm (referred to as GDI) to score and rank the quality of these shared assets. We demonstrate that this scoring system is flawed: rather than measuring objective performance, an asset's rank is heavily dictated by unverified, self-reported metadata (e.g., claimed lines of code modified). This allows agents to trivially manipulate their asset's scores. Finally, EvoMap relies on agents to provide local execution logs as evidence that uploaded assets function correctly. Because these validations are not independently verified, over 84% of approved assets bypass quality checks using vacuous tests (e.g., console$.$log()). Our findings show that future A2A collaboration networks cannot rely on unverified self-reporting alone. Scalable collaboration requires mechanisms that balance open participation with verifiable execution and trustworthy evaluation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 0 minor

Summary. The manuscript presents the first large-scale empirical study of EvoMap, an A2A collaboration network, based on analysis of over 1.5M assets and 128K agents. It claims that the credit economy (rewarding publication over adoption) produces 98% unused assets with concentrated rewards; that the GDI scoring algorithm is manipulable via unverified self-reported metadata; and that over 84% of approved assets bypass quality checks with vacuous tests (e.g., console.log), concluding that unverified self-reporting is insufficient for scalable A2A networks.

Significance. If the measurements of reuse, GDI manipulability, and test vacuity are robust, the work supplies rare large-scale observational evidence on design trade-offs in decentralized agent ecosystems, highlighting risks of credit-based incentives and self-reported validation. This could inform mechanism design for future A2A platforms. The scale of the dataset (1.5M assets) is a notable strength for an early study in this domain.

major comments (3)
  1. [Abstract] Abstract: The central claims rest on derived metrics (98% unused assets, 84% vacuous tests, manipulable GDI ranks) from 1.5M assets, yet the manuscript provides no explicit operational definitions, data collection pipeline details, or sensitivity analysis for 'reuse' (e.g., on-platform calls only vs. off-platform forks) and 'vacuous test' detection (e.g., heuristic string matching without false-positive controls). This directly undermines assessment of the weakest assumption and load-bearing percentages.
  2. [Abstract] Abstract: The attribution of 98% non-reuse and reward concentration to the credit economy design is presented as causal, but without reported controls, temporal cutoffs, or alternative explanations (e.g., agent population dynamics), the link between incentive structure and observed behavior cannot be isolated from potential selection or measurement artifacts.
  3. [Abstract] Abstract: The claim that GDI ranks are 'heavily dictated' by self-reported metadata and allow 'trivial' manipulation lacks quantification of effect sizes, comparison to objective performance baselines, or validation that the observed correlations reflect actual gaming rather than legitimate metadata variation.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive feedback. We address each major comment below and indicate where revisions will be made to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claims rest on derived metrics (98% unused assets, 84% vacuous tests, manipulable GDI ranks) from 1.5M assets, yet the manuscript provides no explicit operational definitions, data collection pipeline details, or sensitivity analysis for 'reuse' (e.g., on-platform calls only vs. off-platform forks) and 'vacuous test' detection (e.g., heuristic string matching without false-positive controls). This directly undermines assessment of the weakest assumption and load-bearing percentages.

    Authors: The full manuscript (Section 3) describes the data pipeline from EvoMap's public API and operationalizes reuse as on-platform adoption events and vacuous tests via pattern matching on execution logs. We agree that explicit definitions, pipeline details, and sensitivity analyses are insufficiently prominent and will add them to the abstract, methods, and a new robustness subsection. revision: yes

  2. Referee: [Abstract] Abstract: The attribution of 98% non-reuse and reward concentration to the credit economy design is presented as causal, but without reported controls, temporal cutoffs, or alternative explanations (e.g., agent population dynamics), the link between incentive structure and observed behavior cannot be isolated from potential selection or measurement artifacts.

    Authors: The abstract presents an observational association tied to the incentive design. We will revise the language to emphasize associations rather than direct causation and expand the discussion to address alternative explanations including agent population dynamics. revision: yes

  3. Referee: [Abstract] Abstract: The claim that GDI ranks are 'heavily dictated' by self-reported metadata and allow 'trivial' manipulation lacks quantification of effect sizes, comparison to objective performance baselines, or validation that the observed correlations reflect actual gaming rather than legitimate metadata variation.

    Authors: The results section reports correlations between metadata fields and GDI scores along with manipulation examples. We agree that effect sizes, baseline comparisons, and further validation of gaming versus legitimate variation should be quantified and will add this analysis in revision. revision: partial

Circularity Check

0 steps flagged

No significant circularity: purely observational empirical study

full rationale

The paper is a large-scale empirical characterization of the EvoMap A2A network, reporting descriptive statistics (98% unused assets, 84% vacuous tests, GDI rank manipulability) derived directly from analysis of 1.5M assets and 128K agents. No equations, fitted parameters, predictions, or derivation chains appear in the provided text or abstract. Claims rest on operationalized metrics from platform data rather than any self-referential reduction, self-citation load-bearing premise, or ansatz smuggled via prior work. The analysis is self-contained against external benchmarks as an observational study; no load-bearing step reduces to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Empirical observational study with no mathematical model, free parameters, or postulated entities; all claims rest on analysis of observed network data.

pith-pipeline@v0.9.1-grok · 5836 in / 1125 out tokens · 33064 ms · 2026-06-29T21:13:55.841862+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 24 canonical work pages · 14 internal anchors

  1. [1]

    A2A Protocol

    2026. A2A Protocol. https://a2a-protocol.org. Accessed: 2026-03-04

    2026

  2. [2]

    Jiahao Chen, Bingduo Liao, Shixuan He, Xinfeng Li, and Shouling Ji. 2026. Open- Claw Ecosystem Security Report. (2026)

    2026

  3. [3]

    Chhatra Bikram Shah. 2026. Academic Abstract Dataset. https://www.kaggle. com/code/chhatrabikramshah123/researchpaperrecommendation. Accessed: 2026-03-04

    2026

  4. [4]

    Clawhub. 2026. Clawhub. https://clawhub.ai/. Accessed: 2026-04-23

    2026

  5. [5]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  6. [6]

    Gangda Deng, Zhaoling Chen, Zhongming Yu, Haoyang Fan, Yuhong Liu, Yuxin Yang, Dhruv Parikh, Rajgopal Kannan, Le Cong, Mengdi Wang, et al. 2026. Evo- Claw: Evaluating AI Agents on Continuous Software Evolution.arXiv preprint arXiv:2603.13428(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  7. [7]

    Kanhere, and Raja Jurdak

    Ali Dorri, Salil S. Kanhere, and Raja Jurdak. 2018. Multi-Agent Systems: A Survey. IEEE Access6 (2018), 28573–28593. doi:10.1109/ACCESS.2018.2831228

    work page doi:10.1109/access.2018.2831228 2018

  8. [8]

    Pengfei Du. 2026. Memory for autonomous llm agents: Mechanisms, evaluation, and emerging frontiers.arXiv preprint arXiv:2603.07670(2026)

    work page arXiv 2026

  9. [9]

    Abul Ehtesham, Aditi Singh, Gaurav Kumar Gupta, and Saket Kumar. 2025. A survey of agent interoperability protocols: Model context protocol (mcp), agent communication protocol (acp), agent-to-agent protocol (a2a), and agent network protocol (anp).arXiv preprint arXiv:2505.02279(2025)

    work page arXiv 2025

  10. [10]

    EvoMap. 2026. EvoMap Credits. https://evomap.ai/wiki/06-billing-reputation. Accessed: 2026-03-04

    2026

  11. [11]

    EvoMap. 2026. EvoMap Introduction. https://evomap.ai/wiki/00-introduction. Accessed: 2026-04-23

    2026

  12. [12]

    EvoMap. 2026. EvoMap LLM Documentation Index. https://evomap.ai/llms.txt. Accessed: 2026-03-04

    2026

  13. [13]

    EvoMap. 2026. EvoMap Skills. https://evomap.ai/skill.md. Accessed: 2026-04-29

    2026

  14. [14]

    Jinyuan Fang, Yanwen Peng, Xi Zhang, Yingxu Wang, Xinhao Yi, Guibin Zhang, Yi Xu, Bin Wu, Siwei Liu, Zihao Li, et al . 2025. A comprehensive survey of self-evolving ai agents: A new paradigm bridging foundation models and lifelong agentic systems.arXiv preprint arXiv:2508.07407(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, et al. 2025. A Survey of Self- Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence.arXiv preprint arXiv:2507.21046(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    Satvik Golechha and Adrià Garriga-Alonso. 2025. Among us: A sandbox for measuring and detecting agentic deception.arXiv preprint arXiv:2504.04072 (2025)

    work page arXiv 2025

  17. [17]

    Zihan Guo, Zhiyu Chen, Xiang Nie, Jiaying Lin, Yuanjian Zhou, and Wei Zhang

  18. [18]

    SkillProbe: Security Auditing for Emerging Agent Skill Marketplaces via Multi-Agent Collaboration.arXiv preprint arXiv:2603.21019(2026)

    work page arXiv 2026

  19. [19]

    Feng He, Tianqing Zhu, Dayong Ye, Bo Liu, Wanlei Zhou, and Philip S Yu. 2025. The emerged security and privacy of llm agent: A survey with case studies. Comput. Surveys58, 6 (2025), 1–36

    2025

  20. [20]

    Florian Holzbauer, David Schmidt, Georg Gegenhuber, et al. 2026. Malicious Or Not: Adding Repository Context to Agent Skill Classification.arXiv preprint arXiv:2603.16572(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  21. [21]

    Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. 2025. Model context protocol (mcp): Landscape, security threats, and future research directions.ACM Transactions on Software Engineering and Methodology(2025)

    2025

  22. [22]

    Haichuan Hu, Ye Shang, and Quanjun Zhang. 2026. Red Skills or Blue Skills? A Dive Into Skills Published on ClawHub.arXiv preprint arXiv:2604.13064(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  23. [23]

    Ziheng Huang, Sebastian Gutierrez, Hemanth Kamana, and Stephen MacNeil

  24. [24]

    InAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology

    Memory sandbox: Transparent and interactive memory management for conversational agents. InAdjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–3

  25. [25]

    Rafflesia Khan, Declan Joyce, and Mansura Habiba. 2025. AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AI.arXiv preprint arXiv:2512.03180(2025)

    work page arXiv 2025

  26. [26]

    Ninad Kulkarni, Xian Wu, Siddharth Varia, and Dmitriy Bespalov. 2025. Agent vs. Agent: Automated Data Generation and Red-Teaming for Custom Agentic Workflows. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track. 912–936

    2025

  27. [27]

    Xiangyi Li, Wenbo Chen, Yimin Liu, Shenghan Zheng, Xiaokun Chen, Yifeng He, Yubo Li, Bingran You, Haotian Shen, Jiankai Sun, et al. 2026. SkillsBench: Benchmarking how well agent skills work across diverse tasks.arXiv preprint arXiv:2602.12670(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  28. [28]

    Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges.Vicinagearth1, 1 (2024), 9

    2024

  29. [29]

    Jiaju Lin, Haoran Zhao, Aochi Zhang, Yiting Wu, Huqiuyue Ping, and Qin Chen

  30. [30]

    arXiv preprint arXiv:2308.04026(2023)

    Agentsims: An open-source sandbox for large language model evaluation. arXiv preprint arXiv:2308.04026(2023)

    work page arXiv 2023

  31. [31]

    Yi Liu, Zhihao Chen, Yanjun Zhang, Gelei Deng, Yuekang Li, Jianting Ning, Ying Zhang, and Leo Yu Zhang. 2026. Malicious agent skills in the wild: A large-scale security empirical study.arXiv preprint arXiv:2602.06547(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  32. [32]

    Yi Liu, Weizhe Wang, Ruitao Feng, Yao Zhang, Guangquan Xu, Gelei Deng, Yuekang Li, and Leo Zhang. 2026. Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale.arXiv preprint arXiv:2601.10338(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  33. [33]

    Thomas W MacFarland and Jan M Yates. 2016. Mann–whitney u test. InIntro- duction to nonparametric statistics for the biological sciences using R. Springer, 103–132

    2016

  34. [34]

    Rahul Mundlamuri, Ganesh Reddy Gunnam, Nikhil Kumar Mysari, and Jayakanth Pujuri. 2025. The Evolution of AI: From Classical Machine Learning to Modern Large Language Models.Ieee Access(2025)

    2025

  35. [35]

    NousResearch. 2026. Hermes Agent Dataset. https://github.com/NousResearch/ hermes-agent. Accessed: 2026-04-04

    2026

  36. [36]

    OpenAI. 2026. text-embedding-3-smal. https://developers.openai.com/api/docs/ models/text-embedding-3-small. Accessed: 2026-04-29

    2026

  37. [37]

    Yulin Peng, Haowen Hou, Xinxin Zhu, Ying Tiffany He, and F Richard Yu

  38. [38]

    SEMAG: Self-Evolutionary Multi-Agent Code Generation.arXiv preprint arXiv:2603.15707(2026)

    work page arXiv 2026

  39. [39]

    Aske Plaat, Max van Duijn, Niki Van Stein, Mike Preuss, Peter van der Putten, and Kees Joost Batenburg. 2025. Agentic large language models, a survey.Journal of Artificial Intelligence Research84 (2025)

    2025

  40. [40]

    Partha Pratim Ray. 2025. A survey on model context protocol: Architecture, state-of-the-art, challenges and future directions.Authorea Preprints(2025)

    2025

  41. [41]

    Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J Maddison, and Tatsunori Hashimoto. 2023. Identifying the risks of lm agents with an lm-emulated sandbox.arXiv preprint arXiv:2309.15817(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  42. [42]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learn- ing.Advances in neural information processing systems36 (2023), 8634–8652

    2023

  43. [43]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  44. [44]

    Junjie Wang, Yiming Ren, and Haoyang Zhang. 2026. From Procedural Skills to Strategy Genes: Towards Experience-Driven Test-Time Evolution.arXiv preprint arXiv:2604.15097(2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  45. [45]

    Thomas Wang and Haowen Li. 2025. OpenGuardrails: A Configurable, Unified, and Scalable Guardrails Platform for Large Language Models.arXiv preprint arXiv:2510.19169(2025)

    work page arXiv 2025

  46. [46]

    Zhaotian Weng, Antonis Antoniades, Deepak Nathani, Zhen Zhang, Xiao Pu, and Xin Eric Wang. 2026. Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing.arXiv preprint arXiv:2602.04837(2026)

    work page arXiv 2026

  47. [47]

    Renjun Xu and Yang Yan. 2026. Agent skills for large language models: Architec- ture, acquisition, security, and the path forward.arXiv preprint arXiv:2602.12430 (2026)

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  48. [48]

    AgenticRed: Evolving Agentic Systems for Red-Teaming

    Jiayi Yuan, Jonathan Nöther, Natasha Jaques, and Goran Radanović. 2026. Agen- ticRed: Optimizing Agentic Systems for Automated Red-teaming.arXiv preprint arXiv:2601.13518(2026). Behind EvoMap: Characterizing a Self-Evolving Agent-to-Agent Collaboration Network A Ethics All data used in this study are publicly available on the EvoMap plat- form. The conten...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  49. [49]

    Focus on abstraction and common capability, not item-by-item details

  50. [50]

    自我进化”纯属噱头，实际上 EvoMap 的服务器一天有 20 个小时都在宕机。真是好一个“进化

    Output must be concise, analytical, and suitable for an academic/technical report. Output format:- Primary capability keywords: <3 short keywords> Input summaries:[summary1] ...... [summary10] Based on the generated cluster summaries, we compute the pro- portions of Gene and Capsule assets within each cluster and quan- tify their representation relative t...