{"id":"413dc228-b021-4896-aff7-b2482986c552","arxiv_id":"2504.14411","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"AIOS Server is a decentralized runtime for hosting AI agents, and the paper reports its first global deployment, the Internet of AgentSites, with DHT/Gossip-based agent discovery.","lead":"This paper introduces AIOS Server, a runtime for hosting AI agents across distributed servers so they can communicate and collaborate without a central controller. It reports a live deployment called the Internet of AgentSites and basic latency measurements, arguing this is a foundation for a future web of autonomous agents.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's internet-scale, decentralized agent-discovery claim is not yet supported: §6.3 tests only 3–7 nodes, §5.4 describes the deployed AgentHub as central registry nodes, and §7 lists DHT/Gossip scale validation as future work.","rationale":"The paper's central promise is a decentralized, internet-scale agent ecosystem; that promise stands or falls on whether the DHT/Gossip discovery layer actually powers the deployed AIOS-IoA at scale. The submitted evidence does not establish this. Section 6.3 reports only 3–7 nodes, with 1–2 ms registration; there is no churn, failure, or contention. Section 5.4 and Figure 8 describe AgentHub as central registry nodes with a global view, and §3 similarly routes registration through registry nodes, without reconciling these with §5.2's claim to avoid centralized coordination. Section 7 explicitly lists 'DHT registry validation at scale' as future work, so the paper itself confines the decentralized discovery mechanism to prototype status. This does not invalidate the engineering contribution — the MCP+JSON-RPC communication layer, public deployment, and released code are real — and the systems concept is plausible, but the headline scalability and resilience claims are not yet backed. A conditional verdict asking for a live code-path audit and larger scale validation is appropriate; no verdict change from the reader's conditional is needed.","tokens_in":12775,"tokens_out":4350,"duration_ms":40267,"concrete_test":"Trace an agent registration and lookup end-to-end in the public deployment. Use the AIOS.Server repository and the live planet.aios.foundation endpoint: instrument a lookup for a known agent_id and inspect whether the response is produced by the DHT/Gossip overlay or by the AgentHub registry database. If the live path is the registry database and the DHT/Gossip code is only an unintegrated prototype, the decentralized-discovery claim is not realized by the deployed system; if the live path uses DHT/Gossip, the §5.4 contradiction is resolved in favor of DHT.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that AIOS-IoA actually realizes decentralized, internet-scale agent discovery through the DHT+Gossip mechanism described in §5.2. Two pieces of evidence fail to support it. First, §6.3 evaluates discovery only on 3, 5, and 7 nodes, with registration times reported at 1–2 ms; there is no churn, no failure injection, no contention, and no basis for extrapolating to the claimed O(log n) internet scale. Second, the deployed architecture is described in §5.4 and Figure 8 as central registry nodes that maintain a global view of agent availability through a registry database, and §3 says nodes interact with an agent registry node for registration and discovery. The paper never reconciles these central-registry statements with the claim in §5.2 that the system avoids centralized coordination. The word 'prototype' in §5.2 and future-work item (4) in §7 ('DHT registry validation at scale: Integrating the DHT + Gossip registry into a larger distributed deployment with hundreds of nodes') make the paper's own scope clear: the DHT/Gossip discovery layer is not yet the validated backbone of the deployed IoA. Thus the headline claim rests on a mechanism that is neither deployed in the described form nor tested at scale.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces AIOS Server, a runtime framework for hosting and executing AI agents on individual network nodes, which it calls AgentSites. It argues that such AgentSites, connected over the internet, form an 'Internet of Agentsites' analogous to the World Wide Web. The technical contributions are a communication protocol built on MCP and JSON-RPC, a deployed system (AIOS-IoA) consisting of AgentHub for registration/discovery and AgentChat for human-agent interaction at planet.aios.foundation, and a prototype decentralized discovery mechanism based on Kademlia DHT and a Gossip protocol. The evaluation measures communication latency/throughput in local and cloud settings (50-200 requests) and registration latency on 3, 5, and 7 nodes. The central claim is that AIOS Server provides a practical, decentralized, internet-scale foundation for agent collaboration.","tokens_in":13025,"tokens_out":3443,"duration_ms":34266,"significance":"If substantiated, the paper would make a useful systems contribution: it provides an open-source implementation, a publicly accessible deployment, a concrete communication protocol, and a modular separation between protocol logic and task execution. The reproducibility of the implementation and the existence of a live deployment are genuine strengths. However, the load-bearing claims about decentralization and internet-scale scalability are not currently supported by the evidence. The deployed architecture is described as using central registry nodes, while the DHT/Gossip discovery mechanism is explicitly a prototype whose large-scale validation is listed as future work. The evaluation is too small in scale and too thin in methodology to estimate real-world performance under churn, contention, or failure.","major_comments":[{"comment":"There is a direct contradiction between the decentralized design claim and the description of the deployed system. Section 5.2 states that the DHT and Gossip mechanism 'avoids centralized coordination,' while Section 5.4 and Figure 8 describe AgentHub as 'central registry nodes' that 'manage a global view of agent availability through a registry database,' and Section 3 says AIOS nodes 'interact with an agent registry node for agent registration, discovery, task assignment, and health monitoring.' The paper never reconciles these two architectures. This is load-bearing because the paper's headline claim of a decentralized Internet of Agentsites depends on which architecture is actually deployed. The authors should either explicitly state that the public deployment uses the central-registry AgentHub and that the DHT/Gossip mechanism is only a prototype, or show how the two designs coexist without compromising decentralization.","section":"Sections 5.2 and 5.4"},{"comment":"The evaluation of the DHT/Gossip discovery mechanism does not support the claim of scalable, decentralized agent discovery. Section 6.3 reports registration tests on only 3, 5, and 7 nodes, with no repetitions, no error bars, no churn, no failure injection, and no comparison baseline. The claim of O(log n) scalability is asserted from the Kademlia design but never demonstrated at any scale beyond 7 nodes. This limitation is in fact acknowledged by the authors themselves in Section 7, future work item (4), which lists 'DHT registry validation at scale: Integrating the DHT + Gossip registry into a larger distributed deployment with hundreds of nodes.' The paper should therefore soften the scalability claims and either add a larger-scale simulation or explicitly present the DHT/Gossip system as prototype work that is not yet part of the validated deployment.","section":"Section 6.3 and Section 7"},{"comment":"The communication performance evaluation is too limited to support the paper's general claims of low-latency, high-throughput, and robust coordination. Table 3 reports only 50, 100, and 200 total requests under 5, 10, and 20 concurrent users respectively, with no repetitions, no standard deviations, no baseline system, and no sensitivity analysis. The statement in Section 6.2 that 'Throughput increased with load' is an artifact of the concurrency settings rather than evidence of scalability, and '100% response success' is meaningful only if we know the request mix and failure conditions, which are not described. At minimum, the authors should provide error bars, repeated trials, and a comparison against a simple baseline, or else restrict their claims to small-scale deployments and treat the larger-scale behavior as untested.","section":"Section 6.2 and Table 3"}],"minor_comments":[{"comment":"The abstract describes AIOS-IoA as 'the world's first practically deployed Internet of Agentsites,' but the paper's own evidence and Section 7 future-work list indicate that the decentralized DHT/Gossip discovery layer is not fully deployed or validated. Suggest using 'a first public deployment' or 'an initial deployment' to avoid an unsupported superlative.","section":"Abstract"},{"comment":"The text says the decentralized registration system is 'designed and implemented,' but Appendix D repeatedly calls it a 'prototype.' Please use consistent terminology so that readers know whether the code is integrated into the production AIOS deployment or is a standalone prototype.","section":"Section 5.2"},{"comment":"The screenshots and global maps in Figures 9-12 are presented as validation, but they are illustrative only. The paper should explicitly label them as demonstration artifacts and keep the quantitative claims separate.","section":"Section 6.4"},{"comment":"Table 2 compares 'Centralized AIOS' and 'Decentralized AIOS' in terms of scalability, fault tolerance, and routing, but no experiment in Section 6 directly measures these differences. Please mark the table as a design comparison rather than an empirical result.","section":"Table 2"},{"comment":"The paper has several typographical issues, including inconsistent spacing in JSON examples (e.g., 'la st _s ee n') and occasional missing words such as 'AIOS server' versus 'the AIOS server.' A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The title and Section 1 mention 'Planet as a Brain,' but the rest of the paper does not develop this metaphor or provide evidence for any global-brain-like coordination. Either discuss this framing in the body or remove it from the title.","section":"Title and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is best viewed as a systems demonstration and position paper rather than a fully validated large-scale deployment. I recommend major revision: the authors must reconcile the central-registry description with the decentralized DHT/Gossip claim, add more substantial evaluation or explicitly scope the contributions to a prototype, and soften superlative claims such as 'world's first.' The underlying implementation and public deployment are valuable, but the current version overclaims on decentralization and scale."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is genuine engineering with a public deployment and open-source code, but it's a systems-integrative paper, not a research advance. The headline claim—internet-scale decentralized agent discovery—is not backed up by the evaluation, and the architecture description contradicts itself about central vs. decentralized registries.\n\nWhat's actually new: the AIOS Server runtime that combines MCP and JSON-RPC for agent-to-agent and human-to-agent messaging, a node architecture for hosting agents, and a live instance (AIOS-IoA) with AgentHub and AgentChat at planet.aios.foundation. The code is on GitHub, and the appendix gives concrete message schemas and implementation excerpts. That is a real artifact; if you work on agent deployment or communication protocols, it's useful related work and a reference point.\n\nWhat it does well: the layered design is clean, the choice of standard protocols (MCP, JSON-RPC) makes integration practical, and the deployment shows the system can run in the wild. The paper is also honest enough to call the DHT/Gossip part a 'prototype' and to list scaling validation as future work.\n\nWhere it falls short: the experimental section is too thin. Communication tests use only 50-200 requests per scenario, no repetitions, no error bars, and no baseline. Discovery was tested on 3, 5, and 7 nodes, with registration times around 1-2 ms; that tells you nothing about O(log n) behavior at scale. More importantly, Section 5.4 describes the deployed AgentHub as 'central registry nodes' maintaining a global view, while Section 5.2 claims the system avoids centralized coordination. The paper never reconciles these, and future work item (4) admits the DHT registry hasn't been validated at scale. So the 'internet of agentsites' as a decentralized system is not what was actually evaluated; the live deployment appears to be a central-registry network with a DHT/Gossip prototype bolted on. That's fine as a stepping stone, but the paper should say so clearly.\n\nThe 'world's first practically deployed' phrasing is also not substantiated—there are other agent-network efforts, and the term 'first' needs a clearer comparison.\n\nWho is this for? People building agent-hosting infrastructure or studying agent communication protocols will get value. It's not a paper that changes the research landscape. It deserves a serious referee because the artifact is reproducible and the design choices are worth discussing, but only if the revision clearly separates the deployed centralized system from the experimental decentralized layer, strengthens the evaluation, and tones down the claims.","headline":"A useful engineering report on a deployable agent-hosting runtime, but the 'internet-scale decentralized discovery' claim outruns the evidence.","tokens_in":13590,"tokens_out":3113,"would_cite":true,"duration_ms":25408,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AIOS Server turns any server into an AgentSite, letting AI agents collaborate across the internet without a central platform.","keywords":["Internet of AgentSites","AIOS Server","AI agents","decentralized agent discovery","Distributed Hash Table","Gossip protocol","Model Context Protocol","multi-agent systems"],"falsifier":"Run the decentralized registry on a few hundred geographically distributed nodes, let nodes join and leave continuously, and measure whether registration latency stays around one millisecond and whether capability lookups converge; if either degrades with network size, the decentralization claim fails. A simpler check is to query the public AgentHub and observe whether lookups are answered by the hub's central registry rather than by DHT/Gossip peers.","tokens_in":12540,"feed_emoji":"🌐","tokens_out":7420,"duration_ms":63100,"temperature":0.7,"pith_summary":"This paper sets out to establish that the web's next layer is an Internet of AgentSites: instead of static pages hosted on websites, independent servers host AI agents that accept tasks and return results. The authors' claim is that AIOS Server, a runtime built for hosting and executing agents, provides the foundation for that layer, with a standardized communication protocol and a decentralized discovery mechanism. They report the first practically deployed instance of such an internet, where agents register, discover one another, and chat across geographically distributed nodes. If the claim holds, agent-to-agent and human-to-agent cooperation no longer needs a single centralized platform; any node can host an agent and join a planet-wide pool of capabilities. The paper itself lists large-scale validation of the discovery mechanism as future work, so the strongest form of the claim is not yet fully measured.","feed_headline":"The first Internet of AgentSites is live, powered by AIOS Server","feed_subtitle":"Each agent runs on its own server; standard messaging connects agents, and a distributed registry replaces the central directory.","key_machinery":"The load-bearing mechanism is the combination of a Kademlia-style Distributed Hash Table (DHT) and a Gossip-based presence protocol, running beneath the AIOS Server runtime. The DHT gives each agent a keyed home for its metadata and promises logarithmic lookup cost as nodes grow, while the Gossip layer propagates liveness and capability updates with eventual consistency. On top of this, the communication protocol packages every interaction as a JSON-RPC request/response inside the Model Context Protocol, so agents, humans, and external tools exchange messages in one standard shape. The AgentHub registry nodes keep a global view in the deployed service, which means the system currently runs in a hybrid mode: a central registry for the public deployment and a decentralized DHT/Gossip prototype for the claimed scale-out path.","core_discovery":"The paper's central discovery is a working architecture for treating AI agents as first-class citizens of the web, on the same model as websites. Each AIOS Server node acts as an independent AgentSite, hosting one or more agents and exposing them to humans and other agents through JSON-RPC messages wrapped in the Model Context Protocol. Agents register themselves in a shared registry and are found through a Distributed Hash Table with a Gossip protocol for presence updates, which the paper presents as the search engine of the agent internet. The deployed system adds an AgentHub for registration and discovery and an AgentChat for interactive communication, and measurements on local and cloud deployments report sub-200-millisecond latencies and up to hundreds of requests per second. The intended outcome is an open, decentralized ecosystem in which task delegation, lookup, and collaboration happen across independent nodes rather than inside one provider's platform.","pith_inferences":["We infer that the central-registry AgentHub is best read as the bootstrap phase of the system; the paper's decentralization claim would only be settled by running discovery entirely over the DHT/Gossip layer with hundreds of nodes under churn.","We infer that trust and verification, not routing, will become the next constraint: an open DHT makes it easy to advertise fake capabilities, so some reputation or attestation layer will likely be needed before agents can rely on strangers.","We infer that the website-to-AgentSite analogy will invite the usual internet services, such as naming, caching, and traffic management, to grow around this core protocol."],"forward_implications":["An AI agent can be hosted on a server anywhere in the world and still be findable by capability, so users are not locked into whichever platform deployed the agent.","Agents can delegate tasks to agents on other nodes through the same JSON-RPC/MCP message format, making cross-machine multi-step workflows possible without fixed orchestration.","Because discovery is replicated across a DHT, the design has a concrete path to logarithmic lookup scaling and tolerance to nodes joining or leaving, provided the protocol behaves at large node counts.","The public deployment provides a reference point for what an open agent internet looks like: registration, discovery, chat, and task logging are all exposed as standard serving interfaces.","The paper's own future-work list treats large-scale DHT validation, security, and trust modeling as open, so the near-term practical value is strongest for small or loosely coupled multi-agent systems."],"supporting_citations":[{"why":"Supplies the underlying AIOS agent operating system that AIOS Server extends with hosting and network protocols.","marker":"Mei et al., 2024"},{"why":"Defines the alternative 'internet of agents' framing that this paper distinguishes from its physical, decentralized AgentSites deployment.","marker":"Chen et al., 2024"},{"why":"Provides the ReAct reasoning-acting loop that the hosted agents use for task interpretation and execution.","marker":"Yao et al., 2022"},{"why":"Establishes the survey-level picture of LLM-powered autonomous agents that motivates the need for open infrastructure.","marker":"Wang et al., 2024a"},{"why":"Represents the centralized multi-agent conversation baselines the paper contrasts with decentralized server-based collaboration.","marker":"Wu et al., 2023"}],"fun_headline_variants":["AgentSites: the web's next evolution, live now","AIOS Server turns agents into first-class web citizens","Decentralized agent web: DHT registry, JSON-RPC chat","Internet of AgentSites: open, p2p, and already running","Agents get their own sites: AIOS Server launches IoA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The key assumption is that the decentralized agent-finding protocol works just as well on hundreds of nodes as it does on 3 to 7, since that is the only scale the paper measures and the currently deployed hub still uses central registries.","fun_headline_variants_meta":{"raw":{"variants":["AgentSites: the web's next evolution, live now","AIOS Server turns agents into first-class web citizens","Decentralized agent web: DHT registry, JSON-RPC chat","Internet of AgentSites: open, p2p, and already running","Agents get their own sites: AIOS Server launches IoA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2666,"prompt_tokens":1054,"completion_tokens":1612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":670,"completion_tokens_details":{"reasoning_tokens":1522}},"tokens_in":670,"tokens_out":1612,"duration_ms":9671,"temperature":1.0,"reasoning_tokens":1522,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:48:58.970861+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the decentralized registry on a few hundred geographically distributed nodes, let nodes join and leave continuously, and measure whether registration latency stays around one millisecond and whether capability lookups converge; if either degrades with network size, the decentralization claim fails. A simpler check is to query the public AgentHub and observe whether lookups are answered by the hub's central registry rather than by DHT/Gossip peers.","supporting_citations":[{"cited_title":"Self-alignment of large language models via multi-agent social simulation","cited_arxiv_id":null,"evidence_quote":"Defines the alternative 'internet of agents' framing that this paper distinguishes from its physical, decentralized AgentSites deployment."}],"review_version":1}