{"id":"2542c10f-c2d2-4462-8ec4-dbeabb587772","arxiv_id":"2506.08507","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"MasHost uses reinforcement learning to autonomously construct query-adaptive multi-agent graphs, and its authors report the best average accuracy across six LLM benchmarks.","lead":"MasHost is a reinforcement learning system that builds a query-specific team of AI agents from scratch, choosing each agent's role and which agents talk to each other. On six standard math, reasoning, and coding benchmarks, the authors report it edges past prior automatic agent-design methods while using slightly fewer tokens.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'full autonomy' claim rests on sampling roles from an unconstrained role space, but the implementation uses a hand-curated global role pool (Sec 6.9), so MasHost is closer to candidate-pool sampling than the paper's Sec 3 dichotomy admits.","rationale":"The reader's weakest_assumption correctly identifies the role pool, and my stress test agrees. This is the most load-bearing issue because it targets the claimed novelty and autonomy, not just the numerical margins. The method could still be a useful extension of GRPO/PPO to graph construction, but the claim of being the first fully autonomous RL-driven Mas designer is not supported while the role vocabulary is human-authored and benchmark-aligned. The concern is not an attack on the authors; it is a mismatch between Sec. 5's guarantee and Sec. 6.9's implementation. A concrete experiment can settle it. Until then, CONDITIONAL remains the right verdict, so I leave the reader's verdict unchanged.","tokens_in":16230,"tokens_out":5158,"duration_ms":56078,"concrete_test":"Release the full role pool and its creation date/selection process, and run the following controlled experiment: construct a second pool by deleting every role whose name or Assist/Reject Conditions reference math, coding, or science (Algebra Solver, Coding Algorithm Specialist, Molecular Biologist, etc.) and replacing them with generic, task-agnostic roles (e.g., 'Assistant,' 'Critic'). Retrain MasHost with identical hyperparameters on GSM8K and HumanEval and report accuracy, token cost, and the role-query correlation. If accuracy drops substantially or the 100% correlation breaks, the reported results depend on human-curated, benchmark-aligned roles, and the 'entire role space without human-imposed restrictions' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty and autonomy claims rest on Sec. 5's assertion that role sampling 'operates over the entire role space' without 'human-imposed restrictions.' The implementation contradicts this. Algorithm 1 requires a 'full-scale role pool'; Sec. 6.9 states MasHost 'relies on a global role pool, which includes all known applicable roles,' with hand-written Responsibilities, Assist Conditions, and Reject Conditions. Every visualized system (Figs. 6-10) uses roles that are textbook fits to the benchmark (Algebra Solver for GSM8K, Coding Algorithm Specialist for MBPP/HumanEval, Applied Mathematician for MATH). The 100% role-query correlation in Fig. 3(middle) is therefore not strong evidence of emergent autonomy; it may simply reflect the policy learning to invoke roles whose Assist Conditions already encode the task-query mapping. Under this reading, MasHost is a larger, RL-driven candidate-pool method of exactly the kind Sec. 3 criticizes. Since the headline is 'first RL-driven framework for autonomous Mas graph construction' and the abstract promises freedom from human biases, this disconnect is load-bearing: if the pool is benchmark-adapted, the performance numbers do not validate autonomous construction, only RL over a hand-selected role set.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MasHost, a reinforcement learning framework that formulates multi-agent system (MAS) construction as a graph-search problem. It introduces Joint Probabilistic Space Sampling (JPSS) to jointly sample agent roles and connectivity, and Hierarchical Relative Policy Optimization (HRPO) to optimize a reward combining performance, token efficiency, and component rationality. Experiments on six benchmarks (GSM8K, MATH, GPQA, MMLU, HumanEval, MBPP) with GPT-4o-mini report an average accuracy of 74.20, claimed to be the best among all compared methods, and the paper argues that this demonstrates autonomous, query-adaptive, and human-bias-free MAS construction.","tokens_in":16438,"tokens_out":3712,"duration_ms":47894,"significance":"If the central claims were fully supported, the paper would make a useful contribution: it extends adaptive MAS design beyond candidate-pool sampling by treating role and edge selection as a jointly learned probabilistic policy, and it introduces a hierarchical reward structure that is not present in the main baselines. The authors also provide an unusually broad baseline comparison across single-agent, handcrafted, workflow, and semi-autonomous MAS methods, and they report cost comparisons. However, the paper's strongest claims—full autonomy, freedom from human biases, consistency of best performance, and 100% role-query correlation—are not currently supported by the evidence, because the role space is hand-curated, several hyperparameters are tuned on the same test benchmarks, and the performance margins over the strongest baseline are small and reported without variance or significance tests.","major_comments":[{"comment":"The autonomy claim is contradicted by the implementation. Section 5 states that role sampling 'operates over the entire role space' without 'human-imposed restrictions,' but Algorithm 1 requires a 'full-scale role pool' and Sec. 6.9 describes a hand-curated global role pool whose entries contain human-written Responsibilities, Assist Conditions, and Reject Conditions. The visualized systems in Figs. 6–10 use roles such as Algebra Solver for GSM8K and Coding Algorithm Specialist for HumanEval, which are direct matches to the benchmarks. Under this reading, MasHost is an RL-driven candidate-pool method of the same general kind criticized in Sec. 3, and the 100% role-query correlation in Sec. 6.4 may simply reflect the policy learning to invoke roles whose Assist Conditions already encode the query-to-task mapping. The authors should either provide evidence that the role pool is not adapted to the six benchmarks (e.g., out-of-domain queries, systematic removal of task-specific roles, or a role pool built without access to benchmark categories) or substantially soften the 'full autonomy' and 'free from human biases' claims.","section":"Sec. 5 and Sec. 6.9"},{"comment":"Hyperparameters are tuned on the same benchmarks used for the headline numbers. Section 6.6 selects TE=3 and nr=4 from sensitivity curves on GSM8K and HumanEval, and Sec. 6.8 sets β separately for GSM8K (0.0001) and for the other datasets (0.00001). Since these very benchmarks appear in Table 1, the reported improvements may reflect test-set tuning rather than a generally applicable configuration. The authors should move hyperparameter selection to a validation split or an auxiliary set, or report results for a fixed configuration chosen without access to the evaluated test sets.","section":"Sec. 6.6 and Sec. 6.8"},{"comment":"The claim that MasHost 'consistently achieves the best performance among all compared methods' is not supported by the table. On HumanEval, MasHost scores 89.96 while MaAS scores 90.09, so MasHost is not the best on that benchmark. Moreover, Table 1 reports 'average performance across five independent runs' but gives no standard deviations, confidence intervals, or significance tests; the average margin over MaAS is only 0.64 points (74.20 vs. 73.56), and on several datasets the difference is below 1 point. The authors should report per-run variance and conduct paired significance tests (e.g., bootstrap or matched-pair tests) before claiming consistent superiority.","section":"Table 1 and Sec. 6.2"},{"comment":"The 'correlation matching strategy' used to claim 100% role-query correlation is not described. Without a precise definition of how a role is deemed 'relevant' to a query, a perfect correlation score is not interpretable, especially because the role definitions in Sec. 6.9 already contain Assist Conditions such as 'General mathematics questions' or 'General biology questions.' The authors should specify the matching algorithm, report the actual role-query pairs, and ideally compare against a baseline that samples roles uniformly from the same pool to show that the policy's role choices are informative beyond the pool's built-in category structure.","section":"Sec. 6.4"},{"comment":"The action-wise reward in Eq. (5) requires judging the intermediate output O_t against the ground truth Y at every construction step, but the paper does not explain how O_t is obtained or evaluated for free-form generation tasks such as MATH, GPQA, or code generation. It is also unclear whether the policy receives this reward only at training time (which is legitimate) or whether the reward computation itself relies on evaluators that could leak information during test-time construction. The authors should clarify the protocol for comparing partial agent outputs to ground truth, and should state how O_t is defined when the current MAS has not yet produced a final answer.","section":"Eqs. (4)–(6) and Sec. 2.2"},{"comment":"Algorithm 1 samples a relative group of four MAS instances at every step (line 5) and then updates the policy inside the same loop, but the group-relative advantage in Eq. (4) is defined over a group G of L full MAS instances sampled from the old policy. This discrepancy between the per-step online group construction in Algorithm 1 and the batch-level formulation in Sec. 4.2 is not reconciled, and L is never specified. The authors should clarify whether the group is formed per step or per episode, and how the group rewards are aggregated over time.","section":"Sec. 4.1 and Algorithm 1"}],"minor_comments":[{"comment":"The abstract says 'consistently outperforms most competitive baselines' while Sec. 6.2 says 'consistently achieves the best performance among all compared methods'; these statements are inconsistent and should be aligned.","section":"Abstract and Sec. 6.2"},{"comment":"The token 'DELTE' appears in the enumeration of actions and should be corrected to 'DELETE'.","section":"Sec. 4.1"},{"comment":"The figure caption contains '(fight)' instead of '(right)', and the label 'M+ M M+' in the middle panel is unclear; the axes and legend should be described more carefully.","section":"Sec. 6.4 and Fig. 3"},{"comment":"The efficiency comparison is reported only on MATH; since cost is one of the three claimed objectives, the authors should provide cost results for all six benchmarks or explain why MATH alone is representative.","section":"Table 2"},{"comment":"The paper states that the global role pool 'includes all known applicable roles' but does not give the total number of roles, the process by which they were collected, or the exact list used in the experiments; the appendix shows only a subset of role prompts.","section":"Sec. 6.9"},{"comment":"In Table 3, 'Cost' is defined as a relative proportion of total token consumption during training, but the text and table do not specify which configuration is the reference for all rows or whether token counts include policy-network training and LLM inference separately.","section":"Sec. 6.5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of cs.MA and the core algorithmic idea—joint probabilistic sampling of roles and edges with a hierarchical RL objective—is worth considering after revision. My main concern is that the authors frame the contribution as 'fully autonomous' and 'free from human biases' while the role pool is demonstrably hand-curated; this mismatch affects the central novelty claim and should be addressed head-on rather than through wording changes alone. I would also ask the editor to encourage the authors to release the role pool, code, and hyperparameter selection protocol, since the current paper does not provide enough detail to reproduce the 100% correlation or the exact training budget."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a workable RL recipe for building multi-agent system graphs from a role pool, and it deserves referee time, but the \"full autonomy\" framing and the benchmark claims run ahead of the evidence.\n\nWhat is actually new: JPSS treats role selection and edge connectivity as one joint sampling step, and HRPO stacks group-relative advantage on per-action absolute rewards with an exemption time for early agents. That is a genuine departure from MaAS, MasRouter, and AFlow, which search over predefined structure pools or fixed workflows. The ablation table is useful: removing JPSS or HRPO hurts, and the cost-efficiency comparison is competitive.\n\nWhere it is soft: the numbers. Table 1 gives five-run averages with no variance, and the largest win over MaAS is 1.47 points on GSM8K; on HumanEval MasHost is actually below MaAS (89.96 vs 90.09). Beta is set per dataset on the six benchmarks, and TE and nr are chosen from sensitivity curves on GSM8K and HumanEval—the same sets that appear in the headline table. That makes the reported gains hard to interpret. The 100% role-query correlation in Fig. 3 is asserted with no metric described.\n\nThe autonomy claim is the biggest overreach. Section 5 says role sampling operates over the entire role space with no human-imposed restrictions, and Section 3 criticizes candidate-pool methods. But the implementation requires a \"global role pool\" whose prompts, Assist Conditions, and Reject Conditions are hand-written (Sec. 6.9). That is a large candidate pool with human-designed definitions, not an unrestricted role space. The stress-test has it right; the fix is to soften the claim and compare against a role pool that is not tuned to the benchmarks.\n\nNone of this kills the paper. The central comparison is against external baselines on standard benchmarks, and the method is not circular in its core. The fixes are standard: report variance or confidence intervals, tune hyperparameters on a held-out split, specify the rationality metric, and release code. If those land, this is a solid contribution.\n\nRecommendation: send to peer review, require those revisions.","headline":"A workable RL recipe for building MAS graphs from a role pool, but the 'full autonomy' and benchmark claims run ahead of the evidence; worth refereeing rather than desk-rejecting.","tokens_in":17073,"tokens_out":2340,"would_cite":true,"duration_ms":28824,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MasHost claims to be the first reinforcement-learning framework that constructs query-adaptive multi-agent systems from scratch, reporting the best average accuracy across six benchmarks.","keywords":["multi-agent systems","reinforcement learning","graph construction","role sampling","policy optimization","LLM agents","query adaptation","multi-objective reward"],"falsifier":"Run MasHost on queries from domains outside the role pool, such as legal reasoning or creative writing, without adding roles; if performance collapses to single-agent levels or the constructed graphs use obviously irrelevant roles, the full-space autonomous role-selection claim is falsified.","tokens_in":15930,"feed_emoji":"🤖","tokens_out":5671,"duration_ms":62055,"temperature":0.7,"pith_summary":"MasHost is a reinforcement-learning framework that builds a multi-agent system as a directed graph from scratch for each incoming query. It jointly samples agent roles and inter-agent connections through a unified probabilistic mechanism, and optimizes construction with a hierarchical reward scheme that balances answer accuracy, token cost, and structural rationality. The paper reports that MasHost outperforms all compared baselines on six benchmarks, with the largest absolute gain of 1.47% on GSM8K and the highest average score, 74.20, under GPT-4o-mini execution. If these results hold, RL can replace hand-crafted and semi-autonomous multi-agent system design with query-adaptive systems that are also cost-competitive.","feed_headline":"RL builds multi-agent teams that beat handcrafted workflows","feed_subtitle":"MasHost jointly samples agent roles and connections, and reports top scores across math, code, and QA benchmarks.","key_machinery":"Two mechanisms carry the argument. Joint Probabilistic Space Sampling treats role selection and connectivity as one differentiable action: the node policy produces a softmax over roles plus DELETE and EXIT, and the edge policy produces per-edge sigmoid scores that are multiplied by the selected role's probability before sampling. Hierarchical Relative Policy Optimization is the training objective: it adds a group-relative advantage, comparing accuracy and token cost across a group of constructed multi-agent systems, to an action-wise absolute reward that rewards reaching the correct answer and penalizes disrupting it, with an exemption time protecting early structure-building actions.","core_discovery":"The central claim is that the full construction of a multi-agent system, including which agents exist and how they communicate, can be posed as a graph-search problem and solved by reinforcement learning. MasHost's policy samples node-level actions from the union of a role pool and special DELETE and EXIT actions; whenever a role is added, a second policy samples edges to existing agents using joint probabilities that combine the role's selection probability with per-edge scores. Training uses Hierarchical Relative Policy Optimization, which combines a group-relative advantage over a batch of constructed systems, rewarding correct answers while penalizing token use, with an action-wise absolute reward that penalizes actions breaking a correct answer and protects early exploration through an exemption time. The paper reports that this yields the best accuracy among all compared methods on GSM8K, MATH, MMLU, GPQA, MBPP, and HumanEval, with an absolute improvement of up to 1.47% on GSM8K and an average of 74.20, while using fewer tokens than GPTSwarm, AFlow, and MaAS on MATH.","pith_inferences":["The 'entire role space' in the autonomy argument is, in implementation, the hand-curated pool of the role prompts section, with human-written responsibilities and assist or reject conditions; a decisive test would be whether MasHost can handle queries whose useful roles are absent from that pool.","Because reported averages come from five runs and no significance intervals are given, the 1.47% GSM8K gap over MaAS could be within run-to-run noise; a re-analysis with variance would settle how robust the lead is.","The joint probability scaling couples role relevance with edge confidence, so low-probability roles are less likely to receive connections even when they are the right collaborator; an unnormalized or calibrated variant might behave differently.","If the learned policies transfer across executor models, the same RL-trained graphs could be reused with cheaper or stronger LLMs, turning multi-agent system construction into a meta-learned capability rather than a per-benchmark artifact."],"forward_implications":["Reinforcement-learning-discovered multi-agent graphs can beat hand-crafted workflows and prior semi-autonomous search on standard math, QA, and code benchmarks using a single cheap executor, GPT-4o-mini.","Joint sampling of roles and edges makes full-graph construction tractable as a differentiable RL problem, removing the need for a predefined structure pool or task-level workflow.","Adding structural rationality as an explicit reward objective prevents both redundant and oversimplified systems: removing one agent from a constructed system drops performance, while adding one degrades it slightly.","MasHost is cost-competitive: on MATH it uses 3,630 prompt tokens and 3,698 completion tokens, cheaper than GPTSwarm, AFlow, and MaAS.","Ablations attribute the largest performance drop to removing the hierarchical reward, indicating that the multi-level reward design is the main driver of the reported gains."],"supporting_citations":[{"why":"Models workflows as RL-optimizable graphs; the graph-based RL baseline that MasHost extends and compares against.","marker":"[47]"},{"why":"Uses MCTS for automated workflow generation; a strong workflow baseline MasHost must beat.","marker":"[43]"},{"why":"Samples from an agentic supernet with a predefined structure pool; the state-of-the-art autonomous MAS baseline whose GSM8K score MasHost claims to exceed by 1.47%.","marker":"[41]"},{"why":"Trains an LLM to emit multi-agent systems as code; the seemingly autonomous approach the paper contrasts with RL-based exploration.","marker":"[38]"},{"why":"Designs communication topologies via graph neural networks; a semi-autonomous baseline in the comparison.","marker":"[42]"},{"why":"Automatic agent generation baseline in the autonomous multi-agent system category.","marker":"[3]"},{"why":"Source of the group-relative policy optimization idea that HRPO adapts into its hierarchical reward structure.","marker":"[26]"},{"why":"Source of the clipped importance-ratio objective that HRPO uses to stabilize policy updates.","marker":"[25]"}],"fun_headline_variants":["RL auto-builds multi-agent systems, tops all six benchmarks","MasHost: RL designs agent roles and links, beats handcrafted","Reinforcement learning learns to assemble agent networks from scratch","RL-driven multi-agent construction surpasses manual workflows","Auto multi-agent design via RL achieves 74.2 average accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The autonomy claim rests on the assumption that the hand-curated global role pool in the role prompts section is not implicitly fitted to the six evaluation benchmarks; if it is, MasHost is a larger candidate-pool sampler rather than a free explorer of the full role space.","fun_headline_variants_meta":{"raw":{"variants":["RL auto-builds multi-agent systems, tops all six benchmarks","MasHost: RL designs agent roles and links, beats handcrafted","Reinforcement learning learns to assemble agent networks from scratch","RL-driven multi-agent construction surpasses manual workflows","Auto multi-agent design via RL achieves 74.2 average accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000297,"raw_usage":{"total_tokens":1735,"prompt_tokens":974,"completion_tokens":761,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":676}},"tokens_in":590,"tokens_out":761,"duration_ms":9463,"temperature":1.0,"reasoning_tokens":676,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:09:59.801343+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MasHost on queries from domains outside the role pool, such as legal reasoning or creative writing, without adding roles; if performance collapses to single-agent levels or the constructed graphs use obviously irrelevant roles, the full-space autonomous role-selection claim is falsified.","supporting_citations":[{"cited_title":"Gptswarm: Language agents as optimizable graphs","cited_arxiv_id":null,"evidence_quote":"Models workflows as RL-optimizable graphs; the graph-based RL baseline that MasHost extends and compares against."}],"review_version":1}