{"id":"4ad0863f-401f-4a0a-bda5-605ae3070757","arxiv_id":"2607.10836","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"GRADE coordinates heterogeneous frozen LLM experts with four jointly trained gates and critic-free CoGRPO, outperforming multi-agent baselines on three reasoning benchmarks at ~17B average active parameters.","lead":"GRADE is a hierarchical multi-agent LLM system with four learned gates that choose which experts to run, how deep to go, whether agents communicate, and which branches to prune. It beats stronger multi-agent baselines on GSM8K, MMLUPro and GPQA at roughly half the active compute and supports hot-swapping experts via per-agent calibration.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Shared group-relative credit may be too noisy for hierarchical discrete gates, so the efficiency claim may rest more on architecture than on CoGRPO.","rationale":"The reader correctly flags the shared group-relative advantage (Eqs. 11–12) as the weakest assumption behind the strongest claim. The paper supplies useful supporting evidence (Table 4 objective comparison, regularizer statistics, leave-one-out architecture ablations in Table 9) but never directly measures the variance or bias of the collaborative credit signal for the four discrete gates. That leaves open the possibility that the hierarchy and masked cross-attention, not the particular form of CoGRPO, are doing most of the work. The concrete counterfactual-credit experiment would settle the issue without requiring a full independent reproduction of every baseline. Because the architecture results themselves look solid and the authors already acknowledge AIME limitations and the need for calibration, the appropriate stance remains CONDITIONAL rather than REJECT; the concern simply tightens the condition on what must be verified before the efficiency claim can be taken as fully established.","tokens_in":25179,"tokens_out":634,"duration_ms":7734,"concrete_test":"Re-train the identical GRADE architecture and agent pool with a counterfactual leave-one-gate-out baseline (or per-gate advantage estimated by one extra forward pass that freezes that gate) for 200 steps; recompute Table 1 / Table 4. If MMLUPro / GPQA drop by more than ~2 points relative to full CoGRPO while hierarchy ablations remain large, the shared-advantage mechanism is load-bearing; if accuracy is essentially unchanged, the efficiency claim rests primarily on architecture, not on CoGRPO credit assignment.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Table 1: ~17B active params beating Puppeteer by 4.8 on MMLUPro / 3.3 on GPQA) is attributed to the four gates trained end-to-end by CoGRPO. Section 2.4 (Collaborative Credit Assignment) and Eqs. 11–12 assign one scalar group-normalized advantage Â^(i) uniformly to every gate decision and every surviving agent in a rollout. The authors note this is noisy but unbiased and that regularizers keep ρ≈0.5, mean depth ≈1.9, k≈2–4. Table 4 shows CoGRPO ahead of MAGRPO-obj / Shared Reward / MAPoRL-obj, yet those ablations still use the same joint hierarchical action space; they do not isolate whether the shared-advantage signal itself is low-variance enough for the discrete Depth / Assignment / Cross-Read / Prune factors. If credit is too coarse, the observed gains could be driven mainly by the hierarchy + masked cross-attention (Table 9) rather than by the claimed joint RL recipe, weakening the causal link between CoGRPO and the efficiency numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes GRADE, a hierarchical multi-agent LLM system whose four learned gates (Assignment, Depth, Cross-Read, Prune) jointly decide which experts to activate, how deep a query should go, which agent pairs may communicate, and which branches to drop before combination. Training uses CoGRPO, a critic-free multi-agent adaptation of GRPO that samples groups of joint coordination rollouts and assigns a single group-relative advantage to every participating gate and surviving agent. Experts live in a hot-swappable registry with per-agent affine calibration maps so that a backend can be replaced at inference with only 64 anchor queries. Empirically, at ~17B average active parameters GRADE reports 94.8/78.2/57.4 on GSM8K/MMLUPro/GPQA, beating the strongest baseline (Puppeteer, ~28B) by 4.8 and 3.3 points on the latter two while remaining competitive on AIME-2025; ablations attribute the largest accuracy drops to hierarchy and masked cross-attention, and show that calibration is required for safe hot-swapping.","tokens_in":25597,"tokens_out":1403,"duration_ms":14658,"significance":"If the results hold under broader scrutiny, the work is a concrete step toward compute-adaptive multi-agent reasoning: it jointly learns depth, routing, selective communication and pruning inside one hierarchical policy, rather than fixing topology by prompt or always running a full ensemble. The Expert Registry plus per-agent calibration is a practical contribution for production systems that must swap models without full retraining. Strengths include three-seed reporting with standard deviations, leave-one-out component ablations (Table 9), RL-objective and fixed-k sweeps (Table 4), hot-swap recovery curves (Table 3, Figure 2), cross-read frequency and token-usage analyses (Figures 3–4), and a significance table against Puppeteer (Table 11). The central efficiency claim is therefore empirically well-supported even if the precise credit-assignment story remains partly architectural.","major_comments":[{"comment":"Section 2.4 (Collaborative Credit Assignment) and Eqs. (11)–(12) assign one shared group-normalized advantage Â^(i) uniformly to every gate decision and every surviving agent. Table 4 shows CoGRPO ahead of MAGRPO-obj / Shared Reward / MAPoRL-obj / MAPPO, but those variants still operate on the same joint hierarchical action space; they do not isolate whether the shared scalar is low-variance enough for the discrete Depth / Assignment / Cross-Read / Prune factors. Because Table 9 already attributes the largest accuracy drops to hierarchy and masked cross-attention, the causal link between CoGRPO’s credit assignment and the ~17B efficiency numbers in Table 1 is not fully established. A targeted ablation (e.g., per-gate counterfactual baselines on a subset of queries, or gate-wise advantage decomposition) would strengthen the claim that the joint RL recipe, not only the architecture, drives","section":null},{"comment":"Active-parameter accounting (Section 3.1, Tables 1–2) defines “~17B average active parameters” as the expected total parameters evaluated across forward passes, ranging from 0.5B (depth 0) to ~32B (all six sub-agents). The comparison to Puppeteer’s fixed ~28B is central to the efficiency claim, yet the paper does not report the empirical distribution of depth / k / cross-read decisions on the evaluation sets, nor a FLOPs or wall-clock breakdown that would let a reader recompute the average under alternative definitions (e.g., counting only unique model weights resident in VRAM). Without that distribution, the half-compute claim is harder to audit and may be sensitive to the particular agent pool (three Qwen-7B copies + Phi-3-mini + Llama-3.2-3B).","section":null},{"comment":"On AIME-2025 (Table 1), GRADE trails Puppeteer by 1.9 points (25.3 vs 27.2) and the authors correctly note that per-agent capacity dominates. The discussion (Section 3.3) suggests adding math-specialist backends, but the manuscript does not test whether the Depth / Assignment gates still allocate compute efficiently when the registry contains stronger specialists, nor whether CoGRPO’s cost term continues to prevent collapse onto the strongest model. A short specialist-swap experiment would clarify whether the adaptive-depth story generalizes beyond the current heterogeneous but relatively weak pool.","section":null}],"minor_comments":[{"comment":"Figure 1 caption and Section 2.1: managers are described as “trainable two-layer MLP” modules that “generate no tokens”; a one-sentence clarification that they only supply the grouping term in Eq. (1) would prevent readers from expecting manager-level generation.","section":null},{"comment":"Eq. (6) and the surrounding text: the Cross-Read mask R is applied as an element-wise product inside the softmax argument; state explicitly whether invalid pairs are set to −∞ or to 0 before the softmax, as the two conventions differ in gradient behavior.","section":null},{"comment":"Table 11 reports a pooled two-sample t-test with df=4 (three seeds). Note that multiple-comparison correction is not applied; a brief remark that the MMLUPro result remains significant under Bonferroni would help cautious readers.","section":null},{"comment":"Appendix A.1 lists many free hyperparameters (λbal, λD, λR, τP, α, etc.). A short sensitivity paragraph or a pointer to which of these were tuned vs. fixed a priori would improve reproducibility.","section":null},{"comment":"Typographical: “affinity” / “efficient” appear with ligature artifacts in several places; normalize to ASCII “affinity” / “efficient” for arXiv rendering consistency.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is a solid systems/ML contribution with unusually thorough ablations for the multi-agent LLM literature. The shared-advantage concern is real but not fatal; I would not block acceptance over it if the authors add a short clarifying experiment or a clearer limitation statement. Fit for a methods-oriented AI venue is good; novelty relative to Puppeteer / MAPoRL / MAGRPO is incremental but the combination (hierarchy + selective communication + hot-swap calibration) is useful."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The punchline is practical: a hierarchical multi-agent stack with four small learned gates (assignment, depth, cross-read, prune) plus per-agent affine calibration, trained end-to-end with a critic-free GRPO variant, beats Puppeteer on GSM8K/MMLUPro/GPQA at roughly half the active parameters (~17B vs ~28B). On AIME it trails, which the authors own.\n\nWhat is actually new is the combination, not any single piece. Managers that only emit grouping signals (no tokens), a Cross-Read mask inside masked multi-head fusion, a Prune gate before the expensive combine, and κa maps that let you hot-swap a 7B for GPT-4o-mini with 64 anchors and recover in a few batches. CoGRPO’s per-component importance ratios on a joint discrete policy are a clean adaptation of GRPO; Table 4 shows it ahead of MAGRPO-obj, shared-reward, and MAPoRL-style critics on the same architecture. Ablations (Table 9) isolate hierarchy and cross-attention as the biggest accuracy drivers; calibration removal is catastrophic (18-point drop, >20 queries to recover). Multi-seed means, fixed-k sweeps, token trajectories, and a public GitHub link make this more reproducible than most multi-agent LLM systems papers.\n\nSoft spots, in proportion. The shared group-relative advantage (Eqs. 11–12) is noisy by design; the stress-test is right that Table 4 does not fully isolate whether that signal is low-variance enough for four discrete factors. Gains could lean more on architecture than on the exact credit recipe. The free-parameter surface is large (G, η, β, λc, λm, λa, regularizers, τP, etc.), the agent pool is small and fixed (six agents, three backends), and AIME shows capacity still matters more than routing for olympiad math. None of these sink the central efficiency claim on the multi-domain suites.\n\nMath and citation pattern look fine: no circular fitting, rewards are correctness-plus-cost, related work covers AutoGen/MetaGPT/Puppeteer/MAPoRL/MAGRPO and adaptive-depth/MoE lines without obvious omissions. This is for people building multi-agent routers who care about active-parameter budgets and runtime expert swaps. I would send it to peer review; a serious referee can pressure the credit-assignment story and demand a broader pool, but the empirical package already earns the time.","headline":"Solid systems paper: four jointly trained gates plus hot-swap calibration deliver real efficiency gains on three of four benchmarks; CoGRPO credit is the softest link but not a load-bearing collapse.","tokens_in":26192,"tokens_out":634,"would_cite":true,"duration_ms":8833,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Four learned gates can make a multi-agent LLM system more accurate than fixed ensembles while using roughly half the active compute.","keywords":["multi-agent LLMs","adaptive depth","gated routing","selective communication","group-relative policy optimization","hot-swappable experts","efficient reasoning"],"falsifier":"Retrain or re-evaluate the same architecture with a method that gives distinct counterfactual credit to each gate (or that removes std-normalization and clipping) and check whether the accuracy and compute advantages over Puppeteer-style fixed ensembles disappear on MMLUPro and GPQA.","tokens_in":26047,"feed_emoji":"⚙️","tokens_out":906,"duration_ms":11382,"temperature":0.7,"pith_summary":"Multi-agent LLM setups usually pay for every agent on every query, without learning which specialists to call, how deep to go in a hierarchy, or when agents should talk to each other. This paper claims that four small learned gates—assignment, depth, selective cross-reading, and branch pruning—can answer those questions per query inside one hierarchical system called GRADE. The gates and adapters are trained end-to-end with CoGRPO, a critic-free group-relative policy method that gives every participating gate and agent the same shared advantage from a group of rollouts. Experts live in a hot-swappable registry; a short per-agent calibration map lets a model be replaced at inference time without retraining the gates. Empirically, at about 17B average active parameters GRADE beats listed baselines on GSM8K, MMLUPro, and GPQA, including a 4.8-point gain over the strongest baseline on MMLUPro at roughly half the active compute, while remaining competitive on AIME-2025 where raw model depth still matters. Ablations single out hierarchy and masked cross-attention as the largest accuracy drivers, and show that calibration is required for safe expert swaps.","feed_headline":"Four gates cut multi-agent LLM compute roughly in half","feed_subtitle":"GRADE beats fixed ensembles on reasoning benchmarks at ~17B average active parameters","key_machinery":"GRADE’s coordination harness: four lightweight MLP gates plus CoGRPO, which samples a group of joint coordination rollouts, normalizes their dense rewards into a shared advantage, and applies that advantage to every gate and surviving agent that participated, without a learned critic.","core_discovery":"A hierarchical multi-agent system controlled by four jointly trained gates (agent selection, hierarchy depth, selective inter-agent communication, and branch pruning), optimized with a shared group-relative advantage under CoGRPO, can outperform fixed multi-agent ensembles and monolithic baselines on standard reasoning benchmarks while activating only about 17B parameters on average and supporting runtime expert substitution via per-agent calibration.","pith_inferences":["The same gate stack could be attached to stronger math specialists to close the remaining AIME gap without permanently paying for those specialists on easy queries.","Shared-advantage training may scale poorly if the number of discrete gate decisions grows much larger; sparse counterfactual baselines might then become necessary.","Calibration maps suggest a practical pattern for any multi-model system that mixes open-weight and API backends under one learned router."],"forward_implications":["Per-query adaptive depth and agent count can replace always-on multi-agent stacks for many reasoning workloads, cutting active parameters roughly in half without accuracy loss on multi-domain benchmarks.","Selective, gated communication between specialists is more useful than full broadcast; forcing all pairs to communicate hurts accuracy.","Runtime expert swaps become practical if each backend keeps a cheap affine calibration map refreshed on a few dozen anchor queries.","Harder tasks naturally recruit more agents and deeper hierarchy levels once the gates are trained, so difficulty-aware compute allocation can be learned rather than hand-tuned."],"fun_headline_variants":["Four gates halve multi-agent compute while lifting scores","GRADE gates select agents and depth for 17B-avg efficiency","Adaptive routing and pruning beat fixed ensembles at half cost","Joint gates for selection depth communication and branch cuts","Hierarchy plus CoGRPO gates cut params without losing accuracy"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That giving every gate and surviving agent the same shared group-relative advantage from a rollout is stable and fair enough credit assignment for a hierarchy of discrete routing, depth, communication, and pruning decisions.","fun_headline_variants_meta":{"raw":{"variants":["Four gates halve multi-agent compute while lifting scores","GRADE gates select agents and depth for 17B-avg efficiency","Adaptive routing and pruning beat fixed ensembles at half cost","Joint gates for selection depth communication and branch cuts","Hierarchy plus CoGRPO gates cut params without losing accuracy"]},"model":"grok-4.5","effort":"low","cost_usd":0.006298,"raw_usage":{"total_tokens":1579,"prompt_tokens":794,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":62980000,"prompt_tokens_details":{"text_tokens":794,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":722,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":794,"tokens_out":63,"duration_ms":10019,"temperature":1.0,"reasoning_tokens":722,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T08:53:24.975094+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Retrain or re-evaluate the same architecture with a method that gives distinct counterfactual credit to each gate (or that removes std-normalization and clipping) and check whether the accuracy and compute advantages over Puppeteer-style fixed ensembles disappear on MMLUPro and GPQA.","supporting_citations":[],"review_version":1}