{"id":"f63adaa3-5552-470e-a5d1-f06bd116c4f7","arxiv_id":"2608.09251","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"MoRSE trains role- and subtask-specific LoRA experts with a prototype router and hierarchical GRPO, improving LLM code-generation benchmarks and held-out task generalization.","lead":"MoRSE makes AI agents specialize in two ways: each agent gets a unique subtask in a task graph, and each uses its own small set of trained expert modules on a shared language model. Tests on code generation benchmarks show it beats existing multi-agent systems and generalizes to tasks it wasn't trained on.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 3.1's variance-reduction equality does not follow from the stated proof: the cross term between reward deviation and score norm is dropped without justification, so HGRPO's claimed theoretical guarantee is unsupported.","rationale":"The reader's weakest assumption is the validity of the rule-based verifier and DAG decomposition. That concern is real, but the paper partially mitigates it with an execution-verified evaluation on SRDD and by using direct unit-test rewards on SciCode. The more decisive, checkable soft spot is Proposition 3.1: the paper's formal variance-reduction theorem appears to be mathematically unsupported. The proof treats A and C as uncorrelated and then implicitly treats A∇ and C∇ as uncorrelated, but the cross covariance contains E[(u-μ_b)||∇ log p||²|b], which is not zero from the score-function identity alone. A concrete synthetic counterexample can settle this immediately. If the theorem is false, the theoretical contribution of HGRPO is invalid, although the empirical results, including the C-vs-D ablation and training-dynamics plots, could still be correct and useful. I therefore keep the overall verdict at CONDITIONAL rather than moving to REJECT: the authors should either supply a correct variance-reduction proof (with the cross term bounded or canceled) or remove the 'provable' claim and present HGRPO's benefit as empirical. The reader's conditions (multi-seed error bars, released code) still stand. This concern is not an ad hominem or a disagreement with consensus; it is an internal inconsistency in a central derivation.","tokens_in":27127,"tokens_out":15574,"duration_ms":157979,"concrete_test":"Verify Eq. (10) on the minimal instance of §3.3: let b∈{1,2} with uniform router probabilities, p(y|b)=N(y; φb, 1), reward u=y² (or its bounded version min(y²,1) if a [0,1] reward is preferred), and φ>0. Compute, by closed form or 10^6-sample Monte Carlo, the variances of g_HGRPO=(u-μ_b)∇log p and g_GRPO=(u-μbar)∇log p at the paper's group sizes (B=4, M=4). If Var[g_HGRPO]≥Var[g_GRPO] for any φ, or if the claimed equality with σ_C² E||∇log p||² fails, Proposition 3.1 and Corollary G.1 must be withdrawn or substantially repaired.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central mechanism is HGRPO, and Proposition 3.1 is its formal support. The proof argues that A=(u-μ_b) and C=(μ_b-μbar) are orthogonal under conditional expectation, so Var[(A+C)∇]=Var[A∇]+Var[C∇]. This step is invalid: orthogonality of A and C gives E[AC]=0, but the gradient cross term is E[AC||∇_Φ log p||²], which does not factor. Condition on the route b, the cross term is E_b[C_b E[(u-μ_b)||∇_Φ log p||²|b]], and the score-function identity only guarantees E[∇ log p|b]=0, not E[(u-μ_b)||∇ log p||²|b]=0. For any reward correlated with gradient magnitude (e.g., u=y² under a Gaussian policy), this term is nonzero, so Eq. (10) and Corollary G.1 can fail; the variance difference can even be negative. Algorithm 1's empirical within-route mean μhat_b also includes the current candidate, adding a further dependence the proof does not model. The C-vs-D ablation and Appendix H still provide empirical support for HGRPO, but they do not establish the claimed 'provable' variance reduction, which is a headline theoretical contribution and a load-bearing reason the method is proposed.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MoRSE, a multi-agent LLM framework that combines (i) a task-oriented DAG decomposition assigning each agent an explicit (role, subtask) pair, (ii) a mixture of role and subtask LoRA experts selected by a prototype-based semantic router, and (iii) a hierarchical group-relative policy optimization (HGRPO) with within-route and cross-route advantages. The central claims are that the task-structure DAG alone improves held-out inference, that the LoRA expert/router module adds further parameter-level specialization, and that HGRPO provides a provable variance reduction for the expert updates while stabilizing co-optimization. The method is evaluated on SRDD and SciCode across three backbones, with budget-matched ablations, OOD splits, and an execution-verified evaluation protocol.","tokens_in":27520,"tokens_out":10904,"duration_ms":100545,"significance":"If the empirical results hold, the paper makes a meaningful contribution: it moves multi-agent LLM specialization beyond prompt-level role differentiation to parameter-level (role, subtask) adaptation, and it provides a careful evaluation design. The execution-verified independent SRDD evaluation, the budget-matched trained baselines, and the OOD category/domain splits are genuinely useful strengths, as are the three-backbone comparisons and the explicit ablation map in Appendix J. However, the paper's headline theoretical contribution, Proposition 3.1, is not supported by the proof as written, and this undermines one of the three stated pillars of the method. The empirical case for HGRPO also rests on single-seed runs, so the quantitative strength of the claims is weaker than the text suggests. The core empirical idea remains defensible and the theoretical overreach is fixable, but the current version overclaims.","major_comments":[{"comment":"The variance-reduction claim in Proposition 3.1 and Corollary G.1 does not follow from the proof. The proof treats the LoRA gradient as (u−μ_b)∇log p and the single-baseline gradient as (u−μ̄)∇log p, then asserts that the cross term between u−μ_b and μ_b−μ̄ vanishes because these two reward deviations are uncorrelated. Uncorrelatedness gives E[(u−μ_b)(μ_b−μ̄)]=0, but the gradient cross term is E[(u−μ_b)(μ_b−μ̄)∥∇Φ log p∥²], which does not factor as E[(u−μ_b)(μ_b−μ̄)]·E[∥∇Φ log p∥²]. Conditioning on the route b reduces the issue to E_b[(μ_b−μ̄) E[(u−μ_b)∥∇Φ log p∥² | b]], and the score-function identity only guarantees E[∇Φ log p | b]=0, not E[(u−μ_b)∥∇Φ log p∥² | b]=0. A concrete counterexample exists: for a Gaussian policy with mean f(x) and reward u=y², E[(u−μ_b)(y−f)² | b] is nonzero, so the claimed variance reduction can fail and can even reverse. Corollary G.1 inherits the same flaw, and it is compounded by the fact that Algorithm 1 computes the within-route mean μhat_b from the same M candidates that include the current sample, so the baseline is not independent of the action and the unbiasedness argument is only heuristic. The C-vs-D ablation (Table 2) and the training dynamics in Appendix H are legitimate empirical evidence, but they do not establish the 'provably reduces variance' statement. The authors should either prove the result under explicit additional assumptions (for example, conditional independence of the score norm and reward deviation given the route) or remove the formal guarantee and present HGRPO as a heuristic whose benefits are empirical.","section":"§3.3, Eq. (10); Appendix G"},{"comment":"There is a mismatch between the object measured empirically and the object used in Theorem L.2. The diagnostic in App. L.1 computes rank-ρ subspaces of input activations to each target layer (A^(L) ∈ R^{Np×d}, the last-token input activation), whereas Assumption L.1 and Theorem L.2 concern 'group-optimal rank-ρ LoRA update subspaces' Va, Vb. The paper states in the 'Connection' paragraph that no bridging assumption is needed because Δ is 'exactly the rank-ρ subspace divergence reported in Figure 2(a,b)', but Figure 2 measures activation subspaces, not weight-update subspaces. Activation subspace divergence and update subspace divergence are different quantities; a low-rank adapter could in principle compensate for divergent input subspaces or fail despite similar input subspaces. The theorem should be restated as a conditional result on update-subspace divergence, and the activation diagnostic should be presented as suggestive evidence rather than as an exact measurement of the theorem's premise.","section":"Appendix L, §L.1–L.3 and Figure 2"},{"comment":"All headline results are reported from a single seed, and the text states this only in the limitations appendix. With stochastic decoding at temperature 0.2, the differences in Table 1 and Table 2 have no confidence intervals, and some reported OOD deltas (e.g., Table 8 Mean Step Pass deltas of −0.63 and +0.51) are clearly within plausible run-to-run noise. The paper should either provide multiple seeds with standard deviations or, at minimum, a paired bootstrap or Wilcoxon analysis over benchmark instances, for the key comparisons (MoRSE vs. MoRSE base, MoRSE vs. fixed-LoRA, and HGRPO vs. standard GRPO). Without this, the quantitative claim that MoRSE 'attains the column-best score on every reported test cell' is not yet statistically supported.","section":"§4, Appendix C; Tables 1, 2, 7, 8"}],"minor_comments":[{"comment":"The sentence 'On the All set, the untrained MoRSEbase achieves the best overall performance comparing with all baselines' is contradicted by the Gemma-4-31B rows, where the Single base model has ECI Mean 0.763 vs. 0.734 and ECI Product 0.338 vs. 0.316. Please qualify this claim or state explicitly that 'best overall' refers to a majority of metrics rather than all metrics.","section":"§4.1, Table 1"},{"comment":"The router loss uses log πψ(E_s,i | s_i), where E_s,i is a top-K subset of subtask experts, but Eq. (5) only defines a per-expert softmax πψ(k | s_i). The paper should specify the distribution over subsets used for stochastic top-K sampling and the exact log-probability formula in Eq. (8).","section":"§3.2, Eq. (8)"},{"comment":"The bridge argument says both normalized estimators are divided by the same route-constant scale σ(b)+ε, but σ(b) is itself estimated from the same M candidates that define u(b,m), so it is not a fixed constant given the data. The finite-sample effect should be stated explicitly and not folded into the same conditional-independence argument.","section":"Appendix G, Corollary G.1"},{"comment":"The left panel's axis label appears garbled in the preprint rendering ('010 2'); please correct the typesetting of the symlog axis.","section":"Figure 6"},{"comment":"On the Test set, the Problem Pass value for MoRSE is 20.00, tied with Single, MacNet, and AFlow. The abstract's 'improvements in both whole-task and step-wise performance' should be nuanced to account for ties on this cell, or the claim should focus on the metrics where the gain is unambiguous.","section":"Table 1, SciCode Problem Pass, Gemma-4-31B"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong empirical package and a fixable theoretical overreach. I recommend major revision rather than rejection: the authors should be asked to either prove the HGRPO variance-reduction claim under explicit assumptions or reframe it as an empirically motivated design choice, to align App. L's theorem with the actual measured quantity, and to add multi-seed or bootstrap-based uncertainty estimates for the headline comparisons. The execution-verified evaluation and budget-matched ablations are valuable and should be preserved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"MoRSE is a strong systems paper whose empirical contribution is real and whose headline theoretical claim does not hold up as stated. The stress-test note on Proposition 3.1 is correct: the proof drops a cross term it is not entitled to drop. The \"provable variance reduction\" language needs to be fixed or walked back. The paper would still be worth publishing, because the empirical case for HGRPO does not rest on the theorem.\n\nWhat is actually new: the (role, subtask) factorization of LoRA experts with a prototype-based router, and the two-layer credit assignment that gives experts and router separate advantages. The pieces are known (DAG decomposition, GRPO, LoRA-MoE), but the combination is a real step beyond prompt-level role differentiation, and the system delivers the gains it claims on held-out splits of SRDD and SciCode across three backbones. The evaluation is the best part: budget-matched ablations, including a total-parameter-matched fixed-LoRA control, category/domain-disjoint OOD splits, a router-utilization analysis, and an execution-verified evaluation whose checks come from a model family disjoint from the trained backbone. The limitations appendix is unusually honest about single-seed results and the partial reward/ECI alignment.\n\nThe soft spot that matters is the math in Appendix G. The proof writes u−μ̄ = (u−μ_b) + (μ_b−μ̄), calls the two terms uncorrelated, and concludes Var[(u−μ̄)∇] = Var[(u−μ_b)∇] + σ²_C E[||∇||²]. Orthogonality of the reward deviations gives E[AC]=0, but the gradient cross term is E[AC||∇ log p||²], which does not factor. The score identity only guarantees E[∇ log p | b] = 0, not E[(u−μ_b)||∇ log p||² | b] = 0. When reward correlates with gradient magnitude, the cross term is nonzero and Eq. (10) can fail with either sign. Corollary G.1 inherits the gap.\n\nHappily, this does not sink the paper. The C-vs-D ablation and the Appendix H training dynamics show standard GRPO destabilizing with MoLE while HGRPO stays controlled; that is direct empirical evidence for the credit-assignment design. The fix is straightforward: prove a correct variance statement under explicit assumptions (e.g., conditional independence of reward and gradient norm within routes) or downgrade the claim to an empirical finding. The subspace-coverage theorem in Appendix L checks out, and the rest of the math is fine. The remaining concerns are minor: single-seed runs, and trust in the rule-based verifier's step rewards, which is reasonable for SciCode's unit tests and coarser for SRDD, though the independent execution-verified evaluation mitigates the latter.\n\nVerdict: this deserves real peer review, and it would make a good reading-group paper — the proof gap is exactly the kind of thing worth dissecting. I would cite it for the MoLE+HGRPO design and the careful empirical protocol, and I would send it to referees with a request that the theoretical claim be corrected or softened.","headline":"Solid empirical paper; the headline variance-reduction theorem has a real proof gap that needs correcting or downgrading before this goes out.","tokens_in":28017,"tokens_out":7575,"would_cite":true,"duration_ms":64025,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MoRSE claims that conditioning each agent's parameters on both role and subtask, not just prompts, is the key to multi-agent LLM performance on complex code tasks.","keywords":["multi-agent systems","large language models","mixture of experts","low-rank adaptation","reinforcement learning","credit assignment","code generation","task decomposition"],"falsifier":"Train MoRSE exactly as reported on SRDD with Qwen3-4B, then rerun the same HGRPO objective with each node's verifier reward replaced by a shuffled version of the same reward values. If whole-task and step-wise scores still reach the Table 2 row-D levels, the claim that valid step-level credit drives the gains is false; if performance drops to the MoRSE-base level, the verifier-based credit is confirmed as load-bearing.","tokens_in":26975,"feed_emoji":"🧩","tokens_out":7460,"duration_ms":59111,"temperature":0.7,"pith_summary":"Large language model multi-agent systems currently differentiate agents mostly by prompts, and the paper's motivating diagnosis is that this leaves outputs redundant and capabilities underspecialized. MoRSE aims to show that spelling out each agent's responsibility as a (role, subtask) pair, and then giving agents parameter-level specialization through a prototype-routed mixture of role and subtask LoRA experts, substantially outperforms prompt-level role differentiation. A hierarchical group-relative policy optimization (HGRPO) with two-layer credit assignment is introduced to train experts and router stably under sparse step rewards. The paper reports best held-out whole-task and step-wise scores over all baselines on two code-generation benchmarks across three backbones, with trained specialization transferring to held-out task categories and domains. If correct, the result identifies parameter-level heterogeneity, not agent count or prompt design alone, as the key lever in multi-agent LLM systems.","feed_headline":"Role-subtask experts beat prompt-only agents on code tasks","feed_subtitle":"MoRSE tunes each agent's parameters by role and subtask and tops held-out scores on SRDD and SciCode.","key_machinery":"The load-bearing object is a dual-factorized mixture of LoRA experts: a small pool of role experts (one per role such as execute or merge) plus a shared pool of subtask experts, selected per agent call by a prototype-based semantic router that scores subtask embeddings against learnable prototypes and picks the top-$K$ subtask experts. The active set is composed as a LoRA combination on a frozen backbone, so each (role, subtask) call obtains its own low-rank adaptation subspace at near single-model cost. Training is carried by HGRPO, whose two-layer credit assignment computes a within-route advantage for the expert update and a cross-route advantage for the router update; this conditional-baseline structure removes the cross-route component of reward variance from expert gradients. The task-oriented DAG decomposition supplies the (role, subtask) labels and rule-based step rewards that condition and supervise this machinery.","core_discovery":"On its own terms, the paper establishes that (role, subtask)-conditional specialization at both the task-structure and parameter levels yields measurable gains over untrained structural decomposition and over prompt/role-only multi-agent baselines. The central claim is that router and expert quality have to be decoupled: HGRPO assigns a within-route advantage to the LoRA experts and a cross-route advantage to the router, and Proposition 3.1 states this strictly reduces the LoRA-expert gradient variance relative to a single shared GRPO advantage. The supporting subspace analysis argues that different roles and subtasks occupy divergent rank-$\\rho$ update directions, so no single shared LoRA can cover them, and the paper formalizes this as a coverage limit (Theorem L.2). Empirically, MoRSE attains the best held-out test scores over all baselines in both whole-task and step-wise performance on SRDD and SciCode across three backbones, and the trained specialization outperforms an untrained variant and a fixed-LoRA fine-tuning baseline on held-out categories and domains.","pith_inferences":["If the LoRA subspace-coverage limit generalizes, then any setting where agent subgroups need divergent low-rank updates should favor conditional mixtures of adapters over one shared adapter; this predicts similar gains outside code generation, for example in multi-step planning or tool use.","HGRPO's router-expert credit split is a structural rather than temporal decomposition, so it could be composed with rollout-time hierarchies to assign credit along both axes at once.","A testable corollary of the variance formula is that HGRPO's advantage over standard GRPO should grow as cross-route reward variance increases; experiments that manipulate router-induced reward spread could verify this directly.","Because the whole pipeline leans on rule-based step rewards, the magnitude of the reported gains should depend on verifier quality; comparing training under shuffled versus faithful verifier rewards would isolate that dependence."],"forward_implications":["Multi-agent LLM systems can shift from prompt-level role descriptions to actual parameter-level specialization while keeping the backbone frozen and training only small adapters.","Learned subtask experts compose for held-out task categories and domains, suggesting that specialization transfers as reusable skills rather than memorized training examples.","HGRPO makes routed-mixture training stable under sparse step rewards, removing a known fragility of standard GRPO when experts and router are co-optimized.","Because gains appear at the step level as well as end-to-end, per-subtask credit assignment is meaningful enough to drive process-level improvement.","The framework's cost profile stays close to a single model, since only LoRA experts and router parameters are trained on the shared backbone."],"supporting_citations":[{"why":"supplies the SRDD benchmark, a role-based baseline pipeline, and the Exec/ECI evaluation metrics.","marker":"[24]"},{"why":"provides the large-population role-prompted DAG baseline whose output redundancy MoRSE diagnoses and outperforms.","marker":"[25]"},{"why":"supplies the SciCode benchmark with executable step-level unit tests used for rewards and evaluation.","marker":"[32]"},{"why":"is the search-based workflow-generation baseline MoRSE is compared against.","marker":"[40]"},{"why":"defines the group-relative policy optimization (GRPO) objective that HGRPO extends with two-layer credit assignment.","marker":"[28]"},{"why":"supplies the conditional-baseline variance-reduction argument underlying Proposition 3.1.","marker":"[7]"},{"why":"provides the LoRA parameter-efficient adaptation used to build role and subtask experts.","marker":"[13]"},{"why":"motivates the dependency-aware DAG decomposition used for per-node (role, subtask) structure.","marker":"[34]"},{"why":"motivates the prototype-based semantic routing mechanism for subtask experts.","marker":"[30]"},{"why":"grounds the sparsely-gated mixture-of-experts design the LoRA expert pools build on.","marker":"[29]"}],"fun_headline_variants":["Role-subtask LoRA experts beat prompt-only agents","Decoupling expert and router gradients boosts code tasks","Mixture of role-subtask experts tops code benchmarks","Role-subtask specialization generalizes across code tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The rule-based verifier is assumed to provide valid step-level rewards, and the planner's DAG decomposition is assumed accurate enough for subtask-level credit assignment to propagate correct signals; if either signal is noisy or misaligned with true subtask quality, the HGRPO training signal is corrupted and the reported gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Role-subtask LoRA experts beat prompt-only agents","Decoupling expert and router gradients boosts code tasks","Mixture of role-subtask experts tops code benchmarks","Role-subtask specialization generalizes across code tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000299,"raw_usage":{"total_tokens":1786,"prompt_tokens":1062,"completion_tokens":724,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":663}},"tokens_in":678,"tokens_out":724,"duration_ms":6154,"temperature":1.0,"reasoning_tokens":663,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:25:06.133660+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MoRSE exactly as reported on SRDD with Qwen3-4B, then rerun the same HGRPO objective with each node's verifier reward replaced by a shuffled version of the same reward values. If whole-task and step-wise scores still reach the Table 2 row-D levels, the claim that valid step-level credit drives the gains is false; if performance drops to the MoRSE-base level, the verifier-based credit is confirmed as load-bearing.","supporting_citations":[{"cited_title":"Variance reduction techniques for gradient estimates in reinforcement learning.Journal of Machine Learning Research, 5(Nov): 1471–1530, 2004","cited_arxiv_id":null,"evidence_quote":"supplies the conditional-baseline variance-reduction argument underlying Proposition 3.1."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"motivates the prototype-based semantic routing mechanism for subtask experts."}],"review_version":2}