{"id":"c9b1dfe3-8932-4beb-ad8c-78ac92ce7676","arxiv_id":"2606.31413","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Hard top-1 token routing over frozen, independently RLVF-trained LoRA experts matches or beats soft-routing mixtures with about 8–10x fewer trainable parameters.","lead":"The paper tests whether a large language model can reuse several separately trained reasoning add-ons (LoRA adapters) by picking one add-on per token at full strength instead of blending them. On five benchmarks it matches or beats soft-blending baselines while training far fewer parameters, though a simple prompt-level classifier does just as well when each input belongs to one domain.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scale-mismatch mechanism is confounded with expert retraining: no frozen-expert soft-routing baseline is reported, so hard routing's advantage may reflect freezing experts rather than unit-scale selection.","rationale":"We read the paper in good faith. It is a carefully ablated two-stage framework with honest limitations; the empirical system (frozen experts + hard top-1 + STE) is plausible and the parameter-count advantage is real by construction. The reader's CONDITIONAL verdict is appropriate. The load-bearing uncertainty is whether the paper's mechanism (unit-scale hard routing is the cause of the gains) is actually identified. As documented, every soft-routing comparison also retrains the experts, so the data cannot distinguish scale-matching from freezing effects. This is a correctable experimental gap rather than a fatal flaw: a single frozen-expert soft-routing baseline would settle it. We therefore keep the verdict unchanged while flagging the required control. We also note the paper's own prompt-level classifier result and scale-mismatch zero-gradient analysis, which are honest and partially supportive; they do not resolve the confound.","tokens_in":22593,"tokens_out":5527,"duration_ms":51047,"concrete_test":"Re-run Stage II on LLaMA-3B with the same frozen RLVF experts, the same shared linear router, the same attention LoRA (R=128, Table 16), the same 1000-sample distillation data, and the same training hyperparameters, but replace hard top-1 + STE with two soft variants: (a) top-1 soft weighting (apply p_j E_j(x)) and (b) normalized soft top-2 (weights sum to 1), both trained by standard softmax backprop with no preservation loss, all experts frozen. Compare average accuracy over the five tasks against the reported 72.07. If either soft-frozen variant lands within 0.5 points of 72.07, the unit-scale mechanism is not the driver and the central explanation is unsupported; if both drop by >1.5 points, the scale-mismatch mechanism is corroborated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism claim (§4.1, Eq. 8) is that soft routing's weighting (ω_i < 1) breaks the unit-scale LoRA update, and that this mismatch is why soft mixtures underperform. The evidence in Tables 6/7 compares Hard-Routed MoR-LoRA (frozen experts) against LoRAMixer top-1/top-2 (soft routing with continued expert training via an L2 preservation loss). Routing type and expert-retraining status therefore vary together; the ~1.2–1.5 point gaps (72.07 vs 70.81 at 3B; 79.80 vs 78.33 at 8B) are equally consistent with 'retraining experts degrades them' as with 'soft weighting is harmful.' No baseline with soft routing and frozen experts is reported: LoRAMixer without preservation loss, or a simple soft-weighted combination of the same frozen experts with a trained router, would isolate the mechanism. Appendix E's soft-train/hard-inference (66.49 vs 72.07 at 3B) changes the training-time output scale, not the routing family, so it does not disentangle the two. Table 1 adds MoLE, but MoLE also retrains adapters. Thus the paper's own claim that 'routing quality dominates expert adaptation' (§5.2) is not actually supported by a controlled comparison; the alternative explanation — freezing experts alone is sufficient, regardless of hard vs soft routing — remains viable. This matters because the paper's stated contribution is the identification of the scaling mismatch, not merely a parameter-efficient way to compose frozen adapters.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Hard-Routed MoR-LoRA, a two-stage framework for composing independently trained LoRA adapters. Stage I trains domain-specific reasoning experts with RLVF/GRPO; Stage II freezes all experts and trains only a shared token-level top-1 router with a straight-through estimator plus a small attention LoRA on distilled expert traces. The central claim is that soft MoE-style routing scales frozen LoRA updates by routing probabilities, violating the unit-scale additive update assumed during LoRA training, whereas hard top-1 routing preserves each expert's trained function and yields better or comparable performance with far fewer trainable parameters. The headline results are on LLaMA-3B and 8B with five benchmarks: 72.07 vs 70.81 (3B) and 79.80 vs 78.33 (8B) against LoRAMixer normalized top-2, with roughly 73M vs 606M and 109M vs 1.133B trainable parameters, respectively.","tokens_in":22849,"tokens_out":4891,"duration_ms":49008,"significance":"If the central mechanism were cleanly established, the work would be a useful contribution to modular and parameter-efficient LLM adaptation: it combines RLVF-trained experts, a frozen-expert composition principle, and an STE-based token-level router, and it ships code plus detailed hyperparameters. The manuscript also contains honest and useful control analyses: the prompt-level routing baseline, the mixed-domain and unseen-dataset evaluations, the Stage II data-budget ablation, and Appendix C's proof that normalized top-1 soft routing has zero router gradient are all valuable. However, the significance is currently tempered by the fact that the main mechanistic claim is not isolated from expert retraining, and by the absence of any variance estimates in a reinforcement-learning pipeline.","major_comments":[{"comment":"The central scale-mismatch mechanism is confounded with expert retraining. The LoRAMixer baselines in Tables 6/7 continue training the adapters with an L2 preservation loss, while Hard-Routed MoR-LoRA freezes them. Thus the reported gains (72.07 vs 70.81 at 3B; 79.80 vs 78.33 at 8B) are equally consistent with the alternative explanation that retraining degrades independently trained experts, independent of soft weighting. A soft-routing baseline over the same frozen experts is missing — e.g., a learned soft top-1/top-2 over the frozen FFN LoRAs, or LoRAMixer without the preservation loss. The Appendix E soft-train/hard-inference variant does not disentangle this, since it stays within the hard-routing family. Note also that the prompt-level frozen-expert baseline in Table 1 achieves 72.79/79.87, comparable to or above the proposed method on clean single-domain tasks, suggesting that fre","section":"§4.1, Eq. (8) and Tables 6/7"},{"comment":"The RLVF-versus-SFT expert comparison is not controlled: RLVF uses learning rate 5e-6 and 2 epochs, while SFT uses 1e-6 and 10 epochs. The claim that 'RLVF produces stronger reasoning experts' is therefore confounded with optimization budget and learning-rate schedule. Since Stage I expert quality is load-bearing for the entire two-stage result, please either match hyperparameters or provide a sweep showing the effect is due to the RLVF objective rather than the training configuration.","section":"§5.1, Table 5 vs Table 16"},{"comment":"No seeds, error bars, or confidence intervals are reported anywhere. This is especially problematic for an off-policy GRPO pipeline with temperature 1.1, four sampled generations per prompt, and small Stage II budgets (1000 samples per dataset). The headline margins over LoRAMixer normalized top-2 are ≈1.2–1.5 points, which may be within run-to-run noise. Please report means and standard deviations (or bootstrap intervals) over at least 3–5 seeds for the main tables, including Tables 5–13.","section":"All experimental tables"},{"comment":"The histogram of dominant routing weights under normalized soft top-2 is presented as evidence that soft routing 'collapses' toward single-expert selection, but it is computed for LoRAMixer, where the experts have been retrained under the soft objective. It therefore does not characterize what a soft router over the original frozen experts would do, and it is not causally connected to the performance gap. Moreover, a mean dominant weight of 0.71 leaves 0.29 on a second expert; whether that residual weight is harmful is exactly what the missing frozen-expert soft baseline would test.","section":"§5.4, Fig. 4"}],"minor_comments":[{"comment":"There is a garbled sentence: 'There is a rich body of research have Several works, therefore, studiedy how to combine multiple pretrained adapters.' Please rewrite.","section":"§2, Related Work"},{"comment":"'SV AMP' should be 'SVAMP' (the dataset name) throughout the appendix and in Table 14.","section":"Appendix F.1 and Table 14"},{"comment":"The 1B baseline GSM8K value is shown as '03.49' and the RLVF value as '00.05'; the leading zeros are inconsistent with the rest of the table.","section":"Table 5"},{"comment":"Typo: 'the routerselects a single expert' should be 'the router selects a single expert'.","section":"Figure 1 caption"},{"comment":"The BoolQ reference title is misspelled as 'BooQ: Exploring the Surprising Difficulty...'.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the manuscript is clearly written, but the main mechanistic claim is not yet supported by a controlled comparison. I would like to see either a frozen-expert soft-routing baseline or a clear statement that the paper only claims 'frozen experts plus learned selection' as an engineering recipe, not that soft weighting per se is harmful. The absence of seeds is also a serious concern for a journal decision. If the authors add the missing baselines and variance estimates, the paper could become a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper deserves a serious referee, but the mechanism claim in the title is not yet supported. The construction is clean: train reasoning LoRA experts independently with RLVF, freeze them, and learn only a shared token-level top-1 router plus a small attention LoRA via distilled traces, using a straight-through estimator. That specific combination is new, and the scale-mismatch framing in Eq. 8 — soft weights change the unit-scale additive update LoRAs trained under — is a genuinely useful viewpoint in the LoRA-composition literature.\n\nCredit where it's due: the ablations are well designed and informative (attention LoRA rank saturates quickly, 1000 samples per domain is nearly enough, deterministic STE beats Gumbel-softmax under limited supervision), and Appendix C correctly shows why normalized top-1 soft routing gives zero router gradient. The Limitations section is refreshingly candid: integration is not zero-shot, and prompt-level routing is 'already highly competitive' on clean single-domain inputs.\n\nThe soft spots are in attribution, not in execution. The headline comparisons (Tables 6/7) pit hard routing on frozen experts against LoRAMixer, which retrains the adapters with a preservation loss. Routing type and expert-retraining status vary together, so the 1.2–1.5 point gap is equally consistent with 'retraining degrades experts' as with 'soft scaling is bad.' There is no frozen-expert soft-routing baseline anywhere — that's the missing control for the central claim. Second, no seeds or error bars anywhere, which is a real problem for a pipeline that involves GRPO, distillation, and STE. Third, the RLVF-vs-SFT expert comparison changes learning rate and epochs between the two (Table 16), so that comparison is also confounded. Most tellingly, the paper's own prompt-level BERT classifier matches or beats the token-level router on all five clean single-domain benchmarks (72.79 vs 72.07 at 3B; 79.87 vs 79.80 at 8B); token-level routing only wins on the constructed GSM8K+BoolQ mixed prompt. So the practical advantage of the method's distinctive feature is, on this evidence, confined to a narrow setting.\n\nNone of this kills the paper. The method is plausible, honestly reported, and parameter-efficient enough to be practically useful. But the mechanism story — unit-scale hard routing is the cause of the gains — needs a controlled test with soft routing on the same frozen experts before I'd believe it. If that test holds, this is a genuinely useful result.\n\nRecommendation: send to peer review. Ask for the missing frozen-expert soft baseline, seed-level variance, and a cleaner RLVF comparison. I'd read the revision.","headline":"A clean construction and honest evaluation, but the unit-scale mechanism is confounded with expert freezing and the token-level advantage only shows on a contrived mixed-domain test.","tokens_in":23530,"tokens_out":2524,"would_cite":true,"duration_ms":23968,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Composing frozen LoRA adapters works better with hard per-token selection than with soft weighted blending, and needs far fewer trainable parameters.","keywords":["hard routing","LoRA composition","mixture of experts","straight-through estimator","unit-scale update","RLVF reasoning experts","token-level routing","parameter-efficient fine-tuning"],"falsifier":"Run the same frozen expert on the same task with its update scaled by 1.0 versus, say, 0.5, keeping everything else identical; if accuracy does not drop, the scale mismatch is not the reason soft routing loses. Alternatively, train a soft-routing mixture with frozen experts and no preservation loss and compare directly with hard routing.","tokens_in":22304,"feed_emoji":"🧩","tokens_out":4664,"duration_ms":38455,"temperature":0.7,"pith_summary":"The paper argues that when independently trained LoRA adapters are combined into a single model, mixing them with soft routing weights violates the unit-scale additive update under which each adapter was trained. It proposes Hard-Routed MoR-LoRA: freeze all experts, distill reasoning traces from them, and train only a lightweight router that selects exactly one expert per token at full scale, using a straight-through estimator for gradients. On five reasoning benchmarks across multiple model sizes, this matches or beats soft-routing baselines while training roughly an order of magnitude fewer parameters. A reader should care because it suggests modular reuse of domain experts is mostly a selection problem, not a relearning problem.","feed_headline":"Hard top-1 routing wins for frozen LoRA composition","feed_subtitle":"Selecting exactly one adapter per token keeps trained behavior intact and needs ~10x fewer parameters than soft mixtures.","key_machinery":"The key object is a token-level top-1 router trained with a straight-through estimator: the forward pass selects exactly one expert, applies its LoRA update at weight 1, and the backward pass uses the softmax probabilities as a surrogate gradient. The paper also formalizes the scale-mismatch it is correcting: a soft mixture computes W = W_base + sum_i ω_i A_i B_i, so each frozen adapter's update is multiplied by ω_i < 1, which the authors argue breaks the unit-scale assumption the LoRA training used.","core_discovery":"The central claim is that the integration stage for frozen LoRA experts should be a pure routing problem: keeping every expert fixed and learning only when to apply which one, with the chosen expert applied at unit scale. The paper identifies a mechanism for why soft mixtures lose: routing weights below one rescale each LoRA update and change the effective function the adapter performs. Hard top-1 routing through a straight-through estimator restores the original unit-scale application and still allows end-to-end training of the router. Empirically, on instruction-tuned models from 1B to 8B parameters across five distinct tasks, the hard-routed mixture achieves average accuracy at least as h","pith_inferences":["If the scale-mismatch mechanism is the real cause of soft routing's losses, then even a properly selected expert applied at fractional scale should underperform the same expert at unit scale; this is a directly testable prediction the paper does not run.","The unit-scale principle suggests a natural extension to an abstention route: when no expert is a good match, the model could fall back to the base computation graph, which the authors list as future work but do not evaluate.","The parameter-efficiency advantage should widen with larger expert pools, since soft-routing baselines that retrain experts incur per-expert costs that hard routing avoids."],"forward_implications":["Frozen LoRA experts can be composed into one model without access to the original training data and without retraining the experts, as long as a small labeled set for routing is available.","The number of trainable parameters in the integration stage does not grow with the number of experts, since only a shared router and small attention LoRA are trained.","Normalized soft top-2 mixtures often behave like near top-1 selection, suggesting hard routing captures the same behavior more directly and with sparser computation.","Token-level routing handles inputs that mix multiple domains better than prompt-level routing, which must commit to a single expert for the whole prompt."],"fun_headline_variants":["Hard top-1 routing beats soft mixtures for frozen LoRA experts","Pick one LoRA per token: unit-scale routing preserves reasoning","Frozen LoRA experts: hard routing needs ~10x fewer params","Select, don't relearn: hard-routed MoR-LoRA keeps expert behavior","Unit-scale hard routing: simple composition for frozen LoRAs"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that applying a frozen LoRA update with a weight less than one degrades the adapter's trained behavior; the paper never isolates this effect from the simultaneous choice of not retraining the experts.","fun_headline_variants_meta":{"raw":{"variants":["Hard top-1 routing beats soft mixtures for frozen LoRA experts","Pick one LoRA per token: unit-scale routing preserves reasoning","Frozen LoRA experts: hard routing needs ~10x fewer params","Select, don't relearn: hard-routed MoR-LoRA keeps expert behavior","Unit-scale hard routing: simple composition for frozen LoRAs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1172,"prompt_tokens":760,"completion_tokens":412,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":317}},"tokens_in":504,"tokens_out":412,"duration_ms":3757,"temperature":1.0,"reasoning_tokens":317,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T09:16:42.796196+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same frozen expert on the same task with its update scaled by 1.0 versus, say, 0.5, keeping everything else identical; if accuracy does not drop, the scale mismatch is not the reason soft routing loses. Alternatively, train a soft-routing mixture with frozen experts and no preservation loss and compare directly with hard routing.","supporting_citations":[],"review_version":2}